## Getting started

Image segmentation is the process of partitioning a digital image into multiple segments (sets of pixels or image objects). In the context of plant image analysis, segmentation is used to simplify the representation of an image into something easier to analyze. For example, when using count_objects() to count crop grains, first the grains need to be isolated (segmented) from the background. In pliman the following functions can be used to segment an image.

In pliman the following functions can be used to segment an image.

• image_binary() to produce a binary (black and white) image
• image_segment() to produce a segmented image (image objects and a white background).
• image_segment_iter() to segment an image iteratively.

Both functions segment the image based on the value of some image index, which may be one of the RGB bands or any operation with these bands. Internally, these functions call image_index() to compute these indexes. The following indexes are currently available.

Index Equation Band
R R RGB
G G RGB
B B RGB
NR R/(R+G+B) RGB
NG G/(R+G+B) RGB
NB B/(R+G+B) RGB
GB G/B RGB
RB R/B RGB
GR G/R RGB
BI sqrt((R2+G2+B^2)/3) RGB
BIM sqrt((R2+G2+B*2)/3) RGB
SCI (R-G)/(R+G) RGB
GLI (2G-R-B)/(2G+R+B) RGB
HI (2*R-G-B)/(G-B) RGB
NGRDI (G-R)/(G+R) RGB
NDGBI (G-B)/(G+B) RGB
NDRBI (R-B)/(R+B) RGB
I R+G+B RGB
S ((R+G+B)-3*B)/(R+G+B) RGB
VARI (G-R)/(G+R-B) RGB
HUE atan(2(B-G-R)/30.5(G-R)) RGB
HUE2 atan(2(R-G-R)/30.5(G-B)) RGB
BGI B/G RGB
L R+G+B/3 RGB
GRAY 0.299R + 0.587G + 0.114*B RGB
GLAI (25*(G-R)/(G+R-B)+1.25) RGB
CI (R-B)/R RGB
SHP 2*(R-G-B)/(G-B) RGB
RI R**2/(B*G**3) RGB
G-B G-B RGB
G-R G-R RGB
R-G R-G RGB
R-B R-B RGB
B-R B-R RGB
B-G B-G RGB
DGCI ((h - 60)/60 + (1 - s/100) + (1 - b/100)) / 3 HSB
GRAY2 ((R2.2+(1.5G)^2.2+(0.6B)2.2)/(1+1.52.2+0.62.2))^1/2.2 RGB
L* (0.2126 * R + 0.7152 * G + 0.0722 * B) CIELab
a 0.55( (R - (0.2126 R + 0.7152 * G + 0.0722 * B)) / (1.0 - 0.2126)) CIELab
b* 0.55( (B - (0.2126 R + 0.7152 * G + 0.0722 * B)) / (1.0 - 0.0722)) CIELab
L*-a ((0.2126 * R + 0.7152 * G + 0.0722 * B)) - (0.55( (R - (0.2126 R + 0.7152 * G + 0.0722 * B)) / (1.0 - 0.2126))) CIELab
L*-b ((0.2126 * R + 0.7152 * G + 0.0722 * B)) - (0.55( (B - (0.2126 R + 0.7152 * G + 0.0722 * B)) / (1.0 - 0.0722))) CIELab
b*-a (0.55( (B - (0.2126 R + 0.7152 * G + 0.0722 * B)) / (1.0 - 0.0722))) - (0.55( (R - (0.2126 R + 0.7152 * G + 0.0722 * B)) / (1.0 - 0.2126))) CIELab

Here, I use the argument index" to test the segmentation based on the RGB and their normalized values. Users can also provide their own index by explicitly providing the formula. e.g., index = "R-B".

library(pliman)
#> The legacy packages maptools, rgdal, and rgeos, underpinning the sp package,
#> which was just loaded, will retire in October 2023.
#> Please refer to R-spatial evolution reports for details, especially
#> https://r-spatial.org/r/2023/05/15/evolution4.html.
#> It may be desirable to make the sf package available;
#> package maintainers should consider adding sf to Suggests:.
#> The sp package is now running under evolution status 2
#>      (status 2 uses the sf package in place of rgdal)
#> |==========================================================|
#> | Tools for Plant Image Analysis (pliman 2.0.1)            |
#> | Author: Tiago Olivoto                                    |
#> | Type citation('pliman') to know how to cite pliman     |
#> | Visit 'http://bit.ly/pkg_pliman' for a complete tutorial |
#> |==========================================================|
soy <- image_pliman("soybean_touch.jpg")
# Compute the indexes
# Only show the first 16 to reduce the image size
indexes <- image_index(soy, index = pliman_indexes()[1:16], plot = FALSE)

# Create a raster plot with the RGB values
plot(indexes) # Create a density plot with the RGB values
plot(indexes, type = "density") In this example, we can see the distribution of the RGB values (first row) and the normalized RGB values (second row). The two peaks represent the grains (smaller peak) and the blue background (larger peak). The clearer the difference between these peaks, the better will the image segmentation.

### Segment an image

The function image_segmentation() is used to segment images using image indexes. In this example, I will use the same indexes computed below to see how the image is segmented. The output of this function can be used as input in the function analyze_objects().

segmented <- image_segment(soy, index = c("R, G, B, NR, NG, NB")) plot(segmented$NB) It seems that the "NB" index provided better segmentation. "R" and "NR" resulted in an inverted segmented image, i.e., the grains were considered as background and the remaining as ‘selected’ image. To circumvent this problem, we can use the argument invert in those functions. image_segment(soy, index = c("R, NR"), invert = TRUE) ### Iterative segmentation The function image_segment_iter() provides an iterative image segmentation, returning the proportions of segmented pixels. This is useful when more than one segmentation procedure is needed. Users can choose how many segmentation perform, using the argument nseg. seg_iter <- image_pliman("sev_leaf_nb.jpg", plot = TRUE) Using a soybean sample leaf (above), I will use the function image_segment_iter to segment the diseased tissue from healthy tissue. The aim is to segment the symptoms into two classes, namely, necrosis (brown areas) and chlorosis (yellow areas), and compute the percentage of each symptom class. First, we need to find a suitable index for segmentation. # First segmentation seg <- image_segment(seg_iter, index = "all") # second segmentation seg2 <- image_segment(seg$VARI,
index = "all") Note that the same results can be obtained with image_segment_iter() using an iterative section.

# Only run iteratively
image_segment_iter(seg_iter, nseg = 2)

The "VARI" seems to be a suitable index to segment symptoms (necrosis and chlorosis) from healthy tissues. The "GLI" can be used to segment necrosis from chlorosis. Knowing this, we can now use image_segment_iter() explicitly indicating these indexes, as follows


image_segment_iter(seg_iter,
nseg = 2, # two segmentations
index = c("VARI", "GLI"),
ncol = 3)
#>      image  pixels   percent
#> 1 original 1317600 100.00000
#> 2     seg1  397044  30.13388
#> 3     seg2  102621  25.84625 It can be observed that 30.28% of the original image were characterized as symptoms (both necrosis and chlorosis). Of out this (symptomatic area), 25.92% are necrotic areas. So 7.85% of the total area were considered as necrotic areas (30.288 $$\times$$ 0.2592 or 103464/1317600 $$\times$$ 100) and 22.43% (30.28 - 7.85 or (399075 - 103464) / 1317600 $$\times$$ 100) were considered as chlorotic areas.

Users can use the argument threshold to controls how segmentation is made. By default (threshold = "Otsu"), a threshold value based on Otsu’s method is used to reduce the grayscale image to a binary image. If a numeric value is informed, this value will be used as a threshold. Inform any non-numeric value different than "Otsu" to iteratively chosen the threshold based on a raster plot showing pixel intensity of the index. For image_segmentation_iter(), a vector (allows a mixed (numeric and character) type) with the same length of nseg can be used.

seg_iter1 <-
image_segment_iter(seg_iter,
nseg = 2, # two segmentations
index = c("VARI", "GLI"),
threshold = c(0, "Otsu"),
ncol = 3,
plot = FALSE)
#>      image  pixels   percent
#> 1 original 1317600 100.00000
#> 2     seg1  103605   7.86316
#> 3     seg2   82395  79.52802
seg_iter2 <-
image_segment_iter(seg_iter,
nseg = 2, # two segmentations
index = c("VARI", "GLI"),
threshold = c(0.5, "Otsu"),
ncol = 3,
plot = FALSE)
#>      image  pixels   percent
#> 1 original 1317600 100.00000
#> 2     seg1  321999  24.43830
#> 3     seg2  101175  31.42091

image_combine(seg_iter1$images$seg1,
seg_iter2$images$seg1) Users can then set the argument threshold for their specific case, depending on the aims of the segmentation.

## Producing a binary image

We can also produce a binary image with image_binary(). Just for curiosity, we will use the indexes "B" (blue) and "NB" (normalized blue). By default, image_binary() rescales the image to 30% of the size of the original image to speed up the computation time. Use the argument resize = FALSE to produce a binary image with the original size.

binary <- image_binary(soy) # original image size
image_binary(soy,
index = c("B, NB"),
resize = FALSE) ## A little bit more!

At this link you will find more examples on how to use {pliman} to analyze plant images. Source code and images can be downloaded here. You can also find a talk (Portuguese language) about {pliman} here. Lights, camera, {pliman}!