Machine Vision Detection Parameters for Plant Species Identification

George E. Meyer, Tim Hindman, and Koppolu Laksmi

University ofNebraska, Lincoln NE 68583
Department of Biological Systems Engineering

ABSTRACT

Machine vision based on classical image processing techniques has the potential to be a useful tool for plant
detection and identification. Plant identification is needed for weed detection, herbicide application or other
efficient chemical spot spraying operations. The key to successful detection and identification of plants as
species types is the segmentation of plants from background pixel regions. In particular, it would be
beneficial to segment individual leaves from tops of canopies as well. The segmentation process yields an
edge or binary image which contains shape feature information. Results indicate that red-green-blue (RGB)
formats might provide the best segmentation criteria, based on models of human color perception. The
binary image can be also used as a template to investigate textural features of the plant pixel region, using
gray image co-occurrence matrices. Texture features considers leaf venation, colors, or additional canopy
structure that might be used to identify various type ofgrasses or broadleaf plants.

Keywords: Plants, Image analysis, Segmentation, Shape, Texture, Color

1. INTRODUCTION
Research in the area of weed or plant detection and identification has focused on two separate optical
electronic methods. One method is to just detect the presence of a weed using a simple photodiode detection
method. Another method is to use machine vision and image analysis to actually discriminate between crop
and weed species. The latter method is more difficult as it requires feature extraction in the form of shape and
textural feature analysis. The feature extraction process needs to be developed further by segmenting weeds
from crop plants and other background features, such as soil, residue, and rocks within the image.

Shape feature analysis has been used with some degree of success to identify plant types. Guyer et al 2 used
four shape parameters to evaluate binary images of leaves of eight different plant species. His methodology
was successful in identifying an individual plant from a set containing two separate species, but individual
species identification was not achieved. Guyer et al further developed shape feature analysis methodology
by attempting to build a rule-based knowledge system to simulate human interpretation of shape features.
The study used 17 basic primitive features based primarily on plant morphology. Individual species were
classified with a success rate of 69 percent, compared to 65 percent for a group of human evaluators.
Accuracy would therefore be considered low or poor.

Gerhards et al acquired Fourier transforms and shape parameters for leaf images from ten weed species
commonly found in cereal crops. The methodology was applied to binarized images of each species at
several growth stages. The success rate for individual species identification ranged from 41 to 100 percent,
with an average accuracy rate of 8 1 percent.

Woebbecke et al used five dimensionless shape features obtained from near-binary plant images to separate
monocots from dicots. These features were shown to effectively separate individual monocot species from all
dicot species after approximately ten days after emergence for a three week growing period. No attempt was
made to distinguish between individual species.

Yonekawa et al 6 applied five simple dimensionless shape factors to a set of idealized leaf image shapes
consisting of 28 overall leaf and petal types, seven leaf lobe and three compound leaf types, and twelve leaf
margin types. The system successfully distinguished among sixteen overall leaf and petal types, ten leaf lobe
and three compound leaf types, and eight leaf margin types for only a theoretical success rate of 38 percent.
No attempt was made to apply this methodology to actual leaf images.

Part of the SPIE Conference on Precision Agriculture and

Biolooical Quality • Boston. Massachusetts • November 1998 327
SPIE Vol. 3543 S 0277-786X/99/$1 0.00

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/11/2015 Terms of Use: http://spiedl.org/terms

 Robert Haralick ' gave an excellent overview of various statistical and structural approaches to textural
features in image analysis, which included optical transforms, digital transforms, textural edginess, structural
element, gray tone co-occurrence, run lengths, and so on. He also developed some important probability
definitions that apply to the statistical nature of texture as a surface property. Those statistical and structural
definitions are often used in many image analysis research applications.

Textural feature analysis has more recently received significant attention as a means of plant species
identification. Shearer and Holmes ' used color co-occurrence matrices based on hue, saturation, and
intensity. Co-occurrence matrices define the spatial dependence of gray tones within an image. Plants were
isolated with small manually placed windows. Obviously, backgrounds were carefully controlled. Individual
cultivar samples were successfully classified with an accuracy rate of over 90 percent. Mehta et al 8 used
four statistical texture measurements to distinguish between broad leaves and grasses. Compared to the
previous work, an automatic soil-background segmentation method, based on excess green was used to
isolate plant areas from background. This was followed up with digital filtering to clean up each area. The
result was a binary template image used to guide the calculation of the grayscale co-occurrence matrices
independently for plant and background. Even so, this methodology was only successful in separating broad
leaves and grasses. It was not used to determine individual plant species.

Review of the literature indicates a range of plant identification success that rates from 40% to 90%. This
indicates that there remains a random element of the machine vision process not yet fully investigated. The
most serious problem that appears during shape and textural feature analysis are extraneous pixels that are
picked up from the background. These extraneous details show up in either black and white or color images.
Only when Shearer et al, manually forced analysis or plant pixels, did the textural identification success rate
appeared to increase. Consequently, one of the main problems is the isolation or segmentation of the plant
surface from background surface of the image. Segmentation problems include chromatic adaptation, which
includes the ability of the segmentation system to adapt changes in color dues to changing lighting. Another
problem is how to encode the color. The primary two choices are red-green-blue (RGB) or hue-saturation-
value (HSV) color separations. Hue and saturation components have reduced textural details. Only the value
component has significant textural detail. Color human vision works quite differently than what we have
been programming. Humans require complimentary colors to bring out details.

The sole use of textural features has been somewhat discouraging, as results are different for various studies
according to species, surface topography and plant canopy features. Most traditional botanical keying
procedures involve shape features of individual leaves. In general, a rigorous database of combined shape
and textural features needs to be investigated to rank the importance of each feature.

Although an effective means of individual species identification has yet to be determined, development of a
commercial spot spraying system using simple optical sensors has been ongoing for several years. The
optical sensor system closest to achieving commercial success is the WeedSeeker system developed by
Patchen Inc ' o Currently, no image analysis weed detection system has been identified commercially.

The objectives ofthis research were to study:

(a) factors which affect color segmentation in plant and soil images,

(b) quantitative shape and textural features based on individual leaves that best identify different plant types.

2. MATERIALS AND METHODS

A Kodak DC 120 digital camera was used to obtain uncompressed electronic images of various plant leaves:
soybean, velvetleaf, prayer plant, and ivy. Some plants were grown in an environmental chamber at optimum
temperature, humidity, and soil moisture conditions. Other plants such as weeds were found naturally in the
parking lot. Individual leaves were harvested from the plant to obtain the images. Digital resolution was 25
dots/inch with a width of 1280 pixels and height of 960. The camera was adjusted to include an entire leaf in
the field of view. Two different lighting conditions were used, forward lighting with soil background and

328

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/11/2015 Terms of Use: http://spiedl.org/terms

 backlighting without soil background. The leaf was placed either on the soil or on the backlighting
arrangement box. The automatic flash option of the camera was used for all the backlit images. The auto
exposure and flash options can automatically adjust for light from f3.8 to f21.4 up to 1/500 ofa second.

Digital images were transferred from the camera to a Gateway 2000, 266 MHz, Pentium II computer with
64 megabytes memory running Windows NT 4.0 Workstation® for image analysis. Original images were
compressed and saved as 24-bit colorjpeg files on CD-ROM. During processing, each digital image was
loaded from the CD-ROM and resized to a width of 384 and a height of 288 pixels to stay within the memory
limitations of the computer..

Shape and texture feature analyses were calculated using algorithms written in Visual Basic 5.O®, which was
interfaced with Image-Pro 3.0 ipc32.bas ® software and Windows APlfunctions. Each color image was
converted to 256 gray scale image. The color image was also segmented to a binary template image. The
template provided the boundaries between plant and background regions. The binary template also provided
the plant edge for spatial feature analysis.. Image-Pro ipc32.bas provide the primitives for basic shape
features of maxferret, minferret, perimeter and area of the leaf maximum length, maximum width, aspect
ratio and roundness used to calculate more advanced shape factors. Five advanced dimensionless shape
features independent of size and orientation given by Yonekawa et al 6 were calculated using Visual Basic.
These were compactness, elongation, lobation, roundness and roughness. From the gray scale image,
Haralick's texture features of angular second moment, local homogeneity, inertia and entropy were computed
using Visual Basic, based on the co-occurrence matrices scanning all four directional orientations: 0°, 45 0
90°, and 1350.

The computed feature data for each leaf was transferred and stored as a record in a Microsoft Access ®
database. The feature data was then statistically analyzed using SAS® STEPDISC and CANDISC
discriminate analysis procedure. Figure 1 is a diagrammatic representation of the image analysis scheme
used in the study. The binary image provided shape features and guided the texture analysis.

Digital Image-Pro
Image Resize images, black and Visual
Kodak DC white segmentation Basic
120 routines, __________
other parameters
calculation and plant
outline extraction

SAS Text Microsoft

file
Access
STEPDISC and
CANDISC analysis OutPFeData

Figure 1: Image Analysis Scheme

Se2mentation between leaf and soil background

Several approaches to segmenting the leaf from the soil background were considered for obtaining a binary
template image. One method was to use a threshold value such that all pixels with greater value were selected
as leaf (white) and all pixels with less value were taken as soil background (black). This was done after
analyzing the histogram. Segmentation based on individual RGB and HSV color channels were considered.
One of these was excess green, defined as 2*GRB. The other is excess red described later.

Another solution was to use analyze all of the candidate objects and develop a shape selection criteria. With
this approach, the outlines or boundaries of all candidate "leaf' objects were identified using Image-Pro's
automatic color segmentation method. All outlines were saved and analyzed for size using Visual Basic

329

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/11/2015 Terms of Use: http://spiedl.org/terms

 code. The outline with the largest area was selected and assumed to be the leaf or plant. That outline was
then checked with excess green. If the outline was the leaf, the object was then painted or filled with white
pixels leaving the remaining area black as the background to create the binary template image.

Shape Features
Five dimensionless shape features independent of size and orientation identified by Yonekawa et al 6 were
used in this research. These included compactness, elongation, lobation, roundness and roughness. The basic
shape parameters are repeated here and shown in Figure 2. Convex hull (h) and diameter of inscribed circle
(D) shown in the figure are approximated by maxferret and minferret functions, respectively using Image-
Pro.
Convex hull (H)

Maximum length (L)

width (W)

Figure 2. Shape features of a leaf (after Yonekawa et al.)

Compactness (C), indicting the roundness or a compactness relating, is defined as:

C- 4pA (1)
where,
C = compactness (dimensionless)
A = area(mm square)
P = perimeter(mm)

Compactness has a maximum value of one for a circle.

Roundness (R) is a reciprocal value and indicates the roundness-compactness relating area. The value R has a
maximum value of one for a circle.
4A
R= (2)
pL
where,
R = roundness (dimensionless)
L = maximum length (mm)

Elongation (E) is defined by equation (3). The value of E has a maximum value of one for a circle and a
square.
w
(3)
L
where,
E = Elongation (dimensionless)
W = maximum width (mm)

330

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/11/2015 Terms of Use: http://spiedl.org/terms

 Lobation (B), as defined by equation (4), reflects the amplitude of the border. The value B has a maximum
value ofone for a circle.
D
B=— (4)
w
where,
B = lobation (dimensionless)
D = diameter of inscribed circle (mm)

Roughness (G), as defined by equation (5), reflects the frequency of the border and amplitude of the border.
The value G has a maximum value of one for convex figures.

(5)
G=J
where,
G = roughness (dimensionless)
H = convex hull (mm)

Texture Features

Texture features were determined from co-occurrence matrices. These matrices define the intensity and
frequency relationships between nearest neighbor pixels in the segmented image. The elements of these
matrices are the relative frequencies F1 with which two pixels, of intensities 'i' and 'j ' separated by 'd'
pixels (usually d=l), occur in the region. Co-occurrence matrices were obtained for all four scanning
directions: O, 450 900, and 135°. The textural features were calculated from the co-occurrence matrices
using Visual Basic. The code was verified with sample data provided in Haralick ' using a Mathcad 7.O®
program. Features used: angular second moment, entropy, inertia and local homogeneity are given as:

Angular Second Moment (ASM) Is a measure of the homogeneity of a region and is given by:

ASM=p (6)
where,
P entry in the co-occurrence matrix
i, j = pixel index

P..
p1
— is the (i,j) entry in the normalized co-occurrence matrix

ii 'J
Entropy (EN) is a measure of the non-homogeneity of the region and is given by:

EN=p11og(p) (7)

Inertia (IN) is given by equation (8).

IN = (i— j)2p1 (8)

Local homogeneity (LoH) is a measure of the contrast between features in the region and is given by:

LoH=(1(.)2) (9)

331

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/11/2015 Terms of Use: http://spiedl.org/terms

 Statistical Analsis

Ihe data obtained br the combined shape and texture feature analysis s'. crc analv7ed using AS. I he
procedure as used to perform a .sti'pu'lst' /l.sc'ulnhlil1nt analysis. SlFPDIS(' uses forward
selection, backward elimination, or stepsvlse selection of quantitative variables that might he useful for
discriminating among several classes. The (ANDISC procedure in SAS performs a canonical dmscriininant
analysis, computes squared \lahalanobis distances. and does both univariate and multivarmate one—st a
analyses of variance. ()utput data sets containing canonical coefficients and scores on the canonical variables
were created.

3. RESU1TS AND DtSCt'SSlO

Multiple criteria may he needed fOr successful segmentations of plant regions from background. Excess
green with successful application of median 6lters works only part of the time. Bright pixels in color images
can contain considerable green. although not appearing green phvsiologmcall\ . However, the histogram
method usuall brings in background pixel cnors in multiple modal frequencies. The intensity of light
icliection at some image locations iii the soil background may he at the same level as or highei than that of
leaf pixels. These intensity errors can shos up iii gravscale images as svell as any color rasters of red—green—
blue R(i13 ) or hue—saturation value ( HS\ ) images. Excess green segmentation was also tested, hut the
previous background problems occassionall show up. Ho ever. L'XCi'.sS green will identil\ S5 per cent or
more of the plant pixels Murch' indicates that in human visual perception that the retina of the human eye
4? blue. 32' green and 640 red cones in the retina, although there is a non-unform spatial distribution.
Figure 3 shows a comparison of excess green with an excess rid de1ned as 1 .3*R_G. Evct'ss it'd did identify
part of leaf in the upper right hand corner. Both plant red and blue are about 60 percent of the green pixel
intensity using the DC12O flash.

Figure 3a. Excess green binarized image of dandelion. Figure 3b. Excess red binarized image of dandelion.

Hue saturation and value images do not seem to provide any better charmels of potential contrast information.
\'alue is just a gray scale image. Hue shows the niost contrast as shoii in figure 4a.

Figure 4a. Hue component of dandelion color image. Figure 4b. Value component of dandelion color image.

332

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/11/2015 Terms of Use: http://spiedl.org/terms

 The use of histograms or color cubes requires a set of different rules for each new set of images. Figure 5a is
especially interesting because the soil had a lot of white perlite which showed up as background noise. The
soybean is distinctive enough to be segmented as the largest object and then fill it and the background with
white and black, respectively.

Figure 5a. Binary image of soybean leaf obtained Figure 5b. Binary image of soybean leaf
by threshold segmentation obtained by object selection and painting
method

The plant features database was large enough that this file had to broken into two separate infiles for SAS.
Thirty-six textural features and five shape features were analyzed together with Stepdisc. The procedure
executed a total of ten steps before settling on candidate features. Roundness as a leaf shape feature was most
often selected. Local homogeneity and inertia were most often selected as candidate textural features for the
four plants selected.

Figure 6 shows the statistical clustering of five plant species as a SAS canonical plot. The canonical
functions are linear combinations of the feature variables that have the highest multiple correlation among
groups. The maximum multiple correlation is called the first canonical correlation CAN1, which was 0.96.
The second canonical correlation CAN2 is found from the linear combination uncorrelated with the first
canonical variable. The CAN2 correlation was 0.78. Clustering is evident in Figure 6. Table 1 summarizes
the multivariate statistical parameters Wilk's Lambda, Pillai trace, Hotellmg-Lawley Trace, and Roy's
maximum root calculated during CANDISC. However, the database of plants is really too small at this point
and the feature variables that are thought to be good discriminant features may show up with many other
kinds of plants. The SAS 12 manual actually presents as its example, a plant species identification problem.

4. CONCLUSIONS

The research reported in this paper is only the beginning to a long process of imaging plants and studying
their shape and textural features. While many other researchers have concentrated on just processing a few
images for features, our work has concentrated on finding new methods for segmenting plant pixels from
background pixels under various environmental and lighting conditions. This work will look into some the
physiological concepts of vision. That process while highly complex is also very sensitive and accurate.

ACKNOWLEDGEMENTS

This research is supported in part by an USDA National Needs Ph.D. Fellowship and an USDA Special
Water Quality grant. Mention of specific trade names is for reference only and not to the exclusion of others
that may be suitable. This article has been approved for publication at this conference by the Agricultural
Research Division, University of the Nebraska, Lincoln. The authors are George E. Meyer, Professor, Tim
Hindman, pH Fellow, and Koppolu Lakshmi, pH graduate assistant. Some of the research was performed as
a special problem for BSEN 951, Advanced Plant Environmental Engineering during Spring Semester 1998.

333

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/11/2015 Terms of Use: http://spiedl.org/terms

 Table 1: Multivariate Statistics and F Approximations

Statistic Value F Num. DF Den DF Pr> F

Wilk's Lambda 0.016 9.94 27 85 0.0001
Pillai's Trace 1.99 6.84 27 93 0.0001
Hotelling-Lawley Trace 14.82 15.19 27 83 0.0001
Roy's GreatestRoot 12.41 42.73 9 31 0.0001

NOTE: F Statistic for Roy's Greatest Root is an upper bound.

CAN2
5

4 4

2 3
2 4
1 12

0 1 1
12 33
1

2 2 3
—1 1 2 2 3
2
33
3 33
-2 3

-3

+-
-6 -4 -2 0 2 4 6 8

CAN 1

Figure 6. Plot of CAN2*CAN1. Symbol is the leaf SPECIES identifier: 1 =Ivy, 2 = Velvet leaf,
3 = Soybean, 4 = Prayer plant.

334

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/11/2015 Terms of Use: http://spiedl.org/terms

 5. REFERENCES

1. Haralick, R. M.. Statistical and Structural Approaches to Texture. Proc. IEEE, 67(5), 786-804, 1979.

2. Guyer, D. E., G. E. Miles, M. M. Schreiber, 0. R. Mitchell, and V. C. Vanderbilt. Machine vision and
image processing for plant identification. Trans. ofthe ASAE 29(6),1500-1507, 1986.

3. Guyer, D. E., G. E. Miles, L. D. Gaultney, and M. M. Schreiber. Application of machine vision to shape
analysis in leafand plant identification. Trans. ofthe ASAE 36(1), 163-171, 1993.

4. Gerhards, R., A. Nabout, M. Sokefeld, W. Kuhbauch, and H. A. Nour Eldin. Automatische erkennung von
zehn unkrautarten mit hilfe digitaler bildverarbeitung und Fouriertransformation (Automatic
identification of 10 weed species in digital images using Fourier descriptors and shape parameters). J.
Agronomy and Crop Science 171(5), 321-328, 1993.

5. Woebbecke, D. M., G. E. Meyer, K. Von Bargen, and D. A. Mortensen. Shape features for identifying
young weeds using image analysis. Trans. of the ASAE 38(1), 271-281, 1995.

6. Yonekawa, S., N. Sakai, and 0. Kitani. Identification of idealized leaf types using simple dimensionless
shape factors by image analysis. Trans. of the ASAE 39(4), 1525-1533, 1996.

7. Shearer, S. A., and R. G. Holmes. Plant identification using color co-occurrence matrices. Transactions.
of the ASAE 33(6), 2037-2044, 1990.

8. Mehta, T., G. E. Meyer, M. F. Kocher, D. A. Mortensen, and A. Samal. Comparison of optical color and
textural imaging methods for distinguishing plants from soil background. Transactions of the ASAE
41(4), 1189-1197, 1998.

9. Smart sprayers ready for rows. Farm Industry News 30(6), 44, 1997.

10. A "smart" sprayer. Farm Chemicals 160(9), 26, 1997.

11. Murch, G. Physiological principles for effective use of color. IEEE CG&A, 49-54, 1984.

12. SAS/STAT User's Guide, SAS Institute, Inc., Cary, N.C., Version 6., 387-404 and 707-77 1.

335

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 05/11/2015 Terms of Use: http://spiedl.org/terms