來自法國農(nóng)業(yè)科學(xué)院等研究機(jī)構(gòu)的專家，利用Hiphen公司開發(fā)的Airphen多光譜成像系統(tǒng)（集成與表型車上），發(fā)表了題為“Pixelwise instance segmentation of leaves in dense foliage”的文章，文章發(fā)表于Computers and Electronics in Agriculture，Volume195,April 2022,106797。
 

摘要

葉片的像素級實例分割是在植物種類的混合體上執(zhí)行的。
提出了一種新的基于形狀的損耗函數(shù)，并將其應(yīng)用于每個連接部件。
分割輸出使用分水嶺和深度索引方法進(jìn)行細(xì)化。
該方法的基準(zhǔn)測試是在“葉片分割挑戰(zhàn)”上進(jìn)行的。
共享了一個新的多光譜數(shù)據(jù)集，其中包括在自然光下采集的300幅圖像。
 

使用圖像分析檢測和識別植物是精準(zhǔn)農(nóng)業(yè)（從表型到特定地點(diǎn)雜草管理）中許多應(yīng)用的關(guān)鍵步驟。實例分割通常用于檢測整個植物。然而，被檢測物體的形狀在個體和生長階段之間會發(fā)生變化。減少這些變化的一個相關(guān)方法是縮小對葉片的檢測范圍。然而，當(dāng)圖像包含混合植物種類時，當(dāng)個體重疊時，尤其是在不受控制的室外環(huán)境中，分割葉片是一項困難的任務(wù)。為了充分解決這個問題，本研究基于最近的卷積神經(jīng)網(wǎng)絡(luò)機(jī)制，提出了一種基于像素的實例分割方法來檢測密集樹葉環(huán)境中的樹葉。它結(jié)合了“深度輪廓感知”（從其邊緣的內(nèi)部分離大葉子）、“葉子分割槽、邊緣的分類”（以特定的內(nèi)邊緣分離實例）和“密集葉的金字塔CNN”（考慮不同尺度的邊緣）。但分割輸出也會使用分水嶺和計算優(yōu)化植被指數(shù)（deepindex）的方法進(jìn)行優(yōu)化。該方法與其他運(yùn)行葉片分割挑戰(zhàn)（由PPPN國際植物表型網(wǎng)絡(luò)提供）的方法進(jìn)行了比較，并應(yīng)用于小松屬植物的外部數(shù)據(jù)集。此外，還介紹了一個新的多光譜數(shù)據(jù)集，包含300幅豆類植物圖像（具有濃密的葉子、個體重疊、物種混合和自然光照條件）。地面真值（例如樹葉邊界）由標(biāo)記的多邊形定義，可用于培訓(xùn)和評估各種專門用于樹葉檢測或作物/雜草分類的算法的性能。在通常的數(shù)據(jù)集上，該方法的性能與通常的葉片分割方法相似。在新的數(shù)據(jù)集上，它們的結(jié)果明顯優(yōu)于通常的RCNN方法。剩余的誤差是相鄰區(qū)域之間的融合不良和多葉葉片的過度分割造成。為了克服這些不足，可以研究結(jié)構(gòu)分析方法。
 

關(guān)鍵詞：精準(zhǔn)農(nóng)業(yè)、遙感、葉片分割、濃密葉片、邊界檢測，語義分割，CNN，多光譜
 

Pixelwise instance segmentation of leaves in dense foliage

Highlights

Pixelwise Instance segmentation of leaves is performed on mixes of plant species.
A new shape-based loss function is proposed and applied to each connected component.
The segmentation output is refined using a Watershed and a DeepIndices approach.
A benchmark of the method is performed on “leaf segmentation challenges”.
A new multi-spectral dataset of 300 images acquired in natural light is shared. 

Detecting and identifying plants using image analysis is a key step for many applications in precision agriculture (from phenotyping to site specific weed management). Instance segmentation is usually carried on to detect entire plants. However, the shape of the detected objects changes between individuals and growth stages. A relevant approach to reduce these variations is to narrow the detection on the leaf. Nevertheless, segmenting leaves is a difficult task, when images contain mixes of plant species, and when individuals overlap, particularly in an uncontrolled outdoor environment. To leverage this issue, this study based on recent Convolutional Neural Network mechanisms, proposes a pixelwise instance segmentation to detect leaves in dense foliage environment. It combines “deep contour aware” (to separate the inner of big leaves from its edges), “Leaf Segmentation trough classification of edges” (to separate instances with a specific inner edges) and “Pyramid CNN for Dense Leaves” (to consider edges at different scales). But the segmentation output is also refined using a Watershed and a method to compute optimized vegetation indices (DeepIndices). The method is compared to others running the leaf segmentation challenge (provided by the International Network on Plant Phenotyping) and applied on an external dataset of Komatsuna plants. In addition, a new multispectral dataset of 300 images of bean plants is introduced (with dense foliage, individuals overlapping, mixes of species and natural lighting conditions). The ground truth (e.g. the leaves boundaries) is defined by labelled polygons and can be used to train and assess the performance of various algorithms dedicated to leaf detection or crop/weed classification. On the usual datasets, the performances of the proposed method are similar to those of the usual methods involved in the leaf segmentation challenges. On the new dataset, their results are strongly better than those of the usual RCNN method. Remaining errors are bad fusion between neighboring areas and over segmentation of multi-foliate leaves. Structural analysis methods could be studied in order to overcome these deficiencies.