Precision agriculture: Exploration of machine learning approaches for assessing mango crop quantity

A machine vision based system is proposed to replace the current in-orchard manual estimates of mango fruit yield, to inform harvest resourcing and marketing. The state-of-the-art in fruit detection was reviewed, highlighting the recent move from traditional image segmentation methods to convolution neural network (CNN) based deep learning methods. An experimental comparison of several deep learning based object detection frameworks (single shot detectors versus two-staged detectors) and several standard CNN architectures was undertaken for detection of mango panicles and fruit in tree images. The machine vision system used images of individual trees captured during night time from a moving platform mounted with a Global Navigation Satellite System (GNSS) receiver and a LED panel floodlight. YOLO, a single shot object detection framework, was re-designed and named as MangoYOLO. MangoYOLO outperformed existing state-of-the-art deep learning object detection frameworks in terms of fruit detection time and accuracy and was robust in use across different cultivars and cameras. MangoYOLO achieved F1 score of 0.968 and average precision of 0.983 and required just 70 ms per image (2048 × 2048 pixel) and 4417 MB memory. The annotated image dataset was made publicly available. Approaches were trialled to relate the fruit counts from tree images to the actual harvest count at an individual tree level. Machine vision based estimates of fruit load ranged between -11% to +14% of packhouse fruit counts. However, estimation of fruit yield (t/ha) requires estimation of fruit size as well as fruit number. A fruit sizing app for smart phones was developed as an affordable in-field solution. The solution was based on segmentation of the fruit in image using colour features and estimation of the camera to fruit perimeter distance based on use of fruit allometrics. For mango fruit, RMSEs of 5.3 and 3.7 mm were achieved on length and width measurements under controlled lighting, and RMSEs of 5.5 and 4.6 mm were obtained in-field under ambient lighting. Further, estimation of harvest timing can be informed by assessment of the spread of flowering. Deep learning object detection methods were deployed for assessment of the number and development stage of mango panicles, on tree. Methods to deal with different orientations of flower panicles in tree images were implemented. An R2 >0.8 was achieved between machine vision count of panicles on images and in-field human count per tree. Similarly, mean average precision of 69.1% was achieved for classification of panicle stages. These machine vision systems form a foundation for estimation of crop load and harvest timing, and for automated harvesting.