Deep Learning in Estimation of Fruit Attributes Using Near Infrared Spectroscopy

Determining the Dry Matter Content (DMC) of mango fruit is important for assessing harvest maturity and ensuring the quality of the ripened fruit. Near Infrared (NIR) spectroscopy offers a non-invasive approach to estimate fruit attributes, including DMC, by deducing chemical information from spectra collected from the fruit utilising predictive models, termed chemometrics. However, heterogeneity in growing conditions, fruit varieties, and instrumental differences impacts model performances, which has hindered the widespread practical adoption of NIR spectroscopy within the fruit industry. This research aims to address these challenges by developing robust, global models that accommodate seasonal, conditional, and instrumental variations, allowing for the simplified deployment of new instruments without local calibration. 

A review of existing literature on chemometrics for fruit quality evaluation was conducted, with a particular focus on one dimensional (1D) Convolutional Neural Networks (CNNs). Although CNNs are a relatively recent addition to this field, their success in analogous domains such as image and audio processing positions them as a potent tool for enhancing the accuracy and robustness of NIR-based fruit quality prediction models. The literature review systematically dissects the advantages and shortcomings of the use of 1D CNNs in spectroscopy, spotlighting the necessity for further research such as the spectral wavelength range and pretreatment, and the optimal CNN architecture for this specific application. 

The initial empirical phase of this research validated the superiority of CNN models over traditional methods like Partial Least Squares (PLS) and shallow Artificial Neural Networks (ANNs) through a rigorous comparison using a multi-season mango NIR dataset from a single instrument that has been used in previous publications. This study established a new benchmark for prediction accuracy, with the CNN model achieving a Root Mean Square Error of Prediction (RMSEP) of 0.77 %FW when tested on a subsequent season of fruit. The interplay between model type and spectral preprocessing proved to be critical, with the CNN model able to make particular use of a concatenation of spectral pretreatments. The sensitivity of neural network model training to the use of different seeds for generating pseudo-random sequences was documented, with a standard deviation in RMSEP for 50 ANN and CNN models, trained with varied random seeds, of 0.03%FW and 0.02%FW, respectively. It is recommended that this variation be documented in future studies. 

To build upon these findings from a single instrument dataset, an extensive dataset encompassing over 85,000 spectra from numerous spectrometers, fruit populations, and seasons, was compiled and made publicly accessible. This initiative aims to catalyse further chemometric research by providing a common ground for comparison and validation of diverse analytical techniques. Utilising this dataset, the optimal methodologies for PLS, ANN, and CNN, as established in earlier comparisons with a single instrument dataset, were implemented on this broader dataset, initially without additional optimisation. The models were trained using over 55,000 spectra from three seasons and 21 instruments and tested on a separate validation set from two subsequent seasons, comprising 12,539 spectra from 25 instruments, including 8 instruments not used during training. This approach resulted in RMSEP scores of 1.19, 0.94, and 1.24 %FW for the PLS, ANN, and CNN models, respectively. Following Bayesian Optimisation of spectral preprocessing methods, wavelength selections, and CNN hyperparameters, the CNN model's RMSEP improved to 0.93 %FW. This process identified the optimal wavelength range for the CNN model as 402 to 990 nm, an increased range compared to the previously established range of 684 to 990 nm for PLS modelling for a similar dataset. Subsequently, the architecture and hyperparameters of the CNN model were optimised, which improved the calibration results but resulted in a slightly worse RMSEP on the unseen validation set (0.96 %FW). Optimised PLS and ANN models achieved RMSEPs of 1.19 and 0.96 %FW respectively. Finally, the optimised PLS, ANN and CNN models were retrained using the combined training and validation set (five seasons) and tested on a new holdout set consisting of populations from a subsequent two growing seasons. The RMSEP on this set was 0.93, 1.24 and 1.37 %FW for the optimised PLS, ANN and CNN models respectively indicating that the ANN and CNN models were overfit after optimisation. This result was attributed to further variation in the holdout set despite the extensive dataset used in training. The PLS result demonstrates that a global model for prediction of mango fruit DMC is feasible, capable of accommodating a wide range of growing conditions, cultivars, physiological stages, and instruments. 

Further research on the design and training of deep learning architectures to achieve generalisability as well as calibration accuracy is recommended, to fully explore the potential of this modelling type. To this end, the public release of the extensive dataset is anticipated to be a valuable resource for the chemometric community, fostering innovation and collaboration in the pursuit of enhancing fruit quality evaluation using NIR spectroscopy.