关键词: 枸杞, 开花期识别, 果实成熟期识别, 发育期识别, Faster R-CNN, 图像识别

Abstract: From 2018 to 2019, 16 sets of Lycium barbarum farmland monitoring systems had been built in Ningxia. Each system took 10 images every day, and over 30,000 images of the growth of Lycium barbarum trees were taken in two years. To study the recognition technology of the flowering period and fruit ripening period of Lycium barbarum based on these images, three methods were used in this paper to judge the developmental stage of Lycium barbarum. The first one was the field observation method. In this method, two fields where the real-life monitoring system was installed were selected, and the Lycium barbarum trees in the two fields were manually observed once in every two days during the growing season. The Lycium barbarum trees selected by manual observation should be consistent with the ones photographed by the farmland monitoring systems. The second method was expert visual judgment, in which 5 experienced experts were invited to judge all the images. The judgment standard was as follows. If there were 5 features in a certain developmental period in an image, it was considered that this Lycium barbarum tree had reached the universal period of this developmental period. If 5 out of 10 images on a certain day reached the universal period of this developmental period, it was considered that the Lycium barbarum population in the filed had entered this developmental period. Based on the opinions of the experts, the result of the expert visual judgment was given. The third method is the automatic recognition method. In this method, more than 3000 images with characteristics of Lycium barbarum flowering and fruit ripening were screened out from all the images. Removed the images with lens fouling or unsatisfactory field of view, and finally, the number of remaining image samples was 1210. To avoid the phenomenon of underfitting or overfitting due to too few or too many images of a certain category involved in training, rotation, cropping and flipping were used for data enhancement. The data enhanced samples were divided according to the format of the PASCAL VOC2007 data set. Finally, a total of 7260 experimental samples were obtained, including 5808 images in the training set and 1452 images in the test set. According to the significant image characteristics of Lycium barbarum in the flowering and fruit ripening periods, the labelImg label tool was used to label all the flowers and fruits in the image samples, marking 12100 ‘flower’ labels and 11602 ‘fruit’ labels. Then, faster region-based convolutional neural network (Faster R-CNN) was utilized to train and classify the selected images, and to construct the algorithm for identifying the flowering period and fruit ripening period of Lycium barbarum. In the constructed algorithm, the judgment standard was the same as that in the second method, and the time series judgment was introduced when judging the different stages of flowering or fruit ripening. Taking AP and mAP as the evaluation indicators of the automatic recognition model, the results showed that the mAP value could reach 0.74 on the test set when the important hyperparameters batch size and the number of iterations in the network structure were set to be 64 and 20000 respectively, which outperforms other hyperparameters setting. Comparing the results of the three methods, it could be found that the difference between the automatic recognition results and the field observation records during the same period was 0-12 days. The main reason for the difference was that the observation objects and standards of the two methods were inconsistent. The observation object of the automatic recognition method was a two-dimensional image, and it could not be judged when the feature was occluded. The object of field observation is the Lycium barbarum tree, which is not affected by occlusion. Besides, the standards of these two methods were different. The standard of the automatic recognition method was based on the number of feature points observed in the image, while the field observation method was based on the ratio of the observed feature points to the expected feature points of the Lycium barbarum tree that could not be obtained in the automatic recognition method. The difference between the two methods could not be eliminated fundamentally, so it was difficult to optimize the automatic recognition algorithm using the results of the field observations method. The comparison results also showed that the difference between the automatic recognition results and the expert visual judgments was within 2-5d. The judgment objects and standards of these two methods were consistent, so the results were highly comparable. The results of expert visual judgment could be used as the verification standard to optimize and adjust the automatic recognition method.

Key words: Lycium barbarum, Flowering period recognition, Fruit ripening period recognition, Growth stages recognition, Faster R-CNN, Automatic image recognition