针对剪纸图像分类中文本与图像模态差异大、类原型表达能力弱的问题,提出了一种基于CLIP模型的文本特征增强方法(CLIP visual text enhancer,C-VTE)。该方法通过人工提示模板提取文本特征,设计了一种视觉文本增强模块,并利用Cross Atten...针对剪纸图像分类中文本与图像模态差异大、类原型表达能力弱的问题,提出了一种基于CLIP模型的文本特征增强方法(CLIP visual text enhancer,C-VTE)。该方法通过人工提示模板提取文本特征,设计了一种视觉文本增强模块,并利用Cross Attention和比例残差进行连接和融合图像特征与文本特征,以减小模态差异,增强类别特征表达能力。在剪纸数据集及Caltech101等4个公开数据集上进行了实验,验证其有效性:在剪纸数据集的基类分类任务中,C-VTE平均准确率达到了72.51%,较现有方法提升3.14百分点;在公开数据集的小样本分类任务中,平均准确率达到了84.78%,提升2.45百分点。消融实验表明,模态融合模块与比例残差对性能提升影响显著。该方法为视觉语言大模型在下游分类任务中的高效适配提供了新思路,尤其适用于小样本与基类主导的场景。展开更多
[Objective]Accurate prediction of tomato growth height is crucial for optimizing production environments in smart farming.However,current prediction methods predominantly rely on empirical,mechanistic,or learning-base...[Objective]Accurate prediction of tomato growth height is crucial for optimizing production environments in smart farming.However,current prediction methods predominantly rely on empirical,mechanistic,or learning-based models that utilize either images data or environmental data.These methods fail to fully leverage multi-modal data to capture the diverse aspects of plant growth comprehensively.[Methods]To address this limitation,a two-stage phenotypic feature extraction(PFE)model based on deep learning algorithm of recurrent neural network(RNN)and long short-term memory(LSTM)was developed.The model integrated environment and plant information to provide a holistic understanding of the growth process,emploied phenotypic and temporal feature extractors to comprehensively capture both types of features,enabled a deeper understanding of the interaction between tomato plants and their environment,ultimately leading to highly accurate predictions of growth height.[Results and Discussions]The experimental results showed the model's ef‐fectiveness:When predicting the next two days based on the past five days,the PFE-based RNN and LSTM models achieved mean absolute percentage error(MAPE)of 0.81%and 0.40%,respectively,which were significantly lower than the 8.00%MAPE of the large language model(LLM)and 6.72%MAPE of the Transformer-based model.In longer-term predictions,the 10-day prediction for 4 days ahead and the 30-day prediction for 12 days ahead,the PFE-RNN model continued to outperform the other two baseline models,with MAPE of 2.66%and 14.05%,respectively.[Conclusions]The proposed method,which leverages phenotypic-temporal collaboration,shows great potential for intelligent,data-driven management of tomato cultivation,making it a promising approach for enhancing the efficiency and precision of smart tomato planting management.展开更多
文摘针对剪纸图像分类中文本与图像模态差异大、类原型表达能力弱的问题,提出了一种基于CLIP模型的文本特征增强方法(CLIP visual text enhancer,C-VTE)。该方法通过人工提示模板提取文本特征,设计了一种视觉文本增强模块,并利用Cross Attention和比例残差进行连接和融合图像特征与文本特征,以减小模态差异,增强类别特征表达能力。在剪纸数据集及Caltech101等4个公开数据集上进行了实验,验证其有效性:在剪纸数据集的基类分类任务中,C-VTE平均准确率达到了72.51%,较现有方法提升3.14百分点;在公开数据集的小样本分类任务中,平均准确率达到了84.78%,提升2.45百分点。消融实验表明,模态融合模块与比例残差对性能提升影响显著。该方法为视觉语言大模型在下游分类任务中的高效适配提供了新思路,尤其适用于小样本与基类主导的场景。
文摘[Objective]Accurate prediction of tomato growth height is crucial for optimizing production environments in smart farming.However,current prediction methods predominantly rely on empirical,mechanistic,or learning-based models that utilize either images data or environmental data.These methods fail to fully leverage multi-modal data to capture the diverse aspects of plant growth comprehensively.[Methods]To address this limitation,a two-stage phenotypic feature extraction(PFE)model based on deep learning algorithm of recurrent neural network(RNN)and long short-term memory(LSTM)was developed.The model integrated environment and plant information to provide a holistic understanding of the growth process,emploied phenotypic and temporal feature extractors to comprehensively capture both types of features,enabled a deeper understanding of the interaction between tomato plants and their environment,ultimately leading to highly accurate predictions of growth height.[Results and Discussions]The experimental results showed the model's ef‐fectiveness:When predicting the next two days based on the past five days,the PFE-based RNN and LSTM models achieved mean absolute percentage error(MAPE)of 0.81%and 0.40%,respectively,which were significantly lower than the 8.00%MAPE of the large language model(LLM)and 6.72%MAPE of the Transformer-based model.In longer-term predictions,the 10-day prediction for 4 days ahead and the 30-day prediction for 12 days ahead,the PFE-RNN model continued to outperform the other two baseline models,with MAPE of 2.66%and 14.05%,respectively.[Conclusions]The proposed method,which leverages phenotypic-temporal collaboration,shows great potential for intelligent,data-driven management of tomato cultivation,making it a promising approach for enhancing the efficiency and precision of smart tomato planting management.
