摘要
利用场景图的上下文信息可以帮助模型理解目标之间的关联作用;然而,大量不相关的目标可能带来额外噪声,进而影响信息交互,造成预测偏差。在嘈杂且多样的场景中,即使几个简单的关联目标,也足够推断目标所处的环境信息,并消除其他目标的歧义信息。此外,在面对真实场景中的长尾偏差数据时,场景图生成(SGG)的性能难以令人满意。针对上下文信息增强和预测偏差的问题,提出一种基于关联信息增强与关系平衡的SGG(IERB)方法。IERB方法采用一种二次推理结构,即根据有偏场景图的预测结果重新构建不同预测视角下的关联信息并平衡预测偏差。首先,聚焦不同视角下的强相关目标以构建上下文关联信息;其次,利用树型结构的平衡策略增强尾部关系的预测能力;最后,采用一种预测引导方式在已有场景图的基础上预测优化。在通用的数据集Visual Genome上的实验结果表明,与3类基线模型VTransE(Visual Translation Embedding network)、Motif和VCTree(Visual Context Tree)相比,所提方法在谓词分类(PredCls)任务下的均值召回率mR@100分别提高了11.66、13.77和13.62个百分点,验证了所提方法的有效性。
Utilizing contextual information of scene graphs can help models understand the correlation effect among targets.However,a large number of unrelated targets may introduce additional noise,affecting information interaction and causing prediction biases.In noisy and diverse scenes,even a few simple associated targets are sufficient to infer environmental information of the target and eliminate ambiguity information of other targets.In addition,Scene Graph Generation(SGG)faces challenges when dealing with long-tailed biased data in real-world scenarios.To address the problems of contextual information optimization and prediction biases,an association Information Enhancement and Relationship Balance based SGG(IERB)method was proposed.In IERB method,a secondary reasoning structure was employed according to biased scene graph prediction results,to reconstruct association information under different prediction angles of view and balance the prediction biases.Firstly,strongly correlated targets from different angles of view were focused on to construct the contextual association information.Secondly,the prediction capability for tail relationships was enhanced using a balancing strategy of tree structure.Finally,a prediction-guided approach was used to optimize predictions based on the existing scene graph.Experimental results on Visual Genome dataset show that compared with three baseline models Visual Translation Embedding network(VTransE),Motif,and Visual Context Tree(VCTree),the proposed method improves the mean Recall mR@100 in the Predicate Classification(PredCls)task by 11.66,13.77 and 13.62 percentage points,respectively,demonstrating the effectiveness of the proposed method.
作者
李林昊
韩冬
董永峰
李英双
王振
LI Linhao;HAN Dong;DONG Yongfeng;LI Yingshuang;WANG Zhen(School of Artificial Intelligence,Hebei University of Technology,Tianjin 300401,China;Hebei Province Key Laboratory of Big Data Computing(Hebei University of Technology),Tianjin 300401,China;Hebei Data Driven Industrial Intelligent Engineering Research Center(Hebei University of Technology),Tianjin 300401,China)
出处
《计算机应用》
北大核心
2025年第3期953-962,共10页
journal of Computer Applications
基金
国家自然科学基金资助项目(62306103)。
关键词
场景图生成
信息增强
有偏预测
关系平衡
预测优化
Scene Graph Generation(SGG)
information enhancement
biased prediction
relationship balancing
prediction optimization
作者简介
李林昊(1989-),男,山东威海人,副教授,博士,CCF会员,主要研究方向:机器学习、计算机视觉、知识推理;韩冬(1998-),男,黑龙江齐齐哈尔人,硕士研究生,主要研究方向:机器学习、计算机视觉;通信作者:董永峰(1977-),男,河北定州人,教授,博士,CCF会员,主要研究方向:人工智能、计算机视觉、智能信息处理,电子邮箱dongyf@hebut.edu.cn;李英双(1986-),女,河北衡水人,工程师,硕士,主要研究方向:人工智能;王振(1989-),男,河北唐山人,副教授,博士,主要研究方向:机器学习、计算机视觉、可信学习。
