融合双目与ToF的高速三维视觉感知系统

High-speed 3D Vision Perception System Incorporating Binocular and ToF Fusion

在线阅读下载PDF

导出

摘要通过三维视觉感知系统获取高质量彩色图像与三维点云信息是移动机器人等应用的关键,针对现有三维视觉感知系统难以同时满足实时性、高精度、高鲁棒的需求,设计并实现了双目与ToF融合的高速三维视觉感知系统。系统主要由相机板、FPGA板、GPU板与ToF板组成,通过FPGA板实现高速图像采集及预处理,以及提出并实现适应FPGA的Fast-SGM算法来获得高速双目立体匹配深度图;利用GPU采集ToF深度图并基于置信度对双目立体匹配深度图与ToF深度图进行融合,以解决双目立体匹配在弱光照、弱纹理情况下失效与ToF在强光照、远距离情况下失效的问题,进一步提高测量精度与鲁棒性。系统通过FPGA与GPU的并行加速特点对数据处理进行加速,能够输出分辨率为1024 pixels×1024 pixels、帧率为120 fps的双路RGB图像与立体匹配深度图像,同时能够输出分辨率为1024 pixels×1024 pixels、帧率为60 fps的融合深度图像,能够满足移动机器人等实时应用对环境深度估计的需求。 Obtaining high-quality color images and 3D point cloud information through a 3D visual perception system is the key to applications such as mobile robots.Since the existing 3D visual perception system is difficult to meet the requirements of real-time,high precision and high robustness at the same time,a binocular high-speed 3D visual perception system integrated with ToF(Time of Flight)is designed and realizedin this paper.The system is mainly composed of camera board,FPGA board,GPU board and ToF board.High-speed image acquisition and preprocessing are realized through FPGA board,and Fast-SGM algorithm adapted to FPGA is proposed to obtain high-speed binocular stereo matching depth map.Collect the ToF depth map and fuse the binocular stereo matching depth map with the ToF depth map based on the confidence,so as to solve the problem that the binocular stereo matching fails in the case of weak light and weak texture and the ToF fails in the case of strong light and long distance,to further improve the measurement accuracy and robustness.The system accelerates data processing through the parallel acceleration characteristics of FPGA and GPU,and can output dual-channel RGB images and stereo matching depth images with a resolution of 1024 pixels×1024 pixels and a frame rate of 120 fps,and can output a resolution of 1024 pixels×1024 pixels,fused depth images with a frame rate of 60 fps,which can meet the needs of real-time applications such as mobile robots for environmental depth estimation.

作者姜宇扬黄开基潘前锋杨华尹周平 JIANG Yuyang;HUANG Kaiji;PAN Qianfeng;YANG Hua;YIN Zhouping(School of Mechanical Science and Engineering,State Key Laboratory of Digital Manufacturing Equipment and Technology,Huazhong University of Science and Technology,Wuhan 430074,China)

机构地区华中科技大学机械科学与工程学院数字制造装备与技术国家重点实验室

出处《机械科学与技术》北大核心 2025年第8期1408-1417,共10页 Mechanical Science and Technology for Aerospace Engineering

基金国家重点研发计划(2020YFA0405700)。

关键词现场可编程门阵列(FPGA) 双目立体视觉高速视觉系统飞行时间(ToF) field programmable gate array(FPGA) binocular stereo matching high-speed vision systems time of flight(ToF)

分类号 TP391 [自动化与计算机技术—计算机应用技术]

作者简介姜宇扬,硕士研究生,jyy_hust@hust.edu.cn;通信作者:杨华,教授,博士生导师,huayang@hust.edu.cn。