摘要
深度估计任务的一种解决思路是使成像光学系统的点扩散函数呈现深度关联的形式,即不同深度处的点扩散函数呈现出不同的空间或光谱结构,通过成像过程将点扩散函数中包含的深度信息编码进传感器图像中,并通过算法反演场景深度。提出一种成像系统的设计方法,使用单透镜成像系统实现上述深度估计思路。通过搭建可微深度场景成像仿真模型,设计约束方法,对单透镜光学系统中多波长、深度变化、视场变化的4维点扩散函数进行调控,并设计神经网络学习传感器编码图像中包含的深度信息,最终设计出包含极简光学系统的单目深度估计模型,以替代现有包含透镜组和相位调制光学元件的复杂光学系统。使用NYU Depth V2数据集在1~5 m的深度范围内进行训练和测试,深度估计的平均相对误差可达8.41%,该设计在低成本小型化深度成像探测领域具有一定的应用潜力。
Objective The awareness of depth is important for many computer vision tasks,such as autonomous driving and 3D reconstruction.Existing depth estimation methods include methods using structured light,time of flight,stereo-based depth estimation and monocular depth estimation.Monocular depth estimation has significant advantages in terms of power consumption,cost,and size.There are two ways to achieve monocular depth estimation.One is to learn the depth clues contained in the scene itself in the image through the neural network,such as texture gradients,shading,occlusions,and the type and size of the object.But this method is lack of interpretability.The other is depth from defocus(DFD)or depth estimation based on coded aperture,which estimates depth by identifying depth-related optical features in the optical system.This method often requires the addition of a phase mask to the existing camera,and does not utilize the depth information encoding ability of the lens itself.Therefore,we propose a depth estimation method based on single-lens optical system,which further reduces the size and volume of the optical system on the basis of existing single-lens depth estimation,while improving the accuracy of depth estimation.Methods A monocular depth estimation system usually consists of a camera and diffractive optical elements.The implementation method involves designing a diffraction optical element such that the point spread functions(PSFs)at different depths present different spatial or spectral structures.Using optical system to encode scene depth information into 2D images,then a decoding algorithm is used to decode the encoded image to estimate the scene’s depth map.The singlelens optical system also has the capability to present different spatial or spectral structures for objects at varying depths.For this specific imaging system of a single lens,we propose a simulation model for the four-dimensional point spread function that varies with multi-wavelength,depth-aware,spatially-variant four-dimensional point spread functions along with a differentiable optical imaging model.We then introduce an optimization constraint method for depth estimation tasks,which regulates the point spread function in both depth and field dimensions.For the depth estimation algorithm,we should consider that depth estimation methods relying on different spatial or spectral structures of PSFs are quite dependent on the object’s texture in the scene.Since the semantic information of the scene can compensate for this limitation,we propose a depth estimation network that includes a semantic information extraction preprocessing model.We connect the imaging model and depth estimation algorithm,jointly designing the single-lens optical system and the depth estimation algorithm.Finally,a visible light depth detector which includes an aspherical single-lens optical system and corresponding depth estimation algorithm is designed.Results and Discussions To verify our method,we train and test it on the NYU Depth V2 dataset,set the target depth range from 1.0 m to 5.0 m,The initial single-lens optical system,characterized by a focal length of 31.4 mm and a field of view of approximately 10°,is optimized along with its corresponding depth estimation algorithm.we compare our method with three alternative approaches:1)the conventional depth from defocus model,which treats the lens as a thin lens;2)phase coded-aperture model implemented with a diffractive optical element size of 256×256;3)phase coded-aperture model which has a phase mask with several concentric rings.Our designed single-lens depth estimation model achieved a relative error of as low as 0.083 on the NYU Depth V2 dataset,demonstrating the lowest relative error among the compared methods.To further evaluate the contribution of our proposed method,we conducted ablation experiments.Specifically,we replaced the optimized single-lens optical system with an unoptimized version and substituted the semantic information extraction preprocessing step with a neural network lacking this preprocessing capability.Both modifications resulted in a degradation of depth estimation accuracy,thus substantiating the effectiveness of our method in improving the depth estimation model.Conclusions We introduce an end-to-end single-lens depth estimation model.Firstly,in order to accurately simulate the out-of-focus and off-axis aberrations in the real camera lens in the depth scene,we propose a differentiable imaging model.Then,we introduce a single lens optimization constraint method to regulate the point spread functions of a single lens optical system to improve the depth dependence of the imaging response features of the optical system,so that the single lens can be optimized along the direction of maximizing the depth estimation performance of the model.In this paper,a preprocessing method combining semantic information is proposed to make up for the lack of dependent image texture in decoding process.Finally,by jointly optimizing the single-lens optical system and the depth estimation algorithm,the depth estimation model based on the minimalist optical system is realized,and the simulation and test are carried out on the NYU Depth V2 dataset.The results show that the design method can greatly reduce the volume of the depth estimation system while maintaining a high depth estimation performance.It has certain significance in the application of unmanned aerial vehicle platform distance sensor and other fields.
作者
孙再武
谭凡教
于鹏亮
李宗岭
张荣帅
杨昌健
侯晴宇
Sun Zaiwu;Tan Fanjiao;Yu Pengliang;Li Zongling;Zhang Rongshuai;Yang Changjian;Hou Qingyu(Research Center for Space Optical Engineering,School of Astronautics,Harbin Institute of Technology,Harbin 150001,Heilongjiang,China;Harbin Xinguang Optic-Electronics Technology Co.,Ltd,Harbin 150036,Heilongjiang,China)
出处
《光学学报》
北大核心
2025年第11期92-101,共10页
Acta Optica Sinica
基金
国家自然科学基金(62375067)。
关键词
成像系统
计算成像
单透镜
成像仿真
单目深度估计
点扩散函数
imaging system
computational imaging
single lens
imaging simulation
monocular depth estimation
point spread function
作者简介
通信作者:侯晴宇,houqingyu@126.com。
