The increasing popularity of the metaverse has led to a growing interest and market size in spatial computing from both academia and industry.Developing portable and accurate imaging and depth sensing systems is cruci...The increasing popularity of the metaverse has led to a growing interest and market size in spatial computing from both academia and industry.Developing portable and accurate imaging and depth sensing systems is crucial for advancing next-generation virtual reality devices.This work demonstrates an intelligent,lightweight,and compact edge-enhanced depth perception system that utilizes a binocular meta-lens for spatial computing.The miniaturized system comprises a binocular meta-lens,a 532 nm filter,and a CMOS sensor.For disparity computation,we propose a stereo-matching neural network with a novel H-Module.The H-Module incorporates an attention mechanism into the Siamese network.The symmetric architecture,with cross-pixel interaction and cross-view interaction,enables a more comprehensive analysis of contextual information in stereo images.Based on spatial intensity discontinuity,the edge enhancement eliminates illposed regions in the image where ambiguous depth predictions may occur due to a lack of texture.With the assistance of deep learning,our edge-enhanced system provides prompt responses in less than 0.15 seconds.This edge-enhanced depth perception meta-lens imaging system will significantly contribute to accurate 3D scene modeling,machine vision,autonomous driving,and robotics development.展开更多
Spatial computing and augmented reality are advancing rapidly,with the goal of seamlessly blending virtual and physical worlds.However,traditional depth-sensing systems are bulky and energy-intensive,limiting their us...Spatial computing and augmented reality are advancing rapidly,with the goal of seamlessly blending virtual and physical worlds.However,traditional depth-sensing systems are bulky and energy-intensive,limiting their use in wearable devices.To overcome this,recent research by X.Liu et al.presents a compact binocular metalens-based depth perception system that integrates efficient edge detection through an advanced neural network.This system enables accurate,realtime depth mapping even in complex environments,enhancing potential applications in augmented reality,robotics,and autonomous systems.展开更多
基金supports from the Research Grants Council of the Hong Kong Special Administrative Region,China[Project No.C5031-22GCityU11310522+3 种基金CityU11300123]the Department of Science and Technology of Guangdong Province[Project No.2020B1515120073]City University of Hong Kong[Project No.9610628]JST CREST(Grant No.JPMJCR1904).
文摘The increasing popularity of the metaverse has led to a growing interest and market size in spatial computing from both academia and industry.Developing portable and accurate imaging and depth sensing systems is crucial for advancing next-generation virtual reality devices.This work demonstrates an intelligent,lightweight,and compact edge-enhanced depth perception system that utilizes a binocular meta-lens for spatial computing.The miniaturized system comprises a binocular meta-lens,a 532 nm filter,and a CMOS sensor.For disparity computation,we propose a stereo-matching neural network with a novel H-Module.The H-Module incorporates an attention mechanism into the Siamese network.The symmetric architecture,with cross-pixel interaction and cross-view interaction,enables a more comprehensive analysis of contextual information in stereo images.Based on spatial intensity discontinuity,the edge enhancement eliminates illposed regions in the image where ambiguous depth predictions may occur due to a lack of texture.With the assistance of deep learning,our edge-enhanced system provides prompt responses in less than 0.15 seconds.This edge-enhanced depth perception meta-lens imaging system will significantly contribute to accurate 3D scene modeling,machine vision,autonomous driving,and robotics development.
基金financially supported by the POSCO-POSTECH-RIST Convergence Research Center program funded by POSCOthe National Research Foundation (NRF) grants (RS-2024-00462912, RS-2024-00416272, RS-2024-00337012, RS-2024-00408446) funded by the Ministry of Science and ICT (MSIT) of the Korean government+2 种基金the Korea Evaluation Institute of Industrial Technology (KEIT) grant (No. 1415185027/20019169, Alchemist project) funded by the Ministry of Trade, Industry and Energy (MOTIE) of the Korean governmentthe Soseon Science fellowship funded by Community Chest of Koreathe NRF PhD fellowship (RS-2023-00275565) funded by the Ministry of Education (MOE) of the Korean government。
文摘Spatial computing and augmented reality are advancing rapidly,with the goal of seamlessly blending virtual and physical worlds.However,traditional depth-sensing systems are bulky and energy-intensive,limiting their use in wearable devices.To overcome this,recent research by X.Liu et al.presents a compact binocular metalens-based depth perception system that integrates efficient edge detection through an advanced neural network.This system enables accurate,realtime depth mapping even in complex environments,enhancing potential applications in augmented reality,robotics,and autonomous systems.
