摘要
针对无人机视频流的实时拼接方法进行深入研究,提出一种多线程并行处理的无人机视频实时拼接方法,主要解决无人机视频流实时拼接问题。该方法充分利用计算机并行运算的特点,将视频实时拼接工作分解成多线程工作模式,主要由图像配准线程、非线性优化线程、闭环检测线程、图像拼接线程组成,其中图像配准线程实现对视频关键帧的自适应提取和配准,包括特征点检测、特征点匹配、图像粗配准;非线性优化线程完成对拼接参数模型的动态优化以最小化配准误差,及时修正实时配准引入的误差;闭环检测线程检测当前帧与历史帧存在的闭环图像对,通过闭环图像对进行非线性全局优化以最小化全局配准误差;图像拼接线程则根据优化后的配准参数动态生成拼接影像。通过4组实验数据验证了该方法的有效性和实用性,能有效消除多帧拼接的累积误差,实现对无人机视频流的实时拼接。
The real⁃time stitching method of UAV video stream is studied in depth,and a UAV video stream real⁃time stitching method based on multi⁃threaded parallel processing is proposed,which aims to cope with the difficulties of real⁃time stitching of UAV video stream.This method decomposes the real⁃time video stitching into a multi⁃threaded working mode by using the characteristic of computer parallel computing fully.The multi⁃threaded working mode is mainly composed of image registration thread,nonlinear optimization thread,closed⁃loop detection thread and image stitching thread.The image registration thread realizes the adaptive extraction and registration of video key frame,including feature point detection,feature point matching and image rough registration.The nonlinear optimization thread completes the dynamic optimization of the stitching parameter model to minimize the registration error and correct the error generated by real⁃time registration timely.The closed⁃loop detection thread detects the closed image pairs existing in the current frame and the historical frame,and performs nonlinear global optimization on the closed image pairs to minimize the global registration error.The image stitching thread generates the stitched image dynamically according to the optimized registration parameters.The validity and practicability of the method are verified by four groups of experimental data.It can be seen that the proposed method can eliminate the cumulative error of multi⁃frame stitching effectively and realize the real⁃time stitching of UAV video stream.
作者
刘彬
梁攀攀
卢迪
靳冉
LIU Bin;LIANG Panpan;LU Di;JIN Ran(The Aviation Military Deputy Office of Army in Luoyang,Luoyang 471000,China;The Aviation Military Deputy Office of Army in Zhuzhou,Zhuzhou 412000,China)
出处
《现代电子技术》
北大核心
2024年第23期49-54,共6页
Modern Electronics Technique
关键词
无人机视频
多线程
并行处理
图像配准
非线性优化
闭环检测
累积误差
实时拼接
UAV video
multithreading
parallel processing
image registration
nonlinear optimization
closed⁃loop detection
accumulated error
real⁃time stitching
作者简介
刘彬(1981-),男,四川邛崃人,博士研究生,工程师,主要研究领域为基于图像的三维重建、图像匹配、相机标定、图像测量等;梁攀攀(1992-),男,河南洛阳人,硕士研究生,助理工程师,主要研究领域为基于深度的学习目标检测与识别、模式识别等;卢迪(1993-),男,江苏徐州人,硕士研究生,助理工程师,主要研究领域为航空发动机、自动控制等;靳冉(1985-),男,河南尉氏人,助理工程师,主要研究领域为光电侦察、火控任务系统等。
