The unmanned aerial vehicle(UAV)swarm technology is one of the research hotspots in recent years.With the continuous improvement of autonomous intelligence of UAV,the swarm technology of UAV will become one of the mai...The unmanned aerial vehicle(UAV)swarm technology is one of the research hotspots in recent years.With the continuous improvement of autonomous intelligence of UAV,the swarm technology of UAV will become one of the main trends of UAV development in the future.This paper studies the behavior decision-making process of UAV swarm rendezvous task based on the double deep Q network(DDQN)algorithm.We design a guided reward function to effectively solve the problem of algorithm convergence caused by the sparse return problem in deep reinforcement learning(DRL)for the long period task.We also propose the concept of temporary storage area,optimizing the memory playback unit of the traditional DDQN algorithm,improving the convergence speed of the algorithm,and speeding up the training process of the algorithm.Different from traditional task environment,this paper establishes a continuous state-space task environment model to improve the authentication process of UAV task environment.Based on the DDQN algorithm,the collaborative tasks of UAV swarm in different task scenarios are trained.The experimental results validate that the DDQN algorithm is efficient in terms of training UAV swarm to complete the given collaborative tasks while meeting the requirements of UAV swarm for centralization and autonomy,and improving the intelligence of UAV swarm collaborative task execution.The simulation results show that after training,the proposed UAV swarm can carry out the rendezvous task well,and the success rate of the mission reaches 90%.展开更多
The guidance strategy is an extremely critical factor in determining the striking effect of the missile operation.A novel guidance law is presented by exploiting the deep reinforcement learning(DRL)with the hierarchic...The guidance strategy is an extremely critical factor in determining the striking effect of the missile operation.A novel guidance law is presented by exploiting the deep reinforcement learning(DRL)with the hierarchical deep deterministic policy gradient(DDPG)algorithm.The reward functions are constructed to minimize the line-of-sight(LOS)angle rate and avoid the threat caused by the opposed obstacles.To attenuate the chattering of the acceleration,a hierarchical reinforcement learning structure and an improved reward function with action penalty are put forward.The simulation results validate that the missile under the proposed method can hit the target successfully and keep away from the threatened areas effectively.展开更多
堆叠覆盖环境下的机械臂避障抓取是一个重要且有挑战性的任务。针对机械臂在堆叠环境下的避障抓取任务,本文提出了一种基于图像编码器和深度强化学习(deep reinforcement learning,DRL)的机械臂避障抓取方法Ec-DSAC(encoder and crop fo...堆叠覆盖环境下的机械臂避障抓取是一个重要且有挑战性的任务。针对机械臂在堆叠环境下的避障抓取任务,本文提出了一种基于图像编码器和深度强化学习(deep reinforcement learning,DRL)的机械臂避障抓取方法Ec-DSAC(encoder and crop for discrete SAC)。首先设计结合YOLO(you only look once)v5和对比学习网络编码的图像编码器,能够编码关键特征和全局特征,实现像素信息至向量信息的降维。其次结合图像编码器和离散软演员-评价家(soft actor-critic,SAC)算法,设计离散动作空间和密集奖励函数约束并引导策略输出的学习方向,同时使用随机图像裁剪增加强化学习的样本效率。最后,提出了一种应用于深度强化学习预训练的二次行为克隆方法,增强了强化学习网络的学习能力并提高了控制策略的成功率。仿真实验中Ec-DSAC的避障抓取成功率稳定高于80.0%,验证其具有比现有方法更好的避障抓取性能。现实实验中避障抓取成功率为73.3%,验证其在现实堆叠覆盖环境下避障抓取的有效性。展开更多
基金supported by the Aeronautical Science Foundation(2017ZC53033).
文摘The unmanned aerial vehicle(UAV)swarm technology is one of the research hotspots in recent years.With the continuous improvement of autonomous intelligence of UAV,the swarm technology of UAV will become one of the main trends of UAV development in the future.This paper studies the behavior decision-making process of UAV swarm rendezvous task based on the double deep Q network(DDQN)algorithm.We design a guided reward function to effectively solve the problem of algorithm convergence caused by the sparse return problem in deep reinforcement learning(DRL)for the long period task.We also propose the concept of temporary storage area,optimizing the memory playback unit of the traditional DDQN algorithm,improving the convergence speed of the algorithm,and speeding up the training process of the algorithm.Different from traditional task environment,this paper establishes a continuous state-space task environment model to improve the authentication process of UAV task environment.Based on the DDQN algorithm,the collaborative tasks of UAV swarm in different task scenarios are trained.The experimental results validate that the DDQN algorithm is efficient in terms of training UAV swarm to complete the given collaborative tasks while meeting the requirements of UAV swarm for centralization and autonomy,and improving the intelligence of UAV swarm collaborative task execution.The simulation results show that after training,the proposed UAV swarm can carry out the rendezvous task well,and the success rate of the mission reaches 90%.
基金supported by the National Natural Science Foundation of China(62003021,91212304).
文摘The guidance strategy is an extremely critical factor in determining the striking effect of the missile operation.A novel guidance law is presented by exploiting the deep reinforcement learning(DRL)with the hierarchical deep deterministic policy gradient(DDPG)algorithm.The reward functions are constructed to minimize the line-of-sight(LOS)angle rate and avoid the threat caused by the opposed obstacles.To attenuate the chattering of the acceleration,a hierarchical reinforcement learning structure and an improved reward function with action penalty are put forward.The simulation results validate that the missile under the proposed method can hit the target successfully and keep away from the threatened areas effectively.
文摘堆叠覆盖环境下的机械臂避障抓取是一个重要且有挑战性的任务。针对机械臂在堆叠环境下的避障抓取任务,本文提出了一种基于图像编码器和深度强化学习(deep reinforcement learning,DRL)的机械臂避障抓取方法Ec-DSAC(encoder and crop for discrete SAC)。首先设计结合YOLO(you only look once)v5和对比学习网络编码的图像编码器,能够编码关键特征和全局特征,实现像素信息至向量信息的降维。其次结合图像编码器和离散软演员-评价家(soft actor-critic,SAC)算法,设计离散动作空间和密集奖励函数约束并引导策略输出的学习方向,同时使用随机图像裁剪增加强化学习的样本效率。最后,提出了一种应用于深度强化学习预训练的二次行为克隆方法,增强了强化学习网络的学习能力并提高了控制策略的成功率。仿真实验中Ec-DSAC的避障抓取成功率稳定高于80.0%,验证其具有比现有方法更好的避障抓取性能。现实实验中避障抓取成功率为73.3%,验证其在现实堆叠覆盖环境下避障抓取的有效性。
