Reinforcement learning has been applied to air combat problems in recent years,and the idea of curriculum learning is often used for reinforcement learning,but traditional curriculum learning suffers from the problem ...Reinforcement learning has been applied to air combat problems in recent years,and the idea of curriculum learning is often used for reinforcement learning,but traditional curriculum learning suffers from the problem of plasticity loss in neural networks.Plasticity loss is the difficulty of learning new knowledge after the network has converged.To this end,we propose a motivational curriculum learning distributed proximal policy optimization(MCLDPPO)algorithm,through which trained agents can significantly outperform the predictive game tree and mainstream reinforcement learning methods.The motivational curriculum learning is designed to help the agent gradually improve its combat ability by observing the agent's unsatisfactory performance and providing appropriate rewards as a guide.Furthermore,a complete tactical maneuver is encapsulated based on the existing air combat knowledge,and through the flexible use of these maneuvers,some tactics beyond human knowledge can be realized.In addition,we designed an interruption mechanism for the agent to increase the frequency of decisionmaking when the agent faces an emergency.When the number of threats received by the agent changes,the current action is interrupted in order to reacquire observations and make decisions again.Using the interruption mechanism can significantly improve the performance of the agent.To simulate actual air combat better,we use digital twin technology to simulate real air battles and propose a parallel battlefield mechanism that can run multiple simulation environments simultaneously,effectively improving data throughput.The experimental results demonstrate that the agent can fully utilize the situational information to make reasonable decisions and provide tactical adaptation in the air combat,verifying the effectiveness of the algorithmic framework proposed in this paper.展开更多
In the air combat process,confrontation position is the critical factor to determine the confrontation situation,attack effect and escape probability of UAVs.Therefore,selecting the optimal confrontation position beco...In the air combat process,confrontation position is the critical factor to determine the confrontation situation,attack effect and escape probability of UAVs.Therefore,selecting the optimal confrontation position becomes the primary goal of maneuver decision-making.By taking the position as the UAV’s maneuver strategy,this paper constructs the optimal confrontation position selecting games(OCPSGs)model.In the OCPSGs model,the payoff function of each UAV is defined by the difference between the comprehensive advantages of both sides,and the strategy space of each UAV at every step is defined by its accessible space determined by the maneuverability.Then we design the limit approximation of mixed strategy Nash equilibrium(LAMSNQ)algorithm,which provides a method to determine the optimal probability distribution of positions in the strategy space.In the simulation phase,we assume the motions on three directions are independent and the strategy space is a cuboid to simplify the model.Several simulations are performed to verify the feasibility,effectiveness and stability of the algorithm.展开更多
With continuous growth in scale,topology complexity,mission phases,and mission diversity,challenges have been placed for efficient capability evaluation of modern combat systems.Aiming at the problems of insufficient ...With continuous growth in scale,topology complexity,mission phases,and mission diversity,challenges have been placed for efficient capability evaluation of modern combat systems.Aiming at the problems of insufficient mission consideration and single evaluation dimension in the existing evaluation approaches,this study proposes a mission-oriented capability evaluation method for combat systems based on operation loop.Firstly,a combat network model is given that takes into account the capability properties of combat nodes.Then,based on the transition matrix between combat nodes,an efficient algorithm for operation loop identification is proposed based on the Breadth-First Search.Given the mission-capability satisfaction of nodes,the effectiveness evaluation indexes for operation loops and combat network are proposed,followed by node importance measure.Through a case study of the combat scenario involving space-based support against surface ships under different strategies,the effectiveness of the proposed method is verified.The results indicated that the ROI-priority attack method has a notable impact on reducing the overall efficiency of the network,whereas the O-L betweenness-priority attack is more effective in obstructing the successful execution of enemy attack missions.展开更多
Combining the heuristic algorithm （HA） developed based on the specific knowledge of the cooperative multiple target attack （CMTA） tactics and the particle swarm optimization （PSO）, a heuristic particle swarm opt...Combining the heuristic algorithm （HA） developed based on the specific knowledge of the cooperative multiple target attack （CMTA） tactics and the particle swarm optimization （PSO）, a heuristic particle swarm optimization （HPSO） algorithm is proposed to solve the decision-making （DM） problem. HA facilitates to search the local optimum in the neighborhood of a solution, while the PSO algorithm tends to explore the search space for possible solutions. Combining the advantages of HA and PSO, HPSO algorithms can find out the global optimum quickly and efficiently. It obtains the DM solution by seeking for the optimal assignment of missiles of friendly fighter aircrafts （FAs） to hostile FAs. Simulation results show that the proposed algorithm is superior to the general PSO algorithm and two GA based algorithms in searching for the best solution to the DM problem.展开更多
The combat survivability is an essential factor to be considered in the development of recent military aircraft. Radar stealth and onboard electronic attack are two major techniques for the reduction of aircraft susce...The combat survivability is an essential factor to be considered in the development of recent military aircraft. Radar stealth and onboard electronic attack are two major techniques for the reduction of aircraft susceptibility. A tactical scenario for a strike mission is presented. The effect of aircraft radar cross section on the detection probability of a threat radar, as well as that of onboard jammer, are investigated. The guidance errors of radar guided surface to air missile and anti aircraft artillery, which are disturbed by radar cross section reduction or jammer radiated power and both of them are determined. The probability of aircraft kill given a single shot is calculated and finally the sortie survivability of an attack aircraft in a supposed hostile thread environment worked out. It is demonstrated that the survivability of a combat aircraft will be greatly enhanced by the combined radar stealth and onboard electronic attack, and the evaluation metho dology is effective and applicable.展开更多
At evaluating the combat effectiveness of the defense system, target′s probability to penetrate the defended area is a primary care taking index. In this paper, stochastic model to compete the probability that targe...At evaluating the combat effectiveness of the defense system, target′s probability to penetrate the defended area is a primary care taking index. In this paper, stochastic model to compete the probability that target penetrates the defended area along any flight path is established by the state analysis and statistical equilibrium analysis of stochastic service system theory. The simulated annealing algorithm is an enlightening random search method based on Monte Carlo recursion, and it can find global optimal solution by simulating annealing process. Combining stochastic model to compete the probability and simulated annealing algorithm, this paper establishes the method to solve problem quantitatively about combat configuration optimization of weapon systems. The calculated result shows that the perfect configuration for fire cells of the weapon is fast found by using this method, and this quantificational method for combat configuration is faster and more scientific than previous one based on principle via map fire field.展开更多
文摘Reinforcement learning has been applied to air combat problems in recent years,and the idea of curriculum learning is often used for reinforcement learning,but traditional curriculum learning suffers from the problem of plasticity loss in neural networks.Plasticity loss is the difficulty of learning new knowledge after the network has converged.To this end,we propose a motivational curriculum learning distributed proximal policy optimization(MCLDPPO)algorithm,through which trained agents can significantly outperform the predictive game tree and mainstream reinforcement learning methods.The motivational curriculum learning is designed to help the agent gradually improve its combat ability by observing the agent's unsatisfactory performance and providing appropriate rewards as a guide.Furthermore,a complete tactical maneuver is encapsulated based on the existing air combat knowledge,and through the flexible use of these maneuvers,some tactics beyond human knowledge can be realized.In addition,we designed an interruption mechanism for the agent to increase the frequency of decisionmaking when the agent faces an emergency.When the number of threats received by the agent changes,the current action is interrupted in order to reacquire observations and make decisions again.Using the interruption mechanism can significantly improve the performance of the agent.To simulate actual air combat better,we use digital twin technology to simulate real air battles and propose a parallel battlefield mechanism that can run multiple simulation environments simultaneously,effectively improving data throughput.The experimental results demonstrate that the agent can fully utilize the situational information to make reasonable decisions and provide tactical adaptation in the air combat,verifying the effectiveness of the algorithmic framework proposed in this paper.
基金National Key R&D Program of China(Grant No.2021YFA1000402)National Natural Science Foundation of China(Grant No.72071159)to provide fund for conducting experiments。
文摘In the air combat process,confrontation position is the critical factor to determine the confrontation situation,attack effect and escape probability of UAVs.Therefore,selecting the optimal confrontation position becomes the primary goal of maneuver decision-making.By taking the position as the UAV’s maneuver strategy,this paper constructs the optimal confrontation position selecting games(OCPSGs)model.In the OCPSGs model,the payoff function of each UAV is defined by the difference between the comprehensive advantages of both sides,and the strategy space of each UAV at every step is defined by its accessible space determined by the maneuverability.Then we design the limit approximation of mixed strategy Nash equilibrium(LAMSNQ)algorithm,which provides a method to determine the optimal probability distribution of positions in the strategy space.In the simulation phase,we assume the motions on three directions are independent and the strategy space is a cuboid to simplify the model.Several simulations are performed to verify the feasibility,effectiveness and stability of the algorithm.
文摘With continuous growth in scale,topology complexity,mission phases,and mission diversity,challenges have been placed for efficient capability evaluation of modern combat systems.Aiming at the problems of insufficient mission consideration and single evaluation dimension in the existing evaluation approaches,this study proposes a mission-oriented capability evaluation method for combat systems based on operation loop.Firstly,a combat network model is given that takes into account the capability properties of combat nodes.Then,based on the transition matrix between combat nodes,an efficient algorithm for operation loop identification is proposed based on the Breadth-First Search.Given the mission-capability satisfaction of nodes,the effectiveness evaluation indexes for operation loops and combat network are proposed,followed by node importance measure.Through a case study of the combat scenario involving space-based support against surface ships under different strategies,the effectiveness of the proposed method is verified.The results indicated that the ROI-priority attack method has a notable impact on reducing the overall efficiency of the network,whereas the O-L betweenness-priority attack is more effective in obstructing the successful execution of enemy attack missions.
文摘Combining the heuristic algorithm （HA） developed based on the specific knowledge of the cooperative multiple target attack （CMTA） tactics and the particle swarm optimization （PSO）, a heuristic particle swarm optimization （HPSO） algorithm is proposed to solve the decision-making （DM） problem. HA facilitates to search the local optimum in the neighborhood of a solution, while the PSO algorithm tends to explore the search space for possible solutions. Combining the advantages of HA and PSO, HPSO algorithms can find out the global optimum quickly and efficiently. It obtains the DM solution by seeking for the optimal assignment of missiles of friendly fighter aircrafts （FAs） to hostile FAs. Simulation results show that the proposed algorithm is superior to the general PSO algorithm and two GA based algorithms in searching for the best solution to the DM problem.
文摘The combat survivability is an essential factor to be considered in the development of recent military aircraft. Radar stealth and onboard electronic attack are two major techniques for the reduction of aircraft susceptibility. A tactical scenario for a strike mission is presented. The effect of aircraft radar cross section on the detection probability of a threat radar, as well as that of onboard jammer, are investigated. The guidance errors of radar guided surface to air missile and anti aircraft artillery, which are disturbed by radar cross section reduction or jammer radiated power and both of them are determined. The probability of aircraft kill given a single shot is calculated and finally the sortie survivability of an attack aircraft in a supposed hostile thread environment worked out. It is demonstrated that the survivability of a combat aircraft will be greatly enhanced by the combined radar stealth and onboard electronic attack, and the evaluation metho dology is effective and applicable.
文摘At evaluating the combat effectiveness of the defense system, target′s probability to penetrate the defended area is a primary care taking index. In this paper, stochastic model to compete the probability that target penetrates the defended area along any flight path is established by the state analysis and statistical equilibrium analysis of stochastic service system theory. The simulated annealing algorithm is an enlightening random search method based on Monte Carlo recursion, and it can find global optimal solution by simulating annealing process. Combining stochastic model to compete the probability and simulated annealing algorithm, this paper establishes the method to solve problem quantitatively about combat configuration optimization of weapon systems. The calculated result shows that the perfect configuration for fire cells of the weapon is fast found by using this method, and this quantificational method for combat configuration is faster and more scientific than previous one based on principle via map fire field.
