This paper addresses the time-varying formation-containment(FC) problem for nonholonomic multi-agent systems with a desired trajectory constraint, where only the leaders can acquire information about the desired traje...This paper addresses the time-varying formation-containment(FC) problem for nonholonomic multi-agent systems with a desired trajectory constraint, where only the leaders can acquire information about the desired trajectory. Input the fixed time-varying formation template to the leader and start executing, this process also needs to track the desired trajectory, and the follower needs to converge to the convex hull that the leader crosses. Firstly, the dynamic models of nonholonomic systems are linearized to second-order dynamics. Then, based on the desired trajectory and formation template, the FC control protocols are proposed. Sufficient conditions to achieve FC are introduced and an algorithm is proposed to resolve the control parameters by solving an algebraic Riccati equation. The system is demonstrated to achieve FC, with the average position and velocity of the leaders converging asymptotically to the desired trajectory. Finally, the theoretical achievements are verified in simulations by a multi-agent system composed of virtual human individuals.展开更多
Aiming at the problem on cooperative air-defense of surface warship formation, this paper maps the cooperative airdefense system of systems （SoS） for surface warship formation （CASoSSWF） to the biological immune s...Aiming at the problem on cooperative air-defense of surface warship formation, this paper maps the cooperative airdefense system of systems （SoS） for surface warship formation （CASoSSWF） to the biological immune system （BIS） according to the similarity of the defense mechanism and characteristics between the CASoSSWF and the BIS, and then designs the models of components and the architecture for a monitoring agent, a regulating agent, a killer agent, a pre-warning agent and a communicating agent by making use of the theories and methods of the artificial immune system, the multi-agent system （MAS）, the vaccine and the danger theory （DT）. Moreover a new immune multi-agent model using vaccine based on DT （IMMUVBDT） for the cooperative air-defense SoS is advanced. The immune response and immune mechanism of the CASoSSWF are analyzed. The model has a capability of memory, evolution, commendable dynamic environment adaptability and self-learning, and embodies adequately the cooperative air-defense mechanism for the CASoSSWF. Therefore it shows a novel idea for the CASoSSWF which can provide conception models for a surface warship formation operation simulation system.展开更多
For multi-agent systems based on the local information,the agents automatically converge to a common consensus state and the convergence speed is determined by the algebraic connectivity of the communication network.T...For multi-agent systems based on the local information,the agents automatically converge to a common consensus state and the convergence speed is determined by the algebraic connectivity of the communication network.To study fast consensus seeking problems of multi-agent systems in undirected networks,a consensus protocol is proposed which considers the average information of the agents＇ states in a certain time interval,and a consensus convergence criterion for the system is obtained.Based on the frequency-domain analysis and algebra graph theory,it is shown that if the time interval is chosen properly,then requiring the same maximum control effort the proposed protocol reaches consensus faster than the standard consensus protocol.Simulations are provided to demonstrate the effectiveness of these theoretical results.展开更多
In multi-agent systems(MAS),finding agents which are able to service properly in an open and dynamic environment are the key issue in problem solving.However,it is difficult to find agent resources quickly and positio...In multi-agent systems(MAS),finding agents which are able to service properly in an open and dynamic environment are the key issue in problem solving.However,it is difficult to find agent resources quickly and position agents accurately and complete the system integration by the keyword matching method,due to the lack of clear semantic information of the classical agent model.An semantic-based agent dynamic positioning mechanism was proposed to assist in the system dynamic integration.According to the semantic agent model and the description method,a two-stage process including the domain positioning stage and the service semantic matching positioning stage,was discussed.With this mechanism,proper agents that provide appropriate service to assign sub-tasks for task completion can be found quickly and accurately.Finally,the effectiveness of the positioning mechanism was validated through the in-depth performance analysis in the application of simulation experiments to the system dynamic integration.展开更多
Theoretical analysis of consensus for networked multi-agent systems with switching topologies was conducted.Supposing that information-exchange topologies of networked system are dynamic,a modified linear protocol is ...Theoretical analysis of consensus for networked multi-agent systems with switching topologies was conducted.Supposing that information-exchange topologies of networked system are dynamic,a modified linear protocol is proffered which is more practical than existing ones.The definition of trajectory consensus is given and a new consensus protocol is exhibited such that multi-agent system achieves trajectory consensus.In addition,a formation control strategy is designed.A common Lyapunov function is proposed to analyze the consensus convergence of networked multi-agent systems with switching topologies.Simulations are provided to demonstrate the effectiveness of the theoretical results.展开更多
To solve the problem of multi-target hunting by an unmanned surface vehicle(USV)fleet,a hunting algorithm based on multi-agent reinforcement learning is proposed.Firstly,the hunting environment and kinematic model wit...To solve the problem of multi-target hunting by an unmanned surface vehicle(USV)fleet,a hunting algorithm based on multi-agent reinforcement learning is proposed.Firstly,the hunting environment and kinematic model without boundary constraints are built,and the criteria for successful target capture are given.Then,the cooperative hunting problem of a USV fleet is modeled as a decentralized partially observable Markov decision process(Dec-POMDP),and a distributed partially observable multitarget hunting Proximal Policy Optimization(DPOMH-PPO)algorithm applicable to USVs is proposed.In addition,an observation model,a reward function and the action space applicable to multi-target hunting tasks are designed.To deal with the dynamic change of observational feature dimension input by partially observable systems,a feature embedding block is proposed.By combining the two feature compression methods of column-wise max pooling(CMP)and column-wise average-pooling(CAP),observational feature encoding is established.Finally,the centralized training and decentralized execution framework is adopted to complete the training of hunting strategy.Each USV in the fleet shares the same policy and perform actions independently.Simulation experiments have verified the effectiveness of the DPOMH-PPO algorithm in the test scenarios with different numbers of USVs.Moreover,the advantages of the proposed model are comprehensively analyzed from the aspects of algorithm performance,migration effect in task scenarios and self-organization capability after being damaged,the potential deployment and application of DPOMH-PPO in the real environment is verified.展开更多
System upgrades in unmanned systems have made Unmanned Aerial Vehicle(UAV)-based patrolling and monitoring a preferred solution for ocean surveillance.However,dynamic environments and large-scale deployments pose sign...System upgrades in unmanned systems have made Unmanned Aerial Vehicle(UAV)-based patrolling and monitoring a preferred solution for ocean surveillance.However,dynamic environments and large-scale deployments pose significant challenges for efficient decision-making,necessitating a modular multiagent control system.Deep Reinforcement Learning(DRL)and Decision Tree(DT)have been utilized for these complex decision-making tasks,but each has its limitations:DRL is highly adaptive but lacks interpretability,while DT is inherently interpretable but has limited adaptability.To overcome these challenges,we propose the Adaptive Interpretable Decision Tree(AIDT),an evolutionary-based algorithm that is both adaptable to diverse environmental settings and highly interpretable in its decision-making processes.We first construct a Markov decision process(MDP)-based simulation environment using the Cooperative Submarine Search task as a representative scenario for training and testing the proposed method.Specifically,we use the heat map as a state variable to address the issue of multi-agent input state proliferation.Next,we introduce the curiosity-guiding intrinsic reward to encourage comprehensive exploration and enhance algorithm performance.Additionally,we incorporate decision tree size as an influence factor in the adaptation process to balance task completion with computational efficiency.To further improve the generalization capability of the decision tree,we apply a normalization method to ensure consistent processing of input states.Finally,we validate the proposed algorithm in different environmental settings,and the results demonstrate both its adaptability and interpretability.展开更多
Two protocols are presented,which can make agents reach consensus while achieving and preserving the desired formation in fixed topology with and without communication timedelay for multi-agent network.First,the proto...Two protocols are presented,which can make agents reach consensus while achieving and preserving the desired formation in fixed topology with and without communication timedelay for multi-agent network.First,the protocol without considering the communication time-delay is presented,and by using Lyapunov stability theory,the sufficient condition of stability for this multi-agent system is presented.Further,considering the communication time-delay,the effectiveness of the protocol based on Lyapunov-Krasovskii function is demonstrated.The main contribution of the proposed protocols is that,as well as the velocity consensus is considered,the formation control is concerned for multi-agent systems described as the second-order equations.Finally,numerical examples are presented to illustrate the effectiveness of the proposed protocols.展开更多
Based on the idea of backstepping design, distributedcoordinated tracking problems under directed topology are discussedfor multiple Euler-Lagrange （EL） systems. The dynamicleader case is considered. First, with the...Based on the idea of backstepping design, distributedcoordinated tracking problems under directed topology are discussedfor multiple Euler-Lagrange （EL） systems. The dynamicleader case is considered. First, with the parameter-linearity property,a distributed coordinated adaptive control scheme is proposedfor EL systems in the presence of parametric uncertainties.Then, subject to nonlinear uncertainties and external disturbances,an improved adaptive control algorithm is developed by usingneural-network （NN） approximation of nonlinear functions. Bothproposed algorithms can make tracking errors for each followerultimately bounded. The closed-loop systems are investigated byusing the combination of graph theory, Lyapunov theory, and BarbalatLemma. Numerical examples and comparisons with othermethods are provided to show the effectiveness of the proposedcontrol strategies.展开更多
The pursuit problem is a well-known problem in computer science. In this problem, a group of predator agents attempt to capture a prey agent in an environment with various obstacle types, partial observation, and an i...The pursuit problem is a well-known problem in computer science. In this problem, a group of predator agents attempt to capture a prey agent in an environment with various obstacle types, partial observation, and an infinite grid-world. Predator agents are applied algorithms that use the univector field method to reach the prey agent, strategies for avoiding obstacles and strategies for cooperation between predator agents. Obstacle avoidance strategies are generalized and presented through strategies called hitting and following boundary(HFB); trapped and following shortest path(TFSP); and predicted and following shortest path(PFSP). In terms of cooperation, cooperation strategies are employed to more quickly reach and capture the prey agent. Experimental results are shown to illustrate the efficiency of the method in the pursuit problem.展开更多
文摘This paper addresses the time-varying formation-containment(FC) problem for nonholonomic multi-agent systems with a desired trajectory constraint, where only the leaders can acquire information about the desired trajectory. Input the fixed time-varying formation template to the leader and start executing, this process also needs to track the desired trajectory, and the follower needs to converge to the convex hull that the leader crosses. Firstly, the dynamic models of nonholonomic systems are linearized to second-order dynamics. Then, based on the desired trajectory and formation template, the FC control protocols are proposed. Sufficient conditions to achieve FC are introduced and an algorithm is proposed to resolve the control parameters by solving an algebraic Riccati equation. The system is demonstrated to achieve FC, with the average position and velocity of the leaders converging asymptotically to the desired trajectory. Finally, the theoretical achievements are verified in simulations by a multi-agent system composed of virtual human individuals.
文摘Aiming at the problem on cooperative air-defense of surface warship formation, this paper maps the cooperative airdefense system of systems （SoS） for surface warship formation （CASoSSWF） to the biological immune system （BIS） according to the similarity of the defense mechanism and characteristics between the CASoSSWF and the BIS, and then designs the models of components and the architecture for a monitoring agent, a regulating agent, a killer agent, a pre-warning agent and a communicating agent by making use of the theories and methods of the artificial immune system, the multi-agent system （MAS）, the vaccine and the danger theory （DT）. Moreover a new immune multi-agent model using vaccine based on DT （IMMUVBDT） for the cooperative air-defense SoS is advanced. The immune response and immune mechanism of the CASoSSWF are analyzed. The model has a capability of memory, evolution, commendable dynamic environment adaptability and self-learning, and embodies adequately the cooperative air-defense mechanism for the CASoSSWF. Therefore it shows a novel idea for the CASoSSWF which can provide conception models for a surface warship formation operation simulation system.
基金supported by the National Natural Science Foundation of China (6087405360574088)
文摘For multi-agent systems based on the local information,the agents automatically converge to a common consensus state and the convergence speed is determined by the algebraic connectivity of the communication network.To study fast consensus seeking problems of multi-agent systems in undirected networks,a consensus protocol is proposed which considers the average information of the agents＇ states in a certain time interval,and a consensus convergence criterion for the system is obtained.Based on the frequency-domain analysis and algebra graph theory,it is shown that if the time interval is chosen properly,then requiring the same maximum control effort the proposed protocol reaches consensus faster than the standard consensus protocol.Simulations are provided to demonstrate the effectiveness of these theoretical results.
基金Projects(61173026,61373045,61202039)supported by the National Natural Science Foundation of ChinaProject(2012AA02A603)supported by the National High Technology Research and Development Program of China+1 种基金Projects(K5051223008,K5051223002)supported by the Fundamental Research Funds for the Central Universities of ChinaProject(513***103E)supported by the Pre-Research Project of the"Twelfth Five-Year-Plan"of China
文摘In multi-agent systems(MAS),finding agents which are able to service properly in an open and dynamic environment are the key issue in problem solving.However,it is difficult to find agent resources quickly and position agents accurately and complete the system integration by the keyword matching method,due to the lack of clear semantic information of the classical agent model.An semantic-based agent dynamic positioning mechanism was proposed to assist in the system dynamic integration.According to the semantic agent model and the description method,a two-stage process including the domain positioning stage and the service semantic matching positioning stage,was discussed.With this mechanism,proper agents that provide appropriate service to assign sub-tasks for task completion can be found quickly and accurately.Finally,the effectiveness of the positioning mechanism was validated through the in-depth performance analysis in the application of simulation experiments to the system dynamic integration.
基金Projects(61075065, 60774045) supported by the National Natural Science Foundation of China Project(CX2010B080) supported by Hunan Provincial Innovation Foundation For Postgraduate,China
文摘Theoretical analysis of consensus for networked multi-agent systems with switching topologies was conducted.Supposing that information-exchange topologies of networked system are dynamic,a modified linear protocol is proffered which is more practical than existing ones.The definition of trajectory consensus is given and a new consensus protocol is exhibited such that multi-agent system achieves trajectory consensus.In addition,a formation control strategy is designed.A common Lyapunov function is proposed to analyze the consensus convergence of networked multi-agent systems with switching topologies.Simulations are provided to demonstrate the effectiveness of the theoretical results.
基金Supported by National Natural Science Foundation of China （61273137, 51209026, 61074017）, the Scientific Research Fund of Liaoning Provincial Education Department （L2013202）, and the Fundamental Research Funds for the Central Universities （3132013037, 3132014047, 3132014321）
基金financial support from National Natural Science Foundation of China(Grant No.61601491)Natural Science Foundation of Hubei Province,China(Grant No.2018CFC865)Military Research Project of China(-Grant No.YJ2020B117)。
文摘To solve the problem of multi-target hunting by an unmanned surface vehicle(USV)fleet,a hunting algorithm based on multi-agent reinforcement learning is proposed.Firstly,the hunting environment and kinematic model without boundary constraints are built,and the criteria for successful target capture are given.Then,the cooperative hunting problem of a USV fleet is modeled as a decentralized partially observable Markov decision process(Dec-POMDP),and a distributed partially observable multitarget hunting Proximal Policy Optimization(DPOMH-PPO)algorithm applicable to USVs is proposed.In addition,an observation model,a reward function and the action space applicable to multi-target hunting tasks are designed.To deal with the dynamic change of observational feature dimension input by partially observable systems,a feature embedding block is proposed.By combining the two feature compression methods of column-wise max pooling(CMP)and column-wise average-pooling(CAP),observational feature encoding is established.Finally,the centralized training and decentralized execution framework is adopted to complete the training of hunting strategy.Each USV in the fleet shares the same policy and perform actions independently.Simulation experiments have verified the effectiveness of the DPOMH-PPO algorithm in the test scenarios with different numbers of USVs.Moreover,the advantages of the proposed model are comprehensively analyzed from the aspects of algorithm performance,migration effect in task scenarios and self-organization capability after being damaged,the potential deployment and application of DPOMH-PPO in the real environment is verified.
文摘System upgrades in unmanned systems have made Unmanned Aerial Vehicle(UAV)-based patrolling and monitoring a preferred solution for ocean surveillance.However,dynamic environments and large-scale deployments pose significant challenges for efficient decision-making,necessitating a modular multiagent control system.Deep Reinforcement Learning(DRL)and Decision Tree(DT)have been utilized for these complex decision-making tasks,but each has its limitations:DRL is highly adaptive but lacks interpretability,while DT is inherently interpretable but has limited adaptability.To overcome these challenges,we propose the Adaptive Interpretable Decision Tree(AIDT),an evolutionary-based algorithm that is both adaptable to diverse environmental settings and highly interpretable in its decision-making processes.We first construct a Markov decision process(MDP)-based simulation environment using the Cooperative Submarine Search task as a representative scenario for training and testing the proposed method.Specifically,we use the heat map as a state variable to address the issue of multi-agent input state proliferation.Next,we introduce the curiosity-guiding intrinsic reward to encourage comprehensive exploration and enhance algorithm performance.Additionally,we incorporate decision tree size as an influence factor in the adaptation process to balance task completion with computational efficiency.To further improve the generalization capability of the decision tree,we apply a normalization method to ensure consistent processing of input states.Finally,we validate the proposed algorithm in different environmental settings,and the results demonstrate both its adaptability and interpretability.
基金supported by the National Natural Science Foundation of China (6093400361074065)+1 种基金the National Basic Research Program of China (973 Program) (2010CB731800)the Key Project for Natural Science Research of Hebei Education Department (ZD200908)
文摘Two protocols are presented,which can make agents reach consensus while achieving and preserving the desired formation in fixed topology with and without communication timedelay for multi-agent network.First,the protocol without considering the communication time-delay is presented,and by using Lyapunov stability theory,the sufficient condition of stability for this multi-agent system is presented.Further,considering the communication time-delay,the effectiveness of the protocol based on Lyapunov-Krasovskii function is demonstrated.The main contribution of the proposed protocols is that,as well as the velocity consensus is considered,the formation control is concerned for multi-agent systems described as the second-order equations.Finally,numerical examples are presented to illustrate the effectiveness of the proposed protocols.
基金supported by the National Natural Science Foundation of China(6130400561174200)the Research Fund for the Doctoral Program of Higher Education of China(20102302110031)
文摘Based on the idea of backstepping design, distributedcoordinated tracking problems under directed topology are discussedfor multiple Euler-Lagrange （EL） systems. The dynamicleader case is considered. First, with the parameter-linearity property,a distributed coordinated adaptive control scheme is proposedfor EL systems in the presence of parametric uncertainties.Then, subject to nonlinear uncertainties and external disturbances,an improved adaptive control algorithm is developed by usingneural-network （NN） approximation of nonlinear functions. Bothproposed algorithms can make tracking errors for each followerultimately bounded. The closed-loop systems are investigated byusing the combination of graph theory, Lyapunov theory, and BarbalatLemma. Numerical examples and comparisons with othermethods are provided to show the effectiveness of the proposedcontrol strategies.
基金the Basic Science Research Program through the National Research Foundation of Korea (NRF-2014R1A1A2057735)the Kyung Hee University in 2016 [KHU-20160601]
文摘The pursuit problem is a well-known problem in computer science. In this problem, a group of predator agents attempt to capture a prey agent in an environment with various obstacle types, partial observation, and an infinite grid-world. Predator agents are applied algorithms that use the univector field method to reach the prey agent, strategies for avoiding obstacles and strategies for cooperation between predator agents. Obstacle avoidance strategies are generalized and presented through strategies called hitting and following boundary(HFB); trapped and following shortest path(TFSP); and predicted and following shortest path(PFSP). In terms of cooperation, cooperation strategies are employed to more quickly reach and capture the prey agent. Experimental results are shown to illustrate the efficiency of the method in the pursuit problem.
