{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,17]],"date-time":"2026-01-17T21:51:38Z","timestamp":1768686698903,"version":"3.49.0"},"reference-count":39,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2022,9,14]],"date-time":"2022-09-14T00:00:00Z","timestamp":1663113600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["52271306"],"award-info":[{"award-number":["52271306"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["31422120"],"award-info":[{"award-number":["31422120"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["6142215200106"],"award-info":[{"award-number":["6142215200106"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Innovative Research Foundation of Ship General Performance","award":["52271306"],"award-info":[{"award-number":["52271306"]}]},{"name":"Innovative Research Foundation of Ship General Performance","award":["31422120"],"award-info":[{"award-number":["31422120"]}]},{"name":"Innovative Research Foundation of Ship General Performance","award":["6142215200106"],"award-info":[{"award-number":["6142215200106"]}]},{"name":"Key Laboratory of Equipment Pre-Research Fund Project","award":["52271306"],"award-info":[{"award-number":["52271306"]}]},{"name":"Key Laboratory of Equipment Pre-Research Fund Project","award":["31422120"],"award-info":[{"award-number":["31422120"]}]},{"name":"Key Laboratory of Equipment Pre-Research Fund Project","award":["6142215200106"],"award-info":[{"award-number":["6142215200106"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The unmanned surface vehicle (USV) has attracted more and more attention because of its basic ability to perform complex maritime tasks autonomously in constrained environments. However, the level of autonomy of one single USV is still limited, especially when deployed in a dynamic environment to perform multiple tasks simultaneously. Thus, a multi-USV cooperative approach can be adopted to obtain the desired success rate in the presence of multi-mission objectives. In this paper, we propose a cooperative navigating approach by enabling multiple USVs to automatically avoid dynamic obstacles and allocate target areas. To be specific, we propose a multi-agent deep reinforcement learning (MADRL) approach, i.e., a multi-agent deep deterministic policy gradient (MADDPG), to maximize the autonomy level by jointly optimizing the trajectory of USVs, as well as obstacle avoidance and coordination, which is a complex optimization problem usually solved separately. In contrast to other works, we combined dynamic navigation and area assignment to design a task management system based on the MADDPG learning framework. Finally, the experiments were carried out on the Gym platform to verify the effectiveness of the proposed method.<\/jats:p>","DOI":"10.3390\/s22186942","type":"journal-article","created":{"date-parts":[[2022,9,14]],"date-time":"2022-09-14T23:16:36Z","timestamp":1663197396000},"page":"6942","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":24,"title":["Dynamic Navigation and Area Assignment of Multiple USVs Based on Multi-Agent Deep Reinforcement Learning"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4287-6644","authenticated-orcid":false,"given":"Jiayi","family":"Wen","sequence":"first","affiliation":[{"name":"Lab of Intelligent Marine Vehicles of DMU, Dalian Maritime University, Dalian 116026, China"}]},{"given":"Shaoman","family":"Liu","sequence":"additional","affiliation":[{"name":"Lab of Intelligent Marine Vehicles of DMU, Dalian Maritime University, Dalian 116026, China"}]},{"given":"Yejin","family":"Lin","sequence":"additional","affiliation":[{"name":"Lab of Intelligent Marine Vehicles of DMU, Dalian Maritime University, Dalian 116026, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1773","DOI":"10.1109\/TCYB.2017.2715228","article-title":"A Sampling-Based Bayesian Approach for Cooperative Multiagent Online Search With Resource Constraints","volume":"48","author":"Xiao","year":"2017","journal-title":"IEEE Trans. Cybern."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1827","DOI":"10.1109\/TMECH.2012.2213607","article-title":"Sensor-Driven Online Coverage Planning for Autonomous Underwater Vehicles","volume":"18","author":"Paull","year":"2012","journal-title":"IEEE\/ASME Trans. Mechatron."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2358","DOI":"10.1109\/TVT.2021.3136670","article-title":"Autonomous Pilot of Unmanned Surface Vehicles: Bridging Path Planning and Tracking","volume":"71","author":"Wang","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"4442","DOI":"10.1109\/JSYST.2019.2891056","article-title":"Persistent Tracking of Maneuvering Target Using IMM Filter and DMPC by Initialization-Guided Game Approach","volume":"13","author":"Qi","year":"2019","journal-title":"IEEE Syst. J."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1174","DOI":"10.1109\/TMECH.2017.2684423","article-title":"Coordinated navigation of surface and underwater marine robotic vehicles for ocean sampling and environmental monitoring","volume":"22","year":"2017","journal-title":"IEEE-ASME Trans. Mechatron."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1106","DOI":"10.1109\/TCYB.2015.2423635","article-title":"A novel extreme learning control framework. of unmanned surface vehicles","volume":"46","author":"Wang","year":"2015","journal-title":"IEEE Trans. Cybern."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Kim, H., Park, B., and Myung, H. (2013). Curvature path planning with high resolution graph. for unmanned surface vehicle. Robot Intelligence Technology and Applications 2012, Springer.","DOI":"10.1007\/978-3-642-37374-9_15"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1109\/TRO.2010.2085790","article-title":"Feasible and Optimal Path Planning in Strong Current Fields","volume":"27","author":"Soulignac","year":"2010","journal-title":"IEEE Trans. Robot."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1504","DOI":"10.1109\/TWC.2019.2891629","article-title":"Liquid State Machine Learning for Resource and Cache Management in LTE-U Unmanned Aerial Vehicle (UAV) Networks","volume":"18","author":"Chen","year":"2019","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/j.oceaneng.2014.09.001","article-title":"Shell space decomposition based path planning for AUVs operating in a variable environment","volume":"91","author":"Zeng","year":"2014","journal-title":"Ocean Eng."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1105","DOI":"10.1109\/TCST.2008.2012116","article-title":"Real-time motion planning with applications to autonomous urban driving","volume":"17","author":"Kuwata","year":"2009","journal-title":"IEEE Trans. Control Syst. Technol."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Karaman, S., Walter, M.R., and Perez, A. (2011, January 9\u201313). Any time motion planning using the RRT*. Proceedings of the 2011 IEEE International Conference on Robotics and Automation, Shanghai, China.","DOI":"10.1109\/ICRA.2011.5980479"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"473","DOI":"10.1109\/TRO.2016.2539377","article-title":"Asymptotically Near-Optimal RRT for Fast, High-Quality Motion Planning","volume":"32","author":"Salzman","year":"2016","journal-title":"IEEE Trans. Robot."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1038","DOI":"10.1016\/j.cja.2013.04.040","article-title":"Real-time trajectory planning for UCAV air-to-surface attack using inverse dynamics optimization method and receding horizon control","volume":"26","author":"Zhang","year":"2013","journal-title":"Chin. J. Aeronaut."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ma, C.S., and Miller, R.H. (2006, January 14\u201316). MILP optimal path planning for real-time applications. Proceedings of the American Control Conference, Minneapolis, MN, USA.","DOI":"10.1109\/ACC.2006.1657504"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1109\/TIV.2018.2843163","article-title":"Mixed-Integer Linear Programming for Optimal Scheduling of Autonomous Vehicle Intersection Crossing","volume":"3","author":"Fayazi","year":"2018","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Khatib, O. (1986). Real-time obstacle avoidance for manipulators and mobile robots. Autonomous Robot Vehicles, Springer.","DOI":"10.1007\/978-1-4613-8997-2_29"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1007\/s11431-017-9198-6","article-title":"Obstacle-avoiding path planning for multiple autonomous underwater vehicles with simultaneous arrival","volume":"62","author":"Yao","year":"2018","journal-title":"Sci. China Technol. Sci."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Xie, S.R., Wu, P., Peng, Y., Luo, J., Qu, D., Li, Q.M., and Gu, J. (2014, January 28\u201330). The obstacle avoidance planning of USV based on improved artificial potential field. Proceedings of the 2014 IEEE International Conference on Information and Automation (ICIA), Hulunbuir, China.","DOI":"10.1109\/ICInfA.2014.6932751"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"898","DOI":"10.1109\/TII.2019.2922823","article-title":"Successive Waypoints Tracking of an Underactuated Surface Vehicle","volume":"16","author":"Wang","year":"2019","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1016\/j.isatra.2018.02.003","article-title":"Multi-objective path planning for unmanned surface vehicle with currents effects","volume":"75","author":"Ma","year":"2018","journal-title":"ISA Trans."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1748","DOI":"10.1109\/TASE.2020.2976560","article-title":"Neural RRT*: Learning-Based Optimal Path Planning","volume":"17","author":"Wang","year":"2020","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_23","unstructured":"Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2016, January 2\u20134). Prioritized Experience Replay. Proceedings of the International Conference on Learning Representations 2016, San Juan, Puerto Rico. arXiv:1511.05952v4."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"108709","DOI":"10.1016\/j.oceaneng.2021.108709","article-title":"The hybrid path planning algorithm. based on improved A* and artificial potential field for unmanned surface vehicle formations","volume":"223","author":"Sang","year":"2020","journal-title":"Ocean Eng."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.oceaneng.2019.05.017","article-title":"A multilayer path planner for a USV under complex. marine environments","volume":"184","author":"Wang","year":"2019","journal-title":"Ocean. Eng."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"6928","DOI":"10.1109\/TVT.2020.2991220","article-title":"Dynamics-Constrained Global-Local Hybrid Path Planning of. an Autonomous Surface Vehicle","volume":"69","author":"Wang","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Gu, Y., Zhang, Y., Rong, Z.W., and Tong, H.Z. (2022). Unmanned Surface Vehicle Collision Avoidance Path Planning in Restricted Waters Using Multi-Objective Optimisation Complying with COLREGs. Sensors, 22.","DOI":"10.3390\/s22155796"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"6782","DOI":"10.1109\/TVT.2020.2991983","article-title":"Cooperative Path Planning for Heterogeneous Unmanned Vehicles in a Search-and-Track Mission Aiming at an Underwater Target","volume":"69","author":"Wu","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1319","DOI":"10.1109\/TVT.2019.2958197","article-title":"A Novel Hybrid Neural Network-Based Multirobot Path Planning With Motion Coordination","volume":"69","author":"Pradhan","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"7464","DOI":"10.1109\/TVT.2021.3093318","article-title":"Swarm-Based 4D Path Planning For Drone. Operations in Urban Environments","volume":"70","author":"Wu","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"107693","DOI":"10.1016\/j.oceaneng.2020.107693","article-title":"Global path planning and. multi-objective path control for unmanned surface vehicle based on modified particle swarm optimization (PSO) algorithm","volume":"216","author":"Guo","year":"2020","journal-title":"Ocean Eng."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1016\/j.eswa.2018.08.008","article-title":"Multi-objective multi-robot path planning in. continuous environment using an enhanced genetic algorithm","volume":"115","author":"Nazarahari","year":"2019","journal-title":"Expert Syst. Appl."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"286","DOI":"10.1109\/LCOMM.2018.2889062","article-title":"Secure UAV Communication With Cooperative Jamming and Trajectory Control","volume":"23","author":"Zhong","year":"2018","journal-title":"IEEE Commun. Lett."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"3163","DOI":"10.1109\/TVT.2019.2897134","article-title":"Deep Reinforcement Learning Based Resource Allocation for V2V Communications","volume":"68","author":"Ye","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2424","DOI":"10.1109\/TVT.2020.2964784","article-title":"Using reinforcement learning to minimize the probability of delay occurrence in transportation","volume":"69","author":"Cao","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"4271","DOI":"10.1109\/TVT.2020.2972999","article-title":"Mode Selection and Resource Allocation in Sliced Fog Radio Access Networks: A Reinforcement Learning Approach","volume":"69","author":"Xiang","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"4069","DOI":"10.1109\/TVT.2019.2900157","article-title":"UAV-Enabled Secure Communications: Joint Trajectory and Transmit Power Optimization","volume":"68","author":"Zhou","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_38","unstructured":"Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., and Mordatch, I. (2017, January 4\u20139). Multi-agent actor-critic for mixed cooperative-competitive environments. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Littman, M.L. (1994, January 10\u201313). Markov games as a framework for multi-agent reinforcement learning. Proceedings of the Eleventh International Conference on International Conference on Machine Learning, New Brunswick, NJ, USA.","DOI":"10.1016\/B978-1-55860-335-6.50027-1"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/18\/6942\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:31:00Z","timestamp":1760142660000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/18\/6942"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,14]]},"references-count":39,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["s22186942"],"URL":"https:\/\/doi.org\/10.3390\/s22186942","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,14]]}}}