AUTHOR=Wang Chengbo , Zhang Xinyu , Yang Zaili , Bashir Musa , Lee Kwangil TITLE=Collision avoidance for autonomous ship using deep reinforcement learning and prior-knowledge-based approximate representation JOURNAL=Frontiers in Marine Science VOLUME=Volume 9 - 2022 YEAR=2023 URL=https://www.frontiersin.org/journals/marine-science/articles/10.3389/fmars.2022.1084763 DOI=10.3389/fmars.2022.1084763 ISSN=2296-7745 ABSTRACT=Reinforcement learning (RL) has shown superior performance in solving sequential decision problems. This study aims to develop a novel intelligent collision avoidance algorithm based on approximate representation reinforcement learning (AR-RL) to realize the collision avoidance of maritime autonomous surface ships (MASS) in a continuous state space environment involving interactive learning capability like a crew in navigation situation. The new algorithm uses an approximate representation model to deal with the optimization of collision avoidance strategies in a dynamic target encounter situation. The model is combined with prior knowledge and International Regulations for Preventing Collisions at Sea (COLREGs) for optimal performance. This is followed by a design of an online solution to a value function approximation model based on gradient descent. This approach can solve the problem of large-scale collision avoidance policy learning in static-dynamic obstacles mixed environment. Finally, algorithm tests were constructed though two scenarios (i.e., the coastal static obstacle environment and the static-dynamic obstacles mixed environment) using Tianjin Port as an example. The results show that the algorithm can improve the large-scale learning efficiency of continuous state space of dynamic obstacle environment by approximate representation. At the same time, the MASS can efficiently and safely avoid obstacles enroute to reaching its target destination. It therefore makes significant contributions to ensuring safety at sea in a mixed traffic involving both manned and MASS in near future.