Vision Transformers & Segmentation for Panoptic Marine Litter Mapping

Poorna Pushkala KSubbulakshmi P^*

• Vellore Institute of Technology (VIT), Chennai, India

Marine litter is composed of various types of debris and poses a significant risk to marine ecosystems, biodiversity, and human life. Effective management and mitigation measures of marine litter, otherwise known as marine waste, can only be achieved through proper classification. The paper presents a new method based on panoptic segmentation and vision transformer (ViT) to perform the overall classification of marine litter. Panoptic segmentation, developed by synthesizing instance and semantic segmentation, can be used to identify both marine litter objects and background objects simultaneously. The image quality is added, and noise removal is provided to the raw input images to provide optimal input to the analysis. With the help of the panoptic segmentation model and a Vision Transformer, marine litter images are divided into semantically coherent segments, which can then be classified and located accurately and reliably as debris objects. Analysis on different datasets has shown good results, and both the quantitative and qualitative analyses support the usefulness of the methodology. These objectives include improving the levels of detection, localization of different kinds of debris under challenging marine environments, and comparing the effectiveness of the technique with the current ones. The proposed methodology can give valuable information regarding marine waste distribution and organization. The approach enables rational decision-making in protecting and managing pollution. Panoptic segmentation is an effective method that can be used in future studies and implementation in marine litter monitoring and mitigation due to its scalability and flexibility.