Abstract
Introduction:
Monitoring oil spills in the port area is very difficult and can spread rapidly, resulting in considerable damage to the environment. This study proposes HyEff-UNet, a deep learning-based semantic segmentation model for the detection of oil spills using drone-captured RGB images of port environments.
Methods:
HyEff-UNet employs a modified version of the U-Net model with an EfficientNet-B4 model as an encoder is used as the model architecture to classify the pixels of the input image as oil, water, or other surface regions. A combined categorical cross-entropy and Dice loss function is used to jointly optimize pixel-wise classification and region overlap.
Results:
The results show that the complete architecture achieves the highest performance among the internally evaluated models, while the sequential ablation indicates that the largest performance improvement is observed after the hybrid categorical cross-entropy and Dice loss is introduced following the EfficientNet-B4 and attention-based architecture.
Conclusion:
These findings demonstrate the effectiveness of the complete architecture within the evaluated configurations. However, the sequential ablation study does not isolate the independent contribution of the hybrid loss. Furthermore, the measured inference latency indicates that further optimization and unmanned aerial vehicle (UAV)-specific benchmarking are required before real-time onboard deployment can be considered.
1 Introduction
Marine and port oil spills pose significant environmental, economic, and ecological threats. Unintentional oil spills in transportation, offshore drilling, and shipping operations can cause serious harm to the aquatic ecosystem, pollution of coastal areas, and an impact on marine biodiversity. Rapid detection and assessment of oil spills are essential to minimize environmental damage and enable timely cleanup operations. The traditional systems of monitoring, like inspection by hand, patrolling boats, and satellite surveillance, are usually time-consuming and laborious. These limitations highlight the need for automated, intelligent, and high-accuracy detection systems. Recent advances in unmanned aerial vehicles (UAVs) and computer vision have enabled efficient large-scale environmental monitoring. UAV-based RGB imaging provides high-resolution aerial images of coastal and port regions, making it suitable for detecting oil contamination. However, detecting oil spills from aerial imagery remains challenging because of varying illumination, water reflections, waves, and the visual similarity between oil slicks and water. Therefore, effective image segmentation methods based on deep learning are needed to improve pixel-level classification in visually challenging marine scenes.
Some other research on monitoring oil spills through Underwater Wireless Sensor Networks (UWSNs) has also been carried out recently. Specifically, a data gathering approach based on priority with multi-mobile sinks has been proposed by ). For effective underwater oil pipeline leakage detection through UWSNs, a GWOWOA algorithm for optimizing data gathering and leakage detection processes has been proposed by ). In mobility-assisted UWSNs. Moreover, ) have devised a deep learning-based methodology with the YOLOv12 algorithm for UAV-based marine oil spill detection. While these methods offer improvements in terms of data gathering, optimization of sensor networks, and object detection, their main emphasis is more on network optimization and object detection with bounding boxes. Conversely, the HyEff-UNet architecture performs pixel-level image segmentation of UAV RGB images.
Convolutional neural networks (CNNs), particularly encoder–decoder architectures such as U-Net, have demonstrated excellent performance in semantic image segmentation tasks. U-Net can capture both large- and small-scale features with skip connections between encoder and decoder layers. However, standard U-Net models often perform poorly on complex textures and small spill regions. To overcome these limitations, the encoder stage can have modern backbone networks introduced with enhanced feature extraction functions. An adapted U-Net design utilizing EfficientNet-B4 as the encoder is suggested for the segmentation of oil spills. The EfficientNet-B4 backbone is incorporated to provide hierarchical and multi-scale feature representations; however, its contribution within the sequential ablation sequence is evaluated empirically rather than assumed a priori. The decoder incorporates attention gates to selectively weight spatial features during skip-connection fusion. Their contribution to segmentation performance is evaluated through the ablation study. The UAV-collected RGB images are preprocessed by resizing, mask color mapping, and one-hot encoding, and then the model is trained via supervised learning with categorical cross-entropy and dice loss function to jointly optimize pixel-wise classification and region overlap.
The system classifies each pixel into one of three classes: Oil, Water, or Other. Performance is measured using Precision, Recall, F1-score, and Intersection over Union (IoU). The experimental results show improved segmentation performance and region overlap compared with the internally evaluated baseline configurations. In general, the HyEff-UNet architecture provides automated pixel-level segmentation of oil spills from UAV RGB imagery, with potential application in marine environmental monitoring. Its suitability for real-time deployment requires further optimization and validation on representative UAV or edge-computing hardware. The framework has potential for supporting automated marine pollution monitoring, although operational effectiveness would require further validation in representative deployment conditions.
2 Main contributions of the proposed work
A hybrid semantic segmentation architecture called HyEff-UNet is introduced for UAV-based marine oil spill segmentation using an EfficientNet-B4 encoder in addition to the U-Net decoder architecture with an attention mechanism. The proposed framework exploits transfer learning and an attention mechanism to help with pixel-wise segmentation of oil spill areas while reducing irrelevant background information. A hybrid loss function combining Categorical Cross-Entropy and Dice Loss is incorporated into the proposed architecture. Among the evaluated sequential configurations, the complete architecture achieved the best performance, although the independent contribution of the hybrid loss could not be isolated. A full semantic segmentation framework for UAV RGB images, including data preparation, attention mechanism for feature extraction, hybrid optimization, and multi-class prediction, is developed. A controlled internal ablation study is conducted to examine the proposed framework for marine oil spill segmentation against the corresponding internally trained U-Net variants using Precision, Recall, F1-score, and IoU on the fixed held-out test partition. These architectural components are assessed using a sequential ablation methodology. Consequently, the reported improvements reflect successive architectural modifications rather than independent factorial effects. This is done by means of a hybrid loss function that utilizes both categorical cross-entropy and Dice loss. Based on the experiments, the best-performing model among the internally evaluated HyEff-UNet configurations is the EfficientNet-B4 + attention + hybrid-loss combination.
3 Literature survey
) introduce a drone-collected RGB image dataset to detect an oil spill in port conditions, filling the gap in high-resolution, low-cost surveillance tools. The dataset was annotated with 1,268 images that are categorized into oil, water, and other categories, and it can be used to train deep learning models. The system can implement oil detection with an F1-score of 0.71 using a U-Net and an EfficientNet encoder, which is practically achievable. The work facilitates real-time, scalable, and low-cost environmental surveillance to manage port safety and pollution.
The authors of the article by ) propose a deep learning model with a dual attention mechanism to improve the detection and classification of oil spills by using unmanned aerial vehicle (UAV) imagery. The system employs two modules, namely one used to extract spatial relationships across the pixels and one used to extract characteristics between channels of the image. The researchers have used a Generative Adversarial Network (GAN) to augment the data in a customized manner to compensate for the lack of data. This method greatly enhances the precision of detection of different types of oil, which is a strong methodology for safeguarding the marine ecosystem using high-resolution, automated aerial observation.
) suggested a laser-induced fluorescence LiDAR to identify an oil spill in real time as well as determine the nature of the oil. The system incorporates the modules of laser emission, signal reception, spectral processing, and wireless transmission to remotely read the fluorescence signatures of the oil films. A BP neural network classifies spectral data and has 96.58 percent prediction accuracy on several types of oils. The method is very sensitive, reliable, and can be applied in the fast and non-contact monitoring of the environment during complicated conditions of the sea.
) introduce LRA-UNet, a single lightweight residual attention network that can be used to detect marine oil spills precisely in SAR images. The model combines depth-wise separable convolutions to minimize computation, SimAM-based residual attention to improve feature extraction, and a Sobel edge-aware joint loss to improve boundary segmentation. The method obtains a high-quality segmentation with 67.36% mIoU, surpassing the results of the baseline UNet and DeepLabV3+ networks based on Sentinel-1 SAR datasets. The technique is cost-effective, robust, and applicable to real-time and large-scale monitoring of the environment.
In a wide-ranging review of the latest developments in marine oil spill surveillance by means of artificial intelligence (AI) and remote sensing (RS) technologies, ) discussed the topic in an in-depth manner. Their research compares different RS data sources such as SAR, optical, and thermal infrared in three key tasks: detection, classification, and thickness estimation. The authors emphasize the importance of multi-source data fusion and publicly available datasets in improving the accuracy of monitoring and response effectiveness by emphasizing the shift from the old system of manual verification to automated deep learning frameworks.
The proposed SAM-OIL (2024) is a compositional oil spill detection model designed to act on SAR images and combines YOLOv8 object detection with an Adapted Segment Anything Model and an Ordered Mask Fusion module. The detector gives bounding boxes, the adapted SAM gives better segmentation of boundaries in the form of masks, and fusion eliminates category conflicts. The method eliminates reliance on finely annotated masks and attains 69.52% mIoU on the M4D dataset, which is better than traditional semantic segmentation methods that rely on efficient and accurate large-scale monitoring ().
) propose an Oil Spill Change Detection (OSCD) system that transitions to bi-temporal SAR change detection to reduce false positives due to look-alike effects that arise after single-image segmentation. They suggest Temporal-Aware Hybrid Inpainting (TAHI), which is a mixture of high-fidelity hybrid inpainting and temporal realism enhancement to produce pre-spill images and build the original OSCD dataset. It has been experimentally demonstrated that change detection models are superior to segmentation baselines, which are more accurate, robust, and scalable to provide consistent monitoring of marine oil spills on large-scale levels.
Full-Scale Aggregated MobileUNet (FA-MobileUNet) is a lightweight model proposed by ) that can be used in the detection of SAR oil spills in real-time. The model is based on MobileNetV3 with Atrous Spatial Pyramid Pooling (ASPP) and Convolutional Block Attention Modules (CBAM) as auxiliary feature extractors to capture multi-scale features. It has a mean Intersection over Union (mIoU) of over 80 percent with various marine targets. The system is highly efficient in its computational performance and accuracy rates in differentiating oil spills and complex look-alikes.
) introduced a deep-learning model for oil spill detection and categorization at scale using Synthetic Aperture Radar (SAR) images. The framework is a computer-aided fully convolutional network (OFCN) on the U-Net architecture, which performs pixel-level segmentation and delivers state-of-the-art results similar to human operators. After the first detection, a second neural network identifies 12 spill types by shape and texture. This is an automated pipeline that enables real-time global surveillance and the past analysis of marine oil pollution.
A marine oil spill-based trauma system using the YOLOv9 deep learning framework is suggested by ). The model manages to overcome issues such as sea clutter and different light conditions by processing high-resolution RGB images through a generalized efficient layer aggregation network (GELAN). The system has a mean average precision (mAP) of 94.45 and 95.6% precision. It is a lean and fast method that improves the effectiveness of surveillance, and it allows a quick response to emergencies to help preserve marine ecosystems against ecological disasters.
CBENet is a contextual and boundary-enhanced network proposed by ) to identify marine oil spills in microwave RS images, especially synthetic aperture radar (SAR) images. The model has an encoder-decoder structure with a contextual fusion block and a boundary-enhanced loss to address the difficulties of blurred boundaries and multi-scale spill distributions. CBENet has been demonstrated to be far superior to state-of-the-art models on a dataset of 1,274 SAR images in terms of detection accuracy and boundary identification.
) introduced a framework of automated oil spill detection in the northern entrance to the Suez Canal based on DeepLabv3+ and Sentinel-1 SAR images. Their study compares the results of models trained using the international EMSA-CSN data with a domestic Egyptian dataset. The localized model was more effective, with an accuracy of 98.14 and mIoU of 0.7872. The study shows that deep learning can be a good tool to monitor the environment instantly and react to an emergency in crucial maritime routes.
) presented a semantic segmentation framework for unmanned aerial vehicle (UAV) RS images based on the fully convolutional neural network (FCNN). To overcome issues such as high resolution and the intricate scale of the target, the authors combined the atrous spatial pyramid pooling (ASPP) module and a coordinate attention (CA) mechanism. Such a combined method enables superior multi-scale feature extraction and accurate localization of target boundaries. It has been shown in experimental work that the model is much more accurate and efficient in terms of segmentation in environmental monitoring as well as disaster assessment.
) assessed semantic segmentation of UAV RS images using fully convolutional networks (FCN). The work compared the segmentation performance of three FCN architectures, FCN-32s, FCN-16s, and FCN-8s, on the AeroScapes dataset for semantic segmentation of pixels into different categories of ground objects. In the work, the authors used a pre-trained FCN based on the VGG16 neural network model. They evaluated the segmentation performance of the networks using the metrics pixel accuracy (PA), mean pixel accuracy (mPA), and mIoU. It was found that FCN models can be effectively used for semantic segmentation of UAV imagery data. FCN-16s and FCN-8s provide comparatively better segmentation performance. However, the work uses the traditional FCN architecture and does not consider marine oil-spill segmentation.
SRCNet, a seminal representation collaborative network to segment marine oil spills in synthetic aperture radar (SAR) images, was also proposed by ). It is based on a generative-adversarial algorithm in which a generative net is used to generate segmentation maps and a discriminative net is used to refine them. The model has high accuracy and efficiency despite the limited training data by incorporating the internal characteristics of SAR imagery, which is referred to as seminal representation. This approach is applicable in dealing with low contrast and complicated sea clutter in marine surveillance.
A hybrid deep learning approach for oil spill detection in Synthetic Aperture Radar (SAR) images has been suggested by ). The strategy combines feature transfer learning with MobileNet, principal component analysis (PCA), and the binary harris hawk optimization (BHHO) algorithm for optimal feature selection. When these methods are combined, the system achieves good dimensionality reduction of features as well as high accuracy in classifying features. This technique increases the efficiency of detecting oil spills on difficult sea backgrounds, which is an effective real-time monitoring tool for marine environments.
) proposed a multimodal fusion system to detect oil spills using synthetic aperture radar (SAR) and hyperspectral imagery (HSI). To address the absence of matched datasets, the authors designed a new coregistration strategy for the data and a deep learning model known as SHNet. This experimental model applies a controlled random sampling approach to improve information from diverse spectral and spatial data. The integration of the structural information of SAR with spectral images of HSI will enhance the accuracy of detection and minimize positives due to look-alikes.
) introduced a smart way of automatically detecting marine oil spills with the help of dual-polarized synthetic aperture radar (SAR) images and deep convolutional neural networks (DCNN). The model is improved by the addition of polarization information from the VV and VH channels, which better differentiates between oil slicks and look-alikes. The suggested DCNN architecture automates the interpretation process, which is much more efficient than the manual one. The accuracy and recall of the results of the experiments are high and offer a reliable method to conduct quick monitoring of the environment and assessment of the ecological risk in the marine ecosystem.
) examined the performance of the Sentinel-1 C-band SAR data in detecting oil spills in the case of the 2022 Ventanilla-Lima ecological disaster. In the paper, polarimetric decomposition methods, namely, Entropy (H) and Alpha ($\alpha) parameters, are used to study the scattering processes at the surface of the seas. Using the dual-polarization H-H decomposition using H-alpha, the researchers were able to differentiate oil slicks and natural sea surfaces. The results show that polarimetric SAR is a reliable and all-weather monitoring tool that enables quick mitigation of environmental impacts and marine sustainability.
) proposed an adaptive network that detects oil spills based on the Polarimetric SAR (PolSAR) data, which is aimed at overcoming the problem of the look-alike phenomena and non-uniform boundaries. This model uses conditionally parameterized convolutions (CondConv) to dynamically change the weights depending on the complexity of the scene and a boundary constraint module to smooth out edge detection. The method is much more precise in segmentation of complex marine environments than any stationary network, which can differentiate between oil spills and algal blooms as well as wave shadows.
) suggested an automated method of detecting oil spills with the help of hyperspectral imaging (HSI) and artificial neural networks (ANNs) operating on the UAV platform. This research involves hyperspectral water and hydrocarbon (diesel, fuel oil, and C10) signatures to train classifiers without large image datasets that need labeling. Dimensionality reduction is performed using an autoencoder (AE), and classification is performed with ANN models. The system performs well, with recall and F-score above 0.90, indicating an efficient and scalable solution for real-time monitoring of marine oil spills.
) offered an object-oriented fuzzy logic-based classification tool to identify striped marine oil spills in scalawag ship lanes using Sentinel-2 multispectral data. The methodology of the study is to select sensitive spectral bands and then perform multi-scale segmentation of the target oil spill objects. The system can differentiate between oil, ship wakes, and waves by incorporating multi-feature parameters into a fuzzy logic framework. This technology increases the accuracy of monitoring within high shipment lanes, which is an important tool in fast marine disaster prevention and control.
) proposed a self-evolving deep learning oil spill detector on Sentinel-1 SAR imagery to overcome look-alike interference and scanty training data. The three-module framework consists of dual-model detection, based on oil area and boundary features, and automatic generation of training data, based on adaptive thresholding and model improvement through repetitive training. With 21 self-evolving cycles, 27 new training samples were created, and the F1-score increased to 0.8896, whereas the F1-score rose to 0.8423 with no human assistance.
) assessed the performance of deep learning models in detecting oil spills in single-polarized SAR images. To solve the issue of small training data, the authors introduce a new preprocessing pipeline to transform single-channel SAR data into three-channel RGB representations. LinkNet and EfficientNet-B4 were found to be the best architecture-encoder combination using experiments. It has been proposed that the preprocessing method achieves significantly higher accuracy (0.064 points in F1-score improvement) than the classical dB-scale methods, but with a higher look-alike sensitivity in Mediterranean Sea experiments.
) focused on problems in PolSAR oil spill detection, such as look-alike interference and detecting long oil slicks. The authors introduce the concept of composite polarimetric scattering entropy (Hc), which is the sum of polarimetric entropy and Freeman-based entropy to enhance the ability to separate oil spills and look-alikes. The multiscale hybrid features fusion network (MSHFFNet) built on a ResNet34 encoder combines square and strip pooling (long slicks) and feature fusion (smaller-scale detection). Experiments on Radarsat-2 data show better performance than four state-of-the-art approaches.
) suggested a deep learning model called MS3OSD, which is used to detect oil spills based on HY-1C/D satellite data. It addresses the problem of a single sensor by combining CZI, UVI, and COCTS data to form high-resolution UV-VIS-NIR data. A two-branch architecture that separates CNN and ViT actively extracts central spectral and surrounding spatial features of patches. Tests in several regions of the sea and under low sun glint reveal higher detection accuracy: 95.24% (oil slicks), 93.04% (emulsions) in the low sun glint, and 90.06% (positive contrast spills) in the strong sun glint.
) suggested a dual-branch EfficientNetV2-S-based solution that combines the SAR and GNSS-R data to detect marine oil spills. An up-to-date data generation procedure generates SAR-GNSS-R datasets in pairs based on SAR-obtained oil contours and ERA5 wind velocities with the Z-V scattering model. The dual-branch design derives complementary information from both payloads, achieving 94.97% accuracy and 3.6% fewer false alarms than SAR-only models. Cross-domain validation on a variety of diverse scenarios illustrates sound generalization and workable look-alike discrimination.
) suggested a semi-empirical dark spot detection threshold model as a step in SAR image segmentation to identify potential oil spills. The model is a correlation between surface wind vectors and incidence angles and SAR threshold backscatter coefficients, using wind-wave generation and electromagnetic scattering principles. Optimized parameters using 193 Sentinel-1 oil spill patches gave an F1 score of 0.8. The model provides a balance between simplicity (no parameter tuning or iteration) and accuracy, which automatically filters out low-wind false positives.
) introduced DATE, an edge-based deep learning method for detecting and estimating the thickness of oil spills (1–10 mm) using C- and X-band radar mounted on a drone. The data handled by two U-Nets are multifrequency backscattering data and are trained on synthetic scenarios in fluid-dynamic simulations. At low wind speeds at which satellite SAR fails, it achieves detection accuracy greater than 90%. The 75% IoU of thickness estimation is twice that of SotA. At less than 1W with under 3s latency, FPGA implementation allows onboard tactical response.
) suggested a two-step deep learning technique for oil spill localization and segmentation on Sentinel-1 SAR images. Faster R-CNN detects the areas of oil spills, then pixel-wise segmentation of extracted patches is done with UNet++. The method produces 87.46 and 87.59 localization and 89.55 and 89.37 segmentation accuracy using 3,166 annotated samples out of 1,000 images. The run time is less than 1 min per image, with an optimum run time of 2–10 m/s winds.
) thoroughly assessed features that were obtained by using dual-polarized Sentinel-1 SAR images in detecting oil spills using DeepLabv3+. The features of intensity, polarization, texture, and wind speed were studied separately and in combinations. The best combination of polarization (anisotropy, entropy), texture (variance, entropy), and intensity (C11) achieved 90.43% mIoU. An artificial attribute through weighted bilinear pooling enhanced efficiency at 88.21 accuracy. The higher performance of 90.89% mIoU of a Squeeze-and-Excitation module was also added to include the wind speed. Effectiveness as a feature that is generalized across models and data sets.
) suggested a prioritization-based data gathering method that uses several mobile sinks for clustered underwater wireless sensor networks (UWSNs) to detect leakage in underwater oil pipelines. The approach prioritizes emergency data regarding leakages by allowing the nearest mobile sink to gather critical data instantly, which helps reduce transmission time, communication cost, and energy utilization. The suggested route selection method increases the effectiveness of underwater leakage monitoring due to optimal mobile sink scheduling and clustered communications. Even though the approach helps improve sensing and data transmission in an underwater environment, it is related to sensor network communications but not to the visual identification of oil spills. Unlike the mentioned solution, HyEff-UNet is designed for oil spill segmentation based on UAV RGB images.
The hybrid Gray Wolf Optimizer–Whale Optimization Algorithm (GWOWOA) technique was proposed by ) for efficient data collection in mobility-assisted Underwater Wireless Sensor Networks for effective oceanography. This optimization algorithm uses transmission distance, remaining energy, buffer occupation, and communication latency to find the optimum path for mobile sinks to detect oil leakage from the oil extraction process on the seafloor quickly. Experiments have shown an increase in the efficiency of remaining energy, accuracy of data collection, and transmission delay when comparing this optimization algorithm with other algorithms. Nevertheless, this study is limited to the optimization of underwater sensor networks and data collection but does not focus on oil spill identification using images.
A deep learning-based framework for marine oil spill detection using UAV imagery was proposed by ) based on the YOLOv12 architecture. The framework incorporates attention-guided feature enhancement, multi-scale feature fusion, pretrained weight initialization, and a cosine annealing learning-rate schedule to improve oil-spill detection under complex marine conditions. The model achieved an F1-score of 0.7929, mAP@0.5 of 0.8334, and mAP@0.5–0.95 of 0.7503 on the test set. However, the study formulates oil-spill identification as an object-detection task, using bounding-box annotations to localize oil-spill regions rather than pixel-level segmentation masks. Conversely, the proposed HyEff-UNet performs semantic segmentation to provide pixel-wise delineation of oil-spill regions, enabling more precise characterization of irregular spill boundaries.
According to ), Anchor-Based Localization using Lion Optimization Algorithm (ALLOA) is an innovative method designed for monitoring oil pipelines in underwater wireless sensor networks. This technique uses time of arrival (ToA), received signal strength (RSS), and angle of arrival (AoA) together with the Lion Optimization Algorithm to perform efficient localization of sensor nodes monitoring oil pipeline leaks. Simulation findings showed that this algorithm led to decreases in localization error by 28%, energy consumption by 7%, and localization delay by 27% when compared with other localization methods, thus increasing the effectiveness of underwater monitoring systems. However, the problem being addressed in this research is the issue of localization and communications, not visualization of oil spills. Conversely, in this paper, HyEff-UNet uses semantic segmentation of UAV RGB images for accurate identification and demarcation of oil spill areas on the water surface.
) proposed PetroAnalyticNet as a task-specific semantic segmentation approach for detecting oil spills in optical marine imagery captured by UAVs. The network utilizes depthwise separable convolution, dilated convolution, residual connections, and spatial-channel attention to capture the boundaries of oil spills while at the same time preserving fine-grained details, considering the continuity of oil slicks. The performance of the proposed architecture was tested on the Oil Spill Drone and LADOS datasets based on the same experimental setup. The result was an F1-score of 0.91 and IoU of 0.84 on the Oil Spill Drone dataset.
However, recent methods for performing semantic segmentation, such as DeepLabV3+, UNet++, SegFormer, HRNet, TransUNet, Mask2Former, and segmentation methods based on the SAM architecture, are quite successful in solving the problems of general computer vision. The DeepLabV3+ method employs an atrous spatial pyramid pooling architecture for capturing multiscale context information, while the UNet++ method uses dense and nested skip connections for more efficient feature aggregation. The transformer-based models, SegFormer and Mask2Former, use self-attention mechanisms to model long-range contextual dependencies.
The suggested HyEff-UNet leverages multi-scale feature extraction via an EfficientNet-B4 encoder alongside attention-skip connections and a custom loss function of CE+Dice for UAV RGB oil spill segmentation. Table 1 contrasts HyEff-UNet with other paradigms based on architecture alone; it should be noted that it is not an empirical benchmark since none of the architectures listed above have been trained on the current dataset. It must be recognized as a limitation of the present research that a comparative analysis of these architectures is essential to validate the state-of-the-art claims of HyEff-UNet and thus becomes a goal for future research.
Table 1
| Model | Encoder | Multi-scale context module | Attention | Decoder | Loss | UAV RGB | Oil spill specific | Hybrid optimization |
|---|---|---|---|---|---|---|---|---|
| U-Net | CNN | × | × | U-Net | CE | ✓ | × | × |
| Attention U-Net | CNN | × | ✓ | U-Net | CE | ✓ | × | × |
| EfficientUNet | EfficientNet | × | × | U-Net | CE | ✓ | × | × |
| UNet++ | CNN | × | × | Nested Decoder | CE | ✓ | × | × |
| DeepLabV3+ | Xception | ✓ (ASPP) | × | Decoder | CE | ✓ | × | × |
| HyEff-UNet (Proposed) | EfficientNet-B4 | ✓ (Encoder hierarchy) | ✓ | Attention Decoder | CE+Dice | ✓ | ✓ | ✓ |
Architectural comparison of the proposed model against representative segmentation paradigms.
The architecture details of the proposed HyEff-UNet are provided in Table 2 along with some state-of-the-art deep learning approaches that have been designed recently for marine oil spill detection. In general, most of the previously discussed methods have mainly focused on the use of SAR images and have used various backbone architectures, attention- or context-based feature extraction blocks, and dedicated loss functions to boost their segmentation efficiency. However, the HyEff-UNet approach is designed explicitly for the semantic segmentation of UAV-acquired RGB images.
Table 2
| Model | Backbone | Attention/context module | Loss | UAV RGB | SAR | Multi-scale mechanism | Hybrid loss | Oil-specific |
|---|---|---|---|---|---|---|---|---|
| LRA-UNet | Lightweight CNN | SimAM-based residual attention | Sobel edge-aware joint loss | No | Yes | Not reported | No | Yes |
| FA-MobileUNet | MobileNetV3 | CBAM | Not reported | No | Yes | ASPP | No | Yes |
| CBENet | CNN | Contextual fusion block | Boundary-enhanced loss | No | Yes | Not reported | No | Yes |
| HyEff-UNet (Proposed) | EfficientNet-B4 | Attention Gates | Categorical Cross-Entropy+Dice Loss | Yes | No | Encoder–decoder skip hierarchy with multi-scale feature extraction | Yes | Yes |
Architectural comparison between recent state-of-the-art marine oil spill segmentation methods and the proposed HyEff-UNet.
Bold values indicate the proposed architecture HyEff-UNet.
Unlike existing U-Net architectures based on EfficientNets, where only the encoder is changed but not the traditional skip connection strategy, the proposed architecture combines an EfficientNet-B4 encoder, an attention-guided skip connection strategy, and a hybrid optimization technique based on Dice loss and Cross-Entropy loss into a unique semantic segmentation framework tailored to UAV RGB oil spill images. Furthermore, the proposed framework uses a simple three-class formulation by combining background and other object classes into one class called “other”. The entire framework, including preprocessing, transfer learning, hybrid optimization, and inference, is specifically designed for UAV-based oil spill segmentation in a marine environment.
4 Proposed methodology
The suggested methodology is designed to identify and delineate oil-spill regions on RGB images of ports and marine settings collected by the UAVs. The overall workflow of the proposed oil-spill detection framework is illustrated in Figure 1. The proposed HyEff-UNet consists of an Attention U-Net architecture with an EfficientNet-B4 encoder for multi-class pixel-level segmentation. The dataset is trained using preprocessing steps including image resizing, mask labeling, and one-hot encoding. The model learns hierarchical spatial features and allocates attention to pertinent areas of oil. Lastly, the trained network is used to predict segmentation maps, which are used to categorize pixels as either Oil, Water, or another surface. The detailed HyEff-UNet architecture, including the EfficientNet-B4 encoder, attention gates, and decoder, is shown in Figure 2.
Figure 1
Figure 2
4.1 Dataset preparation
In the suggested oil spill detection system, the raw inputs are the aerial RGB images that are obtained with the help of unmanned aerial vehicles (UAVs). For each image, a pixel-level ground truth mask is given, which is an indicator of the true class of every pixel. None of the images was relocated or randomly allocated during the pre-processing step. Keeping the original partition makes the work reproducible and allows for comparing results with other works using the same benchmarking protocol. The lack of metadata at the flight level and sequence level tells us that independence of the partitions with respect to UAV flight or scene cannot be verified. Thus, the partition defined beforehand should be considered as the benchmarking protocol.
Let the complete dataset be denoted by Equation 1 as follows:
Where → RGB image, → corresponding segmentation mask, N → total number of samples. Each pair (Ii, Mi) represents one labeled training example. This formulation guarantees that every image has exactly one mask, supervised learning is possible, and pixel-to-pixel correspondence is preserved. Each mask pixel belongs to one of three classes as defined in Equation 2:
where c0 → Oil, c1 → Water, c2 → Other. Thus, the mask-labeling function is defined in Equation 3 as follows:
meaning each spatial coordinate (x, y) contains a valid class label. Before training, the dataset is checked for missing files, corrupted images, mismatched image–mask pairs, and zero-size files. The experiments were performed utilizing the standard train, validation, and test partitions given in the Zenodo version of the Annotated RGB Images of Oil Spills in a Port Environment dataset (doi: 10.5281/zenodo.10555313). This predefined partition contains 811 training images, 203 validation images, and 254 test images, which were not altered at all during this research work. During preprocessing, background pixels were merged with the “Other” category, resulting in the final three-class segmentation problem (Oil, Water, and Other) used consistently for model training and evaluation. The publicly available dataset does not provide flight identifiers or acquisition sequence information; therefore, grouped splitting could not be performed.
Table 3 provides the number of occurrences of each semantic class per image in three different partitions of the dataset. An image is said to have a certain class if there is at least one pixel from the class in the segmentation mask of the image. Class occurrences are not exclusive, meaning an image can have classes like Oil, Water, and Other together. “Other” class exists in all 1,268 images, whereas “Oil” and “Water” exist in 998 and 939 images, respectively. The ratios of occurrences of images of each class are similar for training, validation, and test partitions.
Table 3
| Dataset split | Total images | Images containing oil | Images containing water | Images containing other |
|---|---|---|---|---|
| Training | 811 | 633 (78.05%) | 594 (73.24%) | 811 (100%) |
| Validation | 203 | 163 (80.30%) | 150 (73.89%) | 203 (100%) |
| Test | 254 | 202 (79.53%) | 195 (76.77%) | 254 (100%) |
| Total | 1,268 | 998 | 939 | 1,268 |
Image-level class occurrence across dataset splits.
Table 4 shows the distribution of the pixels for the three semantic classes in each dataset split. The calculations were done based on the total number of annotated pixels in each split. Other has the highest percentage of pixels, taking up 43.10, 40.76, and 41.05% of the three splits, respectively. Water has the lowest percentage of pixels in all three splits, taking up 25.73, 22.24, and 27.19% of the pixels, respectively, while Oil takes up 31.17, 37.00, and 31.77%. The similarities in the distribution of the classes in the three splits suggest that the pre-defined splits have similar class distributions at the pixel level. These figures indicate that there is moderate class imbalance and justify the use of class-based optimization and macro average metrics.
Table 4
| Dataset split | Oil pixels | Water pixels | Other pixels* | Oil (%) | Water (%) | Other (%) |
|---|---|---|---|---|---|---|
| Training | 597,784,812 | 493,327,982 | 826,454,182 | 31.17 | 25.73 | 43.10 |
| Validation | 166,949,058 | 100,361,967 | 183,904,335 | 37.00 | 22.24 | 40.76 |
| Test | 184,036,106 | 157,500,331 | 237,785,931 | 31.77 | 27.19 | 41.05 |
| Total | 948,769,976 | 751,190,280 | 1,248,144,448 | 32.18 | 25.48 | 42.34 |
Pixel-level class distribution across dataset splits.
The * in Other pixels denotes other and background pixels, as background pixels were merged with the “Other” category.
The imbalance in the number of pixels in each of the classes becomes important in relation to the optimization of the segmentation algorithm due to the fact that the contribution of each class toward the training objective is dependent on its frequency in pixels. In this case, the smaller frequency of the Water class in comparison to the Other class will lead to the preference of the more frequent classes by a frequency-driven objective function. The hybrid loss used in this work combines Cross-Entropy and Dice loss during model optimization. While the complete architecture achieved the best performance, the present sequential ablation does not isolate the independent contribution of the hybrid loss. Since there is an imbalance in the number of pixels for the three classes, evaluating only by performance results will not give a proper view of what each individual class has achieved. For this reason, class-wise evaluations were conducted and averaged using the macro-average, treating all three classes equally regardless of the number of pixels for each.
4.2 Data preprocessing
Raw UAV pictures obtained in marine and port places tend to differ in clarity, lighting, color structure, and complexity of the background. Several preprocessing operations are applied to the images to transform the raw data into a formatted form that is easily learned by the model before feeding into the EfficientNet-B4 and Attention U-Net architecture. For the EfficientNet-B4-based configurations, RGB images were retained in their original [0,255] range, as the Keras EfficientNet-B4 implementation incorporates input rescaling internally. Therefore, no additional external normalization was applied to the EfficientNet-B4 inputs.
4.2.1 Image and mask resizing
UAV-acquired images may have different spatial resolutions because of variations in altitude, camera angle, or acquisition parameters. Deep neural networks are sensitive to a fixed scale of inputs to process a batch and apply the same convolution operations in U-Net-based architectures. Therefore, the classification of all images and masks resizes them to the same size of 512 x 512.
Let the original image be defined in Equation 4 as follows:
After resizing, the image dimensions are defined in Equation 5 as follows:
Where
For masks, nearest-neighbor interpolation is used to preserve discrete class labels as defined in Equation 6:
4.2.2 Mask color mapping
Ground-truth masks are in the form of RGB color images. Nonetheless, deep learning models do not accept color labels of classes; instead, they need numeric class labels. Thus, the discrete classes are assigned an index to which each discrete color is projected.
For model training and evaluation, the background pixels and other-object pixels are merged into a single ‘Other' class, consistent with the dataset's own three-category annotation scheme, yielding the final 3-class segmentation problem (Oil, Water, Other) used throughout this study.
Class Definition C = {0:Oil, 1:Water, 2:Other}
Mathematical Mapping in Equation 7 as follows:
where L is the class index mask.
4.2.3 One-hot encoding of labels
In the implementation used in this study, categorical cross-entropy and Dice loss require one-hot encoded target masks. Therefore, each pixel label is converted into a one-hot encoded vector before training. For each pixel, as defined in Equation 8 as follows:
Where Y∈ℝ512 × 512 × 3.
4.2.4 Mini-batch data pipeline
The processed samples are input into a tf.data pipeline after transformation, which executes shuffling, batching, and prefetching. This helps with faster I/O, improved use of the GPUs, and reduced training latency. This can be expressed in Equation 9 as:
where D′ is the preprocessed dataset.
The proposed algorithm provides a pipeline outlining the process of automatic detection and segmentation of oil spills and identifying them on the basis of UAV-captured RGB images. The complete oil-spill detection workflow is summarized in Algorithm 1. It starts with data preparation and preprocessing. Images of raw drone footage and their segmentation masks are loaded initially based on the dataset. The ground-truth masks are turned into numerical labels for a class of Oil, Water, and Other. The labels are then converted into one-hot encoded vectors in order to facilitate multi-class segmentation. The processed samples are piped into a TensorFlow data pipeline that is implemented to shuffle, batch, and prefetch the samples to enable efficient training on the GPU.
Algorithm 1

A segmentation model is trained after preprocessing based on an Attention U-Net architecture with an EfficientNet-B4 encoder with ImageNet pretrained weights. The encoder extracts hierarchical multi-scale features from the input images, while the decoder progressively upsamples these features and combines them with attention-guided skip connections intended to emphasize oil-spill regions while reducing irrelevant background features. The output is a Softmax layer that generates pixel-wise class probabilities. The model parameters are then optimized in the course of training by the Adam optimizer with a learning rate of 1 × 10−4. The model is optimized using a hybrid loss based on categorical cross-entropy and Dice loss. The experiments evaluate this loss only as part of the complete architecture rather than as an independently varied component. The model is trained through 20 epochs of mini-batches, and the model that performs best in terms of validation is saved. Lastly, in the course of testing, the trained model is used to generate segmentation maps of unknown images. The forecasted probability maps are then converted to discrete labels with the help of argmax. The system performance is measured using common segmentation measures such as Precision, Recall, F1-score, and IoU.
Figure 3 presents an intensive Dataset Preparation and Preprocessing process that aims to convert raw drone images into a format that can be fed to deep learning. A Dataset Preparation stage is used, consisting of splitting a UAV RGB image dataset and ground truth mask annotations into training, validation, and testing sets and applying a standardized image size of 512 x 512 pixels. As outlined in Algorithm 2, masks are color-mapped to transform RGB values to certain class indices: Oil (0), Water (1), and Other (2). One-hot encoding of these indices is then performed to convert them into a (512, 512, 3) tensor. Lastly, a tf.data pipeline is used to perform high-performance data processing with a batch size of 2, random shuffling, and preloading to the background to maximize the use of the GPU resources during training.
Figure 3
Algorithm 2

The given oil spill detection framework in Algorithm 3 is based on a complex Attention U-Net architecture that incorporates the use of a pretrained EfficientNet-B4 encoder to ensure a high level of accuracy in semantic segmentation. It starts with Dataset Preparation and a powerful Data Preprocessing pipeline that downsizes images to 512 x 512 and converts masks into one-hot encoded tensors. These mini-batches have undergone preprocessing and are introduced to the EfficientNet-B4 encoder that employs ImageNet transfer learning to obtain hierarchical and multi-scale features in four stages. Attention U-Net decoder is the next step, in which these features are denoted with the help of transposed convolutions because spatial attention gates can enhance the skip connections to pay attention to the areas of oil spills of interest and ignore irrelevant noise. As shown in Figure 4, an end-to-end classification is acquired by finalizing with a 1 x 1 convolution and Softmax activation to yield a probability map of the “Oil, Water, and Other” categories. The objective function to optimize the model will be a hybrid one that consists of Categorical Cross-Entropy and Dice Loss to guarantee the correct pixel-wise classification and overlap. The overall model training and optimization strategy is illustrated in Figure 5.
Algorithm 3

Figure 4
Figure 5
4.3 feature extraction and segmentation network
The proposed system uses a Modified U-Net with an EfficientNet-B4 encoder and an attention-guided decoder. Multi-scale hierarchical features are extracted by the encoder with the help of transfer learning and gradually upsampled by the decoder before being combined with spatial information via skip connections and attention gates. Lastly, a segmentation head produces Softmax probability maps of three classes.
4.3.1 Convolution-based feature extraction
Convolution operations allow the network to learn local spatial dependencies with trainable kernels. For an input feature map X and convolution kernal W, the convolution operation is defined in Equation 10 as follows:
where Fi, j, k → output feature, W → learnable kernel, bk → bias and k → output channel.
This operation enables the network to learn local spatial patterns, including edges and textures.
4.3.2 . Hierarchical encoder representation (EfficientNet-B4)
Single-scale features do not provide enough information to perform the segmentation task. The patterns of oil spills take different shapes and sizes; hence, multi-scale contextual features are needed. EfficientNet-B4 offers hierarchical downsampling, which enhances the size of the receptive field but decreases spatial quality.
EfficientNet-B4 was selected as the backbone encoder because it provides a practical compromise between representational capacity and computational cost. Compared with smaller versions like EfficientNet-B0 and EfficientNet-B2, the larger version, EfficientNet-B4, offers more capacity of the network and can perform richer multiscale feature extraction, which is useful for recognizing oil spill regions with various shapes and complicated water texture backgrounds. While bigger versions like EfficientNet-B5 would further improve the capacity of the model, their computational costs are much higher than EfficientNet-B4. Thus, EfficientNet-B4 was selected as the backbone because it provides a practical balance between representational capacity and computational cost for this study. While it is beyond the scope of this paper to conduct an experimental evaluation of several types of EfficientNet variants, future research will consider encoder selection under the same training conditions to evaluate the balance between segmentation performance and computational overhead. In such a way, EfficientNet is an effective feature extractor backbone. At each stage, as defined in Equation 11:
where Hl is the feature map at level l.
The downsampling operation is performed as given in Equation 12:
4.3.3 EfficientNet-B4 feature extraction
The extracted feature map at each encoder stage can be represented by Equation 13 as follows:
Where F0 = X denotes the input image, El(·) represents the lth EfficientNet-B4 encoder stage, and Fldenotes the extracted hierarchical feature map. Each EfficientNet-B4 stage consists of Mobile Inverted Bottleneck Convolution (MBConv) blocks with squeeze-and-excitation attention, which can be mathematically expressed as given in Equation 14:
Where EX(·) denotes feature expansion, DW(·) represents depthwise convolution, SE(·) denotes squeeze-and-excitation feature recalibration, and P(·) represents the projection layer. This hierarchical feature extraction progressively increases the receptive field while preserving discriminative semantic information at multiple spatial scales. In the MBConv block, the input channels are expanded, and depthwise convolution is applied, followed by recalculation of channel responses via squeeze-and-excitation, and the features are finally projected to the required dimension.
Table 5 details the channel dimensions at each stage of the encoder–decoder pathway. The EfficientNet-B4 encoder outputs skip-connection feature maps with 144, 192, 336, and 672 channels in spatial resolutions of 256 × 256, 128 × 128, 64 × 64, and 32 × 32, respectively, while a bottleneck layer with 1,792 channels operates at 16 × 16 spatial resolution. The decoder processes skip connections in reversed order with respect to depth–that is, the deepest skip connection is processed first and therefore, the first decoding block combines the 672 channels, 32 × 32 skip connection, up-samples it with a 3 × 3 transposed convolution with stride 2, to create an output with a filter of 512 channels; the next blocks then combine the 336-, 192-, and 144-channel skips into filters with 256, 128, and 64 channels respectively. In each attention block, the internal channel width matches the filter size in its respective decoding stage, and therefore, the skip connection as well as the gating tensor get projected to this width through 1 × 1 convolution before the multiplicative sigmoid gate (512 → 256 → 128 → 64, respectively across the four stages). After the four decoder stages, a final transposed convolution upsamples to 512 × 512 with 32 channels, followed by a 3 × 3 convolution retaining 32 channels, before the 1 × 1 softmax head reduces to the 3 output classes.
Table 5
| Stage | Spatial resolution | Encoder skip channels | Decoder filter width | Attention gate channels |
|---|---|---|---|---|
| Input | 512 × 512 | 3 | — | — |
| S1—block2a_expand_activation | 256 × 256 | 144 | 64 | 64 |
| S2—block3a_expand_activation | 128 × 128 | 192 | 128 | 128 |
| S3—block4a_expand_activation | 64 × 64 | 336 | 256 | 256 |
| S4—block5a_expand_activation | 32 × 32 | 672 | 512 | 512 |
| Bottleneck | 16 × 16 | 1,792 | — | — |
| Final upsample | 512 × 512 | — | 32 | — |
| Output head | 512 × 512 | — | 3 | — |
Channel dimensions of the HyEff-UNet encoder–decoder and attention-gate pathway.
4.3.4 Attention-guided skip connections
Standard U-Net simply concatenates features of encoders and decoders. But not all encoder information is helpful. The extraneous background information can be disastrous to the quality of segmentation. Attention gates are biased toward accentuating significant spatial areas and do not emphasize unwanted characteristics. In the attention mechanism, let: x → encoder feature, g → decoder gating signal. The attention gate is used to filter out unnecessary features from the encoder during feature fusion. Rather than passing all the encoder features using skip connections, the gate is intended to emphasize spatial regions relevant to oil-spill segmentation while reducing the contribution of potentially irrelevant background features, including water reflections and port structures. The attention coefficient is computed as given in Equation 15:
Where Wx projects encoder features, Wg projects decoder gating features, ψ is a learnable linear transformation, ReLU introduces non-linearity, and σ denotes the sigmoid activation.
The refined feature becomes as given in Equation 16:
The attention coefficient is intended to suppress irrelevant background activations while emphasizing informative oil-spill regions before feature fusion. The attention coefficient α∈[0, 1]acts as a spatial weighting mask. Regions with larger attention values are emphasized, whereas irrelevant background activations are suppressed before feature fusion.
4.3.5 Decoder upsampling and feature reconstruction
The encoder is space-compressing; hence, the decoder requires the original resolution to get pixel-wise prediction. Learned upsampling is done with transposed convolutions (ConvT). In the process of decoder refinement, the upsampled features from the decoder are combined with the refined features from the encoder at the same spatial resolution level. Such a feature integration ensures that there is preservation of boundary details while retaining the high-level semantics learned by the encoder. The decoder aims to recover spatial details while effectively integrating global contextual information with local features. Thereby, the decoder reassembles segmentation maps bit by bit. Transposed Convolution is given in Equation 17 as:
Where F → low-resolution feature, U → upsampled feature
Then skip features are fused as given in Equations 18 and 19:
Where Up(·) denotes bilinear upsampling, Concat(·) is channel-wise concatenation, Conv(·) represents two successive convolution operations for feature refinement. This equation explains the complete decoder fusion process rather than only the concatenation. The concatenation process combines high-level semantic information from the decoder with fine spatial information from the encoder to improve feature representation through the convolution layers. The decoder progressively upsamples feature maps through five transpose convolution layers, producing feature dimensions of 32 × 32 × 512, 64 × 64 × 256, 128 × 128 × 128, 256 × 256 × 64, and 512 × 512 × 32. At each decoding stage, aligned encoder features are combined with decoder features through skip connections followed by attention-guided feature fusion.
4.3.6 Softmax pixel-wise classification
The final output must assign each pixel to one of three classes. Softmax converts raw logits into probability distributions. SoftMax function for class c is defined in Equation 20 as:
Where zc → logit score, C = 3
Prediction is given in Equation 21 as:
Each pixel gets oil probability, water probability, and other probability. The highest probability determines the final class. The resulting probability map is subsequently optimized using the hybrid Dice and Cross-Entropy loss function described in the training stage.
4.4 Optimization and training strategy
As the classification is conducted pixel by pixel, pixel-wise classification is associated with class imbalance, and thus a hybrid loss function is used. The network parameters are optimized with the help of the Adam optimizer, which dynamically adapts the learning rate to reach convergence faster and more stably. Moreover, mini-batch gradient descent and backpropagation are applied to update weights in a repetitive way.
The EfficientNet-B4 encoder network used pre-trained weights on the ImageNet dataset, but its layers were not frozen during training. In other words, all encoder network layers were trainable, and it could be fine-tuned along with the randomly initialized decoder and attention gate components in an end-to-end fashion. There was no staged freezing of any layers or differential learning rates between the two network parts; both were trained with the same Adam learning rate of 1 × 10−4. In line with Table 6, the training involved a constant learning rate schedule without any regularization other than the internal batch normalization of the pretrained backbone. The training was accomplished using the strategy of transfer learning (ImageNet-pretrained encoder) along with the hybrid CE-Dice loss throughout the whole 20 epochs. It is to be noted that the lack of a learning rate schedule, dropout, and data augmentation is a simple way of training compared to some of the recently developed segmentation algorithms. The model-training and optimization procedure is summarized in Algorithm 4.
Table 6
| Component | Setting |
|---|---|
| Optimizer | Adam (Keras default β1 = 0.9, β2 = 0.999, ϵ = 1 × 10− 7) |
| Learning rate | Fixed at 1 × 10−4; no learning-rate scheduling or decay was applied |
| Weight decay | None applied |
| Batch size | 2 |
| Input resolution | 512 × 512 × 3 |
| Loss function | Hybrid loss: 0.5 × Categorical Cross-Entropy + 0.5 × Dice Loss |
| Dropout | None. Images and masks resized to 512 × 512; mask color-to-class conversion and one-hot encoding applied. RGB scaling was architecture-dependent: external [0,1] scaling for the standard U-Net baseline; EfficientNet-B4 configurations received [0,255] inputs and used the encoder's built-in rescaling. |
| Batch normalization | Inherited from the pretrained EfficientNet-B4 backbone (ImageNet weights); no additional BN layers introduced in the decoder |
| Data augmentation | None. Preprocessing was limited to resizing to 512 × 512 |
| Total epochs | 20, trained in resumable chunks of 5 epochs per session |
| Early stopping | Not used; training ran the full fixed epoch budget |
| Checkpointing | A custom callback saved the latest model state after every epoch (for resumability) and separately tracked/saved the best model by lowest validation loss |
| Random seed | 42 (fixed for NumPy and TensorFlow) |
| Encoder initialization | ImageNet-pretrained weights (EfficientNet-B4); decoder and attention gate weights randomly initialized (Keras default Glorot uniform) |
| Freezing/fine-tuning policy | Full fine-tuning; all encoder layers remain trainable |
Training configuration and implementation details.
Algorithm 4

4.4.1 Forward propagation
In forward propagation, the network layers work out predictions by subjecting input images to the network. The aim is to convert the input image X into a probability map Ŷ. This step extracts features, reconstructs segmentation, and classifies outputs as class probabilities. This is compared with the ground truth when the loss is calculated. For a network with parameters θ, the Equation 22 is as follows:
Where X → input image, θ → network weights and biases, f(·) → encoder–decoder mapping and Ŷ → predicted segmentation probabilities.
4.4.2 Combined hybrid loss function
Pixel-wise classification involves the use of a loss term that penalizes an incorrect probability distribution. Categorical Cross-Entropy (CCE) is a metric that is used to quantify the difference between the probability distributions of the predicted results and the actual labels. For C classes and N pixels, the Categorical Cross-Entropy is given in Equation 23 as follows:
Where yi, c → ground-truth label and ŷi, c → predicted probability.
The regions of oil spills tend to have fewer pixels than the water and the background. Therefore, more effective overlap-based measures like the Dice coefficient are more accurate in determining the quality of segmentation. The Dice Loss is given in Equation 24 as follows:
where y → ground truth mask, ŷ → prediction and ϵ → small constant.
A single loss cannot offer the best segmentation. Thus, the project will combine losses to leverage their advantages. This hybrid loss balances pixel-wise accuracy (CCE), enhances overlap between regions (Dice) and stabilizes convergence. as defined in Equation 25:
4.4.3 Backpropagation and gradient computation
These gradients show the effect of each parameter on the loss. Backpropagation allows propagation of error signals through the output layers to previous layers, thus allowing end-to-end learning. After computing loss, gradients of parameters are calculated using the chain rule as given in Equation 26:
4.4.4 Adam optimizer for weight update
Normal gradient descent involves constant learning rates, and it might slow down. Adam (Adaptive Moment Estimation) is an algorithm that calculates rates of adaptive learning, based on first and second moment estimates Here, learning rate is η = 10−4. The checkpoint of the model was chosen during training based on the lowest validation loss. The selected model would be used in testing. This is because using such an approach reduces the risk of overfitting, as it helps choose the checkpoint with the lowest validation loss and the best observed validation performance.
First moment is given in Equation 27 as follows:
Second moment is given in Equation 28 as follows:
Bias corrections are given in Equations 29 and 30 as follows:
Parameter update is given in Equation 31 as follows:
Where gt = ∇θLHybrid, η is the learning rate, θ denotes the trainable parameters of HyEff-UNet.
4.5 Inference and performance evaluation
Although the optimization component ensures that the network constructs significant parameters, the inference component determines whether the acquired representations apply well in reality when dealing with oil spills. The inference and performance evaluation workflow is summarized in Figure 6.
Figure 6
4.5.1 Forward inference and prediction
The network is applied to each input image to achieve the probability of the target class, which is a class in all pixels. The probability values of each pixel in each of the classes are now three values of the probability of the pixel. Let X∈ℝH×W×3 denote the RGB image and θ* be the trained weights. The output probability map is defined in Equation 32 as follows:
4.5.2 Hard segmentation using argmax
The results of Softmax are continuous probabilities. However, segmentation masks must have discrete class labels. Therefore, the class having the highest probability is chosen. This operation converts probabilities to a categorical mask and produces the final segmentation output.
Thus, the network outputs a binary or multi-class segmentation map as given in Equation 33:
Where Pi, c → probability of pixel i belonging to class c, ŷi → predicted label.
4.5.3 Performance evaluation
Unless otherwise mentioned, all macro averages are considered to be calculated using Precision, Recall, F1-Score, and IoU for the three classes (Oil, Water, and Other). Class-specific metrics for each class are also mentioned individually. The macro average gives equal weight to each class rather than weighting classes according to their pixel frequency. The inference and performance analysis module justifies the efficacy of the suggested oil spill segmentation framework. The system evaluates detection reliability, completeness, balance, and spatial overlap by turning the Softmax outputs into discrete masks and analyzing them using Precision, Recall, F1-score, and IoU. Therefore, the evaluation demonstrates the segmentation effectiveness of the proposed framework for UAV-based oil spill imagery. Its suitability for real-time environmental monitoring cannot be established solely from segmentation metrics and requires additional deployment-specific benchmarking. Performance was measured by Precision, Recall, F1-Score, and IoU. All measurements in the paper, if not specified otherwise, are performed on the macro-average level, meaning that each class (Oil, Water, and Other) is considered separately and its performance is averaged with the other two classes equally. The individual class-based metrics were provided to show how well the network performs segmentation for each semantic category. The macro averaging procedure was chosen to avoid the bias caused by class imbalance. The testing and performance evaluation is summarized in Algorithm 5.
Algorithm 5

4.5.4 Practical deployment considerations
Though the suggested HyEff-UNet model is expected to have high segmentation capability, there are some practical issues to consider before deploying the algorithm into operation on UAVs. The altitude of the flight is a determining factor that affects the resolution of the captured images and, thus, segmentation accuracy. Besides, environmental factors like waves at sea, wind, reflections of sunbeams, cloud shadows, and GPS position accuracy can affect the predictions. Furthermore, it is essential to calibrate cameras and stabilize the images to achieve good results during UAV flight. The actual implementation of the HyEff-UNet architecture is contingent upon the computational power of the hardware employed, its memory resources, energy requirements, power source capacity, communication bandwidth, and camera parameters. The current research uses an NVIDIA Tesla T4 GPU in Google Colab to evaluate the performance of the model rather than UAV-based hardware. Thus, the described speed of the inference process cannot be viewed as the real-time speed in UAVs.
5 Experimental results
5.1 Experimental setup
The oil spill detection framework has been designed and validated using the cloud-based computing environment Google Colab (Google LLC, Mountain View, CA, USA), which is integrated with GPU acceleration for effective training and validation of the deep learning segmentation model. The development environment used included the Intel Core™ (Intel Corporation, Santa Clara, CA, USA) i5-1155G7 processor running at 2.50 GHz with 8 GB RAM and a 64-bit Windows operating system. Dataset handling, model implementation, training, evaluation, and visualization of the experimental results were all accomplished through the Google Colab development environment. The Google Colab runtime includes the Linux operating system with Python version 3.12.12 (Python Software Foundation, Wilmington, DE, USA), around 12.67 GB of system RAM, TensorFlow version 2.19.0 (NVIDIA Corporation, Santa Clara, CA, USA), and the Keras deep learning library. The training and inference of the deep learning model were done with the help of an NVIDIA Tesla T4 GPU with 15 GB of GPU memory and CUDA Version 13.0 (NVIDIA Corporation, Santa Clara, CA, USA). All images were resized to 512 × 512 pixels prior to being fed into the network. The proposed Attention U-Net architecture with an EfficientNet-B4 encoder was trained in batches of 2 for 20 epochs using the Adam optimizer at an initial learning rate of 1 × 10−4. The encoder layers of the network were pre-trained on ImageNet, whereas all decoder parameters were randomly initialized and fine-tuned during the training process. Model fine-tuning was performed using the hybrid loss function with 0.5 Cross-Entropy Loss and 0.5 Dice Loss. To provide reproducibility in the experiments, a constant random seed of 42 was used during data preparation, model initialization, training, and evaluation processes.
The dataset to be utilized in this study is the Annotated RGB Images of Oil Spills in a Port Environment, which is publicly accessible on the open data repository, Zenodo. The photographs were taken using unmanned aerial vehicles (UAVs) of the port and coastal regions where oil pollution can take place. There are 1,268 high-resolution (RGB) images along with pixel-level segmentation masks in the dataset. The masks are known to provide accurate annotations for semantic segmentation assignments, dividing all pixels into three categories, namely, oil, water, and other surface elements. To facilitate machine learning experiments, the dataset is organized into training, validation, and testing subsets as per the predefined dataset split. This structured partitioning enables proper model training, hyperparameter tuning, and unbiased evaluation. Additionally, the annotation masks are provided in CamVid format, which is commonly used for semantic segmentation tasks. The dataset also supports other widely used formats such as COCO and ImageNet-style annotations, making it compatible with multiple deep learning frameworks.
The segmentation mask represents each of the classes by a defined color code in RGB that can be used to differentiate between various regions of surfaces. The RGB value (0, 0, 0) is the background class. The RGB value (255, 0, 124) is used to indicate the oil spill areas on the water surface. The other objects category, which can comprise ships, port structures, or non-water surfaces, is given the RGB value (255, 204, 51). Lastly, the surface of the water is colored with the RGB color (51, 221, 255). The data is around 1.1 GB and is structured in a way that can be used in machine learning experiments. With correct pixel-wise annotations, it is now possible to create and test powerful semantic segmentation models like Attention U-Net using EfficientNet encoders. This information hence offers a good reference point toward the creation of automated oil spill surveillance through UAV imagery. To increase reproducibility, all experiments were done with a fixed random seed (42) used for dataset preparation, model initialization, and training. The tf.data pipeline in TensorFlow was utilized for deterministic loading, shuffling, batching, and prefetching of data. The encoder was initialized with the weights from pretrained EfficientNet-B4 on ImageNet, while the decoder was optimized through supervised training. Checkpoints were saved each time there was an improvement in validation loss, and the best checkpoint based on validation loss was used for evaluation of the model on the test set.
5.2 Qualitative results
Figure 7 shows a Keras model summary for a CNN decoder. It tracks layers, output shapes, and parameters (38.7M total), showing a 256 x 256 to 512 x 512 up-sampling process for image reconstruction or segmentation. Figure 8 shows the proposed system metrics output for F1, precision, recall, and IoU for oil, water, and other, each reported separately.
Figure 7
Figure 8
Figures 9–14 represent predicted oil spill segmentation results on randomly selected test samples. Each image highlights oil in pink, water in cyan, and other regions in yellow, enabling clear visual differentiation of classes. These predictions demonstrate the model's ability to accurately identify and segment oil spill regions under varying conditions.
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14
5.3 Quantitative results
5.3.1 F1 score comparison
The Table 7 and Figure 15 show the F1-score comparison between the models that were trained internally. For the conventional U-Net with categorical cross-entropy, the value of the F1-score was 0.8673. Switching from the conventional encoder to EfficientNet-B4 led to an increase in F1-score to 0.9403, while adding attention gates into the EfficientNet-B4 architecture resulted in a lower F1-score of 0.9213. However, the full HyEff-UNet architecture, with the combination of EfficientNet-B4, attention-guided decoder, and hybrid categorical cross-entropy and Dice loss functions, provided the highest F1-score value of 0.9551. This is an increase of 0.0878 in relation to the conventional U-Net configuration and by 0.0338 in relation to the previous EfficientNet-B4 + Attention configuration. Therefore, the positive effect can be attributed to the full configuration of the model rather than to EfficientNet-B4 or attention gates separately. Thus, the F1-score is calculated using Precision and Recall, as given in Equation 34.
Table 7
| Model | Encoder | Attention | Loss | Precision (macro) | Recall (macro) | F1-score (macro) | IoU (macro) |
|---|---|---|---|---|---|---|---|
| Standard U-Net | Standard CNN | No | CE | 0.8676 | 0.8672 | 0.8673 | 0.7724 |
| EfficientNet-B4 U-Net | EfficientNet-B4 | No | CE | 0.9419 | 0.9396 | 0.9403 | 0.8882 |
| EfficientNet-B4 + Attention | EfficientNet-B4 | Yes | CE | 0.9308 | 0.9178 | 0.9213 | 0.8564 |
| HyEff-UNet (EfficientNet-B4+Attention+Hybrid Loss) (Proposed) | EfficientNet-B4 | Yes | CE+Dice (0.5:0.5) | 0.9539 | 0.9565 | 0.9551 | 0.9143 |
Sequential ablation study showing the controlled metric comparison of the internally trained segmentation models, including Standard U-Net, EfficientNet-B4 U-Net, EfficientNet-B4 with Attention, and the proposed HyEff-UNet. The comparison reports Precision, Recall, F1-score, and IoU under different encoder, attention, and loss configurations to evaluate the individual and combined contributions of the EfficientNet-B4 encoder, attention mechanism, and hybrid CE + Dice loss.
Bold values identify the proposed model results among the evaluated models for the macro-averaged performance metrics.
Figure 15
5.3.2 Precision comparison
The Precision results of the ablation experiments are provided in Table 7 and Figure 16. In terms of Precision, the standard U-Net scored 0.8676. On the other hand, the scores for the EfficientNet-B4 U-Net and the EfficientNet-B4 + Attention designs were 0.9419 and 0.9308, respectively. For the complete HyEff-UNet, the score for Precision was 0.9539 after adopting the hybridization of losses, i.e., categorical cross-entropy and Dice loss. This score is higher by 0.0863 than the standard U-Net model and 0.0231 greater than the EfficientNet-B4 + Attention model. Hence, the Precision results confirm that the highest-performance design is the hybrid loss along with the architecture components. Thus, Precision is calculated as the ratio of correctly predicted positive instances to the total number of instances predicted as positive, as given in Equation 35.
Figure 16
5.3.4 Recall comparison
The Comparison of Recall is given in Table 7 and Figure 17. Standard U-Net had a recall of 0.8672, whereas the EfficientNet-B4 U-Net had a Recall of 0.9396. Adding attention gates to the EfficientNet-B4 resulted in a slight decrease in Recall to 0.9178; The HyEff-UNet had the maximum Recall of 0.9565. It is 0.0893 more efficient than the standard U-Net and 0.0387 more efficient than the EfficientNet-B4 + Attention. The improvement is therefore associated with the complete evaluated configuration, while the independent contribution of each component cannot be isolated by the sequential ablation design. Thus, Recall is calculated as the ratio of correctly predicted positive instances to the total number of actual positive instances, as given in Equation 36.
Figure 17
5.3.5 Intersection over Union (IoU) comparison
Further confirmation of the performance trend can be seen from the IoU results as illustrated in Table 7 and Figure 18. While the conventional U-Net scored an IoU of 0.7724, the scores for EfficientNet-B4 U-Net and EfficientNet-B4 + Attention were 0.8882 and 0.8564, respectively. The IoU for the full HyEff-UNet was 0.9143, which is 0.1419 higher than the IoU of the standard U-Net and 0.0579 higher than EfficientNet-B4 + Attention. This is because IoU measures the degree of overlap between predicted and reference regions. Therefore, this is evidence that the full HyEff-UNet configuration has the highest segmentation overlap compared to other models. Nonetheless, because not all combinations of the encoder, attention, and loss have been tested, the improvement is viewed as evidence for the efficacy of the full configuration but not a cause of the hybrid loss. Thus, the Intersection over Union (IoU) is calculated as the ratio of the intersection between the predicted and ground-truth regions to their union, as given in Equation 37.
Figure 18
To conduct a thorough evaluation and enable comparison with previous works on semantic segmentation, other metrics were calculated as well, such as the mean Dice Similarity Coefficient (DSC) as given in Equation 40, mean Intersection over Union (mIoU), and Matthews Correlation Coefficient (MCC) as defined in Equation 38. As presented in Table 8, the mean DSC of the proposed model was 0.9551, the mIoU was 0.9143, and the MCC was 0.9352, which showed a strong agreement between the predicted segmentation masks and the ground truth masks.
Table 8
| Metric | Value |
|---|---|
| F1-score | 0.9551 |
| Precision | 0.9539 |
| Recall | 0.9565 |
| IoU | 0.9143 |
| Mean Dice Similarity Coefficient (DSC) | 0.9551 |
| Mean Intersection Over Union (mIoU) | 0.9143 |
| Matthews Correlation Coefficient (MCC) | 0.9352 |
| Cohen's Kappa | 0.9351 |
This table contains the mean of every metric of the proposed model of oil spill segmentation, including F1-score, Precision, Recall, and IoU, Mean Dice Similarity Coefficient (DSC), Mean Intersection over Union (mIoU), Matthews Correlation Coefficient (MCC), and Cohen's Kappa values of the Oil, Water, and other surface regions.
In addition to the MCC metric, Cohen's Kappa score was calculated for the predicted vs. ground truth pixels as the second chance-corrected measure of agreement. The model produced a Cohen's Kappa score of 0.9351, which nearly coincided with the MCC score of 0.9352. Given that both metrics are chance-corrected measures but are calculated in different ways using the ratio of observed and expected agreement for Kappa, while using the full confusion matrix for MCC their near equality once again proves that the model results are not due to any class imbalance and reflect real agreement of the pixels beyond random chance. The proposed HyEff-UNet achieved a Cohen's Kappa score of 0.9351, indicating excellent agreement between the predicted and actual pixel labels. Cohen's Kappa is mathematically defined in Equation 39 as:
Class-wise DSC of the different categories are presented in Table 9. In particular, the other class had the largest DSC (0.973), followed by Oil (0.951), and then Water (0.941). This implied that the HyEff-UNet maintained a high segmentation performance in all semantic classes, even for the challenging water region. The Dice Similarity Coefficient measures the similarity between the predicted and ground-truth segmentation masks.
Table 9
| Metric | Oil | Water | Other |
|---|---|---|---|
| F1 score (per-class) | 0.951 | 0.941 | 0.973 |
| Precision (per-class) | 0.940 | 0.934 | 0.988 |
| Recall (per-class) | 0.962 | 0.948 | 0.959 |
| IoU (per-class) | 0.907 | 0.888 | 0.948 |
| Class-wise Dice Similarity Coefficient (DSC) | 0.951 | 0.941 | 0.973 |
This table contains the class-wise performance of the proposed model of oil spill segmentation, including F1-score, Precision, Recall, and IoU values of the Oil, Water, and other surface regions.
Since the proposed framework performs three-class semantic segmentation (Oil, Water, and Other), each evaluation metric is first computed independently for every class and then averaged using macro averaging. For a metric M, the Macro averaging is given in Equation 41 as:
Where C = 3 represents the number of semantic classes, and Midenotes the metric value for class i. Macro averaging assigns equal importance to each class regardless of the number of pixels, making it appropriate for class-imbalanced semantic segmentation problems such as marine oil spill detection.
The results constitute a sequential rather than a factorial ablation. Hence, Table 10 is only used to determine the difference in performance between successive configurations and not to estimate the individual effect of the factors.
Table 10
| Transition | F1 change | IoU change | Interpretation |
|---|---|---|---|
| U-Net → EfficientNet-B4 U-Net | +0.0730 | +0.1158 | EfficientNet-B4 alone improved performance under CE |
| EfficientNet-B4 U-Net → EfficientNet-B4 + Attention | −0.0190 | −0.0318 | Attention alone did not improve performance under CE |
| EfficientNet-B4 + Attention → HyEff-UNet | +0.0338 | +0.0579 | Largest gain occurs after introducing CE + Dice |
| U-Net → HyEff-UNet | +0.0878 | +0.1419 | Final model exceeds the controlled standard U-Net baseline |
Observed performance changes across the controlled ablation sequence.
Bold values identify the improvement between internal baselines and proposed model in the performance changes across the ablation sequence.
Figure 19 shows that the training loss generally decreases throughout the 20 training epochs. The training loss decreases consistently throughout the training process, demonstrating effective optimization and convergence of the network. Similarly, the validation loss shows a substantial reduction during the initial epochs and reaches its minimum value at epoch 10 (approximately 0.0863), indicating the point of best generalization performance. Beyond this epoch, the validation loss exhibits slight fluctuations despite the continued reduction in training loss, suggesting the onset of mild overfitting and diminishing improvements in validation performance. Hence, epoch 10 was selected as the final model for quantitative evaluation and testing. As seen in Figure 20, the validation accuracy is shown. The validation performance should therefore be interpreted in conjunction with the checkpoint-based test results rather than as evidence of robustness across unseen acquisition conditions.
Figure 19
Figure 20
5.4 Sequential component ablation
A sequential component ablation was performed to examine the effect of progressively modifying the standard U-Net architecture. Commencing from a traditional U-Net baseline architecture, followed by the inclusion of EfficientNet-B4 as the encoder, then adding the attention gates, and finally including the hybrid loss function. The existing ablation therefore evaluates the cumulative effect of these modifications rather than constituting a complete factorial design. All ablation models were trained using the same dataset partitions, preprocessing pipeline, and training configuration, with only the architectural or loss component under study changed.
Table 7 presents the ablation study of the proposed HyEff-UNet. The Standard U-Net trained with categorical cross-entropy achieved a macro-Precision, Recall, F1-score, and IoU of 0.8676, 0.8672, 0.8673, and 0.7724, respectively. Replacing the standard encoder with EfficientNet-B4 significantly improved the segmentation performance, increasing the macro F1-score to 0.9403 and the IoU to 0.8882. Introducing attention gates with the same cross-entropy loss resulted in a slight reduction in performance, with the macro F1-score decreasing to 0.9213 and the IoU to 0.8564. While the attention gates are meant to highlight important spatial areas, the ablation done cannot specifically point out what led to the drop in performance. However, when the hybrid loss function combining categorical cross-entropy and Dice loss (0.5:0.5) was employed together with the EfficientNet-B4 encoder and attention gates, the proposed HyEff-UNet achieved the best overall performance. It obtained a macro-Precision of 0.9539, Recall of 0.9565, F1-score of 0.9551, and IoU of 0.9143, outperforming all other configurations. It can be concluded that the complete hybrid loss configuration is responsible for the highest enhancement effect within the ablation process currently implemented. Nevertheless, since no such experiments with hybrid loss configurations without EfficientNet-B4 or without attention have been conducted, it is impossible to separate the independent effects of the loss function.
HyEff-UNet achieves the best overall performance for the Oil class, obtaining the highest F1-score (0.9510) and IoU (0.9065) among the evaluated ablation configurations. It also achieves high precision (0.9403) and recall (0.9618), indicating a favorable balance between correctly identifying Oil pixels and limiting false-positive predictions. The EfficientNet-B4 U-Net configuration achieves lower Oil precision (0.9071), recall (0.9609), F1-score (0.9332), and IoU (0.8748), while the EfficientNet-B4 + Attention configuration obtains 0.8580 precision, 0.9718 recall, 0.9114 F1-score, and 0.8372 IoU. Although the EfficientNet-B4 + Attention configuration achieves the highest Oil recall, its lower precision reduces the F1-score and IoU compared with HyEff-UNet. Overall, the results show that the complete HyEff-UNet configuration provides the strongest balance across precision, recall, F1-score, and IoU for the Oil class. The metrics in Table 11 are global pixel-level measures computed by aggregating predictions across the entire test set, whereas the statistical analysis in Table 12 reports mean per-image Oil metrics, where each test image contributes equally regardless of its Oil-pixel content. Consequently, the numerical values reported in the two tables are not directly comparable
Table 11
| Model | Oil precision (per-class) | Oil recall (per-class) | Oil F1 (per-class) | Oil IoU (per-class) |
|---|---|---|---|---|
| Standard U-Net | 0.8319 | 0.8601 | 0.8458 | 0.7328 |
| EfficientNet-B4 U-Net | 0.9071 | 0.9609 | 0.9332 | 0.8748 |
| EfficientNet-B4 + Attention | 0.8580 | 0.9718 | 0.9114 | 0.8372 |
| HyEff-UNet | 0.9403 | 0.9618 | 0.9510 | 0.9065 |
Global pixel-level precision, recall, F1-score, and IoU for the Oil class across the ablation configurations.
Bold values identify the proposed model results for each metric in the Oil-class performance metrics.
Table 12
| Comparison | Metric (per-image) | HyEff-UNet Oil F1/IoU | Baseline oil F1/IoU | Mean difference | Holm-adjusted p-value | Significant |
|---|---|---|---|---|---|---|
| HyEff-UNet vs. Standard U-Net | Oil F1 | 0.7274 | 0.6404 | +0.0870 | < 0.000001 | Yes |
| HyEff-UNet vs. Standard U-Net | Oil IoU | 0.6901 | 0.5731 | +0.1170 | < 0.000001 | Yes |
| HyEff-UNet vs. EfficientNet-B4 U-Net | Oil F1 | 0.7274 | 0.7187 | +0.0087 | 0.000005 | Yes |
| HyEff-UNet vs. EfficientNet-B4 U-Net | Oil IoU | 0.6901 | 0.6784 | +0.0117 | 0.000005 | Yes |
| HyEff-UNet vs. EfficientNet-B4 + Attention | Oil F1 | 0.7274 | 0.6892 | +0.0382 | < 0.000001 | Yes |
| HyEff-UNet vs. EfficientNet-B4 + Attention | Oil IoU | 0.6901 | 0.6373 | +0.0528 | < 0.000001 | Yes |
Paired statistical comparison of mean-per-image oil-class F1-score and IoU across ablation models.
To evaluate the statistical significance of the improvement in Oil-class segmentation, per-image Oil F1-score and IoU were computed independently for each of the 254 test images after converting the three-class segmentation into a binary Oil-vs.-Non-Oil problem. Unlike the global pixel-level metrics reported in Table 12, which aggregate all pixels across the entire test set, the per-image metrics assign equal weight to each test image before averaging. For images in which the Oil class was absent in the ground truth and/or prediction, metric values were assigned as zero using the zero_division = 0 setting. The resulting per-image values were then used for the paired Wilcoxon signed-rank analysis. As shown in Table 12, HyEff-UNet achieved the highest mean Oil F1-score (0.7274) and IoU (0.6901). The corresponding F1/IoU values were 0.6404/0.5731 for Standard U-Net, 0.7187/0.6784 for EfficientNet-B4 U-Net, and 0.6892/0.6373 for EfficientNet-B4 + Attention. HyEff-UNet showed mean improvements of 0.0870/0.1170, 0.0087/0.0118, and 0.0382/0.0528, respectively. All pairwise differences remained statistically significant after Holm correction, with adjusted p-values below 0.00001. These results provide statistical evidence that the higher per-image Oil-class F1-score and IoU achieved by HyEff-UNet are consistent across the evaluated test set. However, the sequential ablation design does not independently isolate the contribution of the encoder, attention mechanism, or hybrid loss; therefore, these experiments alone cannot conclusively attribute their individual effects.
5.4.1 Hybrid loss weighting
For this study, equal weighting (0.5:0.5) was chosen since both criteria of optimization have equal importance for semantic segmentation in oil spill detection. While Categorical Cross-Entropy enhances classification on a per-pixel basis by punishing incorrect probabilities, Dice Loss directly aims at optimizing overlaps between prediction and ground-truth regions, which is especially effective when dealing with small-sized oil spill regions in class imbalance cases. Equal weighting ensures that neither of the two loss criteria will dominate the optimization and that both pixel classification and regional overlaps will contribute equally to training. It should be acknowledged that the objective of the current study was to test the effectiveness of the suggested HyEff-UNet network architecture and not to find out optimal weighting coefficients for the hybrid loss. Hence, other types of weightings such as 0.3:0.7, 0.4:0.6, 0.6:0.4, adaptive weighting, and dynamic loss balancing were not experimentally tested. Systematic analysis of different weighting coefficients is one of the directions for further studies.
5.5 Computational complexity discussion
The computational characteristics of the proposed model were evaluated using the experimental environment employed in this study. The model comprises 38,766,982 trainable parameters with a storage size of approximately 149.59 MB. The computational complexity was estimated at 1.10 × 1011 FLOPs for an input image size of 512 × 512. The average inference latency was measured as 2335.62 ms per 512 × 512 image on the NVIDIA Tesla T4 GPU used in the Google Colab runtime, corresponding to approximately 0.43 FPS. These measurements characterize inference performance on the experimental GPU and should not be interpreted as onboard UAV performance. Since deployment on UAV or edge-computing hardware was not evaluated, further benchmarking on representative embedded platforms is required. These results indicate that the current implementation prioritizes segmentation accuracy over inference speed and requires further optimization before deployment on resource-constrained UAV platforms.
HyEff-UNet was designed with the objective of optimizing segmentation performance while minimizing computation cost. EfficientNet-B4 was chosen as the encoder backbone because it provides a practical balance between representational capacity and computational cost for the proposed segmentation framework. Attention gates ensure efficient feature propagation while adding minimal complexity to the architecture. Transfer learning accelerates the process of convergence while also lowering the amount of time spent in training as opposed to training from scratch. These measurements provide an initial assessment of computational cost on the experimental hardware; however, they do not represent deployment performance on an actual UAV or resource-constrained edge device. Further benchmarking on representative UAV hardware is required.
5.6 Discussion of challenging scenarios
Even though this approach exhibits high segmentation precision on the test set, several visually challenging conditions may negatively impact the performance. The existence of thin oil layers with low color contrast, oil patches that are broken, sun glare, cloud shadows, texture changes due to waves, the presence of ships, and complex port structures can result in appearance patterns that look like an oil spill. This can lead to the generation of both false positive and false negative results since color alone cannot separate oil from other background features. The performance can be impacted by different parameters when put into practice, including UAV height and camera viewing angles, as well as weather factors such as rain and fog.
6 Limitations
There are some limitations to this study. First, the proposed system has been tested using just one publicly available UAV RGB oil spill dataset; hence, its ability to generalize in other geographic areas and acquisition conditions has not yet been proven. Second, RGB imagery is the only type that has been taken into consideration, while the use of thermal, multispectral, hyperspectral, and SAR imagery has not been explored. Third, the database mainly contains daytime UAV images, and difficult environmental conditions like nighttime imaging, fog, rain, waves at sea, strong reflections of sunlight, and illumination variations have not been studied sufficiently well. Also, deployment on UAV platforms was not part of this research. Deployment is affected by many factors such as the power of the GPU onboard the UAV, battery lifetime, latency, calibration of the camera, altitude of the UAV, accuracy of the GPS, and many others. Latency, energy consumption, memory usage, and on-board performance have not been analyzed experimentally.
While the pre-existing Zenodo splits for training, validation, and testing have been preserved for the sake of consistency, correlated-frame risk cannot be ruled out entirely, as there is no information at the flight level, sequence level, or scene level that could identify whether there is a visual or temporal correlation between different UAV frames of the same acquisition. Thus, no reliable metadata-based clustering can be performed, nor can the leave-one-scene-out assessment be conducted, nor can it be guaranteed that visually and/or temporally correlated UAV frames from the same acquisition are present only in one split. As a result, the presented findings are only applicable to the held-out test partition of this benchmark.
Furthermore, the proposed framework was tested using the same fixed train/validation/test split to maintain consistency with previous benchmarking studies. An additional limitation is that the validation-loss curve exhibited noticeable fluctuations during training, indicating that optimization stability could be further improved through additional regularization, learning-rate scheduling, or repeated experiments with multiple random seeds. More thorough statistical verification using repeated experiments with several random seeds, confidence intervals, and statistical tests was not carried out due to the higher computational cost needed for it. The same applies to the comparative experiments on transformer-based models, which are computationally more expensive, like SegFormer, Mask2Former, TransUNet, and Segment Anything Model (SAM). All these are limitations that give direction for future research in enhancing the HyEff-UNet framework. The Wilcoxon signed-rank test was performed using the 254 test images as paired observations, while each model was trained only once with a fixed random seed. Therefore, the reported p-values reflect prediction consistency across the test set rather than variability arising from repeated training with different random initializations.
7 Conclusion and future improvements
7.1 Conclusion
The proposed HyEff-UNet framework provides an automated system of oil spill identification based on semantic segmentation with deep learning. The model operates on high-resolution RGB images of the marine and port environments through the implementation of a modified Attention U-Net architecture, which is integrated with an EfficientNet-B4 encoder. The EfficientNet-B4 backbone, which was trained using ImageNet, provides a way to extract hierarchical features by capturing low-level texture details and the high-level context information of complicated maritime scenes. The hierarchical feature representations provided by the EfficientNet-B4 encoder are intended to capture low-level texture and high-level contextual information that may be useful for distinguishing oil-spill regions from visually complex backgrounds. In the segmentation results, the use of attention gates in the U-Net architecture allows the model to concentrate on the spatial features of oil spills that are relevant to the model, whilst ignoring the irrelevant background data. The attention mechanism is designed to emphasize spatially informative regions and may assist segmentation of fine-grained oil textures and shapes.
The hybrid loss function, which is the combination of Categorical Cross-Entropy Loss and Dice Loss, represents one of the essential aspects of the suggested framework that is used to streamline the training process. The categorical cross-entropy guarantees precise pixel-wise classification in multiple classes such as oil, water, and other background objects, whereas Dice loss enhances overlap of regions and boundaries between the predicted and ground-truth segmentation masks. This hybrid method is useful in solving the issue of class imbalance that is generally seen in the oil spill data, where the oil regions take considerably less space in the picture than the water and the background. The complete architecture using the hybrid loss achieved better segmentation performance within the evaluated sequential configurations. However, the present experiments do not isolate whether this improvement is attributable specifically to the hybrid loss. The standard semantic segmentation performance measures that were used to evaluate the model comprised standard measures such as Precision, Recall, F1-Score, and IoU.
The experimental results demonstrate effective segmentation performance on the predefined held-out test partition of the evaluated UAV RGB dataset. The imagery provided by UAVs also makes the system more practical, as it is possible to conduct a flexible, cost-effective, and high-resolution surveillance of extensive maritime regions. Overall, the framework provides an automated approach for pixel-level oil-spill segmentation on the evaluated UAV RGB dataset. Given that the model evaluation was done using one publicly available UAV dataset, the presented results cannot be assumed to represent generalization in all maritime environments. In future work, the proposed model will be tested using more than one dataset under various environmental settings.
7.2 Future enhancements
Despite the encouraging outcomes of the proposed system, it can be optimized in the future by introducing several improvements that will contribute to its increased strength and practical implementation. To begin with, one can consider the multisource fusion of the data as a fusion of RGB imaging with any other sensing option, including Synthetic Aperture Radar (SAR) or hyperspectral imagery. These other data sources might offer complementary spectral data, which enhances detection accuracy in adverse environmental conditions, such as low visibility, cloud cover, or nighttime surveillance. Second, the model can be streamlined to run on edge computing on lightweight embedded systems directly attached to UAVs. This kind of optimization would allow the detection of oil spills in the case of an aerial surveillance mission, thus saving a substantial amount of time in responding to environmental emergencies. The other prospective path is the combination of change detection methods and temporal analysis methods. Through multi-temporal UAV imagery, the system would be able to distinguish true oil spills from visually similar events, including algal blooms, wave shadows, or trash at the surface, which would minimize false positives.
Future directions for research would revolve around enhancing accuracy as well as the efficiency of the deployment process. Future research would compare the method with contemporary architectures such as EfficientNetV2, SegFormer, Mask2Former, TransUNet, and other segmentation models that use transformers. Other future research would involve the evaluation of the encoder through ablation studies, robustness testing in challenging environments, and cross-location evaluation using various UAV datasets. Future work will include condition-specific robustness evaluation under controlled variations in illumination, reflections, wave patterns, weather conditions, and scene complexity, together with cross-location evaluation on independent UAV datasets. Furthermore, deployment on edge-computing hardware will also be studied to enable an efficient onboard oil spill detection process.
Also, deep learning methods like Generative Adversarial Networks (GANs) could be applied to automated dataset expansion to create synthetic training data and increase the size of existing datasets. This would assist in addressing the scarcity of good-quality annotated datasets on oil spills and enhance the generalization power of the model. Lastly, future research can be aimed at enhancing the segmentation framework to determine the thickness and volume of oil spills, rather than merely identifying them. This information would be useful to environmental monitoring agencies to be able to estimate the spills better, facilitate planning of clean-up, and decision-making in the event of responding to marine pollution activities.
Statements
Data availability statement
The oil-spill images were obtained from the publicly available Zenodo dataset, “Annotated RGB Images of Oil Spills in a Port Environment” (Source: https://zenodo.org/records/10555314).
Author contributions
DS: Investigation, Writing – original draft, Methodology, Formal analysis, Supervision. SM: Conceptualization, Methodology, Investigation, Writing – review & editing. AB: Validation, Writing – review & editing, Software, Formal analysis. PN: Visualization, Validation, Writing – review & editing, Investigation.
Funding
The author(s) declared that financial support was not received for this work and/or its publication.
Conflict of interest
The author(s) declared that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Generative AI statement
The author(s) declared that Generative AI was not used in the creation of this manuscript.
Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1
AtteiaG.DabboorM.KarantzalosK.AlabdulhafithM. (2025). Hybrid deep learning and optimized feature selection for oil spill detection in satellite images. Comput. Mater. Contin. 84, 1747–1767. doi: 10.32604/cmc.2025.063363
2
BianchiF. M.EspesethM. M.BorchN. (2020). Large-scale detection and categorization of oil spills from SAR images with deep learning. Remote Sens.12:2260. doi: 10.3390/rs12142260
3
BuiN. A.OhY.LeeI. (2024). Oil spill detection and classification through deep learning and tailored data augmentation. Int. J. Appl. Earth Obs. Geoinf.129:103845. doi: 10.1016/j.jag.2024.103845
4
CaiY.SuJ.SongJ.XuD.ZhangL.ShenG. (2025). LRA-UNet: A lightweight residual attention network for SAR marine oil spill detection. J. Mar. Sci. Eng.13:1161. doi: 10.3390/jmse13061161
5
Carrasco-GarcíaM. G.Rodríguez-GarcíaM. I.González-EnriqueF. J.Ruiz-AguilarJ. J.Alcantara PérezJ. M.DomínguezI. J. T. (2026). Hyperspectral imaging and artificial neural networks for oil spills automatic detection. Log. J. IGPL.34:jzaf032. doi: 10.1093/jigpal/jzaf032
6
ChenF.BalzterH.RenP.ZhouH. (2023). SRCNet: Seminal representation collaborative network for marine oil spill segmentation. arXiv [Preprint]. doi: 10.1109/TGRS.2024.3463404
7
ChenY. T.ChangL.WangJ. H. (2024). Full-scale aggregated MobileUNet: an improved U-Net architecture for SAR oil spill detection. Sensors.24:3724. doi: 10.3390/s24123724
8
ChoudharyM.GoyalN.GuptaD.SharmaB.SharmaN. (2024). An oceanographic data collection scheme using hybrid optimization for leakage detection during oil mining in mobility assisted UWSN. Multimed. Tools Appl.83, 89723–89741. doi: 10.1007/s11042-024-19023-z
9
De KerfT.SelsS.SamsonovaS.VanlanduitS. (2024). A dataset of drone-captured, segmented images for oil spill detection in port environments. Sci. Data.11:1180. doi: 10.1038/s41597-024-03993-8
10
DiM.DiX.XiaoH.GaoY.LiY. (2025). CBENet: Contextual and boundary-enhanced network for oil spill detection via microwave remote sensing. Intell. Mar. Technol. Syst.3:7. doi: 10.1007/s44295-025-00056-5
11
DongS.FengJ.GuZ.YinK.LongY. (2025). A review of artificial intelligence and remote sensing for marine oil spill detection, classification, and thickness estimation. Remote. Sens.17:3681. doi: 10.3390/rs17223681
12
DuK.MaY.LiZ.LiuR.JiangZ.YangJ. (2025). MS3OSD: A novel deep learning approach for oil spills detection using optical satellite multisensor spatial-spectral fusion images. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.18, 8617–8629. doi: 10.1109/JSTARS.2025.3550421
13
GoyalN.KumarA.PopliR.AwasthiL. K.SharmaN.SharmaG. (2022a). Priority based data gathering using multiple mobile sinks in cluster based UWSNs for oil pipeline leakage detection. Cluster Comput.25, 1341–1354. doi: 10.1007/s10586-021-03513-y
14
GoyalN.NainM.SinghA.AbualsaudK.AlsubhiK.Ortega-MansillaA.et al. (2022b). An anchor-based localization in underwater wireless sensor networks for industrial oil pipeline monitoring. IEEE Can. J. Electr. Comput. Eng.45, 466–474. doi: 10.1109/ICJECE.2022.3206275
15
HammoudB.MarounC. B.WehnN. (2025). DATE on the edge: On-site oil spill detection and thickness estimation using drone-based radar backscattering. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.18, 523–536. doi: 10.1109/JSTARS.2024.3472908
16
HeL.ZhouY.YangH.SuL.MaJ. (2026). A deep learning-based method for marine oil spill detection and its application in UAV imagery. Mar. Pollut. Bull.222:118889. doi: 10.1016/j.marpolbul.2025.118889
17
HuangX.ZhangB.PerrieW. (2025). A two-stage deep learning method for marine oil spill localization and segmentation from synthetic aperture radar images. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.18, 12315–12327. doi: 10.1109/JSTARS.2025.3567859
18
JiangQ.JiM.WangJ.SunP. (2023). Remote sensing methods for striped marine oil spill detection in narrow ship channels. Ocean Eng.289:116162. doi: 10.1016/j.oceaneng.2023.116162
19
KussulN.SaliiY.KuzinV.YailymovB.ShelestovA. (2025). Transfer learning and single-polarized SAR image preprocessing for oil spill detection. ISPRS Open J. Photogramm. Remote Sens.15:100081. doi: 10.1016/j.ophoto.2024.100081
20
LaiC.ChenS.HuangT.SongS.ChengG.LuoC.et al. (2026). Beyond segmentation: an oil spill change detection framework using synthetic SAR imagery. arXiv [Preprint]. doi: 10.48550/arXiv.2601.02139
21
LiC.KimD. J.ParkS.KimJ.SongJ. (2023). A self-evolving deep learning algorithm for automatic oil spill detection in Sentinel-1 SAR images. Remote Sens. Environ.299:113872. doi: 10.1016/j.rse.2023.113872
22
LiuQ.HuangT.DongY.XiangW. (2025). Enhancing oil spill detection with controlled random sampling: a multimodal fusion approach using SAR and HSI imagery. Remote Sens. Appl.: Soc. Environ.38:101601. doi: 10.1016/j.rsase.2025.101601
23
MoncadaW.Rojas AcuñaJ.AldanaC.LloretJ. (2025). Detection of oil spilled at sea with entropy and alpha polarimetric in Sentinel-1 C-band SAR data: Lima-Peru case. Remote Sens. Appl.: Soc. Environ.38:101585. doi: 10.1016/j.rsase.2025.101585
24
ParkS.LiC.KimH.KimD. J. (2024). A semi-empirical threshold model for oil spill detection by analyzing microwave backscatter of ocean surface from Sentinel-1 C-band SAR data. IEEE Geosci. Remote Sens. Lett.21, 1–5. doi: 10.1109/LGRS.2024.3355464
25
SongD.HuangQ.GaoH.WangB.ZhangJ.ChenW. (2024). Adaptive oil spill detection network for scene-based PolSAR data using dynamic convolution and boundary constraints. Int. J. Appl. Earth Obs. Geoinf.130:103914. doi: 10.1016/j.jag.2024.103914
26
SongW.MaX.SongW. (2024). Automatic detection of marine oil spills from polarimetric SAR images using deep convolutional neural network model. Ecol. Indic.169:112934. doi: 10.1016/j.ecolind.2024.112934
27
TalasilaS.GurralaV. K.VarshiniM.BallaS. S.MadhuriM.KumarC. K. (2025). Real-time oil spill detection with YOLO framework for marine ecosystem surveillance. Int. J. Electr. Electron. Res.13, 463–470. doi: 10.37391/ijeer.130310
28
TongJ.XieD. (2025). On the features extracted from dual-polarized Sentinel-1 images for deep-learning-based sea surface oil-spill detection. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.18, 23045–23060. doi: 10.1109/JSTARS.2025.3604418
29
WanY.PengL.ZhangR.ZhangR.WangH. (2026). A dual-branch EfficientNetV2-S-based method for marine oil spill detection using multisource satellite data fusion. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.19, 1549–1566. doi: 10.1109/JSTARS.2025.3639503
30
WuW.WongM. S.YuX.ShiG.KwokC. Y. T.ZouK. (2024). Compositional oil spill detection based on object detector and adapted segment anything model from SAR images. IEEE Geosci. Remote Sens. Lett.21:4007505. doi: 10.1109/LGRS.2024.3382970
31
XiangD.LuY.GuanD.LiG.ChengJ.LiB. (2025). Oil spill detection in PolSAR imagery using composite scattering power entropy and multiscale hybrid feature fusion network. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.18:13388–13407. doi: 10.1109/JSTARS.2025.3570697
32
XueQ.SongJ.LuF.MaJ.GaoX.XuJ. (2024). Research on marine oil spill identification based on laser-induced fluorescence LiDAR. Intell. Mar. Technol. Syst.2:22. doi: 10.1007/s44295-024-00037-0
33
ZakzoukM.AbdulazizA. M.Abou El-MagdI.DahabA. S.AliE. M. (2025). Automated oil spill detection using deep learning and SAR satellite data for the northern entrance of the Suez Canal. Sci. Rep.15:20107. doi: 10.1038/s41598-025-03028-1
34
ZhaoZ.LiJ.WangZ.BuX.GaoH. (2026). PetroAnalyticNet: Applying separable dilated convolutions for drone-captured oil spill segmentation in ports. Alex. Eng. J.143, 64–75. doi: 10.1016/j.aej.2026.03.048
35
ZhengG.JiangZ.ZhangH.YaoX. (2023). Deep semantic segmentation of unmanned aerial vehicle remote sensing images based on fully convolutional neural network. Front. Earth Sci.11:1115805. doi: 10.3389/feart.2023.1115805
Summary
Keywords
attention U-net, Efficientnet-B4, marine pollution monitoring, oil spill detection, semantic image segmentation
Citation
S. D, M. S. SP, B. A and N. PS (2026) HyEff-UNet: an attention-driven Efficientnet U-Net with hybrid loss optimization for UAV-based marine oil spill detection and segmentation. Front. Water 8:1932830. doi: 10.3389/frwa.2026.1932830
Received
09 July 2026
Revised
24 August 2026
Accepted
30 September 2026
Published
03 November 2026
Volume
8 - 2026
Edited by
Xiaowei Jia, University of Pittsburgh, United States
Updates
Copyright
© 2026 S., M. S., B. and N..
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Siva Priya M. S., sivapriya.ms@vit.ac.in
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.