Introduction

Colorectal cancer is one of the most prevalent tumors worldwide, accounting for approximately 10% of cases and over 930,000 annual deaths¹. The progression from adenomatous polyps to carcinoma typically takes between 10 and 15 years, offering a wide window for prevention through early detection and removal²; however, the detection rate during colonoscopy varies widely, especially due to differences in polyp morphology³.

In this context, data science, particularly machine learning, has much to offer. Polyp segmentation can be treated as a pixel-by-pixel classification problem. Such work is challenging due to the morphological heterogeneity of lesions, low contrast between polyp tissue and adjacent mucosa, and variable lighting conditions in endoscopy; however, advances in convolutional neural networks have driven significant improvements in segmentation accuracy, surpassing traditional methods based on manual features^{4 5}.

In this scenario, U-Net has established itself as the state-of-the-art for such tasks through an encoder-decoder scheme with skip connections⁶.

However, its limitations have motivated variants aimed at improving representation fusion and feature selectivity. This work performs systematic comparisons under controlled experimental conditions, comparing five architectures in polyp segmentation using the Kvasir-SEG dataset⁷.

Related Work

Automated polyp segmentation has evolved significantly from classical methods to specialized deep architectures. Initial approaches relied on manual features and traditional computer vision techniques. Bernal et al.³ proposed WM-DOVA maps combining intensity and shape information, while Tajbakhsh et al.⁵ employed shallow convolutional networks with contextual features for detection in colonoscopy videos.

The introduction of fully convolutional networks (FCN)⁸ revolutionized semantic segmentation by eliminating final dense layers and enabling pixel-by-pixel predictions. Brandão et al.⁴ adapted FCNs for polyp segmentation, demonstrating superiority over traditional methods but still facing difficulties with imprecise edges and morphological variations. The U-Net architecture⁶, with its encoder-decoder paradigm and skip connections, became the reference for biomedical segmentation by preserving spatial information at multiple scales.

Various U-Net extensions have been proposed for polyp segmentation. Zhou et al.⁹ introduced nested dense connections to reduce semantic gaps, while Oktay et al.¹⁰ incorporated spatial attention mechanisms. Jha et al.¹¹ proposed DoubleU-Net, combining two encoder-decoder paths with ASPP (Atrous Spatial Pyramid Pooling) blocks, achieving a Dice score of 82.3% on Kvasir-SEG.

Recent approaches explore sophisticated attention mechanisms and multi-scale aggregation. Fan et al.¹² developed PraNet with parallel reverse attention, reporting 89.8% Dice on Kvasir-SEG. Fang et al.¹³ proposed selective feature aggregation with area-boundary constraints, while Zhang et al.¹⁴ introduced adaptive context selection to handle morphological variability. Srivastava et al.¹⁵ proposed MSRF-Net with multi-scale residual fusion.

More recent work investigates transformers for medical segmentation. Valanarasu et al.¹⁶ introduced Medical Transformer with axial attention, demonstrating competitiveness with CNNs. Huang et al.¹⁷ proposed HarDNet-MSEG, achieving Dice scores above 90% at 86 FPS, balancing accuracy and computational efficiency.

Despite advances, direct comparisons are hindered by methodological variations: different data partitions, augmentation strategies, loss functions, and hyperparameters. Many works report results on multiple datasets but lack rigorous statistical analysis. This study addresses these gaps through systematic comparison of five fundamental U-Net architectures under controlled experimental conditions, with multiple seeds for variance analysis, providing a reproducible benchmark for the domain.

Theoretical Foundations of U-Net Architectures

U-Net establishes the encoder-decoder paradigm with skip connections for biomedical segmentation⁶. The encoder captures multi-scale semantic information through successive convolutions and pooling, while skip connections concatenate features from encoder to decoder, preserving spatial details.

Let \(x\in\mathbb{R}^{H\times W\times C}\) be an input image and \(y\in{{0,1}}^{H\times W}\) its binary reference mask. The network implements a mapping \(f:\mathbb{R}^{H\times W\times C}\rightarrow[0,1]^{H\times W}\) whose threshold at 0.5 produces \(\hat{y}\in{{0,1}}^{H\times W}\). At each encoder level \(l\), the number of channels (width) follows \(c_l = c_0\cdot 2^{l}\).

U-Net++: Nested Dense Connections

U-Net++ addresses the semantic gap of U-Net through nested convolutional blocks⁹. In U-Net, skip connections concatenate semantically distant features (low level in encoder, high level in decoder). U-Net++ resolves this by gradually connecting features through dense paths. Each node \(\mathbf{X}^{i,j}\) is

\[ \begin{aligned} \mathbf{X}^{i,j} = \begin{cases} \mathcal{H}(\mathbf{X}^{i,j-1}), & j = 0 \\ \mathcal{H}\left(\left[\left[\mathbf{X}^{i,k}\right]_{k=0}^{j-1}, \mathcal{U}(\mathbf{X}^{i+1,j-1})\right]\right), & j > 0, \end{cases} \end{aligned} \]

where \(i\) indexes resolution, \(j\) connection density, \(\mathcal{H}(\cdot)\) convolution, \(\mathcal{U}(\cdot)\) upsampling, \([[\cdot]]\) concatenation. This progressively reduces the semantic gap through successive transformations.

In addition to U-Net++, we also adopt a version with deep supervision, which adds multi-scale supervision \(\mathcal{L}_{\text{total}} = \sum_{j=1}^{J} \omega_j \mathcal{L}(\mathbf{y}, \mathbf{\hat{y}}^{(j)})\), mitigating vanishing gradients and accelerating convergence.

Attention U-Net: Spatial Attention Mechanisms

Attention U-Net incorporates attention gates that spatially weight features before concatenation¹⁰, suppressing irrelevant information transmitted by skip connections. Given \(\mathbf{x}\) from the encoder and gating signal \(\mathbf{g}\) from the decoder, attention coefficients are

\[ \hat{\mathbf{x}} = \sigma_2\left(\psi^T \sigma_1(\mathbf{W}_x^T \mathbf{x} + \mathbf{W}_g^T \mathbf{g})\right) \odot \mathbf{x} \]

where \(\sigma_1\) and \(\sigma_2\) are ReLU and sigmoid respectively, allowing suppression of irrelevant regions and emphasis of salient structures.

ResU-Net: Residual Connections for Deeper Networks

ResU-Net integrates the U-Net architecture with residual blocks inspired by ResNet¹⁸, addressing the performance degradation that occurs when training very deep networks¹⁹. While standard U-Net uses double convolution blocks, ResU-Net replaces these blocks with residual blocks that implement identity connections. A residual block learns a residual function \(\mathcal{F}(\mathbf{x})\) instead of the desired direct mapping \(\mathcal{H}(\mathbf{x})\), through the shortcut connection

\[ \mathbf{y} = \mathcal{F}(\mathbf{x}, {W_i}) + \mathbf{x}, \]

where \(\mathbf{x}\) and \(\mathbf{y}\) are input and output vectors, and \(\mathcal{F}(\mathbf{x}, {W_i})\) represents the residual transformation to be learned. This architecture facilitates training deeper networks by allowing the model to learn incremental transformations instead of complete mappings, resulting in better gradient propagation and potential for capturing richer representations.

Methodology

Dataset and Preprocessing

Kvasir-SEG⁷ contains 1,000 colonoscopy images with pixel-by-pixel segmentation masks and bounding boxes, covering wide diversity in polyp size and shape. Images have varying resolutions (from 332\(\times\)487 to 1920\(\times\)1072), and are resized to 256\(\times\)256 pixels and normalized with ImageNet statistics. The partition considers 80% of images for training, 10% for validation, 10% for testing.

Implementation and Training

Our study examines five U-Net variants: standard U-Net (double 3\(\times\)3 convolution blocks, batch normalization, ReLU, five levels with \({64, 128, 256, 512, 512}\) channels); U-Net++ (nested dense paths, with and without deep supervision); Attention U-Net (with attention gates); ResU-Net (residual blocks with identity connections, five levels with \({64, 128, 256, 512, 512}\) channels).

All models were trained for a maximum of 100 epochs, with batch size 8, using AdamW optimizer (initial learning rate \(10^{-4}\), weight decay \(10^{-5}\)). Learning rate scheduling used 5 warmup epochs followed by reduction on plateau (factor 0.5, patience 5, minimum rate \(10^{-6}\)). The combined loss function is \(\mathcal{L} = 0{,}4 \mathcal{L}_{\text{BCE}} + 0{,}6 \mathcal{L}_{\text{Dice}}\). Augmentation includes flips, rotation, affine transformations, color jitter, Gaussian blur. Training adopts early stopping monitoring validation Dice score with patience of 10 epochs, stopping when no improvement occurs. To ensure reproducibility, each architecture was trained with three random seeds (42, 123, 456).

Evaluation Metrics and Statistical Analysis

Our evaluation framework employs five complementary metrics that capture different aspects of segmentation performance, providing a comprehensive appreciation of model capabilities in various clinical scenarios, using information from false negatives (FN), false positives (FP), true positives (TP), and true negatives (TN).

The Dice Similarity Coefficient (Dice score) measures regional overlap between predicted \(P\) and ground truth \(V\) segmentations — for pixel-wise binary segmentation, the Dice score is equivalent to the F1 score. It is obtained with

\[ \frac{2 \mid P \cap V \mid}{\mid P \mid + \mid V \mid}. \]

The Jaccard Index (Intersection over Union, or IoU) provides an alternative overlap measure from

\[ \frac{\mid P \cap V \mid}{\mid P \cup V \mid}. \]

In addition to these main metrics, we calculate sensitivity, which measures the model's ability to detect polyp pixels; specificity, which quantifies the proportion of non-polyp pixels correctly identified; and precision, which evaluates the proportion of positive predictions that are correct.

Results

Standard U-Net achieved the best overall performance (Dice: 83.66\(\pm\)0.52%, IoU: 75.65\(\pm\)0.29%), demonstrating remarkable consistency with the lowest standard deviation among the three seeds. U-Net++ with deep supervision obtained comparable performance (Dice: 83.48\(\pm\)0.89%), presenting the highest sensitivity (89.39\(\pm\)0.88%), indicating superior capability for detecting polyp pixels.

ResU-Net, incorporating residual connections, obtained Dice of 82.43\(\pm\)0.52% and stood out with the highest specificity (97.52\(\pm\)0.28%), indicating excellent capability to reject false detections. Standard U-Net++ and Attention U-Net presented similar performance (Dice: 82.24%), with Attention U-Net exhibiting slightly lower specificity (96.82%), suggesting that attention mechanisms may occasionally compromise non-polyp tissue discrimination.

All architectures maintained specificity above 96.8%, essential for clinical applications where false positives reduce endoscopist confidence. Standard U-Net also achieved the best precision (84.76\(\pm\)0.27%), confirming its robustness. Results reveal that simpler architectures can offer competitive performance with greater consistency, questioning the cost-benefit of complex architectural modifications for this specific domain.

Comparative Performance Analysis by Context

Analysis of architecture performance in the three distinct contexts (training, validation, and test) presented in the figure below reveals fundamental patterns about model generalization behavior. All architectures present a clear performance hierarchy, confirming the progressive task difficulty and adequate methodology.

Standard U-Net demonstrates excellent consistency between validation and test, with minimal gap (\(\sim\)1.5 pp.), indicating robust generalization capability. Its low variability between seeds, visible in compact boxplots, suggests stable and predictable training — a desirable characteristic for clinical applications where reproducibility is critical.

U-Net++ with deep supervision presents similar behavior, maintaining good consistency and reduced gap between validation and test. The multi-scale supervision strategy effectively contributes to training stabilization, resulting in elevated sensitivity without significantly sacrificing other metrics. ResU-Net exhibits intermediate variability, with residual connections providing stable training but without decisive advantages in final performance.

Attention U-Net and standard U-Net++ show slightly larger gaps between validation and test, with Attention U-Net occasionally presenting lower specificity. This behavior questions the effectiveness of spatial attention mechanisms for this domain, suggesting that additional selectivity may not be necessary or may even be counterproductive when the object of interest presents great morphological variability.

Qualitative Analysis of Segmentation Results

Visual inspection of predictions, as shown in the figure below, reveals significant and clinically relevant differences between architectures, particularly in challenging cases. The analysis considers five representative cases that expose distinct performance characteristics of the architectures.

The first row shows a pedunculated polyp with right lateral lobe; only U-Net++ and U-Net++ with deep supervision recover this extension, reproducing the complete morphology. U-Net, Attention U-Net, and ResU-Net restrict themselves to the main body; among those covering integrally, the variant with deep supervision is the most consistent, with a discrete peripheral halo.

In the second row, with shiny mucosa, low contrast, and reflections, all architectures exhibit some degree of leakage or halo. U-Net and U-Net++ (standard and with deep supervision) expand the predicted area, but with diffuse contours; Attention U-Net and ResU-Net mitigate leakages but still lose definition in poorly lit edges. None fully reproduces the annotated contour.

The third row presents a small target near the wall and transition to dark lumen. U-Net undersegments; U-Net++ better approximates the shape; U-Net++ with deep supervision stabilizes the mask. Attention U-Net and ResU-Net provide the cleanest predictions, with fewer false positives in the background.

The fourth row represents a favorable case: clean visual field, relatively homogeneous background, and well-defined polyp boundaries. All achieve high accuracy, with only incremental gains. U-Net++ (with and without deep supervision) refines regularity; Attention U-Net avoids leakages; ResU-Net practically coincides with the ground truth.

Finally, the fifth row includes occlusion by instrument and deep shadows. U-Net undersegments at the base; U-Net++ improves continuity but creates an over-segmentation "collar"; U-Net++ with deep supervision recovers volume, rounding the occluded region. Attention U-Net and ResU-Net prove more robust, preserving lesion mass and continuity.

Architectural differences emerge especially in challenging cases. U-Net and U-Net++ are more susceptible to leakages and edge imprecision; deep supervision attenuates such effects modestly and not universally. In the examples, the incidence of false positives is low in complex backgrounds, which is clinically promising; nevertheless, such impressions must be corroborated by quantitative metrics on the complete dataset.

Conclusion

This study compared five U-Net architecture variants for polyp segmentation in colonoscopy. Standard U-Net achieved the best overall performance (Dice: 83.66\(\pm\)0.52%), demonstrating that architectural simplicity and consistency can be advantageous. Its low standard deviation indicates stable and reproducible training, an essential characteristic for clinical applications where predictability is crucial.

U-Net++ with deep supervision obtained comparable performance (Dice: 83.48\(\pm\)0.89%) and higher sensitivity (89.39%), suggesting that multi-scale supervision can improve polyp detection without compromising other metrics. ResU-Net stood out in specificity (97.52%), indicating that residual connections contribute to better non-polyp tissue discrimination, though without decisive advantages in Dice score.

Standard U-Net++ and Attention U-Net presented similar but inferior performance (Dice: 82.24%). The absence of significant gains for Attention U-Net questions the effectiveness of spatial attention mechanisms in this domain, especially considering the additional complexity. Results suggest that polyp characteristics — high morphological variability and low contrast — may not benefit from additional spatial selectivity.

All architectures maintained specificity above 96.8%, essential for clinical acceptance where false positives reduce endoscopist confidence. Primary differences manifest in sensitivity and edge delineation, critical metrics for precise endoscopic interventions. Results highlight the value of well-calibrated fundamental architectures and question the cost-benefit of complex architectural modifications for polyp segmentation.

Future Work

This study establishes promising directions for future research in automated polyp segmentation. First, cross-validation on additional datasets (CVC-ClinicDB, ETIS-Larib, CVC-ColonDB) would strengthen the generalization of findings and reveal possible biases specific to Kvasir-SEG. Evaluation on data from multiple hospital centers, with different endoscopic equipment and acquisition protocols, is essential to verify clinical robustness.

Future work should incorporate computational cost analysis, including number of parameters, inference time, and memory consumption. Such metrics are critical for deployment in real-time diagnostic aid systems during colonoscopy. Comparison with recent transformer-based architectures and hybrid CNN-transformer methods also merits investigation, considering the balance between accuracy and computational efficiency demonstrated by HarDNet-MSEG¹⁷.

Statistical analysis can be expanded through paired significance tests (Student's t-test, Wilcoxon test) to determine if observed differences between architectures are statistically significant. Confidence intervals would complement variance analysis between seeds, providing more robust estimates of model uncertainty.

Alternative training strategies merit systematic exploration. Investigation of different loss functions (Focal Loss, Tversky Loss, Boundary Loss) may improve edge delineation, a persistent challenge observed in qualitative analysis. Ensemble techniques combining predictions from multiple architectures may increase robustness, particularly in challenging cases with low contrast or occlusions.

Finally, prospective clinical studies with endoscopists evaluating automatic systems in real scenarios are fundamental for translational validation. User-centered metrics, including procedure time, adenoma detection rate, and clinical acceptability, should complement purely technical metrics to guide development of clinically useful and safe systems.

Note on AI Use

The authors declare that no generative artificial intelligence tool was used for study conception, code creation, results analysis, illustration generation, or article writing, except for spelling and grammatical review, for which Claude Opus 4.1 was used.

References