Adaptive Rectangular and Feature‐Aware Mamba for Nuclei Segmentation in H&E Histopathology Images

ABSTRACT Accurate nuclei segmentation in H and Lightweight Feature‐Aware Mamba (LFAM), which couples a variable‐selection scan with a channel Multilayer Perceptron (MLP) to efficiently process downsampled features while preserving long‐range dependencies in linear time and with a low memory footprint. A hybrid P–Dice loss balances pixel fidelity and region‐level overlap. Extensive experiments demonstrate the superiority of the proposed approach. Results on the publicly available CoNSeP and TNBC datasets show that AFM‐Net outperforms the baseline by 0.122/0.081 in Dice Similarity Coefficient (DSC) and 0.185/0.175 in Aggregated Jaccard Index (AJI), and reduces Hausdorff Distance (HD) by 2.840/2.260, respectively.