KD-SAM: Keep-awake and detail-enhanced segment anything model for multi-modality multi-organ medical image segmentation

초록

Medical image segmentation plays a crucial role in computer-aided diagnosis and treatment, which is usually dominated by task-specific models, yet suffering from inadequate generalization and adaptability to unseen data domain and modalities. Recently, the Segment Anything Model (SAM) has emerged as a novel promptable segmentation framework with strong transfer learning capabilities, but adapting it to the medical domain remains challenging since SAM is inherently insufficient to capture local details with its global self-attention-based backbone. In this paper, we propose a novel multi-Organ medical image segmentation framework, KD-SAM, consisting of a keep-awake and detail-enhanced feature encoder, a point(s)/bounding box-based prompt encoder, and a mask decoder. Specifically, KD-SAM encoder equipped with Keep Awake Module can efficiently and effectively unleash its adaptability and transferability to medical images while preserving model generalization. Moreover, a Detail-Aware feature Enhancement module compensates for the limitations of SAM in capturing the fine-grained local details. Thus, a Dynamic Mutual feature Fusion module is introduced to adaptively merge global contextual features with local refined features for balanced feature learning. Benefiting from this, KD-SAM enables flexible and interactive segmentation by using simple points or bounding boxes as task-specific prompts. Experimental results on Synapse, BTCV, Kvasir-Seg and ACDC datasets demonstrate that KD-SAM can efficiently adapt to multiple medical data domain and achieves strong performance on multi-modality multi-organ medical image segmentation tasks. Code is available at https://github.com/ShowUNow/KD-SAM.

키워드