Aortic KD former: aortic multiclass segmentation using SegFormer via knowledge distillation

Résumé

Abstract In this paper, we investigate the effectiveness of knowledge distillation for semantic segmentation of aortic structures using deep learning models. We employ SegFormer B5 as the teacher model and SegFormer B0 as the student model. Knowledge distillation is applied through three methods: output-based distillation, inner feature maps-based distillation, and integrated distillation, which combines both logits and feature maps. Our results show that while the student model (SegFormer B0) has significantly fewer parameters (3,715,686) compared to the teacher model (SegFormer B5) with 84,597,958 parameters, it achieves performance closely approaching that of the teacher. Among the evaluated techniques, inner feature map distillation yields the best performance, with a mean IoU of 0.9994, a mean Dice score of 0.9549, and a loss of 0.0020, demonstrating its superior capability in transferring knowledge effectively. The study highlights the potential of inner feature map distillation as the most effective method for achieving high performance with reduced model complexity.

Citer ce document

Accès au document

Texte intégral en lecture en ligne, réservé aux abonnés SPHAERO et aux membres de l'institution. Se connecter

Auteur(s)

Statistiques

Consultations : 1

Téléchargements : 0