Abstract: Objective By combining the complementary information from spinal CT and MR multimodal medical images, and utilizing detailed features of both bone and soft tissues, to improve the accuracy of identification and enhance the segmentation precision of spinal medical images by soft tissues, thereby providing a more comprehensive assessment of spinal lesions. Methods This paper proposed a multimodal medical image fusion network model for the fusion of spinal CT and MR images and a semi-supervised segmentation network model for the segmentation tasks based on the fused images. The multimodal fusion network retained the shared features of different modalities through a shared encoder, with a basic part extracting global features and a detail part focusing on local details. A dual-network architecture was employed to the segmentation network, which was corrected and constrained by a contrastive difference review module and a dynamic competitive pseudo-label generation module when the network was training. Results The proposed fusion network performed well in preserving image information and features, with less high-frequency noise in the fused images. The semi-supervised segmentation network excelled in both the Dice coefficient and Jaccard index, improving the clarity between spinal soft tissues and bone tissues. Conclusion The proposed multimodal medical image fusion network and semi-supervised segmentation network effectively enhance the fusion and segmentation accuracy of spinal images. The introduction of the contrastive difference review and dynamic competitive pseudo-label generation modules further improved the accuracy of the segmentation results, providing clearer and more reliable image information for the assessment of spinal diseases.

Key words: Multimodal, Image fusion, Image segmentation, Semi-supervised, Spine