p-LDM: Partitioned latent diffusion model for memory-efficient 3D medical image translation

초록

Translation between various imaging modalities in the medical field is essential to compensate for the limitations of individual techniques and enable more accurate diagnosis and examination. Recently, diffusion models have been utilized in medical research due to their stability and superior image-generation quality compared to generative adversarial network (GAN)-based methods. However, applying these models to 3D image translation demands significant graphics processing unit (GPU) memory for computation. Various approaches have been introduced to optimize memory usage by generating individual slices in a 3D image using 2D conditional diffusion models. However, the 2D conditional diffusion models may lead to training instability by applying conditioning requirements to all slices within a single model. This paper proposes a memory-efficient partitioned latent diffusion model (p-LDM) to address these issues. It divides 3D images into multiple partitions containing several slices and translates each partition within the latent space to optimize GPU memory usage. Additionally, our method introduces PART-SPADE and PART-DIFFUSION modules for cyclic learning across partitions, maintaining global consistency while capturing structural differences between them. This approach enhances training stability by minimizing conditioning requirements compared to existing 2D conditional diffusion models. The 3D-to-2D latent projection technique is also incorporated to optimize GPU usage in the diffusion model. Comparative experiments demonstrated that the proposed method achieves performance comparable to state-of-the-art 3D diffusion models while significantly reducing GPU memory consumption by 73%.

키워드