AI-Generated Radiology Report with Supervised Fine-Tuning LLM

초록

Title: AI generative Radiology Report with Supervised Fine Tunning (SFT) LLM Purpose: Recently, the increase in CT and MRI scans has led to an excessive workload for radiologists. Artificial Intelligence (AI) has tremendous potential to assist radiologists in their daily clinical routines. However, there often lacks a seamless, standardized, and time-efficient method to integrate AI into radiology workflows. This limitation restricts the full potential of this technology. To address this issue, we have developed a new reporting system that can automatically fill structured reports with results provided by SFT system. Methods: To create the necessary Conclusion (input) - Findings (output) data pairs, we generated 50,000 synthetic data points using a developed template. These data were used for pseudo labeling and training based on GPT-4o. We fine-tuned the pretrained Llama3-8B model using the pseudo-labeled data from GPT-4o. When comparing the performance of the SFT (Supervised Fine-Tuned) GPT-6B model and the prompted GPT-175B model, no significant difference was observed. For fast inference and low memory usage, we optimized the LLM (Large Language Model) training and inference processes by replacing the commonly used Attention Mask with a Causal Mask. This approach, which restricts the model to only reference tokens from the previous sequence, was applied to the Transformer's attention mechanism during training. Additionally, we employed Low-Rank Adaptation (LoRA), a method designed to efficiently fine-tune pretrained LLMs. Instead of updating all weights, LoRA fine-tunes the model using new parameters (matrices) to make the process more efficient. Results: Systemic radiology reports with the SFT LLM system could be created in significantly less time than free-text reports and conventional structured reports (mean reporting times: 56 s vs. 125 s and 90.8 s, respectively; both p?<?0.001). Reports created with the SFT LLM were rated significantly higher quality o