PEFT QLORA-Based Fine-Tuning of Foundation Models for Vitals Estimation Using PPG and ECG-Based Medical IoT Data: A Feasibility Study

초록

This work investigates the feasibility of leveraging large language models (LLMs) and vision-language models (VLMs), commonly associated with artificial general intelligence (AGI), for artificial narrow intelligence (ANI) tasks such as vital sign estimation using medical Internet of Things (IoT) data. Parameter efficient fine-tuning (PEFT)-based quantized lowrank adaptation method (QLoRA) is utilized to fine-tune three foundational models, namely GPT-2, Cohere command-R chat model, and Idefics-9B, to estimate two vital signs: vascular age and heart rate, from photoplethysmography (PPG) and electrocardiography (ECG) signals. Two custom PPG and ECG datasets are used to estimate vascular age, while the public PhysioNet Pulse Transit Time (PTT) PPG dataset is used for heart rate estimation. All datasets underwent baseline correction, denoising, segmentation, and augmentation. The models are evaluated under two settings: signal-only and signal-plus-demographics, with demographic features consistently improving performance. Idefics-9B achieves a mean absolute error (MAE) of 2.768 for ECG-based vascular age estimation, while GPT-2 obtains an MAE of 10.921 for PPG-based vascular age estimation. For heart rate estimation on the PPG-PTT dataset, Idefics-9B achieves mean absolute errors (MAEs) of 8.53 in the signal-only setting and 7.64 with demographics, outperforming GPT-2. Although these LLMs/VLMs do not yet match state-of-the-art machine learning and deep learning methods for vital estimation, their modest performance with minimal fine-tuning on small, domainspecific datasets indicates that there is some room to adapt LLMs/VLMs for specialized downstream tasks. Thus, domainspecific training on larger datasets remains critical. Looking forward, quantized and pruned LLMs/VLMs may enable realtime, on-device inference on smartphones and wearables, thereby supporting agentic AI workflows for proactive, context-aware health diagnostics. © 2025 IEEE.

키워드