ViT4LPA 2.0: A Smart Meter Pre-Trained Vision Transformer for Aggregated-Level Load Disaggregation

초록

Pre-trained neural networks have significantly advanced machine learning, enhancing performance and reducing reliance on labeled datasets. In this study, we analyze a pre-trained neural network trained with residential smart meter load data and its transferability for aggregated-level load profile analysis. This study introduces ViT4LPA 2.0, a pre-trained hierarchical vision transformer (H-ViT) tailored for load profile analysis. ViT4LPA 2.0 is adapted for heating, ventilation and air conditioning (HVAC) load disaggregation at an aggregated-level, leveraging knowledge from individual smart meter load profiles. The H-ViT model employs a window self-attention mechanism to capture both local and global features. Pre-training is performed using masked image modeling on 2,000 smart meter load profiles, followed by domain adaptation and fine-tuning with limited labeled data. Extensive simulations demonstrate the model's effectiveness, achieving approximately 5% daily accumulated error on aggregated-level load disaggregation with less than 15% HVAC consumption visibility in residential households. The model significantly outperforms state-of-the-art benchmarks, showing a 20-35% performance improvement. Additionally, a comprehensive sensitivity analysis and interpretability study provide insights into hyperparameter selection and the model's decision-making process.

키워드