Evaluating the Generalizability of Machine Learning Models for Detecting Power Grid Measurement Data Manipulation Under Unseen Loading Conditions
Evaluating the Generalizability of Machine Learning Models for Detecting Power Grid Measurement Data Manipulation Under Unseen Loading Conditions
MahboubehMoradtajari1
1) M.Sc. in Computer Engineering (Algorithms and Computation); Lecturer, Department of Computer Engineering, University of National Skills, Tehran, Iran.
IOI Code
:
ACCA-ACAGFK
Publication :
4th International Congress on Science, engineering & New Technologies(secongress.com)
Abstract :
With the development of intelligent measurement and monitoring systems in power grids, the accuracy and integrity of measurement data have become increasingly important. One of the main challenges in applying machine learning models to detect data manipulation is maintaining their performance under conditions not encountered during training. This study investigates the generalizability of machine learning models for detecting synthetic manipulation of measurement data under unseen loading conditions. To this end, data were generated using the IEEE standard 33-bus network in the pandapower software. The dataset covered five loading levels, ranging from 0.8 to 1.2, with 15 baseline scenarios at each level. For each baseline scenario, one clean sample and two manipulated samples with severity levels of 0.03 and 0.08 were generated, resulting in 225 samples organized into 75 source groups. The performance of three algorithms—Logistic Regression, Random Forest, and Extra Trees—was then evaluated using the Leave-One-Load-Out (LOLO) method. The results showed that Logistic Regression with channel-only features achieved a mean balanced accuracy (BA) of 0.8467 and a false positive rate (FPR) of 0.0267. In contrast, at the selected decision thresholds, both tree-based algorithms classified all test samples as manipulated. These findings highlight the importance of evaluating model generalizability, selecting decision thresholds appropriately, and considering errors in detecting both clean and manipulated samples when assessing model performance.
Keywords : power grids measurement data manipulation machine learning generalizability unseen loading conditions anomaly detection