Evaluating Machine Learning Algorithms Using Decision-based Adversarial Attacks in Industrial Control System

Evaluating Machine Learning Algorithms Using Decision-based Adversarial Attacks in Industrial Control System

Narges Babadi1

1) Department of Electrical Engineering, Amirkabir University, Tehran, Iran; Email:

IOI Code : XDHZ-FCABF
Publication : 3 rd Conference on Mechanical, Electrical and Computer Engineering (mecconf.com)
Abstract :
    A growing number of Intrusion Detection Systems (IDSs) are powered by Artificial Intelligence (AI) techniques such as Machine Learning (ML) algorithms to classify benign and malicious data points in order to detect cyber attacks. However, currently, ML algorithms are new subjects to cyber attacks in the both training phase and/or test phase. These attacks called Adversarial Machine Learning (AML) attacks try to misclassify data points and interrupt the operation. The use of IDSs in the Industrial Control System (ICS) makes the applicants of ICS more vulnerable and as a result, it can cause major consequences for the operation as well as its users. AML attacks can be classified into two different categories: White-box and Black-box attacks. It is agreed that Black-box attacks are more practical and similar to real-world situations. Decision-based attacks are the most critical ones that need no knowledge related to either datasets or the parameters of the ML model. This paper discusses how performing adversarial examples by decision boundary attacks and HopSkipJump Attack, two class of decision-based attacks, can affect supervised ML models. To this aim, a power system dataset was used to classify events with the use of Random Forest, Adaboost, Long Short Term Memory (LSTM), and a type of Decision tree classifier under the decision-based attacks. In the test phase, the adversarial examples had been applied. Subsequently, it is concluded that the LSTM is the most robust algorithm under decision-based attacks since the accuracy of this classifier is decreased from 11% to 13% in different circumstances while other classifiers are dropped significantly.

Keywords : adversarial machine learning (aml) industrial control system (ics) intrusion detection system (ids) decision based attacks machine learning (ml)