Many grid operators are using specialized intrusion detection systems (IDSs) for their OT networks. These IDSs have become a key tool to effectively monitor OT systems.
But for grid operators it is often difficult to know how well these IDSs really work. In pilot projects, grid operators can test if the systems give a good view into the OT systems, and if they do not generate to many false alarms. The pilot periods however do not usually include any real cybersecurity incidents. So, operators cannot determine how well the IDSs detect such incidents.
Therefore, we tested 10 of these IDSs in our test lab in 2024 to determine if they can reliably detect incidents and vulnerabilities.
For the test, we injected many different test cases, representing different attack steps, into real traffic captures from our members. Captures from both a central OT system and substation were used. We then fed the capture with the test cases to the IDSs, so that we could determine for different types of incidents how well the sensors detected them.
This report provides a detailed explanation of the methodology used for the testing process.