WP-049-2021 Rhebo Industrial Protector test results

This report presents the test results for the Rhebo Industiral Protector security sensors for a laboratory tests on substations traffic.

Our members have shown an increasing interest in deploying OT security monitoring solutions, which provide specialized intrusion detection capabilities for OT systems. The solutions see real-time network communications and monitor for deviations from known baselines or matches to attack signatures.

We first assessed these solutions in 2017, when they were mostly new and unknown. They were also quite expensive to deploy sensors in each high-voltage substation, which would require installing tens to hundreds of sensors. Therefore, most operators deployed them in their central systems, such as their SCADA system. Because of this, our assessment focused on the detection capabilities for protocols used between substations and central systems, such as IEC 104.

Lately, the cost of OT security monitoring solutions has been decreasing, making them more suitable for deployments in substations. Due to this, we revisited these solutions in 2020, this time focusing on the detection capabilities for protocols used between IEDs and gateways, such as IEC 61850.

We divided the new assessment in two stages:

  1. A market survey based on questionnaires, which allowed us to assess new features in previously assessed solutions, solutions that we did not assess before and how they all support the IEC 61850 protocol. It turned out that all solutions are quite balanced on paper.
  2. A test of the solutions in a virtualized environment at ENCS’ laboratory and in our members’ substations. The results allowed us to validate the claims made by vendors in the first stage and to understand how the features work in practice.

WP-048-2021 Omicron StationGuard test results

This report presents the test results for the Omicron StationGuard security sensors for a laboratory tests on substations traffic.

Our members have shown an increasing interest in deploying OT security monitoring solutions, which provide specialized intrusion detection capabilities for OT systems. The solutions see real-time network communications and monitor for deviations from known baselines or matches to attack signatures.

We first assessed these solutions in 2017, when they were mostly new and unknown. They were also quite expensive to deploy sensors in each high-voltage substation, which would require installing tens to hundreds of sensors. Therefore, most operators deployed them in their central systems, such as their SCADA system. Because of this, our assessment focused on the detection capabilities for protocols used between substations and central systems, such as IEC 104.

Lately, the cost of OT security monitoring solutions has been decreasing, making them more suitable for deployments in substations. Due to this, we revisited these solutions in 2020, this time focusing on the detection capabilities for protocols used between IEDs and gateways, such as IEC 61850.

We divided the new assessment in two stages:

  1. A market survey based on questionnaires, which allowed us to assess new features in previously assessed solutions, solutions that we did not assess before and how they all support the IEC 61850 protocol. It turned out that all solutions are quite balanced on paper.
  2. A test of the solutions in a virtualized environment at ENCS’ laboratory and in our members’ substations. The results allowed us to validate the claims made by vendors in the first stage and to understand how the features work in practice.

WP-047-2021 Nozomi Guardian test results

This report presents the test results for the Nozomi Guardian security sensors for a laboratory tests on substations traffic.

Our members have shown an increasing interest in deploying OT security monitoring solutions, which provide specialized intrusion detection capabilities for OT systems. The solutions see real-time network communications and monitor for deviations from known baselines or matches to attack signatures.

We first assessed these solutions in 2017, when they were mostly new and unknown. They were also quite expensive to deploy sensors in each high-voltage substation, which would require installing tens to hundreds of sensors. Therefore, most operators deployed them in their central systems, such as their SCADA system. Because of this, our assessment focused on the detection capabilities for protocols used between substations and central systems, such as IEC 104.

Lately, the cost of OT security monitoring solutions has been decreasing, making them more suitable for deployments in substations. Due to this, we revisited these solutions in 2020, this time focusing on the detection capabilities for protocols used between IEDs and gateways, such as IEC 61850.

We divided the new assessment in two stages:

  1. A market survey based on questionnaires, which allowed us to assess new features in previously assessed solutions, solutions that we did not assess before and how they all support the IEC 61850 protocol. It turned out that all solutions are quite balanced on paper.
  2. A test of the solutions in a virtualized environment at ENCS’ laboratory and in our members’ substations. The results allowed us to validate the claims made by vendors in the first stage and to understand how the features work in practice.

WP-046-2021 Forescout SilentDefense test results

This report presents the test results for the Forescout SilentDefense security sensors for a laboratory tests on substations traffic and two tests in members' substations.

Our members have shown an increasing interest in deploying OT security monitoring solutions, which provide specialized intrusion detection capabilities for OT systems. The solutions see real-time network communications and monitor for deviations from known baselines or matches to attack signatures.

We first assessed these solutions in 2017, when they were mostly new and unknown. They were also quite expensive to deploy sensors in each high-voltage substation, which would require installing tens to hundreds of sensors. Therefore, most operators deployed them in their central systems, such as their SCADA system. Because of this, our assessment focused on the detection capabilities for protocols used between substations and central systems, such as IEC 104.

Lately, the cost of OT security monitoring solutions has been decreasing, making them more suitable for deployments in substations. Due to this, we revisited these solutions in 2020, this time focusing on the detection capabilities for protocols used between IEDs and gateways, such as IEC 61850.

We divided the new assessment in two stages:

  1. A market survey based on questionnaires, which allowed us to assess new features in previously assessed solutions, solutions that we did not assess before and how they all support the IEC 61850 protocol. It turned out that all solutions are quite balanced on paper.
  2. A test of the solutions in a virtualized environment at ENCS’ laboratory and in our members’ substations. The results allowed us to validate the claims made by vendors in the first stage and to understand how the features work in practice.

WP-045-2021 Cisco Cyber Vision test results

This report presents the test results for the Cisco Cyber Vision security sensors for a laboratory tests on substations traffic.

Our members have shown an increasing interest in deploying OT security monitoring solutions, which provide specialized intrusion detection capabilities for OT systems. The solutions see real-time network communications and monitor for deviations from known baselines or matches to attack signatures.

We first assessed these solutions in 2017, when they were mostly new and unknown. They were also quite expensive to deploy sensors in each high-voltage substation, which would require installing tens to hundreds of sensors. Therefore, most operators deployed them in their central systems, such as their SCADA system. Because of this, our assessment focused on the detection capabilities for protocols used between substations and central systems, such as IEC 104.

Lately, the cost of OT security monitoring solutions has been decreasing, making them more suitable for deployments in substations. Due to this, we revisited these solutions in 2020, this time focusing on the detection capabilities for protocols used between IEDs and gateways, such as IEC 61850.

We divided the new assessment in two stages:

  1. A market survey based on questionnaires, which allowed us to assess new features in previously assessed solutions, solutions that we did not assess before and how they all support the IEC 61850 protocol. It turned out that all solutions are quite balanced on paper.
  2. A test of the solutions in a virtualized environment at ENCS’ laboratory and in our members’ substations. The results allowed us to validate the claims made by vendors in the first stage and to understand how the features work in practice.

 


WP-042-2021: Response to the ACER consultation on the framework guidelines

On April 31, ACER published draft framework guidelines for the network code on cyber-security as part of a public consultation. The framework guidelines set the general principles the network code should meet. They build on the previous work from the Smart Grid Task Force Expert Group 2 and the informal drafting team from ENTSO-E and the four DSO associations (CEDEC, E.DSO, Eurelectric, and GEODE).

The framework guidelines help to clarify the governance for the network code and give some new ideas for its rules. But the guidelines makes different choices from the recommendations of the informal drafting team in several major areas. In some of these choices, we think that the framework guidelines are overlooking practical considerations of the informal drafting team. We think these choices will lead to substantial extra costs, not in proportion to the gains in security.

We therefore think the network code should aim for more rules that are more practical to implement. In particular, it should:

  • determine the scope of the advanced measures through processes
  • set lower minimum security requirements for important undertakings
  • require essential undertakings to have a management system
  • set the minimum requirements in terms of security controls
  • allow alternative assurance methods besides product certification
  • require SOC functions only for essential processes

WP-035-2020: Proposed strategy for OT component certification

The European Commission has made certification of products, services and processes one of the pillars of their cybersecurity strategy. In the 2019 Cybersecurity Act, ENISA was tasked with developing a cybersecurity certification framework. In a rolling work program ENISA will develop harmonized, European certification schemes for products, services, and processes. In 2020, the Joint Research Center (JRC) published requirements for an Industrial Automation & Control Systems Components Cybersecurity Certification Scheme (ICCS), which is also expected to be turned into a candidate scheme.

Given this development, ENCS recommends that its members prepare for a certification meeting the requirements in the JRC ICCS. This whitepaper outlines the proposed strategy.


WP-033-2020: Test results for OT security sensors monitoring inside substations

Many grid operators are using specialized security sensors to monitor their central OT networks. The sensors can automatically learn what normal network communication looks like, and then alert on deviations from normal to detect security incidents. ENCS evaluated such sensors in the 2017 member project on security monitoring . Since then, these sensors have become an important tool for most SOC and CSIRT teams working on OT systems.

Vendors of these sensors are also offering industrial PCs or embedded sensors that could be placed in field locations. The prices of such sensors have dropped considerably since 2017. So, it is becoming possible to deploy sensors in the internal networks of high voltage substations. ENCS has therefore started a project to evaluate the sensors for monitoring inside substations.

A market survey was conducted based on questionnaires to determine their capabilities. Unfortunately, it turns out the sensors are hard to evaluate on paper. Based on the questionnaires, they can all detect all vulnerabilities and all incidents. We found no real differences between sensors in the market survey.

This raises the question of how good the sensors are in practice. Can they detect all the vulnerabilities and incidents that vendors claim they do? And can they present these in a meaningful way to analysts?

To answer this questions, ENCS tested five sensors in our lab. The sensors were trained on substation traffic collected from members. Different test cases were injected in the traffic to see if the sensors detect them.


WP-034-2020: Update on the revised NIS directive

This whitepaper gives an update to ENCS members on the revised NIS directive. On 16 December 2020, the European Commission adopted a proposal for a revised NIS directive: the directive on measures for a high common level of cybersecurity across the Union. The directive is meant to repeal the directive concerning measures for a high common level of security of network and information systems across the Union from 2016, also known as the NIS directive . The goal of the revised NIS directive is to achieve a harmonized, high level of cybersecurity across the European Union by incorporating feedback from the most recent consultations and filling some gaps found in the NIS version from 2016. The new directive, which focuses on enabling resilient infrastructure and critical services, is a key component of the Union’s new Cybersecurity Strategy for the Digital Decade. It was released alongside a proposal for a Directive on the Resilience of Critical Entities, which is the successor of the 2008 European Critical Infrastructure Directive.

The most important changes for grid operators seem to be:

  • The scope was extended to include many parties important to grid stability
  • Supervision and enforcement of the implementation is made stricter
  • It may become mandatory to use of products, services, and processes certified under the Cybersecurity Act
  • Provisions are added to integrate the upcoming network code on cybersecurity

WP-030-2020: Distribution automation RTU hardware security test report

This test report gives the results of testing a distribution automation remote terminal unit (RTU) against the hardware security requirements that ENCS has developes in its member project on hardware security. See:

The report gives a good overview of how the requirements can be tested, what vulnerabilities are typically found when RTU vendors first implement them, and how these vulnerabilities can be mitigated.

The test report is classified as TLP:AMBER. It is only shared with employees at ENCS members that need to know its contents for their work on distribution automation security.

To request a copy of the report, please contact info@encs.eu