Independent assessment of attack visibility, telemetry, and incident response capabilities
We present the results of EDR and XDR solution tests, focusing on attack visibility, telemetry quality, event correlation, and analysis and incident response capabilities.
This year’s edition was designed for organizations, SOC teams, security administrators, and cybersecurity solution vendors who want to verify not only the effectiveness of threat detection but also the quality of data available during post-breach investigations.
Why is detection no longer enough?
Modern security solutions are increasingly achieving high threat detection scores. In practice, this means that differences between products are increasingly less likely to be based solely on the detection of an attack.
However, what happens next is crucial:
- How detailed is the telemetry?
- Is it possible to reconstruct the full attack sequence?
- Does the analyst see the relationships between events?
- Is network, process, and user context data available?
- How quickly can an incident be analyzed?
These areas are the focus of this year’s test.
Scope of test
As part of the test, we analyzed the products’ ability to monitor and correlate activity occurring during real-world attack scenarios.
The techniques tested included:
- phishing and running malicious files
- powershell and scripts
- LOLBins (Living-off-the-Land Binaries)
- scheduled tasks
- remote code execution
- lateral movement
- SMB and WMI
- persistence mechanisms
- data exfiltration
- multi-stage attack scenarios
What did we evaluate?
- Alert quality
- Detection context
- Identification of attack techniques
- Mapping to MITRE ATT&CK
- Parent-child processes and relationships
- Command lines
- File operations
- Registry changes
- Network communication
- User context
- Correlation within a host
- Correlation between hosts
- Attack chain reconstruction
- Incident progression visualization
- Timeline analysis
- Host isolation
- Remediation
- IOC management
- Investigation support
- Threat hunting capabilities
What questions does our test answer?
The purpose of this analysis is to verify whether a given EDR-XDR solution provides security analysts with the information necessary to detect, analyze, and handle security incidents.
1.
2.
3.
4.
5.
Unlike traditional testing, which focuses solely on detection effectiveness, we also assess the quality of information available after a threat is detected. This allows us to determine whether the solution supports analysts in understanding the incident, identifying the source of the threat, and taking corrective action.
Levels of Certificates
LEVEL 1
LEVEL 2
Most important observations from the test
The overall level of threat detection was high. Most tested products successfully identified simulated adversary actions at various stages of the attack.
The biggest differences between the solutions were:
completeness of telemetry
RAW data availability
quality of correlation between hosts
possibilities of attack reconstruction
level of analysis automation
response and investigation functions
In some cases, solutions effectively blocked threats before they were launched or the next stages of the attack were executed. However, depending on the product architecture, this could result in limited availability of telemetry regarding blocked activity, which hindered the full reconstruction of the attack sequence and analysis of some events from a forensic perspective.
At the same time, most of the tested solutions provided a high level of visibility into process activity, system events, and user actions. In many cases, detailed information was available regarding command lines, parent-child relationships, network communications, and mapping to MITRE ATT&CK techniques, significantly facilitating incident analysis.
The development of features supporting SOC and Incident Response teams is also noteworthy. Many platforms offered extensive forensic mechanisms, attack chain visualization, IOC management, remediation, and integration with external security systems.
Despite differences in telemetry and event correlation, most of the evaluated products provided sufficient information to identify the source of the incident, understand the attack sequence, and implement mitigation measures.
Tested Solutions
Detailed reports for individual vendors include: test results, telemetry examples, event correlation analysis, Incident Response evaluation, mapping to MITRE ATT&CK, and AVLab analyst commentary.
Methodology
All scenarios were conducted in a controlled laboratory environment using real-world techniques used by modern adversaries.
Each scenario was previously verified and executed using an identical procedure for all tested products.
The goal of the test was not only to verify the effectiveness of the protection, but also to assess the quality of information available to analysts after an incident.
Since 2012, we have been conducting independent security testing of endpoint protection, EDR, and XDR solutions.
We are a participant in the Microsoft Virus Initiative (MVI), which enables collaboration with Microsoft and access to technical information relevant to security testing.
We operate in accordance with AMTSO standards, which promote transparency, repeatability, and objectivity in security testing. We employ recognized testing practices, including transparency of methodology, verifiability of results, and collaboration with vendors during test preparation and execution.
