The Advanced In The Wild Malware Test has come to an end. This year is the second edition of the test that checks the so-called Remediation Time (the time of reaction to threats and resolve security incident). We measure the so-called Remediation Time for each tested solution in a situation when there is an attempt to neutralize malware, recover from an attack. This allow us to indicate even more accurately the differences between technologies used by developers. We have described in detail the new indicator “Remediation Time” in the previous edition of the test.
Results from the test of security software in March 2023
In March 2023 we used 567 malware samples to test threats in the wild originating from the Internet. We have checked the following solutions:
For home and small office:
- Acronis Cyber Protect Home Office
- Avast Free Antivirus
- Avira Antivirus Pro
- Bitdefender Antivirus Free
- F-Secure
- Malwarebytes Premium
- Microsoft Defender
- Webroot Antivirus
- Xcitium Internet Security
Solutions for business and government institutions:
- Emsisoft Business Security
- Malwarebytes Endpoint Protection
- Xcitium ZeroThreat Advanced
Test of protection against Internet threats in numbers
We have prepared the following summary based on the telemetry data from the test:
- 12 protection solutions took part in the test.
- We used 567 unique URLs with malware in total.
- Websites encrypted with HTTPS (in theory – safe) contained 72 malware samples.
- 495 malware samples were hosted with HTTP.
- During the analysis, each malware sample took on average 21 potentially harmful actions in Windows 10.
- Most malware originated from servers located in USA, Netherlands, Russia.
- The following domains were mostly used to host malware: .com (151), .org (101), .su (93)
- The average detection of URLs or malicious files before launch (the PRE-Launch level) is 72%.
- The average detection of malware after launch (the POST-Launch level) is 28%.
- The average result of blocking malware by all developers is 99%.
HTTPS can be used to spread malware. Remember that the SSL certificate and so-called padlock next to the URL do not guarantee the security of the website!
Methodology
The detailed methodology is systematically developed by us. We need to take into account new trends in the cybersecurity and system protection industry which is why we provide links to relevant web pages:
- How do we acquire malware for testing, and how do we classify it?
- How do we collect logs from Windows and protection software?
- How do we automate our tests?
We are a member of the AMTSO group, so you can be sure that our testing tools and processes comply with international guidelines, and are respected by developers.
Test results
Key technical data and results are available at the RECENT RESULTS website, along with a threat landscape which has been prepared on the basis of telemetry data from the test.
In addition, from now on, we will systematically publish dedicated developer’s pages that will contain more detailed data from the test. The list will be regularly updated.
Conclusions of the test
Note that the so-called Remediation Time is an individual-measured value expressed in seconds, and means the time of reaction to a malware sample. In our tests, we need to configure a product so that it automatically makes the best possible decisions – usually by moving threats to quarantine or blocking the connection through the firewall along with saving logs.
In larger organizations employing security specialists, the Remediation Time can be significantly greater due to human decision-making, so the less time elapses from the intrusion of malware into the system until it is noticed, and the incident is resolved, the better. From the end user’s point of view, the most important thing is to detect suspicious activity. However, from an administrator’s perspective, the most important thing is to log the threat and attack telemetry data in the product console to quickly respond to a security incident.