We have completed the 19th edition of the Advanced in the Wild Malware Test recently. We have included two new configurations in the analysis: Malwarebytes Nebula for companies with EDR (Endpoint Detection and Response) and Microsoft Defender with the SmartScreen technology enabled. This is the answer to your previous comments. Our aim is to provide this kind of analysis to find effective protection by different developers in the long term. We have used 2185 malware samples to check the protection against threats in the wild.
We always test the latest software. We intentionally do not provide version number because the testing system updates them once a day:
- Avast Free Antivirus
- Avira Antivirus Pro
- Emsisoft Business Security (for business)
- G Data Total Security
- Malwarebytes Premium
- Malwarebytes Endpoint Protection (for business, new in our test)
- Mks_vir Endpoint Security (for business)
- Microsoft Defender (Windows 10 with SmartScreen enabled)
- CatchPulse (formerly SecureAPlus Pro, check why)
- Webroot Antivirus
- Xcitium – ZeroThreat Advanced (former name after rebranding: Comodo Advanced Endpoint Protection)
- Xcitium Internet Security (formerly Comodo Internet Security)
Unlike other testing laboratories, we try to make our tests fully transparent, so we provide a complete list of virus samples and never share unnecessary technical information. Key information and all test results are available at https://avlab.pl/en/recent-results.
Additional configuration
We usually use the default settings. If you want to learn more, please check out our methodology.
In July 2022, we chose the default settings for Microsoft Defender, that is with SmartScreen enabled. Because the purpose of the test is to check the effectiveness of the protection software, we usually disable the SmartScreen technology during testing to prevent a situation where 100% threats are blocked without the participation of antivirus software.
SmartScreen is designed to block files downloaded from the Internet and prevent executable files from running in order to protect users against known threats and the most likely scenario of data being exposed to malicious software.
The purpose of the test is to do something opposite – to check the response of antivirus software. In the previous month, the result for Microsoft Defender with SmartScreen enabled was 98,6% for blocked threats.
In this edition of the test, the SmartScreen technology helped to achieve 100% detection. Do readers expect this from our tests? Thus, to fully test Microsoft Defender, we disable SmartScreen. However, there are specific scenarios with SmartScreen enabled that are better considered in manual tests, for example:
- attempt to bypass SmartScreen and Mark of The Web by providing malware with the .ISO extension (as an attachment or download link)
- attempt to bypass SmartScreen and MOTW by embedding malware with the .VHDX or .VHD extensions (virtual machine)
- as described above but using compressed files, archives: .ARJ, .GZIP, .RAR, and others.
Reference: https://attack.mitre.org/techniques/T1553/005/
Finally, disabling SmartScreen in tests has the same effect as bypassing MOTW. However, this time SmartScreen for Microsoft Defender was enabled.
Results
After suggestion from our community, we have changed naming of the protection levels since the last edition of the test. Now there are two levels of protection:
Level 1 and 2 are combined into a single level: Pre-launch
Level 3 remains the same with a new name: Post-launch
Reclassification of levels is for marketing purposes to simplify the methodology and make it easier to understand the tests that users have suggested. Thank you!
So the new classification concerns detecting malware samples:
- before they are launched in the system (Pre-launch)
- and those blocked after launch (Post-launch). This is the most dangerous situation, but our experience shows that such cases require tests in the field.
There were not any surprises in the July edition of the test. All developers have proven the superiority of their solutions over the set of malware samples. Security solutions often include machine learning for detecting threats without using signatures (post-launch level), artificial intelligence that correlates events from the entire system, scanners, and sandboxes in the cloud, etc. When it comes to Malwarebytes Nebula – the solution for business, we have the EDR module. In general, Endpoint Detection and Response is designed to facilitate decision-making based on threat indicators logged in the console. This functionality allows better management of incidents, search for trace of intrusion and drive-by attack at every endpoint. Even so, some EDRs may show more general alerts which will result in overloading an analyst with false positives, that is why a high number of alerts is not always desirable. Collecting such data or configuring a product to be more restrictive is a good practice if the organization’s security is handled by a dedicated group of experts. However, we will not cover EDR here – we plan to prepare such test this year. Stay tuned!
Interesting fact is that Comodo has changed the name to Xcitium – we wrote about it here. The reason for the change is rebranding.
The leaders in malware blocking at the Pre-Launch stage include G Data Total Security and mks_vir Endpoint Protection. That means almost 100% of blocked threats before their launch (this happens 30 seconds after downloading a file in a browser). Other solutions have similar level of threat detection. The only differences are whether threats are blocked immediately after launch or after a deep analysis.
In conclusion, nearly all tested products achieved the maximum score of 100% threats blocked in July 2022 (except Avira and Webroot). Based on the logs that we share with developers, our system summarized tens of thousands of test data (12 products times 2185 threats finally gives 26220 test cases) and returned the result of 100% protection for almost each software.
As usual, the differences are minor, so we encourage you to perform own analysis when choosing software. The effectives of protection is very important, but also other aspects such as warranty, technical support, friendly interface, performance, online banking protection, management console, EDR module, etc.