From the first till the last day of the month of July, we completed another edition of the test of software to protect against threats found on the Internet. The aim of the “Advanced In The Wild Malware Test” is to check protection solutions for Windows 10/11 in the long term in order to identify the best software for securing endpoints at the end of the year.
We want to start the summary of July 2023 by classifying the signatures of malware that have been qualified for the test. Based on the feedback received from the antivirus engine of our technology partner, MKS_VIR sp. z o.o., we distinguish the following types of threats:
- Trojan.Ransom.Loki
- DeepScan:Generic.KillMBR
- DeepScan:Generic.Keylogger
- Generic.BAT.Downloader
- Generic:Remcos
- Generic.Ransom.PyCrypter
- Trojan.MSILZilla
- Trojan.Heur2
- Trojan.Lazy
- Trojan.Mint.Zard
The digital threat landscape is dynamic, that is why many more unique threat samples circulated on the Internet in July, although they are often the same malware families with a unique source code.
Post-incident response and recovery indicator
In our test, in addition to the effectiveness of antivirus protection, we check the so-called response time to a threat and recovery after incident (Remediation Time).
We measure the Remediation Time when there is an attempt to neutralize malware and recover after an attack. Thanks to this, we can more accurately point the differences between technologies used by developers.
Our tests comply with the guidelines of the Anti-Malware Testing Standards Organization. Details about the test are available at this website as well as in our methodology.
How do we evaluate the tested solutions in 3 steps?
In a nutshell, the Advanced In The Wild Malware Test covers 3 big aspects that follow each other and combine into a whole:
1. Selecting samples for the test
We collect malware in the form of real URLs from the Internet on an ongoing basis. We use a wide spectrum of viruses from various sources, and these are public feeds and custom honeypots. As a result, our tests cover the most up-to-date and diverse set of threats.
2. Analyzing logs of malware samples and tested solutions
The analyzed samples in Windows are subject to thorough verification based on hundreds of rules: the most common techniques used by malware creators. We monitor system processes, network connections, Windows registry, and other changes made to the operating and file system to learn more about whether and what malware was doing during the analysis.
In addition to changes made by malware, we monitor whether and how the protection solution responds to threats. We also measure the time of recovery after an incident (Remediation Time).
3. Simulating a real-world scenario
In the test, we simulate a real scenario of threat intrusion into the system through an URL via a browser. It can be a webpage prepared by a scammer or a link sent to a victim via messenger that is then opened in the system browser.
Results in July 2023
In July 2023, we used 468 unique malware samples from the Internet.
Tested solutions to install at home and in a small office:
- Avast Free Antivirus
- Eset Smart Security
- F-Secure Total
- G Data Total Security
- Kaspersky Plus
- Malwarebytes Premium
- Norton Antivirus Plus
- Quick Heal Total Security
- Trend Micro Maximum Security
- Webroot Antivirus
- Xcitium Internet Security
- ZoneAlarm Extreme Security NextGen
Solutions for business and government institutions:
- Emsisoft Business Security
- Malwarebytes Endpoint Protection
- Xcitium ZeroThreat Advanced
Key technical data and results are available at the RECENT RESULTS, along with a threat landscape that has been prepared on the basis of telemetry data from the test in July 2023.
In addition, from now on, we will systematically publish dedicated developer’s pages that will contain more detailed data from the test.
Here are some of them:
The list will be regularly updated. Developers who wish to cooperate with us please contact us via AMTSO or directly at our contact webpage.
Protection test in numbers
We have prepared the following summary based on the telemetry data from the test in July 2023:
- 15 protection solutions took part in the test.
- We used 468 unique URLs with malware in total.
- Websites encrypted with HTTPS (in theory – safe) contained 102 malware samples.
- 366 malware samples were hosted with HTTP.
- During the analysis, each malware sample took on average 35 potentially harmful actions in Windows 10.
- Most malware originated from servers located in the Netherlands, USA, Germany.
- The following domains were mostly used to host malware: .org, .com, .info
- The average detection of URLs or malicious files before launch (the PRE-Launch level) is 72%.
- The average detection of malware after launch (the POST-Launch level) is 28%.
- The average result of blocking malware by all developers is 99%.
Methodology
The detailed methodology is systematically developed by us to be up-to-date with new trends in the cybersecurity and system protection industry, which is why we provide links to the methodology:
- How do we acquire malware for testing, and how do we classify it?
- How do we collect logs from Windows and protection software?
- How do we automate our tests?
As a member of the AMTSO group, you can be sure that our testing tools and processes comply with international guidelines and are respected by developers.
Conclusions from the test
Based on the many editions of this test, we confirm that solutions equipped with multi-layered protection achieve the best results against malware. This is because the use of various mechanisms and techniques is more effective against a wide spectrum of threats from the Internet.
When confronted with Trojans, ransomware, backdoors, and other similar, a mix of traditional scanning of webpages, keywords, hyperlinks, and IP addresses against websites distributing malware works best.
Subsequently, good security solutions should be equipped with heuristic scanning to analyze the behavior of files and applications after running for suspicious activity that may indicate the presence of malware. At the same time cloud analysis will work and send suspicious files to the developer’s servers for more detailed identification. During this time, files are often blocked from being launched, so there is no infection of the system.
Dedicated protection against ransomware is often useful. Usually, such threat disguises itself as Trojan-Downloader, so again we remind you of the multi-layered protection that can sometimes fail. Therefore, some solutions have special mechanisms designed to detect and block ransomware. At the same time, it is important that a product has a firewall that monitors network traffic and block suspicious connections.
Multi-layered protection is designed to increase the effectiveness of capturing various threats and minimize risk. The more advanced solutions for the protection of endpoint devices, the better it can deal with a variety of threats.