In November we conducted the last in 2024 6th edition of the test, completing the long-term series of the “Advanced In-The-Wild Malware Test” series. It is based on the fact that we use the same principles of navigating the Internet as during the daily use of e-mail clients, browsers, and communicators to realistically reproduce the delivery of messages, clicking on links, downloading and running files, and installing applications.
Our tests comply with the guidelines of the Anti-Malware Testing Standards Organization. Details about the test are available at this website as well as in our methodology.
Tested solutions
- Avast Free Antivirus
- Bitdefender Total Security
- Comodo Internet Security 2025
- Emsisoft Enterprise Security
- Eset Smart Security Premium
- F-Secure Total
- Microsoft Defender
- Malwarebytes Premium
- ThreatDown Endpoint Protection
- Quick Heal Total Protection
- Panda Dome Advanced
- Webroot Antivirus
- Xcitium ZeroThreat Advanced
Advanced In-The-Wild Malware Test on Internet Samples
During this edition, we tested 13 protection solutions, and after a detailed analysis, it turned out that we can select leaders who flawlessly detected and blocked malware.
We always select new malware samples so that they actually do something malicious in Windows 11. Thanks to this, we significantly minimize the case of including files that are not malicious, do not work, are old, or outdated. Each sample that is downloaded from a URL before the test undergoes a quick selection based on 5 steps. We collect URLs with potential malware from Telegram groups, honeypots, and public and private feeds.
Therefore, each sample before it goes to the test, must undergo a quick check cycle:
- Is a file online? A threat must be available online for download.
- Is a file unique? We compare the SHA256 file with hashes in a database to eliminate duplicate threats. Thanks to this, we never test two identical threats.
- Does a file match the test? Using the tools in the Linux console, we check the original file extension which must match the type of file running on Windows.
- Static scanning. We use Yara rules and a scanner of technology partner to learn more about a threat: we get better feedback on a file and malware family, we eliminate samples that do not fit the Windows environment.
- Dynamic scanning. A file is run in Windows 11 where we check if it shows malicious activity. In this step, we also use an image recognition tool that helps to catch corrupted software, installers, adware, and other non-malware software that manages to reach this stage.
More information on this topic is described in the methodology in point 4 (selecting samples for tests).
How do we evaluate the protection and neutralization of threats in Windows 11?
We evaluate the tested security solutions on the basis of 3 parameters:
- PRE_EXECUTION: Is the malware file effectively detected and blocked at an early stage? This can be blocking a website or a file while it is being saved or trying to access it.
- POST_EXECUTION: Is malware downloaded to the system, launched, and blocked at the stage of advanced analysis? This stage reflects the most dangerous situation, and shows the real effectiveness of the security software against a threat that is already active in the system.
- REMEDIATION TIME: Is a threat or part of it effectively eliminated? This parameter is related to the previous points, and defines the time expressed in seconds when malware is in the system until it is detected, and the machine is recovered from the security incident.
What protection settings do we use?
Although we avoid using potentially unwanted samples (PUPs and PUAs) in the test, it is always worth activating feature protecting against such threats, and we do it too.
In addition, presets are good, but not always the best. Therefore, for full transparency, we list those that we use for better protection or if required by a developer.
We always configure a solution in such a way that it has a dedicated extension for the Firefox browser that we use in tests (if the extension is available). We configure all software in such a way that it automatically blocks, deletes, and recover from incidents.
- Avast Free Antivirus - default settings + automatic PUP repair + browser protection.
- Bitdefender Total Security - default settings + browser protection.
- Comodo Internet Security 2025 - default settings + no alert for sandbox (block).
- Emsisoft Enterprise Security 2025 - default settings + automatic PUP repair + EDR + Rollback + browser protection.
- Eset Smart Security - default settings + PUP/PUA protection + browser protection.
- F-Secure Total - default settings + browser protection.
- Malwarebytes Premium - default settings + browser protection.
- Microsoft Defender - default settings.
- Panda Dome Advanced - default settings + browser protection.
- Quick Heal Total Security - default settings + browser protection.
- ThreatDown Endpoint Protection - default settings + EDR + browser protection.
- Webroot Antivirus - default settings + browser protection.
- Xcitium ZeroThreat Advanced - preset policy "Windows - Secure Profile v.8.1" + HIPS default action on "Block requests" + EDR enabled