We start the new year with the first edition of the six Advanced In-The-Wild Malware Test, where we will identify the best security solutions for Windows 11 at the end of 2025, as we did for last year. In this new edition of long-term tests, in which we check the comprehensive effectiveness of security against threats from the Internet, we have included solutions from vendors and their flagship products: Acronic Cyber Protect, Check Point with ZoneAlarm Extreme Security NextGen software, Cisco Secure Endpoint Advantage and WatchGuard EPDR (Endpoint Protection + Detection and Response).
In this series of tests, we regularly test the effectiveness of early detection and blocking of malware. We automate the actions users perform during their daily use of email programs, web browsers and instant messaging to realistically deliver messages to the system, followed by clicking on links, downloading and running files, and installing software. Technical details are available in the survey methodology, and if you want to know more, just ask in the comments.
Our tests comply with the guidelines of the Anti-Malware Testing Standards Organization. Details about the test are available at this website as well as in our methodology.
What solutions did we test in January 2025?
What settings do we use?
Although we avoid using potentially unwanted samples (PUPs and PUAs) in the test, it’s always a good idea to activate such a protection feature, and we do that too.
We always configure the protection solution to have a dedicated extension for the Firefox browser we use in the test (if the extension is available). In addition, we set up the entire software to automatically block, remove and fix incidents that occur.
Based on a series of tests already carried out, our opinion is that the predefined settings are good, but not always the best. Therefore, for the sake of full transparency, we list the ones we have implemented for better protection or if required by the manufacturer.
- Acronis Cyber Protect – default settings + EDR + browser protection.
- Avast Free Antivirus – default settings + automatic PUP repair + browser protection.
- Bitdefender Total Security – default settings + browser protection.
- Cisco Secure Endpoint Advantage – default settings + Exploit Prevention (Block) + Enable Event Tracing for Windows + Orbital Enabled + Block and report malicious network connections + Terminate and quarantine unknown applications connected to malicious hosts.
- Comodo Internet Security 2025 – browser protection + block automatically for sandbox (block).
- Emsisoft Enterprise Security – default settings + automatic PUP repair + EDR + Rollback + browser protection.
- F-Secure Total – default settings + browser protection.
- K7 Total Security – default settings + browser protection.
- Malwarebytes Premium – default settings + browser protection.
- ThreatDown Endpoint Protection – default settings + EDR + browser protection.
- WatchGuard EPDR (Endpoint Protection + Detection and Response) – default settings + EDR + browser protection.
- Webroot Antivirus – default settings + browser protection.
- Xcitium ZeroThreat Advanced – preset policy “Windows – Secure Profile v.8.1” + HIPS default action on “Block requests” + EDR enabled.
- ZoneAlarm Extreme Security – default settings + Anti-Keylogger enabled + browser protection.
We are already going to test other popular solutions from the next edition.
Comments Off on January 2025 edition
Advanced In-The-Wild Malware Test
We conducted a comprehensive evaluation of 14 security solutions, assessing their effectiveness in Windows 11 across both home and business environments. After a thorough analysis of the results and consultations with vendors, we were able to identify the leaders that effectively and flawlessly detect and block malware that poses a real threat from the Internet.
We always select new malware samples in such a way that they actually do something malicious on Windows. This significantly minimizes the inclusion in the test of files that are not malicious, not working, old or outdated.
Each sample that is taken from a URL before testing undergoes a quick selection based on 5 steps. We collect URLs with potential malware from groups on Telegram messenger, honeypots, and public and private feeds.
Each URL with a file participates in a rapid verification cycle before it goes to the test:
- Is the file online? The threat must be available online for download.
- Is the file unique? We compare the SHA256 file with hashes in the database to eliminate duplicate threats. This ensures that we never test two identical threats.
- Does the file match the test? Using the tools in the Linux console, we check the original file extension, which must match the file type running on Windows.
- Static scanning. We use Yara rules and a technology partner’s scanner to learn more about the threat: we get better feedback on the file and malware family; we eliminate samples that don’t match the Windows environment.
- Dynamic scanning with OCR analysis. The file is run in Windows 11, where we check if it shows malicious activity. In this step, we also use an image recognition tool (OCR) that helps catch corrupted software, installers, Adware and other non-malware that manages to make it all the way to this step.
Only after passing through these 5 steps is the URL with the file suitable for testing, and is immediately forwarded to all machines simultaneously with security solutions installed. For more information on file URL matching, we describe the methodology in section 4 (selecting samples for testing).
How do we evaluate protection and the so-called Remediation Time parameter in Windows 11?
We evaluate the tested security solutions based on as many as 3 parameters
- PRE_EXECUTION: is the malware file effectively detected and blocked at an early stage? This could be blocking a website or a file while saving, while trying to access the file.
- POST_EXECUTION: has the malware been downloaded to the system, run and blocked at the advanced analysis stage? This stage reflects the most dangerous situation and shows the real effectiveness of the protection software with the 0-day threat already active on the system.
- REMEDIATION TIME: has the threat or part of it been successfully eliminated? This parameter is related to the previous points and determines how long the malware stayed on the system until the security incident was detected and remediated.
About the Advanced In-The-Wild Malware Test
The test is conducted six times during the course. It concerns testing Windows 11 security solutions for effectiveness in blocking malware.
The evaluation of tested solutions is based on three parameters: PRE_EXECUTION (early detection and blocking), POST_EXECUTION (advanced analysis and blocking after launch), and REMEDIATION TIME (threat elimination time). These steps are designed to indicate the product’s strengths in detecting and neutralizing malware.
