In November 2019, the CheckLab.pl organization prepared a list of popular solutions to protect computers in micro and small enterprises. Among the tested solutions, there are 13 specialized top-notch products, and also one solution for big business — Comodo Advanced Endpoint Protection. The Microsoft’s product known as Windows Defender was also qualified for the test as an integral part of the Windows 10 system.
This time, employees at CheckLab want to draw attention to the fact that the prepared comparison of thirteen specialized solutions shows the effectiveness of protection against a specific type of an attack. The test described below uses malicious software, tools, and techniques of bypassing security that are used in real campaigns, although additional modules have not been considered, such as: online banking protection, anti-theft module, embedded encrypted VPN tunnel, home network monitor, storing and generating passwords, webcam protection, and other optional modules of good software to protect a system.
In terms of availability of various functions, one product gives way to others. Whereas external developers of security solutions share modules of remote access, cloud management, setting up VPN connections, scanning IoT devices connected to home network, and many more, the disadvantage of Windows Defender in the present case is technical limitations which lie in the fact that Microsoft provides protection only for the Windows system.
An encryption of files and whole catalogs is an important point. Competitive solutions can offer such a component at no extra charge. Encryption in Windows, known better as the BitLocker feature is available only in the Windows 10 Pro versions and higher. Users of Windows 10 Home should use external software, such as VeraCrypt, private storage in the OneDrive cloud, solutions of antivirus developers mentioned above, or encryption software from third parties.
The fact that Microsoft locking itself to one operating system has led to a situation where Windows Defender is hard to compete with the biggest developers from the industry in terms of the complexity of protection and security of various devices with different operating systems. Microsoft also does not provide, for example, routers to the market, and these devices are offered by developers: Avira, Bitdefender, F-Secure, Symantec, and other.
Microsoft’s portfolio lacks of comprehensive security of mobile operating systems or managing protected devices from one central point. It should be mentioned that family features are available in the Windows system, and they help monitor online activities of a child, but again it lacks the option of managing parental control that would cover few devices.
The AVLab.pl portal and the CheckLab.pl organization, obliged to raise awareness and educate, have to take responsibility for published tests and comparison. A practice shows that a large number of people is unaware of cyber threats. Low awareness extends to the lack of a basic knowledge of pros and cons solutions available on the market. There are so many products that the more a user has to choose, the more difficult to make the right decision.
Each developer presents to end user its own marketing data, and multiplying these information by amount of suppliers, it is not easy to get through such an unimaginable portion of information. The amount of data in the context of the whole IT market can be overwhelming even for experts who find it hard to keep up with products that dynamically evolve. As a result, unaware users instead of ensuring at least a basic protection, they do not go for any of the available.
Explanation of the “Advanced In The Wild Malware Test”
The name of the „Advanced In The Wild Malware Test” perfectly reflects its character. The source of malicious software are honeypots located on all continents of the world. We collect malware, among other, for the Windows system. Samples captured in attacks are checked on the basis of over 100 patterns before they are qualified for testing. These patterns allow us to determine whether a potentially dangerous file is actually a threat to the Windows 10 Pro operating system.
In November 2019, in the third edition of security tests of the „Advanced In The Wild Malware Test” we verified the effectiveness of detecting and blocking malicious software of 13 solutions for protecting computers.
The tests lasted continuously the whole November 2019. The list of tested solutions is as follows:
- Avast Free Antivirus
- Avira Antivirus Pro
- Bitdefender Total Security
- Comodo Advanced Endpoint Protection
- Comodo Internet Security
- ESET Smart Security
- F-Secure SAFE
- G DATA Total Security
- Kaspersky Total Security
- SecureAPlus Pro
- Sophos HOME Premium
- Webroot Antivirus
- Windows Defender
The results of November 2019
CheckLab as the first organization in the word shows such detailed information from tests to all interested people. We share checksums of malicious software by dividing them into protection technologies that have contributed to detect and stop a threat. According to experts, this type of innovative approach of comparing security will contribute to better understanding of differences between available products for consumers and enterprises.
A chart describing differences between individual products is available at http://avlab.pl/en/recent-results
In the fourth edition of the test, we have granted the BEST+++ certificate to:
- Avast Free Antivirus
- Avira Antivirus Pro
- Bitdefender Total Security
- Comodo Advanced Endpoint Protection
- Comodo Internet Security
- ESET Smart Security
- F-Secure SAFE
- G DATA Total Security
- Kaspersky Total Security
- SecureAPlus Pro
- Sophos HOME Premium
- Webroot Antivirus
- Windows Defender
Levels of blocking malicious software samples
The CheckLab employees are probably pioneers in this regard — they show more detailed diagnostic data than any other testing institution, including the largest such as AV-Comparatives and AV-Test. Blocking of each malware sample by tested protection solution has been divided into a few levels:
- Level 1 (P1): The browser level, i.e. a virus has been stopped before or after it has been downloaded onto a hard drive.
- Level 2 (P2): The system level, i.e. a virus has been downloaded, but it has not been allowed to run.
- Level 3 (P3): The analysis level, i.e. a virus has been run and blocked by a tested product.
- Failure (N): The failure, i.e. a virus has not been blocked and it has infected a system.
The results of blocking each sample are available at http://checklab.pl/en/recent-results in the table:
The products and Windows 10 settings: daily test cycle
Tests are carried out in Windows 10 Pro x64. The user account control (UAC) is disabled because the purpose of the tests is to check the protection effectiveness of a product against malware and not a reaction of the testing system to Windows messages.
Additionally, the Windows 10 system contains installed the following software: office suite, document browser, email client, and other tools and files that give the impression of a normal working environment.
Automatic updates of the Windows 10 system are disabled in a given month of the tests. Due to the complicated process and the possibility of a malfunction, Windows 10 is updated every few weeks under close supervision.
Security products are updated one time within a day. Before tests are run, virus databases and protection product files are updated. This means that the latest versions of protection products are tested every day.
Malicious software
We have used 761 malicious software samples for the test, consisting of, among others, banking trojans, ransomware, backdoors, downloaders, and macro viruses. In the contrast to the well-known institutions that verify the security usefulness, the CheckLab tests are much more transparent because the organization share the full list of malware samples.