New: An OCR engine has been added to recognise text from screenshots, allowing us to better and more accurately identify potential malware files before they qualify for testing. This allows us to quickly reject installers, PUAs, PUPs and corrupted files.
We also use OCR to capture alerts and messages from security software. This gives us an additional opinion on how the tested solution reacts to malware. This provides vendors with further irrefutable proof that malware has or has not been detected.
Improved: We have added a SHA256 comparison of the malware sample on each machine with the security solution installed. This is to avoid errors where product X can be tested against another sample downloaded from the same URL.
Improved: The CSV reports provided to producers as evidence have been improved. They now include more detailed information from the database.
Improved: We have reorganised and improved the design of our home page and sub-pages relating to the services we offer.
Improved: We have added new features to the web application to make it quicker and easier for Testers to complete the test summary.
New: Added a delay to the download of a file in the browser so that some installed extensions can load correctly before starting the test on a malware sample.
Improved: Sorting test data on Linux is now better structured. A few additional changes will allow us to conduct technical investigations faster.
Improved: We have located a bug in the latest version of Sysmon 15.12 that causes the Sysmon service to crash. The bug has been reported to Microsoft. The developer does not have a fix for this yet, so we have implemented an individual workaround (after weeks the issue has been fixed in Sysmon 15.14).
Improved: We have changed the way the Remediation Time is calculated. This will not have a significant impact on the results, as so far every solution has been tested under the same conditions and on the same principles.
Namely, until now, the Average Remediation Time was calculated only for samples blocked at the Post-Launch level. From now on, the Average Remediation Time is calculated from the Pre-Launch level, and it is always 0 seconds (detection of an incident with blocking a sample before its launch) which is included in the average for all samples in the month.
New: At the request of one of the developers, we added a custom rule to the testing application which is responsible for generating additional logs in a machine with the installed solution, so that it is easier to make decisions about protection against a specific malware sample.
New: We expanded the testing application with a feature to generate a log with information about installed versions of software on Windows using Powershell.
New: Our web application for managing telemetry data from the tests through a browser was supplemented with additional functionalities to speed up the work for testers when summarizing results for a given period. For example, we added faster search for potentially unwanted samples such as PUPs, PUAs, Adware, and others.
Improved: for one of the developers, we improved a custom search of logs for selected threat indicators.
Improved: we improved Firefox’s automatic update for our everyday testing.
Improved: we added the Sysmon rules with additional detection indicators of several new solutions.
New: We implemented a new feature in the testing application that allows to collect telemetry data from selected logs of the tested solution. Developers can receive more feedback on blocked threats upon special request.
New: We integrated an external scanner of malware samples so that they are better classified even before being included in the test. This change should have a positive effect on the rejection of samples that are not suitable for testing for a variety of reasons. Scanning technology is provided by mks_vir (but also Arcabit) – Polish developer of security software.
New: It is now official that the May edition of the Advanced In The Wild Malware Test is compliant with the AMTSO standards, of which we have been a member since April 2023.
Improved: This month’s changes also include generating improved CSV reports from the test with additional telemetry data that we share with all developers.
New: Since the May edition of our test, we have added an individual summary view for each Vendor on their request and that coming from the community. Better marketing perception.
New: We have started the procedure to join a group of experts from AMTSO (Anti-Malware Testing Standard Organization).
New: Added so-called Remediation Time to comparison.
New: Joining a new cybersecurity expert group, known as Cyber Transparency Forum to develop new standards on transparency and endpoint cyber security.
Improved: Added Yara rule to the testing system to detect and classify malicious activity.
Improved: Added some rules to the testing system to detect malicious activity in Windows.
Improved: Added some new Antivirus Indicator Rules to the testing system.
Improved: Added new rules for better detection of malicious activity on Windows using Sysmon software.
New: Development of the so-called Threat Landscape from the test:
New: Replaced the old SHA-HTML table where we assigned a malware sample to a product and results. Instead, we simply provide a CSV file for everyone to download and browse for better transparency.
New: At the request of the community and Vendors, we have abandoned hosting malware for testing using a local DNS and HTTPS. We replaced these with URLs in the wild downloaded in a browser.
New: We changed the graphical presentation of the results.
New: At the request of our community, we simplified complex levels of blocking malware: Level 1, Level 2 at PRE-Launch, and Level 3 at POST-Launch.
New: Implementation of the visual admin panel for faster and better management of telemetry data from tests in the backend infrastructure.
New: Change of the Process Monitor software for logging changes in Windows to the Sysmon tool with a custom configuration.
New: Added Yara rules to pre-classify malware samples and reject invalid ones.
Improved: Replacement of the old NPM package for DNS with a new package.
Improved: Time zones in the database and files shared with Vendors as feedback from the test.
Improved: Changing file logs structure on backend infrastructure.
