Gone are the days when antivirus software was enough to protect digital assets against IT security risks. Today, you need more effective defenses against the large majority of malware, zero-day, and other threats that are out there. But with a limited budget and/or in-house (cybersecurity) talent, it’s not always easy to determine which security solutions should be given top priority.

In order to make the right choices, you need to understand what the risks are, how malware have evolved, why traditional antivirus software are no longer adequate in fighting them, and what kind of solutions work best against these new threats. Otherwise, you’ll be spending on products that only give you a false sense of security.

The consequences of relying on ineffective security solutions can be disastrous and costly. We see it all the time. A company purchases an antivirus solution and installs it on each of their systems. But only a couple of months after, those same systems already get infected with malware. And we’re not just talking about the kind that causes minor inconveniences.

 We’re talking about the ones that:

• Lock down entire networks (ransomware)
• Ensnare systems to launch DDoS or spam attacks (botnets)
• Steal login credentials and personal information (Keyloggers and spyware)
• Bring down networks (worms)
• And many other destructive types

These are the kinds that can cause lengthy downtimes; massive data breaches; expensive remediation, disaster recovery, and digital forensic operations; or even lawsuits.  But how in the world are these threats able to evade antivirus software? The answer lies in the way most traditional antivirus software detect IT security risks.

An Overview of Signature-Based Malware Detection

Traditional antiviruses typically detect malware through what is known as signature based detection. In this method, antivirus software are able to identify malicious files based on their structure, profile, or through certain string patterns encoded in them.

Let me elaborate on this a bit.

Let’s say a new malware draws the attention of cyber security researchers, perhaps after already crippling dozens of networks. The researchers investigate, find the suspicious file (or files), and analyze it. After performing their analysis, they obtain a signature of the file, i.e. something that can be used to uniquely identify the file.

This can be obtained through a variety of ways that mostly involve what are known as hash functions. We won’t go into the details of how hash functions work, except to say that when you run a sequence of characters (like the contents of a file) through them, they produce as output another, usually shorter, sequence of characters that uniquely identify the original sequence. The most commonly used hash functions for obtaining signatures are SHA1 and MD5.

To get the signature of a malicious file, security researchers typically run either the entire file, code snippets within the file, structural properties/characteristics of the file, or even strings of text found in files, through a hash function. That last one is normally applicable to malware that display some kind of message, like “I love you” or “your system has been encrypted with ransomware”.

What is then obtained, as mentioned earlier, is what we call the malware’s “signature”. This signature is then added to a virus database. This database is where a signature-based antivirus software would look up existing signatures upon scanning a suspicious file. If a match is found, then the suspicious file is identified as a virus and dealt with accordingly.

This is very similar to the way authorities identify “most wanted” criminals. First, a person commits a high profile crime, that person is caught, and then booked at the police station. While there, that person’s fingerprint, pertinent information, and mug shot is obtained and recorded as part of that criminal’s profile.

If the criminal escapes or freed, the police can simply warn the public and share them certain elements of that person’s profile, usually the photo. People can then use that profile to identify the criminal. If someone sees a person that fits that profile, he can report the sighting to the authorities, the authorities can in turn verify the profile, and the criminal can be caught.

If you review what I just explained, you’ll notice that malware signatures can only be added to a malware database (and in turn used for identifying viruses) after an infection has already occurred and cyber security researchers have already analyzed the suspicious file. Meaning, signature-based malware detection can only be effective against known threats.

 However, this type of detection can’t be used against what are known as zero days. Zero-day threats are those that have not yet been analyzed and added to the database. When these threats strike, there is no way a signature based antivirus solution can detect them because their signatures are not yet in the database

 People in the cyber crime community know this. That’s why malware developers either constantly develop new malware (currently estimated to be over 390,000 per day) or create polymorphic varieties. Polymorphic malware are malware that have the ability to change their profiles a.k.a. morph in order to avoid detection. For this reason, malware developers are always one step ahead of antivirus developers

 So how can we defend ourselves against these types of IT security risks?

Introducing Sandboxing and Behavior (Heuristic) Based Scanning

Sometimes, crime fighters have ways to identify possible crooks even if they aren’t in a most wanted list. They do so by first understanding criminal behavior. They try to find out: Where and when do certain criminals operate? What is their modus operandi? Who are their usual victims? And so on.

Of course, the information obtained is not as precise as a criminal’s profile. However, it does enable crime fighters to preempt criminals’ actions and, in some situations, catch them in the act.

There are now some security solutions that operate in this manner. Some of these solutions employ techniques like behavior (heuristic) based scanning and sandboxing, which, as you’ll soon learn, enable them to detect unknown malware before they do any serious damage.

First, let’s talk about behavior based scanning or heuristics.

Heuristic-Based Scanning

In this type of scanning, the solution looks for certain behaviors or even commands that might be indicative of malicious actions. Let me explain.

Let’s say a file arrives at a system protected by a heuristic-based security solution. Maybe it came in as an email attachment or something downloaded from the Web. The security solution performs a heuristic scan on the file for signs of possible threats.

For example, several botnet clients communicate with their Command and Control servers via IRC. IRC or Internet Relay Chat is a protocol that used to be popular in the 80’s and 90’s when people used it for real-time, text-based communications a.k.a. Internet-based “chatting”.

Although mainstream IRC chat rooms have practically gone extinct, the protocol lives on in the cyber crime world, where it’s used in botnet client-C&C server communications. Cyber crooks like it because it’s easy to deploy (there are many open source IRC servers out there) and it readily supports the automated execution of commands through scripts.

So if the file, which might be a small application, is found to make contact with a server on the Internet through IRC, then that might raise a little red flag.

If certain parts of the file are encrypted, then that might also raise a little red flag. It might mean the file is hiding something sinister. If, prior to sending data to a remote server, the application first hooks into Windows APIs to capture keystrokes, that might also raise a little red flag because this behavior is characteristic of keyloggers.

Of course, not all files that communicate via IRC are malicious. For all you know, the file might be that of a legitimate IRC client. Similarly, the encrypted portion of the file might be some sort of obfuscation, simply meant to prevent tampering or reverse engineering. Lastly, a program that captures keystrokes and transmits them to a remote server in the cloud might simply be an instant messaging application.

So, what we’re trying to say is that, a single malicious behavior might not be enough for a file to be classified as malicious. If the security solution labels files that easily, it will end up with a lot of false positives. That’s why anti-malware solutions that use heuristics take into consideration several behaviors and process them through sophisticated machine learning techniques before concluding whether a file is malicious or not.

The main advantage of behavior or heuristic based scanning is that it doesn’t rely on a database of signatures. Hence, it can be used against zero day threats.

Another technique that’s effective at detecting zero-day threats is sandboxing. Let’s talk about it now.

Sandboxing

A sandbox, in cyber security parlance, is a secure virtual environment where executable files are sent for analysis. It confines files sent to it in a sort of virtual machine or container and prevents them from directly interacting with the host system. Sandboxes were traditionally installed on endpoint devices but can now also take the form of a separate appliance deployed on-premise or hosted as a service in the cloud.

Once an executable file is inside a sandbox, it can be allowed to run as if it were in a regular system. In turn, it can be monitored for suspicious behavior. Similar to what was discussed earlier, if the executable file makes unusual changes to the file system or system registry, or if it carries out suspicious activities through network connections or system processes, then it can be classified as malware.

Sandboxing or threat emulation not only enables organizations to detect zero day threats, it also presents the opportunity for those threats to be profiled (i.e. their signature obtained) and added to a malware database. Once those signatures are in the database, they can then be used to enhance future detection exercises. 

The ability to detect both known and unknown or zero-day threats provides a huge benefit for businesses that simply can’t afford the potential downtimes, data breaches, and remediation costs resulting from unmitigated malware attacks.

 Like many security solutions out there, sandboxing comes in a variety of shapes and sizes. In choosing a sandboxing solution, several decision makers like CIOs, Network Directors, and COOs, usually get insights from a reputable research advisory body. In the paper How To Choose Your Next Sandboxing Solution, Gartner offers valuable insights for both budget-constrained organizations and organizations who are looking for a quick path to add sandboxing to their cyber security arsenal.