Analyzing Malicious Files For Writing Network Signatures

Attackers continually modify malware in order to evade antivirus software. Attackers will pack or encrypt malicious files using various packers. Blocking each and every malicious executable is a challenging task for antivirus vendors. Most malicious files, once installed, try to download additional malware or send HTTP GET/POST requests to malicious servers controlled by attackers. Malicious files can gather sensitive information and then send it to the controlled servers. Given the networking capabilities of malware, beyond simply trying to identify the malicious binary itself, we can also detect malware by identifying certain patterns in network traffic. This blog will explain how someone with limited reverse engineering skills can monitor network traffic to identify malicious binaries.

For this tutorial, we will use the example of a real-world malicious server - “hxxp://uaetoon.net/scan/”. This website hosted a fake antivirus program, designed to convince a victim that their PC was infected. The site displayed different security alerts, such as “Your Computer is Infected”. Here is the screenshot of the malicious website once visited,

The website would then convince the victim to download a malicious binary file called “setup.exe” in order to remove fake threats found on the system. Installing the binary was of course actually installing a Trojan on the machine as opposed to an AV solution. The malicious binary would then send various HTTP requests to malicious servers controlled by the attackers. We can study the network traffic by allowing the binary to execute in a virtual machine, but this approach has some limitations:

1. You need a separate controlled environment to run the malicious samples. It is never advisable to run the samples on your production network.
2. There are chances that not all HTTP requests are fired by the malicious binary depending upon the availability of malicious severs.

Due to the above limitations, we sometimes need to first reverse the malicious binary in order to retrieve strings such as the destination servers. This can easily be done, with limited reverse engineering knowledge and without running the sample. Let’s begin with some static analysis. Download the malicious file on your machine or VMware image, but don’t run it.

Let’s first find out if the file has been packed using some well-known packer. To do so, you will need to open the malicious file in a hex or text editor. Here is the screenshot of file when opened in a text editor:
Here, a little knowledge of commonly used packers will be helpful. UPX (Ultimate Packer for Executables) is often used by attackers, as it is a free and open source packer. You will generally find strings like “UPX” and “UPX1” in the malicious file in plain text if the file is packed using UPX. Now that we know the file has been packed by UPX, we need to unpack it and open it in the IDA pro dissembler. To unpack the malicious file, download UPX packer. Unzip the packer and copy the malicious file into the UPX folder. Unpack the malicious file with the simple command shown in below screenshot,
Now, the file has been unpacked and can be opened in IDA pro. We are not going to fully reverse engineer the file for the purpose of this blog, rather we’ll look for potential domains where traffic is sent. In the next screenshot, you can see the binary in IDA Pro:

Allow IDA Pro to complete its analysis and then open the “Strings window” to identify strings used by the file. Going through the strings window, you will learn a great deal about the activities conducted by this binary, such as registry changes or network traffic sent. Let’s look at some interesting strings:
From the above strings, we can conclude that this malicious binary creates registry entries under “Run” in order to run this executable each time the computer is rebooted. This binary can also send HTTP POST requests with content type x-www-form-urlencoded, which refers to the URL format. This is valuable information when writing pattern matching signatures, in order to block further infection. Copy and save the URL format which is “http://%s%s?act%sor&v=1&a=%d&id=%s&hardid=%s”. The malicious file later will substitute domain strings and other data in place of the %s (string) variables.

Here is another set of strings:

We now have all the malicious domains, along with another URL pattern. That’s it! We now have all the domains and URL patterns necessary to write network-based signatures. While this approach won’t work in every case, it does represent an efficient way to write network signatures without the need to conduct a full and time consuming reverse engineering exercise. Now we have 2 URL patterns:

1. http://%s%s?act%sor&v=1&a=%d&id=%s&hardid=%s
2. %s?action=%sgen&v=%s

We can now easily block all HTTP requests, including the above patterns. Even if the attacker changes the domains used, chances are high that the URL format will remain the same. It turns out that this traffic relates to the now well-known Koobface worm.

Blocking all variants of malware can be difficult but we can greatly improve detection rates by additionally monitoring network traffic.

Hope this will be useful.
Umesh