Windows Live Response for Collecting and Analyzing Forensically Sound Evidence

This chapter is from the book

Real Digital Forensics: Computer Security and Incident Response

Learn More Buy

This chapter is from the book

This chapter is from the book 

Real Digital Forensics: Computer Security and Incident Response

Learn More Buy

Analyzing Nonvolatile Data

We would like to obtain several key pieces of information while the machine is still running. The type of data we will discuss in this section is nonvolatile. This means we could also retrieve this information from a forensic duplication if we so desired, but that option may be difficult or impossible. Some of the information we would like to acquire is this:

• System Version and Patch Level

• File System Time and Date Stamps

• Registry Data

• The Auditing Policy

• A History of Logins

• System Event Logs

• User Accounts

• IIS Logs

• Suspicious Files

We will address each set of data in its own subsection and analyze the evidence collected from JBRWWW.

System Version and Patch Level

If you have not figured it out by now, an investigation can be tedious, and sometimes it is difficult to know where to start. One of the important facts we can learn about JBRWWW is its operating system version level and which security patches have been installed. Knowing which patches have been applied to the server will enable us to narrow our initial investigation to areas of high probability. This is not to say that an intruder would not try to install a patch to cover the means of attack, keep his access to the machine, and deter other intruders. A program in our toolkit called PsInfo, distributed from the PsTools suite at http://www.sysinternals.com, will enable us to query JBRWWW for its system information. The system information that PsInfo produces will enable us to see the patches that have been applied. PsInfo is run with the following command, where -h is used to show installed hotfixes, -s is used to show installed software, and -d is used to show disk volume information:

psinfo –h –s -d

PsInfo provides the following results. We have bolded the important pieces of information:

PsInfo 1.34 - local and remote system information viewer
Copyright (C) 2001-2002 Mark Russinovich
Sysinternals - http://www.sysinternals.com


Querying information for JBRHTTP://WWW...
                                       
System information for \\JBRWWW:
Uptime:                   39 days, 6 hours, 27 minutes, 42 seconds
Kernel version:           Microsoft Windows 2000, Uniprocessor Free
Product type:             Professional
Product version:          5.0
Service pack:             0
Kernel build number:      2195
Registered organization:  JBR Bank
Registered owner:         JBR Bank
Install date:             8/23/2003, 12:46:00 PM
IE version:               5.0100
System root:              C:\WINNT
Processors:               1
Processor speed:          435 MHz
Processor type:           Intel Pentium II or Celeron
Physical memory:          126 MB
Volume Type      Format   Label           Size     Free  Free
    A: Removable                                           0%
    C: Fixed     NTFS                   4.0 GB   3.2 GB   80%
    D: CD-ROM    CDFS     CDROM       272.8 MB             0%
OS Hot Fix  Installed
Q147222     8/23/2003
Applications:
 WebFldrs 9.00.3501

We see that only one hotfix (Q147222) has been applied. The named hotfix addresses the Exchange server, the mail server for Microsoft Windows. Doing a little research on http://www.securityfocus.com, we see that JBRWWW is vulnerable to a multitude of attacks, including the "Unicode" (Bugtraq ID #1806) and "Double Decode" (Bugtraq ID #2708) attacks. Because these are both Web server attacks and JBRWWW is running a Web server (as we saw in the netstat and FPort output), we need to acquire the Web server logs to see whether the intruder gained access through the Web server. We will discuss the commands to do this in a later section in this chapter.

File System Time and Date Stamps

Most investigators will use the dir command to capture the file time and date stamps, but we recommend a better tool. The standard dir command produces output that is cumbersome and that cannot easily be imported into a spreadsheet so that we may sort on different attributes of the data. In the UnxUtils package, available from unxutils.sourceforge.net, you will find a command called find. If you are already familiar with Cygwin, you can also use the find utility from that tool set (this is what we used in our response). This command will print, one line for each file, any of the file’s attributes we desire. Therefore, with the following command, we can print the file permissions, last access date, last access time, modification date, modification time, created date, created time, user ownership, group ownership, file size, and the full path of every file on the C: drive:

find c:\ -printf "%m;%Ax;%AT;%Tx;%TT;%Cx;%CT;%U;%G;%s;%p\n"

Notice that with the find command, we are delimiting each of the attributes with a semicolon. This will enable us to import it into our favorite spreadsheet. After we import this data, we can perform sorts for file pathname. Because we already know that C:\WINNT\sytem32\os2\dll is a path where the attacker left his tools, we will examine that directory in Table 1-1:

Table 1-1 Suspicious Files Discovered on JBRWWW

We see that most of the tools were created during the evening of 10\01\2003. If we do a sort on the file metadata by creation time and date stamps, we see that all these files were created approximately at the same time, as in Table 1-2:

Table 1-2 Files Created During the Attack on JBRWWW

We obviously know that iroffer was installed on the system from earlier steps in our investigation. We also saw that the attacker, along with PsExec, established a backdoor with netcat. The files we did not know about are bolded in Table 1-2. All of the files in Table 1-2 are of interest to us, and it would behoove us to copy these files to our forensic workstation to perform additional tool analysis. We will describe the process for acquisition a little later in this chapter so that we may perform tool analysis later.

We could obviously perform a sort on modified and access times and review the files that may have been altered or run around the time of the suspicious files listed in Table 1-2. We will save you that step, however, because there are no interesting results from that investigative action.

Registry Data

There are two main investigative leads we can discover in the registry dump. Although the result of dumping the registry is large (in the case of JBRWWW, it was more than 7 MB long), we can quickly search for the following leads:

• Programs executed on bootup

• Entries created by the intruder’s tools

We are able to capture the complete registry, in a rather cryptic format, by using RegDmp without command-line options. The output is ASCII-formatted such that Microsoft’s registry tools can alter the contents. Because we are interested in only a few lines, we will do our analysis with a standard text editor. After we obtain the output with the regdmp command, we see that the key \HKEY_LOCAL_MACHINE\Software\Microsoft\ Windows\CurrentVersion contains three sub-keys that are of interest to us: Run, RunOnce, and RunOnceEx. Any values in the Run keys signify programs that will be executed when the system starts up. JBRWWW had the following information in this area of the registry:

Run
  Synchronization Manager = mobsync.exe /logon
RunOnce
RunOnceEx

mobsync.exe is a system binary, so we do not see tools the intruder intended to execute at system startup. If the attacker was savvy, he could have placed the following command in the registry to automatically open a backdoor:

nc –d –L –p 10000 –e C:\winnt\system32\cmd.exe

Another thing we may want to look for in the registry is any suspicious artifact from the intruder’s tool. This may sound daunting, but it really isn’t. Most of the time we know the names of the tools because of the entries in the file system. Therefore, in the case of JBRWWW, we may search for "PsExec", "iroffer", or other relevant file names. Searching for these names yielded nothing for JBRWWW.

This step becomes more important when you have a rack of servers and you know one is compromised. After you do a thorough investigation and find the remnants in the registry from an intruder’s tools, you can quickly do a search on other servers to determine whether they were compromised also.

The Auditing Policy

The next series of tools we will be running will depend on JBRWWW's auditing policy. Without proper auditing (and that is the default for Windows NT and 2000, by the way), we will not have security-related logs. The command to determine the auditing policy is auditpol. Auditpol is distributed with Microsoft's resource kits. The following information is returned when we run auditpol without command-line arguments on JBRWWW:

Running ...


(0) Audit Disabled
       
System                   = No
Logon                    = No
Object Access            = No
Privilege Use            = No
Process Tracking         = No
Policy Change            = No
Account Management       = No
Directory Service Access = No
Account Logon            = No

This is disturbing! There are no events generated from logins or other security-related events. Our system event logs will not be a good source of information for us because of the conservative auditing policy. Believe it or not, we see this during a majority of our investigations.

A History of Logins

A history of logins can be obtained with the NTLast command, distributed by http://www.foundstone.com. NTLast can be run in a myriad of ways, but we are interested in all of the logins, so we will not use command-line arguments when we run it on JBRWWW:

- No Records - Check to see if auditing is on

Yikes! This tool depends on the auditing policy to determine the login history. As you can see, it is very important to enable auditing.

System Event Logs

Typically, there are three types of event logs on a Windows machine:

• Security

• Application

• System

The command PsLogList within the PsTools suite distributed at http://www.sysinternals.com will extract these logs into a nice, easy-to-read text format. The following command will dump the Security Event Logs a comma-delimited format suitable for your favorite spreadsheet program:

psloglist –s –x security

The -s switch tells psloglist to dump each event on a single line so that the output is suitable for analysis with a spreadsheet. The -x switch tells psloglist to dump the extended information for each event. You can also replace security with application or system if you want to acquire the other logs on the victim system.

The Security Event Log contains all of the information generated by our auditing policy, discussed in a previous section. Most importantly, we would be interested in the information regarding logons/logoffs and any objects audited on the system. Of course, as we would expect, JBRWWW reports no events in the logs.

The application event log contains data generated from the installed applications. Some events are informational, whereas others indicate application failures. As we peruse JBRWWW’s application logs, all we see are messages created from the installation of standard programs on the system beginning August 23, 2003.

The system event log, as you may have guessed, contains the messages from system services. The system log is the log where you would see device driver failures, IP address conflicts, and other information. As we browse JBRWWW’s system logs, we see only messages created from standard use of the system. It seems that the event logs, in this investigation, do not give us valid leads.

User Accounts

The easiest type of backdoor for an intruder to use is one that will blend into the normal traffic patterns for the victim machine. Therefore, it would make sense for the attacker to create a new user so that he could log into the same services that valid users utilize. It is simple for us to dump the user accounts using the popular pwdump utility, which is well known by administrators and attackers alike. By typing pwdump on JBRWWW, we receive the following information:

Administrator:500:9DCFD05D3688BBBFAAD3B435B51404EE:CB8C5705F92DE9D8D11642948ECCAB72:::
Guest:501:NO PASSWORD*********************:NO PASSWORD*********************:::
IUSR_JBRWWW:1000:B936986BA1C5636B0F28D0549F4A7C10:137C045C1CACAE4B07C6C3B88BF0CE6D:::
IWAM_JBRWWW:1001:DA3DF28964893179378B2EB9047FBA87:A2C8D0EC209C60A48DB9365A51565DC4:::

There are four users for JBRWWW: Administrator, Guest, IUSR_JBRWWW, and IWAM_JBRWWW. Administrator is the super user account (RID 500) that every system must have. Guest is a disabled account that also exists on all Windows systems. IUSER_JBRWWW and IWAM_JBRWWW are normal user accounts that processes, such as the IIS Web server, use to run. These accounts are on the machine to limit the damage an attacker could cause the system through a Web-based attack because he would only have a lowly user account rather than Administrator-level access right away. We see that there are no other accounts on JBRWWW of interest.

IIS Logs

Most attacks in the modern era happen over TCP port 80 (HTTP). Why? you may ask. Because there are literally millions of Web servers running, and incoming port 80 traffic is rarely blocked at the victim’s network boundaries. You cannot block what you must allow in. Because we have not seen the initial method of intrusion, we can only guess at this point that it may have been the IIS Web server.

The IIS Web server writes any activity to logs in the C:\winnt\system32\logfiles directory by default. In this directory, there is another directory named W3SVCn, where n is the unique ID of the Web server. Usually this ID starts at one, but because one Web server can host numerous domains, each W3SVC directory must be analyzed. JBRWWW only hosted one domain, so the directory of interest is W3SVC1.

Inside the W3SVC1 directory there are two files: ex030923.log and ex031001.log. Each of these logs contains the activity for the Web server for a whole day. The file name distinguishes the day:

ffyymmdd.log

. . . where ff is the format, yy is the year, mm is the month, and dd is the day. IIS can log three different types of formats: W3C Extended (ff would be ex in this case), NCSA common (ff would be nc in this case), and Microsoft IIS native format (ff would be in in this case). JBRWWW is using the default extended log formatting and contains activity for the days of September 9, 2003 and October 1, 2003.

The next problem we must overcome is how to transfer the relevant logs to our forensic workstation. We do not want to FTP them or to perform any other intrusive command that would greatly change the state of JBRWWW because we will be performing a forensic duplication in the future. Instead, if you refer to the introduction in this chapter, we presented a method of transferring data from one machine to another. Instead of using command like we initially presented, we will use type file.txt to transfer file.txt from the victim machine to the forensic workstation. Therefore, first execute this command on the forensic workstation:

nc –v –l –p 2222 > ex030923.log

Next, type the following command on JBRWWW to transfer the file named ex030923.log to our forensic workstation:

type c:\winnt\system32\logfiles\w3svc1\ex030923.log | nc __forensic_workstation_ip_address 2222

Press CTRL-C when the file is finished transferring. This can be confirmed with a simple network monitoring session (described in a later chapter). We also performed the same series of commands to transfer ex031001.log to the forensic workstation.

When we open ex030923.log, we see the following header:

#Software: Microsoft Internet Information Services 5.0
#Version: 1.0
#Date: 2003-09-23 22:50:59
#Fields: time c-ip cs-method cs-uri-stem sc-status 

The date and time, the first bolded line, are actually reported in GMT, not EDT (JBRWWW’s local time zone). Keep this in mind because it can trip you up when correlating this information to other auditing material (such as file system time and date stamps). The second bolded line lists the recorded fields. These are the default fields that are recorded by the IIS server, but there are many more available if the Administrator enables them. A good reference for these fields exists at

http://www.microsoft.com/technet/prodtechnol/windowsserver2003/proddocs/standard/ref_we_logging.asp

As we begin to skim the first few lines, we notice something very interesting. First, the accesses happen very quickly, and the source IP address is 95.16.3.79. The speed of the Web accesses is much faster than one person can type. Second, the fourth request has an interesting keyword embedded in it:

22:51:17 95.16.3.79 GET /Nikto-1.30-Y7hUN21Duija.htm 404

Nikto is a well-known Web server vulnerability scanning tool available from http://www.cirt.net/code/nikto.shtml. It would make sense that a Web vulnerability scanning tool would access JBRWWW repeatedly in a short amount of time. Another telltale sign is the status code (the last number 404). Any time this number is in the 400s, the access was unsuccessful. If the status code was in the 200s, the access was successful. Web vulnerability scanners generate numerous result codes in the 400s. Other result codes can be compared to the chart at http://www.iisfaq.com/default.aspx?View=A145&P=230. Upon reviewing the log for September 9, we see that all the activity came from one IP address in less than one minute. JBRWWW was the victim of an HTTP vulnerability scan on that day.

On October 1, 2003, we see the following activity:

#Software: Microsoft Internet Information Services 5.0
#Version: 1.0
#Date: 2003-10-01 22:58:53
#Fields: time c-ip cs-method cs-uri-stem sc-status 
22:58:53 95.208.123.64 GET /NULL.printer 404
23:00:55 95.208.123.64 HEAD /iisstart.asp 200
23:01:18 95.16.3.79 GET /iisstart.asp 200
23:01:18 95.16.3.79 GET /pagerror.gif 200
23:01:18 95.16.3.79 GET /favicon.ico 404
23:03:23 95.208.123.64 GET /NULL.printer 404
23:08:45 95.16.3.79 GET /NULL.printer 404
23:15:09 95.208.123.64 OPTIONS / 200
23:16:30 95.208.123.64 OPTIONS / 200
23:16:30 95.208.123.64 PROPFIND /ADMIN$ 404
23:17:04 95.16.3.79 GET /scripts/../../../../winnt/system32/cmd.exe 200
23:17:54 95.16.3.79 GET /scripts/../../../../winnt/system32/cmd.exe 502
23:20:19 95.16.3.79 GET /scripts/..%5c..%5c..%5c../winnt/system32/cmd.exe 200
23:32:43 95.208.123.64 OPTIONS / 200
23:32:43 95.208.123.64 PROPFIND /ADMIN$ 404
23:33:52 95.208.123.64 PROPFIND /ADMIN$ 404
23:58:16 95.208.123.64 OPTIONS / 200
23:58:16 95.208.123.64 PROPFIND /ADMIN$ 404

The first bolded line is a telltale sign of the ".printer" Microsoft Windows 2000 buffer overflow (Securityfocus.com Bugtraq ID 2674) from IP address 95.208.123.64. Because we are seeing the attack in our logs, we know it was unsuccessful. Typically, when this buffer overflow is used against a vulnerable server, it causes the Web server to crash, so the activity is not logged in the IIS log. The next four lines not bolded are attributed to users at 95.208.123.64 and 95.16.3.79 accessing the default Web page, perhaps checking whether the Web server is available. The second set of bolded lines represents unsuccessful attempts from 95.208.123.64 and 95.16.3.79 using the same ".printer" buffer overflow. Seeing two IP addresses tells us that they may be the same person or more than one person working together.

The third set of bolded lines shows a successful (due to the result codes being 200 and 502) attack. If we dissect the attack, we see that someone accessed the C:\winnt\ system32\cmd.exe executable. The Web server should never access the cmd.exe command shell. In short, 95.208.123.64 was able to run commands on JBRWWW in the context of IUSR_JBRWWW (not Administrator). The first two bolded lines of this set show what is known as the Unicode attack. The last line shows the Double Decode attack (also referenced earlier in this chapter). Both attacks are a directory traversal attack in which the attacker escapes the directory to which the Web server is restricted to run arbitrary programs on the victim machine. To quickly locate similar attacks on other machines, we could easily search for cmd.exe in the IIS logs and see whether the result code was 200. Because JBRWWW did not enable more fields in the W3C extended logs, we cannot see what the attacker ran with the command shell.

Suspicious Files

If we were not acquiring a forensic duplication of JBRWWW, we could transfer any suspicious file with our "Poor Man’s FTP" using netcat. The syntax for the command to run on the forensic workstation is as follows:

nc –v –l –p 2222 > filename

Now, type the following command on JBRWWW to transfer the file named filename to our forensic workstation. Remember that the file named filename does not have to contain ASCII text. You can also transfer binary files on the victim machine in this manner.

type filename | nc forensic_workstation_ip_address 2222

The binaries that were flagged by our file system analysis because they were created during the intrusion include the following, in Table 1-3:

Table 1-3 The Suspicious Binaries Transferred from JBRWWW

We transferred these files to our forensic workstation and they are included on your DVD for further analysis.