The recent webcast “Storming the Breach, Part 1: Initial Infection Vector”, with Incident Response experts Wade Woolwine and Mike Scutt sparked so many great questions from our live attendees that we didn't have time to get through all of them! Our presenters took the time to answer additional questions after the fact... so read on for the overflow Q&A on tips and tricks for investigating common breach scenarios:
Q. “What do you recommend for sandboxing the browser?”
A. I'm a huge fan of Sandboxie, which is free for personal use. Sandboxie allows users to launch individual applications within a comprehensive application sandbox, and provides granular control over the application's access to the underlying operating system.
In enterprise environments, we often recommend that clients implement Microsoft's Enhanced Mitigation Experience Toolkit (EMET). IT and security personnel can implement EMET protection for any application via group policy, allowing the implementation of several anti-exploitation techniques at scale. EMET does a fantastic job of mitigating many browser-based exploits, malicious document exploits, and can be tailored to your environment. We recommend enforcing EMET for web browsers, Flash, Acrobat, and Office/Productivity suites, as they are commonly-targeted platforms.
Q. “A lot of this information assumes that you know when the malware was dropped in a reasonable window. For a company that gets contracted to do investigations is there a good way to efficiently identify that window that the malware was dropped?”
A. This is a fantastic question, with a not-so-fantastic answer (though one I give often): it depends. For this example I will briefly walk through a portion of my analysis methodology for Windows compromises. Note: each investigation is different, so I stress “incident response methodology” and critical thinking when training new analysts. For each incident you respond to, consider the following questions:
“What evidence do I have that indicates the system was compromised?” – Use this as your starting point. If a user received a pop-up message when logging in, there is likely persistent malware on the system that a tool such as Sysinternal's AutoRuns utility will help you identify the sample. If responding to a live system, look through volatile data to determine whether there are malicious processes running on the system, unusual DLLs loaded into processes, unusual open handles, and connections to odd or known bad domains/IPs. Use this data to determine where the malware is on disk or in the registry, and use the associated timestamps from those sources to assist in building a timeline of activity. If the initial alert came from AV / SIEM / IPS alerts, use that time period to temporarily reduce the scope of data that you're looking at, and focus on identifying newly created files (especially PE files), modified registry keys, and event log entries.
“What is the earliest evidence of compromise on the system?” – Once you've identified evidence to support the system being compromised, consider how the system was compromised. At some point either an attacker or malicious executable code interacted with the system. Work back from your earliest piece of evidence, and timeline logs, file system activity, registry activity, etc, until you've identified a potential method of ingress.
Ingress evidence often takes the form of an initial logon (for interactive attackers moving laterally / transferring files), Java or Flash artifacts from a user visiting a malicious site (or a legitimate site hosting malicious code), a user opening a malicious document, etc.
These are a few steps that help me find my bearing when responding to a breach, and are not comprehensive. There are a number of excellent Incident Response and OS internals books on the market that will provide a greater level of detail on Incident Response methodologies.
Focus on answering those questions when initially responding to a compromised system. It is seldom enough to simply determine whether there is malware on a system, identifying the initial ingress method / root cause of the compromise is critical to ensuring appropriate remediation and mitigation strategies are deployed for the compromised system and all vulnerable systems on the network
Q. “On the web application side, does it help to use IIS IP filtering to prevent malicious attacks from the outside?”
A. IP filtering is useful as a temporary remediation strategy when dealing with a novice attacker, however I do not recommend relying solely on this as a defensive measure to prevent web application compromises. Many attackers will use proxies such as TOR, other compromised systems, or larger infrastructure during an attack. I recommend following a “defense-in-depth” measure for protecting internet facing systems, to include placing a firewall tailored to your application (i.e. a Web Application Firewall), and an intrusion prevention system between your system and the internet. The system itself should run a host-based intrusion prevention application, a file integrity monitor, a host-based firewall, and the application should be using the most restrictive permissions possible. The system should be as isolated as possible from internal systems, and should be monitored carefully for anomalous behavior. Consider the following: if an attacker was attacking your application whilst using multiple TOR IPs and successfully compromised the application – is the application running on the system with elevated privileges? Can the attacker access critical files such as a database connector configuration file? We frequently respond to web server breaches, and often the underlying applications are running either with elevated privileges or the application user has access to core configuration files. Once an attacker obtains the configuration files it is trivial for them to connect to backend databases and dump the contents.
Q. “For web servers, do you see specific CMOs/hosting platforms more targeted than others? If so, is targeting because of popularity or insecurity? Ex: Wordpress or Joomla.”
A. In opportunistic attacks, absolutely. Most of the common content management systems on the market today are heavily targeted by automated scanners or opportunistic attackers, as are most of the common plugins. In targeted attacks, we've found that it doesn't matter whether the client is running a well-known CMS or something they've developed in-house, as the attackers will diligently reconnoiter the application and identify methods of exploitation. My mitigation strategies for these attacks remains the same: follow defense-in-depth best practices, patch your CMS and plugins, and ensure that the web application user is running with least-privileges.
Q. “Given that we have to isolate individual machines to mitigate the risk of spreading infection, it doesn't seem feasible to install diagnostic tools or VM platforms on every machine in an office. Would it be more feasible to carry around mobile platforms, like a collection of diagnostic tools stored on USB drives, and are there any risks of the mobile diagnostic tool contracting and spreading that infection?”
A. In these situations, I recommend either using a live-cd with the utilities you need, or purchasing a USB device with a write-block switch. In large environments, consider creating an isolated VLAN to place the compromised systems on, alongside an analysis machine booted from a Live-CD. This allows you to connect to the remote machine, perform your data analysis and remediation, and maintain a clean image remotely. Note: I do not suggest this technique for any work that may be used for litigation purposes. This method is not forensically sound in any way J.
Q. “You mentioned whitelist websites. How to effectively do when sites hosted around the world now. Also with akami and amazon being used to host, IP ranges constentaly change. Would proxy be best or blocked sites/regions via firewall?”
A. Whitelisting is typically only a component of the recommendations we have for mitigating browser based attacks. Specifically only allowing known good sites from the browsers in your enterprise will raise the barrier of entry for attackers. Additionally, we also recommend that customers run an intrusion detection and/or prevention system at the internet egress to alert on potential patterns in traffic that could provide some investigative leads.
Q. “How much of a concern do you have, during these investigations, is there much focus on collecting data in ways that can be used in litigation / prosecution? Or are you mostly concerned with finding breaches and mitigating, etc? Thanks much.”
A. This heavily depends on the client and the type of engagement. In situations where we know that the data will be used for litigation / prosecution we maintain forensically sound images of the affected systems and follow internal chain of custody protocols. In situations where it is unlikely that the data will be used outside of the investigation we attempt to avoid impacting the underlying systems as much as possible while collecting.
Q. “If we are using web-based email like Gmail and not Outlook, where else can we look for where the attacks are coming from or other logs inside the registry?”
A. Enterprise Google Apps clients have a wealth of logging for email. In a situation where a system was suspected of being compromised via a Gmail email we'd correlate timeline data from the user's browser histories, download histories, file system activity, and Gmail logging. These data sources will typically provide enough data to determine whether the system was compromised via a malicious email opened within Gmail.
Q. “What should we investigate on a ransomware infection on a PC?”
A. Root cause. The majority of the ransomware infections we run across are due to the browser rendering a page with an exploit kit that determines what vulnerable plugins and applications the browser is running. Once you've determined when the malware dropped on the system, timeline the file system activity prior to determine whether there were artifacts from Java, Flash, etc. created just prior to the compromise. Use that information to ensure that the rest of your systems are appropriately patched (and that your browsers are sandboxed / protected by EMET). Additionally, correlate the timeline data with the user's browser history to identify the malicious domain if possible, and sinkhole DNS resolution for the domain (at least temporarily) to prevent additional systems from being compromised.
Q. “Do you typically do full forensics for single PC issues or triage with tools like Sysinternals?”
A. This depends on why we're investigating the system. IF we have indications that an attacker interacted with the system, we'll typically perform a full forensic investigation to ensure we fully understand the scope of the attacker's actions. If we've simply identified that the attacker attempted to log into the system and need to confirm whether the attempt was successful and whether they interacted with the system we'll perform a lighter triage wherein we collect targeted data from a few relevant sources.
Q. “Do you have any tips for prioritizing how/where you hunt for malware on a machine that you suspect is infected?”
A. If I don't have an initial indicator of compromise aside from “Thar be dragons malware!” I employ a few techniques (Windows system – Linux methodology is similar but the particulars vary considerably):
1) Check running processes, and their associated network connections. – here I'm looking for unusual process names, paths, arguments, ports, and remote IPs. If I'm still struggling I'll enumerate the process's open handles, loaded DLLs, etc.
2) Review Prefetch - if enabled, the Windows Prefetch can help quickly identify unusual executables that have run recently. Review prefetch files for unusual target paths, unusual accessed files, and last execution time (for timelining).
3) Review persistent binaries – Use a tool such as Mandiant's Redline or Sysinternals Autoruns to collect a list of persistent samples (items that startup on boot / login / scheduled time). Look for unsual items by path, name, digital signature status, etc.
4) Review AppCompat /Shimcache – Review the application compatibility cache to get a rough timeline of executables run on the system. Look for unusual filenames & Paths, as well as “last modified” time. Note: there are intricacies and caveats involved in using this data. I recommend reading Mandiant's whitepaper prior to parsing this data to better understand its use: https://dl.mandiant.com/EE/library/Whitepaper_ShimCacheParser.pdf
Q. “What forensic tools would you recommend for a mobile IR kit? For the field”
1) Computer repair toolkit
2) Flash drives (several large a read-only drive)
3) Hard drives for imaging
4) Hard drive imager (we're big fans of the CRU Ditto – it has night vision mode!)
6) ESD evidence bags
7) Small switch and patch cables
8) Analysis rig with analysis VMs, Encase/FTK/Etc.
It sounds like a lot, but fits nicely into a standard laptop bag with careful organization.
Q. “Are there any known limitations to discovering malicious activity via behavioral analysis software like CrowdStrike?”
A. There are always limitations when automating malicious activity detection. We always recommend a combination of people and technology for effective threat detection. Additionally, we recommend that our customers use a constant feedback mechanism to ensure that lessons learned through threat detection and incident response are fed back into the technology to reduce false-positives and increase analyst efficiencies.
Q. “Do you find many organizations have SIEM tools to help corrolate the information in the logs when responding to incidents?”
A. Yes, we often find customers using SIEMs. In almost all cases, the SIEM doesn't ingest logs directly from the endpoint, but rather from select systems and devices (such as Windows domain controllers and critical network devices), which makes the SIEM less relevant during investigations of individual systems. Further, we typically recommend that customers invest in expanding their SIEM deployment to include endpoint logs to improve log based threat detection.
Q. “For crypto ransom ware what will be prevention step from antivirus concern or other....can we identify any encryption process is working during ransomware”
A. In certain situations, a key can be derived through malware analysis and memory analysis, but we typically recommend that customers be prepared for such events by implementing a good system and configuration backup strategy.
To see the full presentation on the expert's guide to investigating common breach scenarios: view the on-demand webinar now.