Series co-written with Jeremiah Dewey, Rapid7 Director of Incident Response.

Part 4: What to Do Under Fire

In our previous posts on building an Incident Response (IR) plan, we outlined how to draft the plan, how to review the plan, and how to test it. This final chapter will share our recommendations around handling a real incident. This will apply to escalated opportunistic attacks as well as the rare but real targeted threat.

We must emphasize that you, your team, and the employees in your company are the most important factors in responding to an incident. Attackers are people—driven by motives, objectives, and an attack opportunity. Properly piecing the puzzle together requires staying calm and not letting distractions like anger, helplessness, and fear get in the way of a coordinated response.

Breathe. Just Breathe. Now Reach Out.

In many cases, a member of your security or IT team will detect an incident in the form of suspicious activity somewhere on your network. Depending on your detection and response capabilities, this could range from identifying an anomalous network scan to activity present later in the attack chain, such as a high level of egress activity that could be data exfiltration.

First, get into the right gear.

Credit to @aloria for finding this gem.

Then, notify your infosec team lead or security manager. Key objectives: gain context, and scope the impact of the incident.

Build Context, Scope the Incident

Your goal is to define the parameters of the incident: the who, what and where malicious behavior is occurring. For example, if the suspicious activity occurred on an asset, consider the following:

  • What’s the criticality of the asset to the business?
  • What users have authenticated to it—can those users take privileged action?
  • Is your SIEM collecting endpoint data to provide additional context?

If user credentials are suspected to be compromised:

  • What admin privileges does this user have?
  • Contact the user—are they aware of what is happening?
  • What is happening across that user’s accounts?

Nearly every successful attack takes advantage of weaknesses across these three categories: vulnerabilities, misconfigurations, or stolen credentials. If your team has proactively assessed the lay of the land across the company network, that preparation will lead to a quicker severity assessment. If you haven’t, it will be difficult to determine if an alert stems from a misconfigured device, a BYOD from the development team, or an actual attacker using compromised credentials. More on how our Managed Detection and Response team investigates alerts.

Proper triage will help you prioritize, so you won’t mistakenly notify business leaders about a false-positive incident. If you’ve tested your incident response plan and escalate during a true time of need, you will likely draw the attention of your company leaders.

Track Everything

It can be easy to muddle up forensic evidence. Keep it as pristine as possible, not only for your team and your organization, but also for any external IR and legal teams that you loop into response.

That is why it’s important to keep volatile evidence, such as memory, intact. If compromised systems are simply powered down, which can easily happen during a panicked response, that can both tip off the attacker and erase important evidence. Identify in advance who in your organization has forensic expertise, and make sure that their contact info is explicitly listed in the incident response plan.

If you have an outside incident response partner such as Rapid7, contact them as soon as possible. For example, we’d much rather you call us about nothing, instead of delaying the response to a potentially important incident. You’ll have fresh sets of eyes on the situation and added firepower to determine and execute the best course for remediation.

An IR service provider can also share legal considerations gleaned from response experience, which will help the team avoid dangerous pitfalls. Learn more about our incident response services.

Get Counsel

If we’re dealing with an incident that involves customer data and breach of compliance, loop in legal counsel early. The actions you take during initial response may be scrutinized with regulatory and legal issues in mind later. Was your response timely and appropriate? Did negligence by anyone in your organization play a role? What about faulty hardware or vulnerabilities in deployed software? All may be subject to investigation later—legal counsel is here precisely for these moments.

Counsel will have advice to inform the all-important response by your PR team. As the authors of a recent Harvard Business Review article point out, even rumors of breaches can do a lot of damage. Good plan execution involves tight coordination across teams, technical and not.

It's important to get out in front of the crisis with clear and direct communication to the outside world. As well as helping you craft the best response, your legal counsel will help your PR team determine what you are legally required to report about any given incident.

Stick with the Plan

Finally, stick to your plan. This is why you practice it. At the same time, understand that every incident is unique and may require deviations when appropriate—ensure they are justified and documented.

Whatever the circumstances of a given crisis, any effort you front-load into drafting, reviewing, and testing will provide a solid foundation for quick, nuanced reactions that will serve you and your organization well.