welcome, earthling!

Navigate Your Journey

The Introduction

Arming & Aiming

Your Incident
Response Team

Incident Response
Process &
Procedures

The Art of Triage:

Types of Security
Incidents

Incident
Reponse Training

Incident
Response Tools

the
Introduction
The fight to protect your companys data isnt for the faint of heart. As an
embattled IT warrior, with more systems, apps, and users to support than ever
before, keeping everything up and running is a battle in itself. When it comes
to preventing the worst-case scenario from happening, you need all the help
you can get, despite your super-hero status.
Thats why weve developed this incident response guide. Weve collected and
curated decades of infosec war stories and hacker intelligence from across
the galaxy so that youre better armed in the fight against cybercrime. Youll
have an insiders perspective on how to build an incident response plan and
team, and what tools and training you can use to arm those team members.

what exactly is

Incident Response?
Were not Wikipedia or Websters, so if youre looking for a dictionary definition,
this isnt the right place. But if a five year old asked us, we might just say,
incident response is sort of like a fire drill for the IT guy. When the worst-case
scenario becomes reality, its essential to have the right plan in place, the
right people on the job, and the right tools and training to remain vigilant. And
thats what reading this incident response guide can do for you.

Preparation: 

Eradication: 

Preparing users and IT to handle potential

Finding and eliminating the root cause

incidents in case they happen (and lets

(removing

face it, we know they will)

production)

Identification: 

Recovery: 

Figuring out what we mean by a security

Permitting affected systems back into the

incident (which events can we ignore vs.

production environment (and watching

which we must act on right now?)

them closely)

Containment: 

Lessons Learned:

Isolating affected systems to prevent

Writing everything down and reviewing

further damage (automated quarantines

and analyzing with all team members so

are our favorite)

affected

systems

from

you can improve future incident response

efforts

tell me, why...

Do I Need an Incident Response Plan?

The problem with plans is that they are designed to sit on the shelf until the day when
the proverbial oxygen masks drop from the ceiling. Otherwise, they just gather dust
except for the occasional auditor visits or executive reviews.
In this guide, we take the active approach because we know that the investment
of time and resources spent enhancing incident response will have immediate and
ongoing benefits to IT operations. After all, security is a subset of reliability and
everyone wants their systems to be more reliable.
We will walk you through building a basic incident response plan and security
monitoring process, covering skills to acquire and helpful resources along the way.

whos the insider and

What Exactly is an Insiders Guide

to Incident Response?
Think of me as your personal shaman. Ive been an infosec geek for more than two
decades. Some of these gigs included managing a team of 24x7 SOC engineers,
others involved supporting the VP of IT Risk of a rather large company. Along the
way, Ive been able to collect an interesting set of cyber war horror stories to share
with you to enhance your own security programs.
At the same time, Im no innocent. I think whenever youve spent this much time
studying the mind of a cyber criminal - his tactics, his tools, and his mindset sometimes the lines between black and white get a bit fuzzy.
Being an insider means that you can see both sides of it. Youve been there, youve
done that. Just because youre an insider doesnt mean youre a threat. On the
contrary, youre an insider who understands threat. And thats the best kind.
Weve collected the best incident response stories and guidance from an army of
insiders, and basically, thats what the Insiders Guide to Incident Response gives you.

3 examples of insider wisdom...

Straight from the Incident Response Front Lines

On Defining Incident
Response Success.

Dont Panic. 
Stay Focused.

Start with Simple Steps.

There are many levels of success in

Execution is key - the range of ways

Attackers have technical and economic

defensive work the common wisdom

to attack a target can seem limitless

imperatives to use the minimum amount

is that the attacker only has to be right

- expecting to be an expert on all of

of effort and resources to breach their

once, but the defender has to be right

them is pointlessly unrealistic. The most

targets - the more you remove the low-

every time, but thats not always true.

important part of incident response is

hanging fruit on your network, the more

Attacks are not all-or-nothing affairs -

to handle every situation in a way that

you raise the actual level of work an

they happen over time, with multiple

limits damage, and reduces recovery

attacker has to expend to successfully

stages before final success. To remain

time and costs. At the end of the

infiltrate it.

undetected

attentive

day, thats how youll be measured

defender, it is the attacker who must

on a job well done not that youve

make every move correctly; if an astute

covered every angle of every potential

defender detects them even once, they

vulnerability.

against

an

Attackers are Lazy.

have the possibility to locate and stop

the whole attack. You arent going to
immediately detect everything that
happens during an attack - but as long
as you detect (and correctly identify)
enough of an attack to stop it in its
tracks, thats success.

Onto Chapter 1 >

arming & aiming your incident response team

6

one

Arming & Aiming Your

Incident Response Team
As much as we may wish it werent so, there are some things that only people, and in some cases,
only certain people, can do. As one of the smartest guys in cyber security points out above, some
things cant be automated, and incident response is one of them. Thats why having an incident
response team armed and ready to go - before an actual incident needs responding to, well, thats
a smart idea.
There are several things well cover in this chapter of the Insiders Guide to Incident Response. First
of all, your incident response team will need to be armed, and they will need to be aimed. Even
though we cover true armature in terms of incident response tools in Chapter 4, well share some
of the secrets of internal armor - advice that will help your team be empowered in the event of a
worst-case scenario.
And second, your incident response team will need to be aimed. In any team endeavor, goal setting
is critical because it enables you to stay focused, even in times of extreme crisis and stress.
In this chapter, youll learn how to assemble and organize an incident response team, how to arm
them and keep them focused on containing, investigating, responding to and recovering from
incidents.

Incident Response needs people,

because successful Incident
Response requires thinking.
Bruce Schneier, Schneier on Security

lets start with the basics...

Whos on the Incident Response Team?

Weve put together the core functions of an incident response
team in this handy graphic. Since every company will have

Team Leader

differently sized and skilled staff, we referenced the core

 rives and coordinates

D
all incident response
team activity, and keeps
the team focused on
minimizing damage, and
recovering quickly.

functions vs. the potential titles of incident response team

Lead Invesitgator

IT leads with strong executive support

& inter-departmental participation. 

Collects and analyzes

all evidence, determines
root cause, directs the
other security analysts,
and implements
rapid system and
service recovery.

members. So you might find that a single person could fulfill

two functions, or you might want to dedicate more than one
person to a single function, depending on your team makeup.
That said, here are a few other key considerations to keep in
mind:

When it comes to cyber security incident response, IT should

be leading the incident response effort, with executive
representation from each major business unit, especially
when it comes to Legal and HR. While the active members
of the incident response team will likely not be senior
executives, plan on asking executives to participate in major

Communications Lead

Leads the effort

on messaging and
communications for all
audiences, inside and
outside of the company.

recruitment and communications efforts.

Clearly define, document, &

communicate the roles & responsibilities
for each team member. 
While weve provided general functions like documentation,
communication, and investigation, youll want to get more

Documentation and
Timeline Leador

Documents all team

activities, especially
investigation, discovery
and recovery tasks,
and develops reliable
timeline for each stage
of the incident.

HR/Legal Representation
Just as you would guess.
Since an incident may
or may not develop
into criminal charges,
its essential to have
legal and HR guidance
and participation.

specific when outlining your incident response team member

roles. Make sure that you document these roles and clearly
communicate them, so that your incident response team
is well coordinated and knows what is expected of them before a crisis happens.

Establish, confirm, & publish

communication channels &
meeting schedules. 
Effective communication is the secret to success for any
project, and its especially true for incident response. Print
out team member contact information and distribute it widely
(dont just rely on soft copies of phone directories. Chances
are, you may not have access to them during an incident).
Include important external contacts as well, and make sure
to discuss and document when, how, and who to contact at
outside entities, such as law enforcement, the media, or other
incident response organizations like an ISAC.
8

tell me...

What Does an Incident Response Team Do?

An incident response team analyzes information, discusses
observations and activities, and shares important reports and
communications across the company. The amount of time
spent on any of one of these activities depends on one key
question: Is this a time of calm or crisis? When not actively
investigating or responding to an incident, the incident
response team should meet at least quarterly, to review
current security trends and incident response procedures.
The more information that an incident response team can
provide to the executive staff, the better, in terms of retaining
executive support and participation when its especially
needed (during a crisis or immediately after).

i wonder...

Where Should Incident Response

Team Members Be Located?
Most companies span across multiple
locations,

and

unfortunately,

most

incidents do the same. While you might

not be able to have a primary incident
response team member onsite at every
location, strive to have local presence
where the majority of business and IT
operations happen. The likelihood that
youll need physical access to perform
certain

investigations

and

analysis

activities is pretty high even for trivial

things like rebooting a server or swapping
out a HDD.

tell me...

Whats the Goal of an Incident Response Team?

The incident response teams goal is to coordinate and align the key resources and team members during a cyber security incident to
minimize impact and restore operations as quickly as possible. This includes the following critical functions: investigation and analysis,
communications, training, and awareness as well as documentation and timeline development.

investigation / analysis
key questions

key tactics

Is this an incident that requires attention

now? Which assets are impacted?

Determine and document the

scope, priority, and impact.

reporting / communications
key questions

key tactics

Which types of security incidents do we

include in our daily, weekly, and monthly
reports? Who is on the distribution list?
What information can we provide to
the executive team to maintain visibility
and awareness (e.g. industry reports,
user behavioral patterns, etc.)?

Define and categorize security

incidents based on asset value/impact.
Document and educate team members
on appropriate reporting procedures.
Collect relevant trending data and other
information to showcase the value the IR
team can bring to the overall business.

Response / Improvement
key questions

key tactics

Whats the most effective way to

investigate and recover data and
functionality? How do we improve
our response capabilities?

Investigate root cause, document

findings, implement recovery
strategies, and communicate
status to team members.

10

i wonder...

How Should I Choose the Right Incident

Response Team Members?
In terms of incident response team membership recruitment, here are three key considerations based
on NISTs recommendations from their Computer Security Incident Handling guide.

Aim for 24/7

Availability
Chances are, your company is like
most, and youll need to have incident
response team members available
on a 24x7x365 basis. In fact, from my
experience and those of other insiders,
Friday afternoons always seemed to be
the bewitching hour, especially when
it was a holiday weekend. Please note
that you may need some onsite staff
support in certain cases, so living close
to the office can be a real asset in an
incident response team member.

Consider Virtual
or Volunteer Team
Members (if full-time
isnt an option)

Monitor & Bolster

Employee &
Team Morale
Incident response work is very stressful,

You may not have the ability to assign

and

full-time responsibilities to all of your

take a toll on the team. This makes

incident

members.

it easy for incident response team

With a small staff, consider having

members to become frazzled or lose

some team members serve as a part

motivation and focus. It is important to

of a virtual incident response team.

counteract staff burnout by providing

A virtual incident response team is a

opportunities for learning and growth

bit like a volunteer fire department.

as well as team building and improved

When an emergency occurs, the team

communication. You may also want

members are contacted and assembled

to consider outsourcing some of the

quickly, and those who can assist do

incident response activities (e.g. SIEM

so. Typically, the IT help desk serves

monitoring) to a trusted partner or

as the first point of contact for incident

MSSP.

response

team

being

constantly

on-call

can

reporting. The help desk members

can be trained to perform the initial
investigation and data gathering and
then alert the incident response team
if it appears that a serious incident has
occurred.

11

could you explain...

Why Participate on an Incident Response Team?

As we pointed out before, incident response is not for the

d at
overhear infosec

faint of heart. It takes an extraordinary person who combines

intellectual curiosity with a tireless passion for never giving
up, especially during times of crisis. This description sounds
a lot like what it takes to be a great leader. And thats what

THE DIFFERENCE BETWEEN IT AND IR IS

LIKE THE DIFFERENCE BETWEEN BEING A
DERMATOLOGIST AND AN ER DOCTOR....

GIVE ME A CASE OF BAD

ACNE ANYTIME OVER A
GUN SHOOTING AT 2 AM.

attracts many of us insiders to join the incident response

team. The opportunity to become and be seen as a leader
inside and outside of your company is one that doesnt come
often, and can reap more benefits than can be imagined at
first. Youll learn things youve never learned inside of a data
center (e.g. disclosure rules and procedures, how to speak
effectively with the press and executives, etc.) and youll be
seen as a leader throughout your company.

tell me...

How Can the Team be Armed?

ONE WORD: EMPOWERMENT

If an incident response team isnt empowered to do what
needs to be done during a time of crisis, they will never
be successful. Thats why its essential to have executive
participation be as visible as possible, and as consistent as
possible. Otherwise, the incident response team wont be
armed effectively to minimize impact and recover quickly
no matter what the scope of the incident.
The key is to sell the incident response team to the executive
staff. No matter the industry, executives are always interested
in ways to make money and avoid losing it. The stronger
you can tie your incident response team goals and activities
to real, measurable risk reduction (in other words cost
reduction), then the easier it will be for them to say yes, and
stay engaged.
Quantifiable metrics (e.g. number of hours of work reduced
based on using a new forensics tool) and reliable reporting
and communication will be the best ways to keep the incident

68%
OF RESPONDENTS TO

A RECENT SAN SURVEY

CITED A SKILLS SHORTAGE

AS BEING AN IMPEDIMENT

TO EFFECTIVE

INCIDENT

RESPONSE

response team front-and-center in terms of executive priority

and support.

See the Survey: Maturing and Specializing:

Incident Response Capabilities Needed >
12

for those who are new to cyber security & incident response,

What skills are needed?

The incident response teams goal is to coordinate and align

Security Analysis is detective work while other technical

the key resources and team members during a cyber security

work pits you versus your knowledge of the technology,

incident to minimize impact and restore operations as quickly

Security Analysis is one where youre competing against

as possible. This includes the following critical functions:

an unknown and anonymous persons knowledge of the

investigation

training,

technology. Detective work is full of false leads, dead ends,

and awareness as well as documentation and timeline

and

analysis,

communications,

bad evidence, and unreliable witnesses youre going to

development.

learn to develop many of the same skills to deal with these.

Here are five lessons were happy to share:

Look for the Common Denominators.

Look for the Common Exceptions.
Security analysis inevitably involves poring over large sets of data
log files, databases, and events from security controls. Finding
leads within big blocks of information logs, databases, etc, means
finding the edge cases and aggregates what is the most common
thing out there, the least common what do those groups have in
common, which ones stand out?

Make Assertions, Not Assumptions.

Dont make assumptions, common wisdom says theyre right,
assuming that something is there and continuing on that assumption
will lead to poor results in incident response work. But in an effort
to avoid making assumptions, people fall into the trap of not making
assertions. In order to find the truth, youll need to put together some
logical connections and test them.
If I know that this system is X, and Ive seen alert Y, then I should see
event Z on this other system.

A system may make 10,000 TCP connections a day but which hosts
did it only connect to once? When following a trail of logs, always be
looking for the things you can group together, with something they
have in common, then find the one that stands out.

This is an assertion something that is testable and if it proves

true, you know you are on the right track! (assuming your assertion is
based on correct information). Always be testing.

Eliminate the Impossible.

Always Look for a Simpler Explanation.

According to good ol Sherlock Holmes, When you have eliminated

the impossible, whatever remains, however improbable must be
the Truth.

Never attribute to malice, that which is adequately explained by

stupidity. Hanlons Razor.

You are going to encounter many occasions where you dont know
exactly what you are looking for to the point where you might
not even recognize it if you were looking directly at it. In these
circumstances, the most productive way forward is to eliminate the
things that you can explain away until you are left with the things
that you have no immediate answer to and thats where find the
truth.

What makes incident response so rewarding is the promise of

hunting down and stopping that red letter day intrusion before it
can do the real damage. When your job involves looking for malicious
activity, its all too easy to see it everywhere you look.
Sometimes that attack youre sure you have discovered is just
someone clicking the wrong configuration checkbox, or specifying
the wrong netmask on a network range.

Imagine Things from the Attackers Perspective.

What Would You do in Their Position?
Experienced incident response analysts, hunting down intrusions being controlled by live human attackers in
pursuit of major corporate IP theft, have a skill that cannot be taught, nor adequately explained here.
From experience administrating systems, building systems, writing software, configuring networks but also,
from knowing how to break into them you can develop that ability to ask yourself what would I next do
in their position? and make an assertion on that question that you can test (and it may often prove right,
allowing you to jump ahead several steps in the investigation successfully).
Bottom line: Study systems, study attacks, study attackers- understand how they think get into their head.
Be smarter than your opponent.

13

Can you share some of the lessons youve

learned from surviving a data breach?

Give people a place to talk

Here are the things you should know about what a

stress levels will be high, and the PR and legal secrecy

breach looks like, from ground zero, ahead of time.

machine will be in full force. Many employees may

Stress levels will be at an all-time high, interpersonal

have had such a bad experience with the whole affair,

conflicts will boil to the surface, that dry-run disaster

that they may decide to quit. Keeping secrets for other

planning drill youve been meaning to do for months,

people is a stress factor most people did not consider

but never found the time for? That one minor change

when they went into security as a career choice. Invite

request your senior engineers have had sitting on the

your HR department staff to join any NDA discussions,

table for weeks that consistently got deferred in favor

and give employees a place to vent their concerns

of deploying that cool new app for the sales team? You

confidentially and legally. Youll be rewarded with many

betcha, good times.

fewer open slots to fill in the months following a breach.

Here are some of the things you can do, to give yourself

Let others learn from your mistakes

a fighting chance:

Nondisclosure agreements will be flying left and right,

If you are required to disclose a breach to the public,

Dont Let Security Be an Island

work with PR and legal to disclose information in a way

IT departments (and engineers) are notorious for the

something from your experiences. Adam Shostack

ivory tower attitude, we invented the term luser to

points out in The New School of Information Security

describe the biggest problem with any network. Create

that no company that has disclosed a breach has seen

some meetings outside the IT Comfort Zone every so

its stock price permanently suffer as a result. However

often; the first time you meet the legal and PR teams

the fallout of intentionally vague and misleading

shouldnt really be in the middle of a five-alarm fire.

disclosures may hang over a companys reputation

Bring some of the people on the ground into the

for some time. Sharing lessons learned can provide

incident response planning process - soliciting input

enormous benefits to a companys reputation within

from the people who maintain the systems that support

their own industries as well as the broader market.

your business processes every day, can give much

more accurate insight into what can go wrong for your
business/than any book full of generic examples can.
These are the people that spend their day staring at the
pieces of the infrastructure that are held together with
duct-tape and chicken wire.

that the rest of the world can feel like they have learned

It gets better
Famously overheard at a recent infosec conference
- Were only one more breach away from our next
budget increase!. Theres nothing like a breach to put

14

security back on the executive teams radar. Take this

as an opportunity for new ideas and approaches, not

Dont Panic!

just Were finally getting that thing weve been asking

Murphys Law will be in full effect. The information the

for, all year. Use the opportunity to consider new

executive team is asking for, was only being recorded

directions beyond the constraints of the old normal.

by that one system that was down for its maintenance

Now is the time to take Misfortune is just opportunity

window, the report you need right now, will take

in disguise to heart.

another hour to generate and the only person with free

hands you have available, hasnt been trained on how

Test for Impact, not vulnerabilities

to perform the task you need done before the lawyers

If you are spending money on third-party penetration

mistakes, mistakes get in the way of work. This advice

testing, you should be expecting more in return

works from both ends of the command chain - if your

than the output of a vulnerability scanner and some

executive team is expecting a fifteen-minute status

compromised systems - expect reports that show

update conference call every hour, thats 25% less

results in terms of impact to business operations,

work the people on the ground are getting done. Calm

bottom lines and branding - these are the things your

Heads Rule The Day - set expectations early on and

executives need to be aware of - either you look for

dont go into a disaster recovery plan that principally

and determine them ahead of time, or your attacks do.

operates on the impossible expectations.

check in for their hourly status update. Panic generates

Onto Chapter 2 >

incident response process & procedures
15

two

Incident Response Process

& Procedures
When most of us hear terms like incident response process and procedures our eyes tend to wander, and
our attention starts to drift. Yawn, right?
But, at the same time, its a necessary evil these days. How many times do you have to hear that data
breaches are inevitable in a single day? Especially at an RSA conference, not to mention your LinkedIn news
feed or the front page of USA Today.
Consider this chapter your resource guide for building your own incident response process, from an insider
whos realized - the hard way - that putting incident response checklists together and telling other people
about them can honestly make your life easier. In fact, it may even help you keep your sanity. Believe me.

So, what is an incident response process?

At the end of the day, its a business process. In fact, an incident response process is a business process that
enables you to remain in business. Quite existential, isnt it?
Specifically, an incident response process is a collection of procedures aimed at identifying, investigating
and responding to potential security incidents in a way that minimizes impact and supports rapid recovery.
Take it from me and many of my friends who wear these battle scars the more you can approach an incident
response process as a business process - from every angle, and with every audience - the more successful
you will be.

Whats the difference between an incident response

process and incident response procedures?
Even though the terms incident response process and incident response procedures are often used
interchangeably, weve used them in specific ways throughout this guide. An incident response process is
the entire lifecycle (and feedback loop) of an incident investigation, while incident response procedures are
the specific tactics you and your team will be involved in during an incident response process.

16

tell me about the...

Incident Response Process: Preparation

Prioritize your assets,
capture baselines

Direct & document actions,

deliver regular updates

Ask yourself and your leadership, what are our most important

Answer these questions for each team member:

assets? In other words, what servers, apps, workloads, or

network segments could potentially put us out of business if
they went offline for an hour? A day? What information could
do the same if it fell into the wrong hands?
By the way, the assets that you consider as important to the
business may not be the ones that your attacker sees as
important (more on that concept in Chapter Three).

What am I doing?

When am I doing it?

Why am I doing it?

The incident response team members - especially those who

are outside of IT - will need ample instruction, guidance, and
direction on their roles and responsibilities. Write this down

Develop a list of the top tier applications, users, networks,

and review it individually and as a team. The time you spend

databases, and other key assets based on their impact

doing this before a major incident will be worth the investment

to business operations should they go offline, or become

later on when crisis hits. Everyone involved, especially the

compromised in other ways.

executive team, will appreciate receiving regular updates, so

Quantify asset values as accurately as possible

negotiate a frequency that works for everyone and stick to it.

because this will help you justify your budget.

Finally, capture traffic patterns and baselines so

that you can build an accurate picture of what
constitutes normal. Youll need this foundation to
spot anomalies that could signal a potential incident.

Connect, communicate
& collaborate
Meet with executive leadership, share your analysis of the
current security posture of the company, review industry
trends, key areas of concern, and your recommendations.
Set expectations on what the IR team will do, along with what
other companies are doing, as well as what to expect in terms

ard at blackhat
e
h
r
e
v
o
BECOME FRIENDS
WITH YOUR CFO. ITS
THE BEST ADVICE IVE
HEARD HERE SO FAR.

of communications, metrics, and contributions. Find out the

best way to work with the legal, HR, and procurement teams
to fast track requests during essential incident response
procedures.

17

lets talk about...

Methodology: The OODA Loop

Its not unusual to see a lot of InfoSec
warriors use military terms or phrases to
describe what we do. Things like DMZ
and command and control are obvious
examples, but one of the best that Ive
seen for incident response is the OODA
Loop. Developed by US Air Force military

OBSERVE

strategist John Boyd, the OODA loop stands

for Observe, Orient, Decide, and Act.

ACT

Imagine youre a pilot in a dogfight. You

incident
response

ORIENT
DECIDE

need a tool to determine the best way to act

as quickly as possible when youre under
attack. Its a useful analogy when applied to
an incident response process.

THE OODA LOOP

Putting the OODA Loop into Your Incident Response Process

OBSERVE:

USE SECURITY MONITORING TO IDENTIFY ANOMALOUS

BEHAVIOR THAT MAY REQUIRE INVESTIGATION.

tools & tactics

questions to ask

key takeaways

Log Analysis; SIEM Alerts;

IDS Alerts; Traffic Analysis;
Netflow Tools; Vulnerability
Analysis; Application
Performance Monitoring

Whats normal activity on my

network? How can I capture
and categorize events or user
activity that arent normal? And
that require my attention now?
How can I fine-tune my security
monitoring infrastructure?

The more observations you can

make (and document) about
your network and your business
operations, the more successful
youll be at defense and response.
Bonus tip: Share additional
observations with executives
that could improve overall
business operations and
efficiencies - beyond IR.

18

ORIENT:

EVALUATE WHATS GOING ON IN THE CYBER THREAT LANDSCAPE

& INSIDE YOUR COMPANY. MAKE LOGICAL CONNECTIONS & USE
REAL-TIME CONTEXT TO FOCUS ON PRIORITY EVENTS.

tools & tactics

questions to ask

key takeaways

Incident Triage; Situational

Awareness; Threat Intelligence;
Security Research

Is our company rolling out a new

software package or planning
layoffs? Have we (or others
in our industry) seen attacks
from this particular IP address
before? Whats the root cause?
Whats the scope and impact?

Get inside the mind of the

attacker so that you can orient
your defense strategies against
the latest attack tools and tactics.
These are constantly changing
so make sure you have the latest
threat intelligence feeding your
security monitoring tools to
ensure that they are capturing the
right information and providing
the necessary context.
Bonus tip: Avoid the
distraction (and lunacy) of
attack back strategies you
have enough work to do.

DECIDE:

BASED ON OBSERVATIONS & CONTEXT, CHOOSE THE BEST

TACTIC FOR MINIMAL DAMAGE & FASTEST RECOVERY.

tools & tactics

questions to ask

key takeaways

Your Companys Corporate

Security Policy ; Hard copy
documentation (notebook,
pen, and clock)

What do we recommend doing

based on the facts available to us?

Document all aspects of the

incident response process,
especially communications
regarding data collection and
the decision-making processes.
Bonus tip: Use incident response
checklists for multiple response
and recovery procedures,
the more detailed, the better.
We cover the essential
ones in Chapter Three.

19

ACT:

REMEDIATE & RECOVER. IMPROVE INCIDENT RESPONSE

PROCEDURES BASED ON LESSONS LEARNED.

tools & tactics

questions to ask

key takeaways

Data capture and forensics

analysis tools; System backup
& recovery tools; Patch mgmt.
and other systems mgmt;
Security Awareness Training
tools and programs

Whats the quickest way to remedy

affected systems and bring
them back online? How can we
prevent this in the future? How
can we train users better so that
these things dont happen again?
Does our business process get
adjusted based on these lessons?

Training, communication, and

continual improvement are
the keys to success in acting
effectively during an incident.
Team members should know
what is expected of them and
that means in-depth training,
detailed run-throughs, and
keen attention on how to
continually improve teamwork
and the overall process.
Bonus tip: Use incident response
checklists for multiple response
and recovery procedures. The
more detailed, the better.

how about...

Incident Response Procedures:

The Need for Checklists
One of my former bosses was also a former pilot, and so of

down a specific task or activity into bite-site chunks. They

course, we had a checklist for everything. And after going

may also involve a few meandering offshoots or if then

through one too many real fires (not to mention fire drills), I

branches off your main checklist, and thats likely where the

can safely say Im really glad we had them. And I can also

richest detail will be necessary. Keep in mind though that you

safely say that they were constantly being edited for clarity

may not be able to predict all incident scenarios, and these

and efficiency after training exercises, and after real

checklists wont necessarily capture everything that could

incidents. There was always a better way to do something,

happen.

and certainly a better way of explaining how to do it.

Every business operation will dictate whats considered

So what kind of incident

response checklists will I need?

essential for that specific business, because the critical

Yes, thats the right question. Because there will definitely be

that can be considered essential for any business. Here are

more than one single incident response checklist. The best

a few examples, along with a few references for additional

checklists are those that apply to specific scenarios and break

information.

business systems and operations to recover first will be

different. That said, there are a few general types of checklists

20

Forensic analysis checklists (customized for all critical systems)

During the process of investigating an incident youll likely need to look deeper at individual
systems. A checklist that provides useful commands and areas to look for strange behavior
will be invaluable. And if your company is like most, youll have a mix of Windows and Unix
flavors. Customize each checklist on an OS basis, as well as on a functional basis (file server
vs. database vs. webserver vs. domain controller vs. DNS). 
Some useful references: SANS Incident Handling Handbook and Lenny Zeltsers Security
Checklists

Emergency contact communications checklist

Dont wait until an incident to try and figure out who you need to call, when its appropriate
to do so, how you reach them, why you need to reach them, and what to say once you
do. Instead, develop a detailed communication plan with the specifics of when to put it
into place and dont forget to get overall consensus on your approach. The entire incident
response team should know whom to contact, when it is appropriate to contact them, and
why. In particular, review the potential worst case scenarios (e.g. an online ordering system
going down right in the middle of Cyber Monday) and identify the essential staff who can
get these critical systems back online, as well as the management team who will need to
remain updated throughout the crisis.
Bonus tip: Youll also need to document when it is or is not appropriate to include law
enforcement during an incident, so make sure you get the necessary input and expertise
on these key questions.

System backup and recovery checklists

(for all OSes in use,including databases)
Each system will have a different set of checklist tasks based on its distinct operating system
and configurations. Its also important to note the time it takes for each step required to
restore operations, and also test full system backup and full system recovery while youre
documenting each checklist. There should also be specific steps listed for testing and
verifying that any compromised systems are completely clean and fully functional. 

21

Jumpbag checklists
SANS, one of the premier sources of information for the incident responder, recommends
that each incident response team member have an organized and protected jump bag
all ready to go that contains the important tools needed for a quick grab-and-go type of
response. Their recommended items include:

An Incident Handlers Journal

to be used for documenting
the who, what, where, why,
and how during an incident

Your incident response

team contact list

USB drives

A bootable USB drive or Live

CD with up-to-date anti-malware
and other software that can read
and/or write to file systems of
your computing environment
(and test this, please)

A laptop with forensic software

(e.g. FTK or EnCase)

Anti-malware utilities

Computer and network tool kits

to add/remove components,
wire network cables, etc. and
hard duplicators with write-block
capabilities to create forensically
sound copies of hard drive images.

Security policy review checklist (post-incident)

The most important lessons to learn after an incident are how to prevent a similar incident
from happening in the future. In addition to potential updates to your security policy, expect
incidents to result in updates to your security awareness program because invariably, most
incidents result from a lack of user education around basic security best practices. At the
very least, this checklist should capture:

When the problem was

first detected, by whom,
and by which method

The scope of the incident

How it was contained

and eradicated

The work performed

during recovery

Areas where the incident

response teams were effective

Areas that need improvement:

Which security controls failed
(including our monitoring tools)?
How can we improve
those controls?
How can we improve our
security awareness programs?

22

one last thing...

The Need for Incident Response

Forms & Surveys
INCIDENT RESPONSE SURVEY

As weve mentioned several times already, youll

need to document many things during your job as
an incident responder. The best way weve seen to
capture an accurate, standard, and repeatable set
of information is to do it with a form. And, thankfully,
SANS has provided a form for every type of security
incident tidbit youll need from contacts to activity logs
with specific forms for handling intellectual property
incidents.

Onto Chapter 3 >

the art of triage: types of security incidents
23

Incident Response Myth Busting for Executives

Myth #1: An incident response process begins at the time of an incident
Truth: Actually, an incident response process never

Advice: Give your executives some analogies that

ends. Its a continual process, like other business

theyll understand. For example, an incident response

processes that never end.

process is like a subscription-based business model,

e.g. software-as-a-service. Its always on. Its important
to point out that there will be stages of criticality for
incidents, some that will require more serious reporting
and external involvement, and some that wont. See
Chapter 3 for more details on this.

Myth #2: Each incident is a discrete, monolithic event, which may occur 1-2 times a year
Truth: As many of us know, were constantly working

Advice: Explain - at a high level - how incident

on incidents. Evaluating log files, investigating outages,

response works. As a continual process, its a daily

and tweaking our monitoring tools at the same time.

activity, that moves from high level investigations

Some of these are related to each other, and some

and pivots to specific abnormalities or outages,

arent. And again, its constant, daily work.

sometimes developing into something more significant,

and sometimes not. Share an example of a specific
investigation and offer to provide weekly updates on
incident response process metrics, cyber security
threat trends, system performance data, user activity
reporting, or any other information that would be
relevant for the executive team.

Myth #3: We havent had any incidents yet, so why do we even need this tool or that resource?
Truth: Its hard to believe, but there are still skeptics

Advice: Time for more executive education. Point out

about the very real cyber security risks facing us, and

that youve done your best to mitigate major risks up

the even more real possibility of becoming the next

until this point, but the adversary continues to up their

victim. When it comes to cyber security, looking at past

game. Its sort of like that moment in Jaws, youre

experience reveals nothing about what could happen

going to need a bigger boat!

in the future, particularly considering the pace of

innovation happening in cyber crime.
24

three
The Art of Triage: Types of
Security Incidents
not everything is an emergency, but anything could become one.
Understanding whether an event is an actual incident reminds me of that common expression, I
know it when I see it made famous by US Supreme Court Justice Stewart. He was referring to
obscenity rather than incident response, but a common misperception of knowing it when you see
it can often plague the most well intentioned incident responders.
The uncomfortable truth is that you may not know it when you see it, because the latest attacker
tools and techniques are increasingly stealthy, and can often hide in plain sight. The trick is to
view your network and operations from the perspective of an attacker, looking for key indicators
and areas of exposure before theyre exploited. And it all comes down to how artfully you can do
incident triage.
Typically used within the medical community, effective triage saves lives by helping emergency
medical personnel rapidly assess wound or illness severity and establish the right protocols, in the
right order, to reduce trauma and sustain patient health and recovery. All in the midst of crisis, when
every second counts.
In this chapter, well give you the tools to craft your ability to triage information security incident
types. Youll learn how to identify the various types of security incidents by understanding how
attacks unfold, and how to effectively respond before they get out of hand.

security incidents vs. information security incidents

A quick note on the difference between a security incident and an information security
incident In this guide, the assumption is that were focused on the various types of
information security incidents vs. your standard security incident, which might not involve
digital information and could be completely contained within the physical world (e.g.
physical assault). That said, there may be occasions that mix things up - types of information
security incidents or attacks that do involve a physical component (e.g. laptop theft).
25

why do...

Different Types of Security Incidents

Merit D
 ifferent Response Strategies
So what are you protecting against? The best way to determine the appropriate incident response in any given situation is to understand
what types of attacks are likely to be used against your organization. For example, NIST has provided the following list of the different
attack vectors:

External/Removable Media: 

Email: 

An attack executed from removable media (e.g.,

An attack executed via an email message or

flash drive, CD) or a peripheral device.

attachment (e.g. malware infection).

Attrition: 

Improper Usage: 

An attack that employs brute force methods

Any incident resulting from violation of an

to compromise, degrade, or destroy systems,

organizations acceptable usage policies by an

networks, or services.

authorized user, excluding the above categories.

Web: 

Loss or Theft:

An attack executed from a website or a web-

The loss or theft of a computing device or media

based application (e.g. drive-by download).

used by the organization, such as a laptop or

smartphone.

Other: 
An attack that does not fit into any of the other
categories.

bonus tip

bonus tip

REVIEW THE ABOVE LIST WITH AN

IDENTIFY WHICH PIECES OF

EYE TOWARDS ENSURING THAT YOUR

SECURITY POLICIES AND CONTROLS
HAVE MITIGATED THE MAJORITY OF
THE RISKS PRESENTED BY THESE
VARIOUS ATTACK VECTORS. YOULL
ALSO USE THIS LIST TO GUIDE YOUR
TEAM IN DETERMINING HOW TO

EQUIPMENT WOULD CAUSE THE

GREATEST RISK TO THE COMPANY
IN THE EVENT OF LOSS OR THEFT. IN
MOST COMPANIES, THE CFOS LAPTOP
WOULD BE INCLUDED ALONG WITH
ANY SERVER HDD CONTAINING IP OR
OTHER SENSITIVE DATA.

CLASSIFY THE VARIOUS TYPES OF

SECURITY INCIDENTS.
26

how do you...

Categorize Information Security Incident Types

by GettingInside the Mind of the Attacker
One of the biggest fallacies with traditional information

The cyber kill chain is a sequence of stages required for

security is the underlying assumption that you know

an attacker to successfully infiltrate a network and exfiltrate

which path an attacker will take through your network. For

data from it. Each stage demonstrates a specific goal along

example, attackers rarely come through your front door, or

the attackers path. Designing your monitoring and response

in this context, your gateway firewall. But each attack does

plan around the cyber kill chain model is an effective method

generally work through a certain pattern, or what Lockheed

because it focuses on how actual attacks happen.

Martin has called the cyber kill chain.

the cyber kill chain stages: how attacks progress

Attackers Goals:

reconnaissance
& probing

Find target
Develop plan of attack based on opportunities for exploit

Attackers Goals:

Delivery &
attack

Use social engineering to induce target to access malware or other exploit

stage 3

Exploit vulnerabilities on target systems to acquire access

Elevate user priveleges and install persistence payload

Attackers Goals:

System
Compromise

stage 2

Place delivery mechanism online

Attackers Goals:

Exploitation &
Installation

stage 1

stage 4

Ex-filtrate high-value data as quietly and quickly as possible

Use compromised system to gain additional access, steal computing resources,
and/or use in an attack against someone else

27

tell me...

Which Security Events Do I Really

Need to Worry About?
Which security events develop into the type of information security incident that requires my attention now? And what do I do about
it? To help categorize each incident type, align each one against the cyber kill chain to determine appropriate priority and incident
response strategy. You can use this table as a start.

Incident Types & Recommended Actions

incident type

kill chain stage(s)

Port scanning
Activity*
(Pre-incident]

reconnaissance
& probing

malware
infection
Delivery &
Attack
distributed
denial of service
Exploitation &
Installation
distributed
denial of service
(DDos] Diversion

Exploitation &
Installation

priority level

recommended action
Ignore most of these events
UNLESS the source IP has a
known bad reputation , and there
are multiple events from this
same IP in a small timeframe.
Bonus tip: AlienVaults OTX is
an excellent way to check on
an IPs reputation score.

Remediate any malware infections

as quickly as possible before
they progress. Scan the rest of
your network for indicators of
compromise associated with this
outbreak (e.g. MD5 hashes).

Configure web servers to

protect against HTTP and SYN
flood requests. Coordinate
with your ISP during an attack
to block the source IPs.

Sometimes a DDoS is used to divert

attention away from another more
serious attack attempt. Increase
monitoring & investigate all related
activity, and work closely with
your ISP or service provider.

28

Incident Types & Recommended Actions

incident type

kill chain stage(s)

unauthorized
access
Exploitation &
Installation
Insider breach
system
compromise
(An inside job doesnt
require much Recon)

Unauthorized
privilege
escalation

Destructive
attack (systems,
data, etc.)

Advanced
persistent threat
(APT) or multistage
attack

Exploitation &
Installation

system
compromise

all
stages

priority level

recommended action
Detect, monitor and investigate
unauthorized access attempts with
priority on those that are missioncritical and/or contain sensitive data.

Identify the privileged user

accounts for all domains, servers,
apps, and critical devices. Ensure
that monitoring is enabled for all
systems, and for all system events,
and also make sure its feeding
your log monitoring infrastructure
(your USM or SIEM tools).

Configure your critical systems

to record all privileged
escalation events and set alarms
for unauthorized privilege
escalation attempts.

Backup all critical data and systems.

Test, document, and update system
recovery procedures. During a
system compromise - capture
evidence carefully, and document
all recovery steps as well as all
evidentiary data collected.

Any one of the singular events that

are listed here could actually be a
part of the worst type of security
incident imaginable the dreaded
APT. The important thing is to view
each event through a larger context,
one that incorporates the latest threat
intelligence (see below for more on
the need for threat intelligence).
29

Incident Types & Recommended Actions

incident type

kill chain stage(s)

priority level

False alarms**

recommended action
Much of the incident responders
job is spent eliminating
irrelevant information and
removing false positives. Youll
be constantly fine-tuning the
radio of security monitoring to
get to just the right signal.

all
stages

malware
infection

Incident response is a discipline of

continual improvement. As you see
more and more events turn into
incidents, youll discover new ways
to categorize those incidents, as well
as new ways to prevent them from
ever happening in the first place.

all
stages

* A NOTE ABOUT PORT SCANNING:

** A NOTE ABOUT FALSE ALARMS:

Even if youre sure that an attacker is getting no useful

Weve expressed the need to concentrate on what you

information back from their scanning, if they seem to be

know many times in this guide much of the work that

doing a detailed and comprehensive scan of your external

security monitoring discovers is mundane yet vital.

systems, it is reasonable to interpret this as intent to followup the recon with attack attempts later on. If the scanning

to the world, categories of websites that should be

originates from a legitimate organizations networks, then

blocked at the proxy, hosts that were compromised

contacting their security team (if they have one) or network

because they didnt have endpoint security installed.

management personnel is usually the best approach.

Incident Response work is best thought of as quality

As a last resort, if no contact details are apparent, try the

contact details listed in the WHOIS information for the domain.
The email address abuse@domain is often a contact email

Controls Failure: Firewall ports that shouldnt be open

assurance for the rest of your security efforts.

Noise Reduction: If security analysis is about finding

the needle in a haystack, one of the best ways to make

for this kind of communication, but may not be available for

the job easier is to make a smaller haystack. Remove

smaller or younger organizations. BTW, blocking the source

unnecessary traffic, unwanted services, outdated

address may be counterproductive, and merely cause the

client software, and easily-patched vulnerabilities.

attacker to use a different source address.

Policy Violation: Ideally, you hope to be spending

more of your time locating the things happening that
put your network at risk, not cleaning up the results of
that risk being exploited by a hostile party.
30

be sure to...

Combine Local & Global Threat

Intelligence for Effective Triage
We often think of incident response as being detailed,
meticulous forensic work, looking closely at one system at
a time. However, the great majority of security monitoring
work can be addressed through seeing a larger more holistic
picture of the state of, and activity on, your infrastructure.
Understanding where, which, and how your systems are
communicating with other systems, and the changes being
made to them, can reveal attacks that other security controls
cannot.
Threat intelligence allows you to move away from a focus on
vulnerabilities, exploits and patches, and focus on the things

Top 5

truths

about environmental

awareness

Unless your infrastructure is entirely static

and unchanging, new vulnerabilities and
exposures are being created all the time.

Good IT and Security management

processes will do its best to minimize these,
but the security analyst still needs to be
aware of them to place other things into
context.

Unexpected configuration changes to

systems can reveal when a hostile party
has control of the system through valid
credentials and methods.

Many configuration options are related to

certain compliance standards alerting (or
reporting) on these is a far better way to
manage them than waiting for them to be
discovered during your next audit.

You cant do security just by looking for

attacks and exploits you have to look
into whats happening on your network and
know the systems you have deployed.

that are actively causing damage to your companys data

confidentiality, integrity, and availability.
The first step is to understand as much as possible about
your current computing environment. Some people refer to
this as environmental awareness or situational awareness
or even contextual awareness. We like to think of it as local
threat intelligence.
Once you combine rich information about your own network
with the latest global threat intelligence (specifics on attacker
tools, techniques, and trends), youll achieve effective triage.
Youll put your immediate focus on the types of security
incidents that matter vs. wasting your time on false positives
or irrelevant noise.

GLOBAL THREAT INTELLIGENCEBRIDGES THE GAP BETWEEN

DETECTING KNOWN METHOD OF ATTACK, AND DETECTING
KNOWN THREAT ACTORS. EVEN IF WE DONT KNOW ALL THE
METHODS THEY MAY CURRENTLY BE USING, WE CAN SEARCH
OUR OWN NETWORK FOR KEY INDICATORS THAT OTHERS
MIGHT HAVE SEEN OUT IN THE WILD.
31

so heres...

The Bottom Line

By understanding what is happening on your network

Today, computing resources are cheap and plentiful attacks

(environmental awareness) and connecting it to information

can come from anywhere especially from compromised

about known sources of malicious activity (Global Threat

systems on otherwise legitimate remote networks. Attackers

Intelligence), it becomes simple to get large-scale reports on

fight a constant battle trying to make it difficult to locate

active threats within your infrastructure. For example, instead

the systems that control their malware, while still allowing

of searching through massive lists of alerts from various

their malware to reach these systems to receive execution

security controls to determine possible exploits and attacks,

instructions.

and attempting to prioritize them based on asset value, we

look at environmental awareness data that can be connected
to the indicators of compromise associated with threat actors.

See our next chapter Chapter Four - Incident Response

Tools for how-to instructions on uncovering more information
on attack sources.

Onto Chapter 4 >

incident response tools
32

four
Incident Response Tools
its not just about the gear. Its About how, when, and why to use it.
Any discussion of incident response deserves a close look at the tools that youll need for effective
incident detection, triage, containment and response. Well cover the best tools for each function,
well share resources for how to learn how and when to use them, and well explain how to
determine the attack source. That way, youll know the right decision to make at each stage of the
investigation.

The Three As of Incident Response

In order to be effective in defending your companys network, youll need the right Ammunition,
youll aspire to identify proper Attribution, and youll focus on increasing Awareness as a way to
reduce the volume and impact of cyber incidents on your company. Still not clear on the As? Read
on...

Ammunition: Most incident responders will want to spend most of their time here, downloading
and customizing incident response tools - open source as well as proprietary. Why? Because its
fun, and thats what cyber geeks tend to like to do code. Well mostly cover open source incident
response tools in this chapter, and well also use the OODA loop framework from Chapter Two so
youll know when to use which tool and why.

Attribution: Understanding where an attack is coming from can help you understand an attackers
intention as well as their technique, especially if you use real-time threat intelligence to do so. Well
cover the basics of attribution, and include some free and open resources to keep you updated on
who might be attacking your company based on the latest collaborative threat intelligence.

Awareness:

The most fundamental security control is an educated and aware user. While we

plan to go deep into incident response training in the next chapter, in this chapter well cover some
of the highlights youll want to consider as you update your security awareness program. The
biggest takeaway here is that every incident should be examined as a way to improve your overall
security program, with awareness as a key part of that.

33

ammunition:

Incident Response Tools & the OODA Loop

Disclaimer: Our preference is for open source incident

For a refresher on the OODA loop: check out Chapter Two.

response tools, and so weve provided recommendations on

Developed by US Air Force military strategist John Boyd,

some of the best open source options. Keep in mind that your

the OODA loop provides an effective framework for incident

mileage may vary. In some cases, you may need to look at

response.

proprietary options for certain capabilities. That said, youll

have to go somewhere else for recommendations on vendor
tools (unless theyre built by aliens, in which case, youre in
the right place. ;)).

OBSERVE
ACT

incident
response

ORIENT
DECIDE

THE OODA LOOP

34

OBSERVE:
type of IR tool

USE SECURITY MONITORING TO IDENTIFY ANOMALOUS

BEHAVIOR THAT MAY REQUIRE INVESTIGATION.

Log Analysis, Log

Management, SIEM

Logs are your richest source for understanding

whats going on in your network, but youll
need an IR tool that makes sense of all of those
logs, and thats what log analysis is all about.

Intrusion Detection Systems (IDS)

Network and Host-based

IDSes (HIDS and NIDS) monitor server and network

activity in real-time, and typically use attack
signatures or baselines to identify and issue an
alert when known attacks or suspicious activities
occur on a server (HIDS) or on a network (NIDS).

Netflow Analyzers

open source options

why you need it

Netflow analyzers examine actual traffic within

a network (and across the border gateways
too). If you are tracking a particular thread of
activity, or just getting a proper idea of what
protocols are in use on your network, and
which assets are communicating amongst
themselves, netflow is an excellent approach.

Vulnerability Scanners

Vulnerability scanners identify potential

areas of risk, and help to assess the overall
attack surface area of an organization, so that
remediation tasks can be implemented.

Availability Monitoring

The whole point of incident response is to avoid

downtime as much as possible. So make sure
that you have availability monitoring in place,
because an application or service outage could
be the first sign of an incident in progress.

Web Proxies

Web Proxies are thought of as being purely for

controlling access to websites, but their ability
to log what is being connected to is vital. So
many modern threats operate over HTTP being
able to log not only the remote IP address, but
the nature of the HTTP connection itself can
be vital for forensics and threat tracking.

OSSIM open source

security information mgmt

Snort
Suricata
BroIDS
OSSEC

Ntop
NfSen
Nfdump

OpenVAS

Nagios

Squid Proxy
IPFire

35

ORIENT:
type of IR tool

EVALUATE WHATS GOING ON IN THE CYBER THREAT LANDSCAPE

& INSIDE YOUR COMPANY. MAKE LOGICAL CONNECTIONS & USE
REAL-TIME CONTEXT TO FOCUS ON PRIORITY EVENTS.

Asset Inventory

In order to know which events to prioritize, youll

need an understanding of the list of critical systems
in your network, and what software is installed on
them. Essentially, you need to understand your
existing environment to evaluate incident criticality
as part of the Orient/Triage process. The best way
to do this is to have an automated asset discovery
and inventory that you can update when things
change (and as we know, thats inevitable).

Threat Intelligence

Threat intelligence gives you global information

about threats in the real world. Things like
indicators of compromise (IoCs), bad reputation
IP addresses, command-and-control servers and
more, can be applied against your own network
assets, to provide a full context for the threat.

Security Research

DECIDE:
type of IR tool
Your Companys Corporate
Security Policy*
Hard Copy Documentation
(notebook, pen, and clock)

open source options

why you need it

OCS Inventory

AlienVault OTX
OpenIOC
AlienVault Labs

BASED ON OBSERVATIONS AND CONTEXT, CHOOSE THE BEST

TACTIC FOR MINIMAL DAMAGE & FASTEST RECOVERY.
open source options

why you need it

If this section looks familiar, its not deja
vu its because it IS familiar These are
the same recommendations we made in
the Decide section in Chapter Two.

Your good ol noggin

Insider secret: There are no Decide tools,

and until AI is truly a thing, well keep having
to do what humans do, use our brains. Decide
based on the information you have at your
disposal, which includes the tools above, as
well as your own companys security policy.

36

ACT:
type of IR tool
Data Capture & Incident
Response Forensics Tools

System Backup & Recovery Tools

Patch Management and Other
Systems Management

Security Awareness Training

Tools and Programs

REMEDIATE AND RECOVER. IMPROVE INCIDENT RESPONSE

PROCEDURES BASED ON LESSONS LEARNED.
open source options

why you need it

Data Capture & Incident Response Forensics tools
is a broad category that covers all types of media
(e.g. memory forensics, database forensics, network
forensics, etc.). Incident Response Forensics tools
examine digital media with the aim of identifying,
preserving, recovering, analyzing and presenting
facts and opinions about the digital information,
all designed to create a legal audit trail.

System backup and recovery and patch

management tools might be something
youve already got in place, but its important
to include them here since an incident is
when youll likely need them most.

Security awareness training tools and programs are

an essential way to improve your overall security
posture and reduce the likelihood of incidents.

SANS Investigative
Forensics Toolkit (SIFT)
Sleuthkit

AMANDA
Opsi (Open PC Server
Integration)

SANS Securing the Human

* If you havent written a corporate security policy yet, and need assistance, you can contact a few associations for free resources and
guidance like Educause. In addition to Charles Cresson Woods Information Security Policies Made Easy, there are also a number of
vendors who sell information security policy templates, heres one example.

37

attribution:

Identifying Ownership on the

Anonymous Internet
One of the most underrated IR tools is one of the most

with any reliability and certainty on your own. IP address and

obvious, if you start thinking about infosec like Sherlock

domain ownership arent terribly easy to interpret, and as you

Holmes would. Uncovering a mystery for Sherlock started

likely know, anyone can easily anonymize their connection

and ended with the motivation and attribution of the criminal

through proxies and other means.

under investigation.

That said, there are certain tricks and tools you can deploy
to get better insight into who and where these nefarious

Who is this & what do they want?

characters are, and more on what they want and the

The challenge for the incident responder is that someones

techniques they deploy to g

 et it.

identity on the Internet is exceedingly difficult to determine

Q: Which network does an IP address belong to?

Answer:

Resources:

Public IP addresses are sold to organizations in

blocks of varying sizes. Just as how Domain names
have their registration information listed with a
registrar, public IP networks have the information
available publicly via network registrars.

Youre likely familiar with the concept of RFC 1918

addresses that are dedicated for use on trusted
networks, behind firewalls and other gateway
devices vs. the open Internet. If not, you
can read more about this here:

ARIN (North America)

[Link]

APNIC (Asia-Pacific)
RIPE (Europe, Russia and the Middle East)
AFRINIC (Africa)
LACNIC (Latin America)
These registrars maintain their own WHOIS services,
but for networks instead of Domains. Heres a
query against ARIN for the address [Link]
NetRange: [Link] - [Link]
CIDR: [Link]/16
OriginAS:
NetName: PRIVATE-ADDRESS-CBLKRFC1918-IANA-RESERVED*
NetHandle: NET-192-168-0-0-1
Parent: NET-192-0-0-0-0
NetType: IANA Special Use
38

Q: How do I find all networks that belong to an organization?

Answer:

Resources:

Organizations are free to use their assigned IP

space wherever they wish, but to make it reachable
over the Internet, they must inform other major
Internet-connected routers how to reach that IP
space, via Border Gateway Protocol (BGP).

The CIDR Report website is the easiest

publicly accessible tool for listing all networks
currently assigned to an Autonomous System.

BGP assigns traffic destinations on the Internet by

mapping IP networks to Autonomous System (AS)
numbers. Each Internet-connected organization
receives an AS number to identify them by.
AS numbers are assigned to a legal entity
(e.g. a corporation) though a company may
own more than one AS, this is uncommon
exception for backbone carriers.

Q: How do I find what domains point to an IP address?

Answer:

Resources:

Because the resolution of a domain name to

an IP address is controlled by the owner of the
domain, there is no central registry of mappings.

[Link] is an excellent
multi-purpose tool for information about
domains, addresses, and networks

There are however independent projects that map

the Internet and maintain public registries of the
most recently-seen mapping of domain to address.

[Link] provides a free lookup

service for domains pointing to a single IP address
[Link] is a commercial
service that provides a wealth of information
(including historical information) about domains.

39

Q: How do I find the location of an IP address?

Answer:

Resources:

Several services attempt to maintain registries

of approximate mappings of the physical
location of the organization, network or system
an IP address is currently assigned to.

[Link] is recognized as somewhat

of the defacto industry leader for this service
they offer a limited free service with more detailed
information offered on a subscription basis.

Insider tip: Physical Location of an IP address is of

somewhat limited value to the DFIR analyst in most
aspects of their work. The organization that owns
the address space is usually of more relevance for
identifying connections between addresses. Information
networks are not limited by geographic boundaries.

[Link] is a community-funded
service that provides automation services
and detailed location information.

Q: How Accurate is Geolocation Information?

Answer:

Resources:

IP addresses are, by their nature, a logical

not physical identifier networks can be reassigned from one side of the planet to another,
within a few hours at the very most.

AlienVault OTX

Most location information about IP addresses is derived

from the location of the organization that owns it. A
multinational corporation may have networks across
5 continents, but all its address space will likely be
registered to the location of the companys HQ.
Like all information kept up to date via the
aggregation of data from multiple sources, geo
location Information accuracy will vary from
point to point, IP address to IP address.

40

awareness:

Security is Everyones Job

Security awareness is sort of like motherhood. Its one of

from wikipedia

the hardest jobs because its the most important yet least
respected, and if everyone did it properly, wed likely put an
end to war around the world, right?
In all seriousness, every post-incident examination should
include an assessment of your overall security posture
especially, the security awareness program. Regardless of
the root cause of the incident, its still important to revisit

SECURITY

how a more security-savvy employee community could have

IS THE KNOWLEDGE &

even the most sophisticated users. In fact, an estimated 91%

AWARENESS
ATTITUDE MEMBERS

OF AN ORGANIZATION
POSSESS REGARDING

THE PROTECTION OF THE

PHYSICAL, & ESPECIALLY

INFORMATIONAL

ASSETS

OF THAT ORGANIZATION.

averted the crisis.

This isnt the part of the guide where we bash dumb users.
Seriously. Phishing and spear-phishing campaigns can fool
of hacking attacks begin with a phishing or spear-phishing
email.* This type of tactic fooled a White House employee
earlier this year, which is believed to be the source behind
a Russian attack against an unclassified system on the US
White Houses network.
So examine each investigation with the perspective of
understanding where your security awareness program
could have prevented that incident, or minimized its impact, if
only those lessons, guidelines, or tips were shared with your
employees ahead of time.
And speaking of security awareness lessons, guidelines,
and tips, read more in our next chapter, Incident Response
Training.

* Source: [Link]

Onto Chapter 5 >

incident response training
41

five
Information Security Awareness Training:
The Key to Optimizing Incident Response

Security is everyones job. Seriously.

Despite the great leaps in innovation weve witnessed over the past few decades, nothing beats
a human beings common sense and good judgment. In fact, pragmatism, common sense and
good judgment are a few values that arent yet possible to develop in software code or artificial
intelligence.
The truth is, you cant automate intuition. And much of the incident responders job comes down to
relying on your and each employees intuition that something in that email just doesnt look quite
right (as an example). Your goal is to reduce the number and impact of cases when someones
bad judgment, mistakes, and oversights open the gate to a possible breach. It could happen from
clicking on an embedded link in an email, or a social engineering scam over the phone.
However it happens, you wont find the answer in some sort of magic pill - like information security
awareness software downloaded to your brain a la Trinity in the Matrix. Thats why you need an
information security awareness training program. And yes, like many things in incident response,
hearing that phrase is likely to inspire a yawn or two. And a sigh, and maybe throw in a few eye
rolls too, while youre at it.
But it doesnt have to. There are a few tools, resources, and program ideas that can make information
security awareness training effective and engaging for your employees. And thats what well cover
in this chapter.

42

whats the difference...

Incident Response Training vs.

Security Awareness Training
We recommend having two different training programs: one
for the overall employee population and one thats specifically

015 SANS Surve

2
a
m
o
y
fr

for the incident responder. As for any specialized set of skills,

incident response training should focus on all aspects of the
job, the IR process, as well as the specific technical skills
(programming, systems administration, and code analysis) to
support whatever technologies or computing contexts that

IR TEAM TRAINING
AND CERTIFICATION WAS CITED AS
THE SECOND MOST COMMON AREA
FOR IR IMPROVEMENT (57%)

are relevant for your company.

Within this guide, were focused on the more broad topic
of security awareness training, because weve seen that
improving the security awareness of everyone in your
company will have a big impact on reducing the number and
cost of security incidents. Were also hoping that this entire
guide provides a rich foundational resource for training the

See the Survey: Maturing and Specializing: Incident Response Capabilities

Needed >

members of your IR team.

lets chat about...

What Exactly Do Employees Need to Know

About Security?How Much Is Too Much?
Its a great question and one that requires we return to our

Defending against both types of attacks requires vigilance

primary goal for security awareness training: to reduce the

and awareness on the part of every employee. Remember

number and impact of high risk security incidents. So lets

to keep your training content and approach focused on

focus on the biggest risk first: phishing and spear-phishing.

teaching skills and good judgment vs. teaching the technical

Phishing and spear-phishing attacks are the most common

way that employees can be manipulated into exposing
your company to risk. These social engineering scams are

aspects of how phishing works on the back end, or esoteric

topics like the differences between a rootkit, a bot, and a
keystroke logger.

responsible for many of the high profile breaches youve

Show employees a few examples of phishing and spear-

likely already heard of. The key difference between phishing

phishing scams, and encourage them to be suspicious, even

and spear-phishing is that spear-phishing is customized and

if an email may appear to be from someone they know. You

targeted to a specific employee and company, whereas

may also wish to consider incorporating simulated phishing

phishing is more broad and automated, less sophisticated

attacks to educate employees about appropriate security

and less specific.

behaviors, measure the effectiveness of your training

program, and identify any knowledge gaps.

43

whats the difference?

PHISHING

SPEAR-PHISHING

IS A BROAD, AUTOMATED ATTACK

THAT IS LESS SOPHISTICATED.

IS A CUSTOMIZED ATTACK ON A
SPECIFIC EMPLOYEE & COMPANY

how about...

Security Awareness Training Goals & Metrics

Trying to increase awareness around any topic is somewhat
dubious. How do you measure how aware someone

Track help desk tickets, and expect to see

an increase in employees reporting suspicious events

is? Hopefully by their behavior, and with any luck, by the

and activity. This wouldnt necessarily be because

reduction in the number of incidents and exposures you keep

there are more suspicious events happening, simply

having to respond to.

that employees are more sensitive to them, and feel

Creating good security metrics is an art unto itself, and while

confident in reporting them.

there are many things that generate numbers that can be

Explore non-traditional training

methods like simulation exercises to test an

tracked, good metrics dont just speak to what has been

done, but how well it was done they enable the future, not

employees resistance to social engineering scams,

recount the past.

and then measure progress on a quarter-by-quarter

That said, here are a few sample indicators for increased

basis. However, work with your communications

awareness and effective training:

team to give everyone ample heads up, to maintain

trust and transparency between employees and the
infosec team.

and finally...
44

Information Security Awareness Training: Top Seven Tips

1
2
3
4
5
6
7

GET EXECUTIVE BUY-IN. EARLY AND OFTEN.

Its a universal truth that the executive team sets the tone for the entire company, for every team, and every
project. If you want your security awareness training program to be successful, involve the management team
at every stage, and ask for their visible participation and support.

CULTURE, CONTEXT & CONTINUITY ARE ESSENTIAL.

Encourage your management team to instill a security-aware culture where everyone sees security as a part of
their job. Most insiders agree that once and done doesnt work for security so look for teaching moments
in daily business operations. For example, attack simulation exercises provide the most realistic context for
the actual risky situations that employees will find themselves in, and often provide one of the most valuable
teaching methods.

BE CLEAR. USE REAL-WORLD SCENARIOS & APPLICATIONS.

Youre trying to raise awareness and change behavior, and the more real, relevant, and compelling you can
make it, the more traction youll have. Dont overcomplicate things, and dont try to address every possible
situation that could happen, because its simply not possible.

TRY TO AVOID A LONG BORING LIST OF DONTS.

The Just Say No approach is old skool in a bad way, like Nancy Reagan and shoulder pads. And it doesnt work.
Instead, show how to do something securely and opt for a scenario-based education approach. Remember,
your goal is to instill good skills and habits vs. rote memorization. Keep the content fresh and engaging because
if employees are bored, they wont remember anything.

GIVE GOOD REASONS. EXPLAIN SECURITY GUIDELINES.

Explain why a users credentials are so valuable and how important it is to safeguard them. This is a much better
approach than simply being frustrated when you hear users complain about the password policy. Once an
employee understands why there are certain security controls, theyll be more likely to respect them, and apply
similar principles to any new high risk situations.

CONSIDER ROLE-SPECIFIC RISK-BASED SECURITY TRAINING.

Training is at its most meaningful when its tightly linked with an employees role within the company, in the
context of the risks they face in fulfilling that role. For example, someone in sales may need more training on
how to protect company data and equipment while traveling than someone in engineering would.

BE CREATIVE. INCLUDE MULTIPLE CHANNELS AND FORMATS.

There is no one size fits all approach to security awareness, and theres no one single training tool that will
accommodate all topics or audiences. Most companies have also found that the annual death by PowerPoint
approach no longer works. As long as it fits your company culture, think about incorporating a security
awareness game at the next company retreat. Remember to use newsletters, posters, blogs, and other media
as ways to get the message out.

45

Good Luck!
REFERENCE RESOURCES:
SANS Securing the Human - Security Awareness Planning Kit
Tips from the US Computer Emergency Response Team
NISTs Computer Security Incident Handling Guide

About AlienVault:
AlienVaults mission is to enable organizations with limited resources to accelerate and simplify
their ability to detect and respond to the growing landscape of cyber threats. Our Unifi ed Security
Management (USM) platform provides all of the essential security controls required for complete
security visibility, and is designed to enable any IT or security practitioner to see results on day one.
Powered by threat intelligence from AlienVault Labs and the AlienVault Open Threat Exchangethe
worlds largest crowd-sourced threat intelligence network AlienVault USM delivers a unifi ed, simple
and aff ordable solution for threat detection, incident response and compliance management.

W W W . A L I E N V A U L T . C O M
46