Adversary Emulation with MITRE ATT&CK

Chapter 1. Introduction

Technology development and digitalization have expanded the attack surface and introduced new attack vectors. Today’s organization has a broader scope of concern due to factors like globalization, supply chain, and digital assets. A significant transformation in cybersecurity has been notable over the past few years—different stakeholders are heavily investing in finding new technologies to defeat cyber threats. Nonetheless, there is news daily about security breaches arising worldwide, so the question is, why is this happening to even the most defended institutions?

Chances are high that you will be affected by data breaches without your being aware or there being any public information about the breaches. One scenario could be that your credentials get sold on the dark web. Although you may not be a primary target, you risk becoming an entry point for the threat actor, jeopardizing the security of those connected to you. During the reconnaissance phase (see Chapter 4), an adversary will covertly discover and collect information about the target, sometimes investigating individuals that are not directly associated with the organization. Examples of this can be a family member or an organization that supplies the target with a relevant service or product. Then, through infiltration processes, threat actors will access the network and propagate until their goals are achieved. A couple of years ago, I got a call from one of my friends who was stressed because all of his crypto funds had been stolen. Later on, while investigating the case, I discovered that his wife’s laptop was initially compromised, leading to the attacker using lateral movement (see Chapter 4) techniques that compromised another device on the network.

Cybercriminals are evolving. They are always striving to find methods to break into infrastructures while utilizing well-researched vulnerabilities and exploiting the weakest links in an organization’s security chain—humans. According to IBM, in the United States alone, the financial damage caused by inside threats (threats to a company’s security that come from someone who works there or has access to the company’s systems and information) reaches $4.18 billion, making them among the most common causes of data breaches.¹ The budgets some cybercriminals are working with are probably larger than what some companies allocate for their own defense. Thus, threat actors have no need to exclusively rely on resource development for successful campaigns; instead, it has become a standard practice for threat actors to buy an employee’s credentials or even buy incriminating information on an employee and then extort that employee for more information. What makes the situation more complicated for defense teams is that sophisticated tools are often accessible in underground markets, with easy-to-use instructions enabling even attackers with limited budgeting and low skills to cause severe damage. It is also becoming difficult for security researchers to attribute attacks to a specific threat, thus providing camouflage options to the adversary (see Chapter 2).

Note

Lapsus$ is an international hacker group known for cyberattacks against companies and government agencies. The group tries to gain initial access in various ways, including paying an employee of an organization for access to credentials or multifactor authentication approval.²

Another major inroad for attackers is zero-day vulnerabilities; these are software flaws unknown to the vendor, meaning there is no mitigation available to patch them. Until the issue is fixed, hackers can exploit it to adversely affect computer programs, data, additional computers, or networks. Used by the military, nation-states, law enforcement, and cybercriminals, zero-day vulnerabilities have created many opportunities for unregulated markets to sell cyber weapons. Over the coming years, regulating global zero-day trade will become even more complicated. To combat these types of attacks, arrangements between nation-states can help. An example of this includes the Wassenaar Arrangement, which is “a multilateral export control regime with 42 participating states, established in 1996, focused on export controls for conventional arms and dual-use goods and technologies.”³

Know Your Attackers

In the world of attackers, you will find stereotypical hackers who are motivated by financial gain and are usually very contentious. You will also find advanced persistent threats (APTs) that plan operations with a specific objective. APTs are significantly more complex, intending to steal information like intellectual property and state secrets, whereas stereotypical hackers are more hit-and-run. Still, both of these threats can represent challenges to the continuity of your business. Even though traditional hackers are not as advanced and coordinated, they can hide their tracks by using tools that APTs developed and then made public. To challenge the status quo and start building better defenses, you should not simply concentrate on detecting indicators of compromise (IoCs); it is also essential to observe the adversary’s behavior. You have to start thinking about the life cycle of an attack and what is the most time-consuming process for an adversary. As discussed earlier, these threat actors have many financial and technological resources at their disposal, so how will you use that against them?

Maximizing Adversary Cost

To know more about how attackers think and to explore ways to increase the adversary’s cost of operations, let’s review David Bianco’s Pyramid of Pain (see Figure 1-1).

Figure 1-1. David Bianco’s Pyramid of Pain

Bianco’s diagram shows how much “pain” is inflicted on the APT when the defense is based on their behavior. Bianco reasons that not all IoCs are equal because some are much easier for adversaries to bypass, others take more time to bypass—and the more time it takes, the higher the adversary’s operations costs. When adversaries are attacking, if you can respond fast enough and deny them the ability to use those indicators, you will force them to perform the most time-consuming process: learning a new behavior.⁴

See the following list for more details on the indicators provided in Bianco’s pyramid, starting with the hardest:

Tactics, techniques, and procedures (TTPs)

TTPs are the crown jewel on the Pyramid of Pain, and operating at this level means you are not going against the adversary’s tools but their behavior. By forcing them to change their behavior, you are making them obsolete, and they will need to give up or start to learn a new behavior; either way, their operation takes a big financial hit. TTPs can help security researchers link an attack with a threat actor, and studying them will aid counterintelligence efforts for better future detection.

Tools

Sometimes, adversaries depend too much upon specific tools, and if you find a way to block the use of that tool, you may force them to look for other targets. Consider that even if they persist in attacking you, they will need to develop or acquire something that has equivalent capabilities and then master the tool from the ground up.

Network/host artifacts

Network and host artifacts can be any interaction from adversaries, and it is the level where you start to have an impact by causing pain to their operation. For example, suppose they use specific software to enumerate a running HTTP service, and in access logs, you notice a distinguishable User-Agent. In that case, by filtering their requests, you may force them to return to the resource development phase, causing them to spend more time developing their attack.

Domain names

Domain names are a little bit tougher to acquire, so they cause more pain to the adversary compared with hash values and IP addresses. Still, many providers do not require or validate identity, making it simple to purchase domain names using stolen funds or hard-to-trace crypto payments.

IP addresses

If an attacker starts sending stolen information to their control server, they will likely use an IP address. So, you might block IP addresses known for malicious activities. But often, advanced attackers hack into systems of trusted organizations to launch attacks. This tactic uses the connections between different organizations, making it hard to stop the attack by just blocking IP addresses.⁵ Many IP addresses are available for attackers, making it easy for them to keep changing their IP address. They might even hide their identity using services like Tor.

Hash values

Hashing algorithms like MD5, SHA1, and many others are one-way functions that can map data of arbitrary size to fixed-size digests. They are often used in defense, but flipping a bit will result in a completely different and unrelated value, so it will be trivial for APTs to perform this type of change to their arsenal. Sometimes, you can use context-triggered piecewise hashes (CTPH), where moderate changes to the input will be noticed with tools like ssdeep, which is frequently used by malware analysis.⁶

You can use all these layers while planning your defense because though some stages will have a negligible impact on the adversary’s operation, they will provide you with enough time to respond appropriately. Implementing only the TTP layer is not always guaranteed to provide results, and how will you ensure that all configurations to catch Pass-the-Hash attacks are done accordingly?

Adversary-Inspired Testing

That is why assessing the blue team’s implementation (see “Blue and Purple Teams”) is vital to ensuring that the behavior you want to detect is working as it should. To improve security, you can incorporate cyber threat intelligence (CTI) and define which adversaries pose risks to your organization. Thus, by mimicking adversaries’ behavior, you can test your defense coverage with the exact TTPs threat actors use in the real world. You will also discover which types of platforms a specific threat is known to target, and you can classify them as a risk or not. The upshot of this approach will be to help you reflect on different phases of an adversary’s attack life cycle and utilize a common taxonomy of adversary actions. Optimally, it will allow you to prioritize existing technologies and capabilities, helping you develop strategies for future investment while showing you which defense integrations failed and which succeeded in the past.

Drawbacks of Traditional Security Assessments

Assessments technology and preventive controls offer a proactive cybersecurity effort involving consistent, self-initiated advancements based on the reports written by the expert conducting the activity (see “Types of Security Assessments”). Unfortunately, for disciplines such as penetration testing and red team assessments, the offense’s success is often perceived as the defense’s failure and vice versa, creating a toxic relationship between the IT department of the organization being tested and the company delivering the service. As an outcome, the assessments are presented with unrealistic defenses that will not reflect the organization’s factual security posture. For example, I encountered such behavior while performing an internal pentest service for an organization operating in the finance sector. After the test concluded, executives of the company requested that the IT department participate in the presentation of the findings. System administrators became defensive during the meeting, insisting that security was reasonable. After I presented evidence showing a fully compromised infrastructure, they began to point fingers and name names, trying to link the fault to specific individuals in the company. Later, one of the executives disclosed that the IT department knew about the engagement months ahead. The tendency to protect their reputation resulted in the implementation of unrealistic defenses that ended up causing downtime for the client as well as the pentesting team.

During traditional cyber assessments, you will encounter the following problems:

• Most traditional assessments are “time-boxed” (conducted within a predetermined period of time).

• They are highly dependent on the scoping phase.

• The traditional tests execute commands and run tools against a target to exploit a vulnerability but don’t evaluate other tactics like collection, exfiltration, or impact.

• Due to heavy enumeration, many technical incidents can occur.

• You will likely face unrealistic defenses by the IT department, usually set up after the assessment is announced.

These tests do not always represent real-world threats and usually focus on initial access objectives, which are very few. Additionally, clients often require that activity stop after the system is compromised and don’t take into account insider threats. Finally, there is a need for better collaboration among teams in which TTP can easily be shared, effectively supporting growth and continuous learning. Therefore, we can conclude that there is a demand for a more mature security discipline to assess defense holistically throughout the adversary’s entire life cycle. This type of assessment provides organization-specific insights.

Types of Security Assessments

Cybersecurity has become a significant area of job growth in the past decade, and it is gradually transforming into an evergreen industry. As in any profession, learning and adopting critical terminology and vocabulary is vitally important for success in the cybersecurity field. There are numerous terms that cybersecurity experts use daily, some easy to agree upon, others harshly debated. The industry has various arguments about the differences among red team, adversary emulation, and purple team or whether emulation and simulation terms should be separated.

The objective in this book is to equip readers with a comprehensive understanding to facilitate proficient discussions within the industry. This involves delving into the nuances of these terms to provide clarity and insight.

Warning

I firmly believe that every level of assessment has its purpose for an organization’s defense. Therefore, I have not listed assessments in order of importance but to highlight the differences between them so that you know what approach or coverage you have or can utilize.

Vulnerability Scanning

Organizations need a vulnerability management program for identifying, classifying, prioritizing, remediating, and mitigating software vulnerabilities. As a result, vulnerability scanning has become vital to determining potential weaknesses in a computer system or network. Moreover, vulnerability scanning establishes an inventory of the entire infrastructure, seeking to identify the operating system and installed software. Vulnerability-scanning tools can perform authenticated and nonauthenticated scans. Therefore, be sure to configure the credentials during the configuration phase, and the scanner will handle the rest. Many organizations favor using multiple vulnerability scanners interchangeably to offer full coverage of every asset, creating a complete picture of the organizations’ infrastructure. Some scanners are built with a specific scope in mind, whereas others provide broader incorporation of various technologies. Assess your organization’s needs and determine the best choice that supports your defense goals.

The following is a list of commonly known vulnerability scanners:

Nessus

Nessus is a remote security-scanning tool developed by Tenable that examines a computer and shows an alert if it uncovers any vulnerabilities that malicious hackers could exploit. It runs checks on a given host, scanning for over 57,000 Common Vulnerabilities and Exposures (CVE).

Nexpose

Nexpose was developed by Rapid7, and it scans for vulnerabilities while collecting data in real time to provide a live view of an organization’s infrastructure. It has its own scoring scaled from 1 to 1,000, allowing it to provide a more extensive scope and better prioritization of issues. It supports on-premises physical, virtual, mobile, and cloud environments. It is widely used due to its integration with Rapid7’s Metasploit for vulnerability assessment and validation, which helps security teams reduce false positives.

Qualys

Qualys is an advanced cybersecurity tool designed to pinpoint and quantify software security weaknesses and remediate them before threat actors can exploit them. In addition, it supports host discovery and helps internal teams better organize assets by tracking vulnerabilities over time and continually showing status and changes.

Vulnerability Assessment

According to the National Institute of Standards and Technology (NIST), vulnerability assessment is a “systematic examination of an information system or product to determine the adequacy of security measures, identify security deficiencies, provide data from which to predict the effectiveness of proposed security measures, and confirm the adequacy of such measures after implementation.”⁷ Vulnerability assessment is more advanced than scanning; findings are verified manually, sometimes including benchmark comparisons or procedure reviews. Vulnerability assessment evaluates whether the system is exposed to any vulnerabilities, assigns severity levels to those weaknesses, and recommends remediation or mitigation. During the manual verification, it is essential to note that no exploitation is involved; this is the main difference between vulnerability scanning and penetration testing, which is discussed next.

Penetration Testing

A penetration test is a simulated cyberattack against a computer system that is performed for the purpose of inspecting exploitable vulnerabilities and determining whether the technology solutions used by the organization can stand up to a cyberattack. Unlike vulnerability assessors, penetration testers take pride in their ability to validate weaknesses and, under controlled circumstances, exploit those flaws.

During a penetration test, a cyber expert examines any identified issues to determine whether an attacker can utilize them to compromise targeted systems or gain access to sensitive information. Host and service discovery involves compiling a comprehensive list of all accessible systems and their respective services to obtain as many details on assets as possible. It includes initial live host detection, service enumeration, as well as operating system and application fingerprinting. Note, however, that penetration testing is heavily determined by the scope of work and is usually time-boxed, which is one of its distinguishing characteristics when compared with red team assessments (discussed next). Thus, the limited scope restricts the tester to a specific host address or range, and often, it can be significantly concentrated on web applications or API endpoints.

Delivering a successful engagement is not simply a test of vulnerability-finding capabilities but also requires a deeper understanding of why the client is requesting the pentest.⁸ Many cybersecurity companies have their own distinct process for executing penetration-testing assessments. From my vast experience, I have chosen to define focus, paradigm, and methodology when building the scope of work:

Focus

Focus is an essential factor for penetration testing because it will define how restricted the scope of work will be. For example, the focus can be a specific API endpoint or an extensive range of IP addresses with many applications running. If the focus is narrow, you have a restricted scope; if it is wide, you have more freedom, optimally providing a broader perspective of the security posture.

Paradigm

A paradigm, or model, of penetration testing will help an organization identify the type of defense process it wants to invest in. For example, if clients request a test of the internal infrastructure, they have an “assume breach” mentality, orienting their strategy toward active defense that aligns with the mindset, “Do not assume that an attack might occur, but assume that it is occurring.” In comparison, clients who request an external pentest usually want to see whether attackers can compromise their defense. This tests the effectiveness of perimeter security controls to prevent and detect attacks against an organization’s public-facing infrastructure.

Methodology

Having a clear methodology for penetration testing leads to realistic goals, expected outcomes, and sound budgeting. Two of the most common approaches are white-box and black-box testing. If a client requires white-box penetration testing, it has to share details and documentation regarding the target system. This guarantees much more extensive and detailed testing coverage but is not as realistic as other methods. In comparison, black-box testing requires no prior information about the target network or application. It enables security experts to look at different security control levels from an attacker’s perspective. However, it is not the best path to follow when you want to test a specific feature or application.

Red Team

Working on a red team (also known as red teaming or just red team) is a practice as old as the role of the devil’s advocate, dating from the eleventh century.⁹ In medieval Europe, to test a candidate’s merit for canonization (into sainthood), the Vatican appointed an official known colloquially as advocatus diaboli (Latin for “devil’s advocate”). This role came with the responsibility to argue against the canonization candidate, hoping to uncover any character flaws or misrepresentation of evidence.

In everyday language, playing devil’s advocate describes a situation where someone, given a particular point of view, takes an alternative position from the accepted norm to explore the thought further using valid reasoning. Military and intelligence leaders started using a similar concept to appoint people to a group they called the red team, to realistically evaluate the strength and quality of various strategies.¹⁰ Since then, red teaming has become common in many fields, including cybersecurity, and organizations use red teams to assess their security by thinking like adversaries.

Red teaming is a stealthy procedure aiming to test people, processes, and technology (PPT), where defenders are unaware of the engagement. It is an assumption-based assessment structured with engagement objectives that can be specific (for example, extracting sensitive data or accessing business-critical applications) or general (for example, assessing whether an adversary can compromise the corporate domain controller). The activity’s success will be measured by how well a red teamer can accomplish these objectives, showing whether or not the organization is capable of detecting and responding to threat actors. During red teaming, the focus shifts from the CVE approach to executing adversary TTP, not to test technology vulnerabilities but to evaluate whether the adversary can achieve a goal, moving from prevention toward measuring detection and response. As Jorge Orchilles put it, “Red Teams may leverage exploits, but they are just a means to an end. Many times, the Red Team may not even need to exploit anything.”¹¹

Blue and Purple Teams

The blue team is made up of security professionals who focus on managing and improving the defensive capabilities of their organization and frequently work as members of the security operations center (SOC). They have various processes and tools to defend a network from attacks, and mitigation techniques for improving its security posture. These include reviewing logs, conducting traffic analysis, performing audits, running digital footprints, and fixing uncovered security weaknesses in the infrastructure.

The purple team is not a team but a function, representing the mindset of having the red team and blue team work together. It ensures the effectiveness of and cooperation between attackers and defenders as a function rather than a dedicated team. Facilitating steady and consistent knowledge transfer enhances the organization’s capability to prevent real-life attack scenarios.

Adversary Emulation Fundamentals

Advanced threat actors do not just exploit technologies in an organization; they also spend time planning their operations. Therefore, when preparing to attack, they try to see the organization’s security holistically and seek to target all the segments designed to protect, detect, and respond. There is no doubt that you need to understand their behavior to build a better defense, but how can you assess that? AE is a type of red team (or purple team) engagement that leverages TTPs that adversaries use in the real world. The critical component of AE is minimizing the distance between red and blue teams and empowering communication and collaboration to improve cybersecurity.

One of the challenges you will face when discussing AE is that there is not an agreed-upon definition. Over my career, I have heard many professionals use the terms emulate and simulate interchangeably. Let’s examine the Cambridge Dictionary definitions of these two words.

Note

Emulate ​/ˈem.jə.leɪt/ : to copy something achieved by someone else and try to do it as well as they have.¹²

Simulate​ /ˈsɪm.jə.leɪt/ : to do or make something that looks real but is not real.¹³

Adversary emulation thus mimics a known threat to the organization and incorporates CTI to assess the organization’s people, processes, and technology with the same TTPs an adversary uses in the real world. On the other hand, adversary simulation is a constructed representation used to assess systems; it is potentially analogous to other disciplines like penetration testing, allowing more freedom and creativity for the practitioner.

There are many thought leaders proposing ideas and trying to define the borders of what an AE is. As stated by Jorge Orchilles, “Adversary Emulations may be performed in a blind (Red Team Engagement) or nonblind manner (Purple Team) with the Blue Team having full knowledge of the engagement.”¹⁴ In AE, you focus on the behavior of one or multiple threat actors by blending in real-world threat intelligence. It is distinguishable from traditional red team activities that are goal oriented. For example, a traditional red team activity might involve accessing a sensitive server or a business-critical application. The red team’s success would be measured by how well it could achieve this objective, whereas AE evaluates the status of executed TTPs. Blake Strom states, “This is what makes adversary emulation different from penetration testing and other forms of red team. Adversary emulators construct a scenario to test certain aspects of an adversary’s tactics, techniques, and procedures (TTPs).”¹⁵

While contradictions abound, it can be declared that AE inherits red or purple teaming approaches (see Figure 1-2), and based on engagement objectives, organizations can request that the activity be announced or unannounced. No matter the perspective of the enactment, AE contains a detailed plan that models a known threat to the organization.

Figure 1-2. Adversary emulation can be unannounced or announced

Importance of Adversary Emulation

You may be wondering why we need another field to assess and enhance security. Here are a handful of reasons:

• Threat actors all operate differently, and most disciplines do not evaluate real-world threats.

• Other fields focus on identifying Common Weakness Enumeration (CWE), whereas AE has a TTP mentality.

• AE is a mature field, providing a holistic view of the organization’s readiness while measuring security operations as a whole.

• AE considers a known threat to the organization and imitates its behavior to evaluate PPT.

• AE minimizes the distance between blue and red teams, whereas pentesters and red teams are discouraged from working together.

• AE helps the blue team learn to detect threats better.

Here’s an example of the benefits of AE. In 2018, I met an individual working at a genetic research firm that was interested in running multiple pentests for some of its applications. During our conversation, I understood that the firm knew about the threat it was facing. His answer to my question about the firm’s primary source of attack was this: “We are targeted daily by Chinese hacking groups that are primarily focused on the genetic data we collect from our clients”. Knowing this information, I proposed that we check how much coverage the company already had from specific adversaries known to target that specific industry by running some of their TTPs. We then followed up with more traditional tests on critical applications to check for any exploitable vulnerability. A security team that wasn’t focused on AE wouldn’t have taken this extra step, and as a result, regardless of fixing the vulnerabilities found by the pentest team, the main problem would have still remained. This case helped me understand that AE is a necessary addition for companies that are targets of APTs.

Framework and Evaluations for Adversary Emulation

If you already have some level of protection, AE can provide valuable insight by allowing you to measure even the security appliances you use for your defense. Therefore, many vendors share the assessment results of their AE to provide transparency for their clients as well as push the industry forward by highlighting potential gaps. These efforts can be better contextualized using standards and frameworks such as the ones offered by MITRE.

MITRE is a nonprofit organization that operates multiple federally funded research and development centers. One of the resources developed by MITRE is the MITRE ATT&CK framework, which provides a common language and taxonomy for describing cyberattacks and the TTPs used by adversaries. The ATT&CK framework is organized around the various stages of an adversary’s attack life cycle, including reconnaissance, weaponization, delivery, exploitation, installation, command and control (C2), and actions on objectives. Cybersecurity professionals widely use the framework to understand and defend against cyber threats and to evaluate the effectiveness of different cybersecurity measures.

MITRE Engenuity is a division of the MITRE Corporation that focuses on advancing the state of the art in cybersecurity through independent evaluations and assessments. Annually, MITRE Engenuity conducts independent evaluations of cybersecurity products through a systemic methodology to help defenders make better decisions. Adversaries are carefully selected to ensure the assessment is realistic and unbiased when estimating the outcomes. The activities focus on the tools’ ability to prevent and detect cyberattacker behaviors and help reflect on their solutions. There are no scores or rankings because the evaluations are not competitive analyses, but they show how the tools approach threat detection.¹⁶

Before you deploy a specific technology, during the procurement phase, you can request AE results or use platforms like ATT&CK Evaluations and compare how much coverage different technologies offer for a particular APT. To review these evaluations, navigate to the MITRE Engenuity website. Then, on the left side, select the adversary and the vendors you want to compare (see Figure 1-3), then wait for a list of all executed TTPs to show. Products receive one of the following detection categories for every substep, allowing you to filter the results based on whether to include or exclude specific detections and providing context to your analysis:

None

The vendor cannot detect activity due to capacity limitations or other reasons.

Telemetry

Detection of this type is usually basic logging of activity, and telemetry alone might not be sufficient.

General

This leaves the security analyst to investigate and determine the next steps because no details are provided as to why the action was performed.

Tactic

This helps answer why an action occurred and provides information on the potential intent of the activity.

Technique

This provides the context and details required to answer why an adversary performed an action and precisely what action they used to achieve their objective.

Figure 1-3 compares FireEye with CrowdStrike on their detection of and response to Wizard Spider and Sandworm.

Figure 1-3. MITRE Engenuity ATT&CK Evaluation results for Wizard Spider and Sandworm emulation plans

You can choose and review the results for a specific technique by selecting a category from the table. You will only find vendors that have requested MITRE Engenuity evaluations. However, even if they are not on this platform, you can directly ask them to deliver AE reports from other third-party providers. When analyzing a technique’s coverage, you must consider its relevance based on the adversary groups and threats your organization encounters.

Currently available emulations on MITRE Engenuity are:

• Wizard Spider and Sandworm

• Carbanak and FIN7

• APT29

• APT3

As shown in Figure 1-3, I’m reviewing the results for Remote Services (T1021), where the adversary connects to 10.0.0.8 using the Remote Desktop Protocol (RDP) on port number 3389. Based on the data, I can conclude that both vendors have the same classification for this technique. This way, you can compare other behavior in the list and decide which solution meets your organization’s security goals.

Adversary emulation paradigms

AE can guide you in making data-driven decisions by gathering information and assessing resolutions. Because it evaluates the adversary’s attack behavior, it can be hard to presume what it entails and how it can be delivered. If the purple team approach is selected, you will sit with the blue team and go through the emulation plan you build, and during the process, if a TTP is missed, detected, or prevented, you will log the results. If the emulation is occurring from a red team perspective, you will execute the plan, ensure that you record all the findings, and review the blue team’s alertness. Because AE promotes collaboration, it will encourage knowledge sharing and positive change, resulting in defenders becoming better at protecting their networks.

Benefits of Adversary Emulation

On an organizational level, AE benefits range from managerial (people, processes, and technologies) to developmental (teams adding new tools and skills to their repertoire).

The key benefits of AE are as follows:

• AE involves assessing people, which helps you train them to be more vigilant and technologically prepared to defend the organization.

• AE aids in identifying and assessing potential risks an organization may face. Emulating the TTPs of an adversary enhances understanding of likely threats and their potential impact. This insight informs risk assessments, prioritizes risk reduction efforts, and ensures alignment with business objectives. Evaluating defense effectiveness helps identify areas of over- or underinvestment in security, allowing the organization to adjust efforts accordingly.

• Technology provides the tools to build its defenses and is linked with people and processes. Evaluating technology will help you see what coverage you have and understand whether there is a need for supported means so you can have an ideal security posture.

During the emulation, in many cases, you will need to build tools or automate jobs, resulting in a robust list of resources that can aid in future tasks for you and the blue team. It can be an automated list of TTPs you have specifically developed for the scenario you are executing or a script that helps you collect artifacts in a network system. These resources will benefit you in future activities by saving research time and enabling you to become a better cybersecurity practitioner.

This pattern of benefits is the same regardless of the industry in which you operate. Usually, every organization with some security grade is a good candidate and should be interested in this type of assessment. An interesting hypothetical use case for AE would be if you are operating in the finance sector and you have recently seen increased attacks on your employees through spearphishing campaigns. In addition, your threat intelligence department has notified you that a criminal group classified with the name Carbanak is targeting similar organizations. After analyzing their known behavior, you have reason to suspect it is also a genuine concern to you. Working closely with the CTI department, an adversary emulator will build a detailed plan of operation and help you understand how much defense coverage you have while executing real-world TTPs seen during Carbanak activities.

You can apply this approach to any industry, including critical infrastructures with industrial control systems (ICS) that want transparency to clarify anomaly and threat detection capabilities. Suppose you are investigating some abnormalities in an organization that is part of the electric utility sector. During the investigation, you conclude there is a good chance some of your employees have been visiting suspicious links. As a result, an adversary has attempted to gain access to electric utilities by leveraging watering hole attacks. Thankfully, the threat has been prevented, but after CTI analysis, you have decided to emulate the ALLANITE adversary group, which is known to conduct similar operations.

Note

ALLANITE is thought to be associated with Russia, and it is focused on cyber espionage that has primarily targeted the electric utility sector within the US and the United Kingdom (UK). It has been indicated that the group maintains a presence in ICS to analyze processes and maintain persistence.¹⁷

Transparency and Relevance

One of the critical attributes of AE is transparency, and as straightforward as it may sound, this attribute is one of the hardest to achieve. As discussed earlier, in many traditional cybersecurity disciplines, blue and red teams are discouraged from working together. Even though you will receive reports with reproduction steps and details of the verified issues, there is much valuable telemetry that is not shared with the blue team and, thus, not utilized for defense.

Similarly, blue teams build their own knowledge pools that could help red teamers advance their knowledge and procedures if shared with them—but that rarely happens. Moreover, sometimes follow-up tests can be delayed for months, putting the organization at risk of a real-world cybercriminal discovering or exploiting weaknesses. Because transparency is at the core of AE and there is no need to withhold information, the findings can be mitigated immediately.

Relevance is another crucial factor because the tests conducted must conform to the organization’s needs to increase the likelihood of finding security gaps that constitute vulnerabilities. Even if you mimic an adversary’s behavior, you must ensure that the threat actor applies to that industry. A threat intelligence approach ensures that all tests conducted are relevant, indicating that you are focused on the same objectives an APT would have. You achieve this by collecting data about an organization’s cyber threats and performing analytics to extract usable intelligence (see Chapter 9).

Engagement Planning

When planning AE, you must align it with the organization’s goals. First, you need to understand why the client requires an assessment and what is the current state regarding security. As previously discussed, AE is frequently used to evaluate technology and vendor claims or doubts about a process or human capacities. To standardize the engagement, you can employ the following phases:

Defining engagement objectives

Engagement planning starts with defining engagement objectives, which is the process of identifying and clarifying the goals and objectives of a particular engagement or project. In the context of AE, this might involve understanding the client’s business and its specific security needs, as well as determining what the client hopes to achieve through the emulation. Defining engagement objectives also involves evaluating whether the client is eligible for an AE and determining whether other types of security assessments, such as penetration testing, might be more appropriate.

Researching adversary tradecraft

You will then start to break down adversary behavior and potentially use platforms like ATT&CK to track selected threat actor TTPs. This results in a detailed summary of the adversary tradecraft with many citations of CTI reports and other external resources used during various phases.

Engagement planning

After you have determined what strategies you will use, you should list and plan how to procure and use all resources you will need to conduct the activity; these can be human capacities or software or hardware capabilities. Because you will work with many stakeholders, you must plan cross-department collaboration, especially with developers and the infrastructure team. In an emergency, you must have a clear communication plan to define all essential contacts for the engagement.

Implementing adversary tradecraft

During this phase, you will detail the adversary emulation plan (AEP) and build automation scripts to reduce manual errors, helping you to improve quality and focus on primary objectives. Because you will be working with the blue team, it is essential to map all possible detection and mitigation techniques so that you can offer your assistance during the process of tuning security controls. You will often set up lab infrastructures to try out your collected TTPs before running them on the client’s infrastructure, hoping to identify potential issues that can cause damage after execution.

Executing adversary tradecraft

In this phase, you will start running TTPs in the environment. It is vital to have some method for collecting results because you’ll use this information when you prepare the report of your findings. Because this is not a pentest or red team report, the focus will be on the behavior of the attacker and the defender, focusing on whether the attack is detected, prevented, or missed. Finally, you can use tools like ATT&CK Navigator to visualize defense coverage and provide a holistic view of security from the standpoint of the adversary being emulated.

During the activity, unexpected things can always happen, and most of the time, they are out of your control, so it is essential to have a general plan for how you will deal with them. These strategies will help you keep the focus on the assessment and know how to react when you believe something is going wrong. One approach would be to define a policy on actions to take when events unrelated to the ongoing emulation are detected. To minimize issues, you can define these actions as a process that will aid you in many future activities (see Part II).

Due to the ever-increasing number of adversaries, every industry can be threatened, even though some sectors are more attractive for APTs. As a direct result of this, APT behavior has been documented and categorized by several organizations that have been observing these groups, with much of this work serving as ground to perform AE. Through these types of collaborations within the industry, our knowledge of advanced threats is updated daily. Nevertheless, attack tactics and techniques undergo continuous evolution, and these changes take time to detect. This means defense must be built keeping in mind that it has not been tested against all possible attack vectors of an APT. Therefore, AE is a cybersecurity discipline associated with the observed behavior of threat actors, and AE practitioners thus have a collective responsibility to improve through knowledge sharing.

Adversary Emulation Plan

AE practitioners make detailed adversary emulation plans (AEPs) that provide a way of compiling adversary tactics. Outlining adversary behavior in this way allows other cybersecurity practitioners to actively model APT groups. An AEP can be a single resource or divided into multiple companion documents, including diagrams, an intelligence summary for the threat actor, and exact procedures to be executed in the testing environment. The plan should also include resources such as tools, binaries, libraries, configuration files, installation scripts, and more. These will support the execution phase and allow you to reach the engagement goals. Finally, because in most cases you are working closely with the blue team, it can be beneficial for you to map mitigation as well. It will help you to guide the network defenders toward the optimal solution.

When building AEPs, you should consider the communication between teams because many terms for the same technique can be interpreted differently and thus lead to confusion. That is why speaking a common language makes it feasible to estimate and compare what you see when executing a TTP. In addition, finding common ground will help both sides collaborate more comfortably and concentrate on the activity.

Suppose you want to report that an adversary has established persistence through the use of registry run keys or login items. Instead of this terminology, you can revert to using T1547.001 and T1547.015, as noted in the MITRE ATT&CK framework. Thus, even if the different stakeholders employ other wording, you can agree on a common taxonomy that will help you quickly identify security gaps, assess risks, and eradicate vulnerabilities.

MITRE created the plan shown in Figure 1-4 to showcase the use of the ATT&CK framework and what offensive operators and defenders can do with public threat reports.

As seen in Figure 1-4, this emulation is executed in three phases, helping you understand the entire attack life cycle of APT3, a China-based threat group known to have the objective of exfiltrating industrial intellectual property from the compromised target.¹⁸

Figure 1-4. APT3’s three phases of action according to MITRE

In the first phase, you want to accomplish code execution and control of a system to achieve an initial compromise. It has been reported in numerous cases that APT3, through spearphishing, has tricked its victims into clicking a link to a weaponized website that exploits a critical vulnerability that allows remote code execution in Internet Explorer.

In the second phase, you perform discovery techniques to gain local privilege escalation (obtaining higher-level access to systems). You gather credentials to achieve network propagation by lateral movement (moving to other targets that are the focus of the assessment), ultimately achieving persistence (the ability of malware/attackers to remain on a system after restart).

Finally, in the third phase, you employ the knowledge and access you gained in the second phase to start exfiltrating office documents from the target, using WinRAR for compression and encryption. Ultimately, this emulation succeeds in stealing proprietary information, which can then be used for espionage and sabotaging.

In many cases, you will need to build diagrams to analyze and identify common patterns by not focusing on one specific action but instead modeling a series of behaviors with the goal of better understanding the adversary.

Biggest problem with network defense is that defenders think in lists. Attackers think in graphs. As long as this is true, attackers win.¹⁹

John Lambert

I like this quote from John Lambert because it shows how adversaries operate their activities and the importance of adding this viewpoint when building the defense. By using graphs like the one in Figure 1-4, you will create a much more effective way to communicate high-level principles to management and discuss complicated technical details with the threat intelligence team and defenders.

I see a lot of potential in the practicality of projects like Attack Flow for creating next-generation intel-driven AEPs. Attack Flow is a data model built by the Center for Threat-Informed Defense (CTID) to describe TTPs and combine those flows into behavior patterns. It also includes a corpus of examples to guide you through Attack Flow or investigate well-known cyber incidents. To model these cyber incidents, you can use the Attack Flow Builder, an easy-to-use platform where you can visualize your plan or analyze adversary campaigns.

Tip

See Attack Flow Builder for graphs that will help you examine a data breach that happened to the Marriott hotel group in 2018.

Summary

Congratulations, you just finished the first chapter, and now you should have a good understanding of what this book is all about. During the first part, you learned about the many flavors of assessing security and their various focuses. As you discovered, nowadays, it is becoming hard to build a defense that detects an attack based on IP address or hash value because it is easy for adversaries to update their tools to evade these controls. Therefore, you should make it costly for them to attack your organization while going against their behavior, considering that it is the most time-consuming process. For some of us, changing our behavior can take years; just think about something that you have done for a big part of your life and if you had to rapidly change it. Adversaries are people, too, so they will need to learn a new behavior or give up and move on to another target.

As shown in David Bianco’s Pyramid of Pain (see Figure 1-1), you will need to direct your defenses toward the TTPs, thus causing more pain to the adversary. There is a need in the industry to evaluate security holistically, considering that threat actors are becoming more advanced and that they have enough funds to purchase tools or malware instead of having to take the time to develop these capabilities. That is why the domain of AE was created—it is a cybersecurity discipline that focuses on mimicking adversary behavior by blending in threat intelligence to assess people, processes, and technology with the same adversary TTPs observed in the real world.

You explored the AE process briefly and learned about the vital importance of a good emulation plan. You walked through strategies to define engagement objectives so as to make them relevant to the partaking stakeholders. Sometimes, you have to let the client know that it may not need an adversary emulation but something else like penetration testing. For these reasons, there are some prerequisites before conducting an activity, and the most important one is that the network owner must have an existing defense.

Many vendors provide emulation results through a project delivered by MITRE Engenuity called ATT&CK Evaluations. You can compare the defense coverage for different vendors and see if they meet your organization’s goals. In conclusion, AE is a mature process that aims to test a network’s resilience and generates invaluable intelligence to assist the blue team with enhancing security.

¹ “Cost of a Data Breach 2022,” IBM, accessed October 13, 2022, https://www.ibm.com/reports/data-breach.

² Katie McCafferty, “Dev-0537 Criminal Actor Targeting Organizations for Data Exfiltration and Destruction,” Microsoft Security Blog, March 22, 2022, https://www.microsoft.com/security/blog/2022/03/22/dev-0537-criminal-actor-targeting-organizations-for-data-exfiltration-and-destruction.

³ “Wassenaar Arrangement,” accessed October 15, 2022, https://www.wassenaar.org.

⁴ David Bianco, “The Pyramid of Pain,” March 1, 2013, Enterprise Detection & Response (blog), accessed October 15, 2022, https://detect-respond.blogspot.com/2013/03/the-pyramid-of-pain.html.

⁵ “Advanced Persistent Threats (APTs): Threat Actors & Groups,” Mandiant, accessed October 15, 2022, https://www.mandiant.com/resources/insights/apt-groups.

⁶ “SSDEEP Project,” ssdeep, accessed October 15, 2022, https://ssdeep-project.github.io/ssdeep.

⁷ CSRC Content Editor, “Vulnerability Assessment—Glossary: CSRC,” NIST Computer Security Resource Center, accessed October 22, 2022, https://csrc.nist.gov/glossary/term/vulnerability_assessment.

⁸ “Cyber Security & Defense—Penetration Testing & Code Auditing,” February 11, 2020, Sentry, https://sentry.co.com/application-security.

⁹ Micah Zenko, Red Team: How to Succeed by Thinking Like the Enemy (New York: Basic Books, 2015).

¹⁰ “Penetration Testing Versus Red Teaming: Clearing the Confusion,” May 1, 2019, Security Intelligence, accessed July 5, 2022, https://securityintelligence.com/posts/penetration-testing-versus-red-teaming-clearing-the-confusion.

¹¹ Jorge Orchilles, “Shifting from Penetration Testing to Red Team and Purple Team,” March 17, 2022, SANS Institute, accessed May 24, 2022, https://www.sans.org/blog/shifting-from-penetration-testing-to-red-team-and-purple-team.

¹² Cambridge Dictionary, “Emulation,” accessed November 6, 2022, https://dictionary.cambridge.org/dictionary/english/emulate.

¹³ Cambridge Dictionary, “Simulate,” accessed November 6, 2022, https://dictionary.cambridge.org/dictionary/english/simulate.

¹⁴ Jorge Orchilles, “Cuddling the Cozy Bear, Emulating APT29 by Jorge Orchilles—Cyber Junegle,” June 28, 2020, SCYTHE, accessed October 29, 2022, https://www.scythe.io/library/cuddling-the-cozy-bear-emulating-apt29-by-jorge-orchilles-cyber-junegle.

¹⁵ Blake Strom, “Getting Started with ATT&CK: Adversary Emulation and Red Teaming,” July 17, 2019, Medium, October 26, 2020, https://medium.com/mitre-attack/getting-started-with-attack-red-29f074ccf7e3.

¹⁶ “ATT&CK® Evaluations,” “MITRE ENGENUITY,” accessed October 29, 2022, https://attackevals.mitre-engenuity.org/get-evaluated.

¹⁷ “ALLANITE Threat Group,” October 13, 2022, Dragos, https://www.dragos.com/threat/allanite.

¹⁸ Michael Mimoso, “Emergency Adobe Flash Patch Fixes Zero Day Under Attack,” June 23, 2015, Threatpost, accessed November 2, 2022, https://threatpost.com/emergency-adobe-flash-patch-fixes-zero-day-under-attack/113434.

¹⁹ John Lambert, “Defender’s Mindset,” November 21, 2021, Medium, https://medium.com/@johnlatwc/defenders-mindset-319854d10aaa.