Mark de Bruijne, Michel van Eeten, Carlos Hernández Gañán, Wolter Pieters
Towards a new cyber
threat actor typology
A hybrid method for the NCSC cyber
security assessment
Towards a new cyber threat actor typology
A hybrid method for the NCSC cyber security assessment
By
Mark de Bruijne, Michel van Eeten, Carlos Hernández Gañán,
Wolter Pieters
Faculty of Technology, Policy and Management
Delft University of Technology
1
Preface
This report could not have been made without the help of a large number of people. We
cannot mention all of these people by name, but our thanks extends to all of them. First of all,
the researchers would like to thank all the interviewees, who were promised anonymity, for
their precious time and valuable feedback. They have contributed a lot to the report and our
understanding of cyber actors and the methods which can be used to classify them. We,
furthermore, would like to extend these thanks to the members of the supervisory committee.
The committee consisted of Prof. Stijn Ruiter (chair), drs. Olivier Hendriks, drs. Noortje
Henrichs, dr. Jan Kortekaas, Prof. Eric Verheul, and drs. Wytske van der Wagen. We
appreciated their critical and highly constructive feedback during the entire process.
Needless to say, the usual disclaimer applies: The contributions from respondents or
members of the supervisory committee do not mean that the respondents, members of the
supervisory committee or these institutions automatically agree with the complete content
of the report. Also, we would like to emphasize that the report does not necessarily reflect
the opinion of or the Minister or the Ministry of Security and Justice.
Mark de Bruijne
Delft, July 2017
2
Contents
Executive summary 5
Leeswijzer 5
1
2
3
4
5
Introduction 6
1.1
Research aim, research questions and delineation 6
1.2
Reader’s guide 7
Designing a method for a cyber threat actor typology 9
2.1
What is a cyber actor typology? 9
2.2
What should the cyber actor typology do? 10
2.3
The CSAN typology and its shortcomings 11
2.4
Criteria for a good threat actor typology 14
2.5
A method to develop a typology – building the framework 15
The deductive approach – threat actor typology framework 19
3.1
Literature review: in search of threat actor dimensions 19
3.2
Operationalizing the dimensions: developing the framework 25
3.3
Feedback on the framework from experts and stakeholders 30
3.4
Observations and feedback from NCSC/NCTV workshop 35
3.5
Final threat actor typology framework 38
The inductive approach – data analysis 44
4.1
Spam trap data 44
4.2
Honeypot data 48
4.3
Darknet data 51
4.4
Cyber criminal markets 52
A tentative new threat actor typology 54
5.1
Key features of the method to develop a threat actor typology 54
5.2
Application: combining the deductive and inductive cycles 55
3
5.3
A first version of a new threat actor typology 57
5.4
CSAN 2016 typology and new threat actor typology compared 62
5.5
Reflection and some final thoughts 64
Bibliography 67
4
Executive summary
For some years, the NCSC/NCTV has been using a cyber threat actor typology in its annual
Cyber Security Assessment Netherlands. It has evolved over time and captures a set of
actors with different motives, intentions and capabilities. In view of its age and rather intuitive
development process, the NCSC/NCTV is considering whether the current typology needs to
be updated and improved in light of recent insights from science and cyber security practice.
This report, which was commissioned by the WODC (Research and Documentation Centre)
of the Ministry of Security and Justice, sets out to develop a new and systematic method to
enable NCSC/NCTV to continuously update its cyber actor typology. Section 3.5 contains a
concise description of the framework, to be used as a standalone document. As part of the
method description, we also develop a tentative new typology. This can be found in Section
5.3.
Leeswijzer
Het NCSC/NCTV gebruikt deze in haar jaarlijkse cyber security beelden een zogenaamde
cyber actor typologie. De typologie die momenteel gebruikt wordt bestaat al weer enkele
jaren en is gedurende deze periode geëvolueerd. Op een vrij intuïtieve wijze vangt de
huidige typologie een aantal actorgroepen met uiteenlopende motieven, intenties en
capaciteiten. NCSC/NCTV vraagt zich af of deze typologie nog steeds valide is, hoe deze
zich verhoudt tot recente inzichten uit theorie en praktijk en hoe deze eventueel verbeterd
kan worden. Dit rapport, geschreven in opdracht van het WODC van het Ministerie van
Veiligheid en Justitie, ontwerpt een nieuwe en systematische methodiek die het NCSC/NCTV
in staat stelt om de typologie voortaan zelf regelmatig up-to-date te houden. Paragraaf 3.5
bevat een compacte beschrijving van de methodiek die bedoeld is om als losstaand
document gebruikt te worden door analisten. Als onderdeel van de methode wordt een
eerste versie van een nieuwe typologie ontwikkeld. Die is opgenomen in paragraaf 5.3.
5
1 Introduction
In the Netherlands, the responsibility for threat analysis in the digital domain is allocated to
the National Coordinator for Security and Counterterrorism (NCTV). The National Cyber
Security Centre (NCSC) is part of the Cyber Security Department of the NCTV and publishes
an annual Cyber Security Assessment Netherlands (CSAN) (cf. NCSC, 2015; 2016). This
assessment has been compiled since 2011.
The CSAN offers “insight into the developments, interests, threats and resilience in the field
of cyber security over the past year. It is aimed at policymakers in government and the
critical infrastructure sectors to help enhance the digital resilience of the Netherlands or to
help improve current cyber security programmes” (NCSC, 2015:15).
Both public and private organizations contribute to this annual cyber security assessment, as
well as make use of it. The CSAN features a cyber actor typology to provide insight in the
threats and threat actors. In the 2016 Cyber Security Assessment Netherlands (CSAN) the
actors in this typology are defined as individuals or groups “who adversely affect the
reliability and security of information and information systems” (NCSC, 2016:25).
The current cyber actor typology has been existence for some years. It evolved over time
and it intuitively captures a set of actors with different motives, intentions and capabilities. In
view of its age, NCSC/NCTV inquired whether the current cyber actor typology is still valid
today and supported or rejected by recent insights from science and cyber security practice
and in need of improvement. This research project, which was commissioned by the WODC
(Research and Documentation Centre) of the Ministry of Security and Justice aims to
address this knowledge gap.
1.1
Research aim, research questions and delineation
This research develops two distinctive products to fill the knowledge gap. First of all, a new
method to develop a threat actor typology is constructed. The method is based upon state-ofthe art insights in cyber actor typologies, designed to be more transparent than the
typologies used in CSAN 2016, and features a structured way to classify threat actors.1 The
method is designed in such a way that it can be repeated over time. In line with the CSAN,
our assignment was to restrict the threat actor typology to the description of actors who either
operate from the Netherlands or attack targets in the Netherlands. We will discuss the
implications of this delineation in subsequent chapters of the report.
Second, the research aims to develop a new tentative threat actor typology from the events,
threat intelligence, and data that were reported in the 2016 CSAN (NCSC, 2016). The report
shows how the method can be used to include input from diverse data sources about cyber
attacks. The researchers do not claim to present a completely new threat actor typology, nor
to have drawn up a final version. Rather, the principal aim of this report is to provide threat
intelligence analysts and security practitioners with a transparent, systematic and repeatable
1
See https://www.ncsc.nl/english/current-topics/Cyber+Security+Assessment+Netherlands, last
visited May 15, 2017.
6
method to develop the cyber actor typology on an ongoing basis. In view of their national
responsibility for threat analysis in the digital domain, this research particularly supports
practitioners in the National Coordinator for Security and Counterterrorism (NCTV) and the
National Cyber Security Centre (NCSC) in performing this crucial function. However, the
method and typology presented are explicitly designed to be more broadly applicable as well.
The research questions which accompany the project goals were:
1. To what extent is the current cyber actor typology validated by recent insights from
science and cyber security practice and what design criteria for a new cyber actor
typology can be identified?
2. What method to develop a cyber actor typology satisfies the identified design criteria
and enhances or enriches the current cyber actor typology different cyber actors?
3. To what extent can a typology be constructed based upon state-of-the art knowledge
on cyber actors and empirical data on cyber incidents, and what would the resulting
typology look like?
In response to this research project proposes the development of a new method to
incrementally improve the current cyber actor typology. As a practical limitation, the cyber
actor typology should be restricted to the description of actors who either operate from The
Netherlands or (intend to) focus their attacks on The Netherlands.
The method features a structured analysis of (potential) cyber threat actors as well as a
structured approach on how to use more (diverse) data sources to update the cyber actor
typology in the (near) future. The claim, nor the intention of the report is the complete
development of a new cyber actor typology. Instead, the report describes the first cycle that
would lead to the design of a new cyber actor typology. The report and the method outlined
in it are explicitly designed to facilitate use by threat intelligence analysts and other experts to
continuously improve and update the Dutch cyber actor typology.
A third practical limitation is that the research pays particular attention towards the possibility
for potential collaboration between different cyber threat actors, which has been reported as
an increasingly complexifying trend in cybersecurity (cf. CSAN, 2016). This focus is
highlighted in the research questions (in particular research question 3), which means that
this element features prominently in the analysis of cyber actors and the search for key
characteristics to analyze them. The overarching goal is to develop a design method that
supports ongoing, incremental development and improvement of the cyber actor typology.
We will reflect on this design choice and the implications for the long-term validity of (design
of) the threat actor typology in the report.
1.2
Reader’s guide
In the first chapter, the main method to develop a cyber actor typology is designed. The
report unpacks and articulates the various terms and terminologies that surround the
typology and identifies the intended use of the typology. The report subsequently explores
the underlying complexity and challenges of the design of such a typology. Next, we outline
the limitations of the CSAN typologies. Criteria are drawn up to identify quality indicators for a
cyber threat actor typology. Finally the new method is proposed to fulfil these criteria and to
allow for the creation of a valid and useable cyber threat actor typology. The method is based
on a combined ‘deductive’ and ‘inductive’ approach, which is cyclical in nature and supports
7
an ongoing, incremental development and improvement of the CSAN cyber threat actor
typology—a hybrid approach.
In Chapter 3, the first part of the method is developed: the deductive cycle. To bootstrap the
design of a threat actor typology, a literature review identifies common dimensions from
existing typologies of threat actors. To enrich the literature research and ensure the
development of a threat actor typology that is fit-for-purpose, recent insights and feedback on
the theoretically deduced dimensions were collected via interviews with cyber security
experts and stakeholders. The result is a ‘deductively’ developed set of key dimensions that
forms the starting point of the new method to develop the threat actor typology.
With the key dimensions in hand, the report proceeds to combine them into a framework and
operationalize them for use by threat intelligence analysts and other experts. The framework
is explicitly designed to support practitioners in the threat classification process. Section 2.2
describes the design and subsequent updates which culminated in a final version of the
threat actor typology framework.
Chapter 4 turns towards the second part of the proposed method to develop a cyber actor
typology: the inductive cycle. This cycle draws on empirical data about incidents and attacks,
available information on online behavior, which is analyzed and fed in the threat actor
typology. Using several datasets which the researchers had at their disposal, it is illustrated
how incident and attack data can be used to gain more insight into certain dimensions of the
actor typology – and is less informative about other dimensions. The chapter reflects on the
added value of large-scale measurement data and how it contributes to current knowledge
and understanding of attackers and their routines.
Chapter 5 presents the culmination of the previous chapters: a tentative new threat actor
typology resulting from a completed deductive and inductive cycle. Since the proposed
method for the development of a threat actor typology in this research project has only
completed a single development cycle, and is thus limited in terms of the underlying data, the
chapter ends with a condensed set of development guidelines and discussion points to
support the subsequent threat actor typology design cycle by NCSC/NCTV.
8
2 Designing a method for a
cyber threat actor typology
As a starting point for the development of the new method to generate a cyber actor
typology, this report first defines the concept ‘typology’. Next the report explicates on the
intended use of this cyber actor typology in the annual Cyber Security Assessment
Netherlands (CSAN). This is necessary to align what the final products—the method and the
resulting cyber actor typology—actually need to ‘do’.
2.1
What is a cyber actor typology?
The on-line Merriam-Webster dictionary defines a typology as: “a system used for putting
things into groups according to how they are similar: the study of how things can be divided
into different types.” In other words, a typology is a specific form of classification. Bailey
(1994:4) claims that “two characteristics distinguish typologies from generic classifications. A
typology is generally multidimensional and conceptual.” A typology is appealing because it
promises to yield a concise yet parsimonious framework to describe and classify observed
patterns. Bennett & Elman (2006:466, Table 1) identify three different subtypes with distinctly
different goals (cf. Clinard, Quinney & Wildeman, 1999:13):
1. Descriptive typologies which answer the question: ‘what constitutes this type’?
2. Classificatory typologies which answer the question: ‘what is this a case of’?
3. Explanatory typologies which allow researchers to extend—if my theory is correct:
‘what do I expect to see? Do I see it’?
The definition and identification of different goals, that can be served by typologies also
forces us to briefly consider and distinguish typologies from other terms can be encountered
in cyber actor research literature, such as the terms ‘taxonomy’ and ‘profiles’. A ‘taxonomy’ is
defined by Merriam-Webster as: “the process or system of describing the way in which
different living things are related by putting them in groups” and a ‘profile’ as: “a brief written
description that provides information about someone or something”. For the intents and
purposes of this report, both cyber actor taxonomies as well as profiles of methods from
cyber attacks or cyber attackers provide valuable input on important characteristics of cyber
attacks or cyber actors which seem relevant for the creation of a cyber actor typology. Yet,
they are not the same. The report returns to this issue later. Sufficient for now is that there
exists a clear distinction between taxonomies and typologies and that typologies are
generally used in the social sciences (cf. Seebruck, 2015:37). In a typology, the dimensions
are made up of concepts which should be considered as “as ideal types rather than empirical
cases, meaning typologies are not necessarily exhaustive” (Ibid.). Typologies can thus be
defined as “conceptually derived interrelated sets of ideal types” (Doty & Glick, 1994:232).
Taxonomies on the other hand “categorize dimensions based on empirical observation and
measurable traits” (Seebruck, 2015:37).
9
After having shortly identified what a typology is, and having identified its various subsets
and distinguished it from other related terms, the research continues and explicates and
aligns its terminology with intended use of NCSC/NCTV and the employed method to build
such a cyber actor typology.
2.2
What should the cyber actor typology do?
A logical second question of the report would be to establish the intended goal that the cyber
actor typology would serve. In the introduction the project’s research goal was identified
based on the tender request: to asses and if needed update or improve the NCSC/NCTV
typology to help security professionals in their efforts to identify and assess threats from
actors who “adversely affect the reliability and security of information and information
systems” in the Netherlands (NCSC, 2016:25).
Obviously, the cyber actor typology and its underlying method need to produce a reliable
output, i.e., when different analysts use it, they should identify a more or less consistent set
of threat actors. Typology and underlying method therefore need to adhere to scientific
design criteria such as consistency, dependability and replicability. That being said, analysts
will face certain trade-offs during the use of the method, such as more precisely
distinguishing different threat actors versus ending up with a manageable number of types in
the typology. Different analysts might make these trade-offs differently based on how the
resulting typology is to be used.
Given the central role that the cyber actor typology plays in threat assessment in The
Netherlands and the highly dynamic environment in which it is embedded, NCSC/NCTV staff
members will have to work with the typology on a day-to-day basis. This requires not only a
reliable, but also a concise typology.
The typology needs to be unambiguous, i.e. (intuitively) clear to its (wide range of) intended
users and must be able to capture the key characteristics of all (potential) cyber actors in a
small set of dimensions which in turn would systematically lead one to identify a threat actor
type based on the available data or assumptions on each of the dimensions. To be more
precise, the cyber actor typology only needs to categorize threat actors who are defined as
actors who (intend to) “adversely affect the reliability and security of information and
information systems” in the Netherlands (NCSC, 2016:25).
Various online activities such as child pornography distribution, copyright infringement, and
cyberbullying do not infringe on those security requirements and are therefore not included in
the typology as a threat actor even though obviously they are conducting illegal activities.
The cyber actor typology is therefore not the same as a cyber criminal typology. To ensure
this crucial distinction is more intuitively kept, the term ‘threat actor typology’ will be used
from here on in the report.
To conclude: the threat actor typology for NCSC/NCTV creates a framework of dimensions
and classifications that enables a reliable and speedy identification and classification of
threat actors and the resulting threat actor landscape that “adversely affect the reliability and
security of information and information systems in The Netherlands” (NCSC, 2016:25).
10
Table 1: 2016 CSAN threat actor typology. Source: NCSC, 2016:12. Table 1: Threat matrix
2.3
The CSAN typology and its shortcomings
After having identified and articulated the intended use of the desired cyber threat actor
typology, and its design requirements, it is time to consider the typology which NCSC/NCTV
uses in its annual Cyber Security Assessment Netherlands (CSAN) in more detail (cf.
NCSC, 2016).
11
The origins of the typology used in the 2016 version of the Cyber Security Assessment
Netherlands (CSAN) can be traced back to the CSAN 2011 (Govcert.nl, 2011). The original
typology identified 6 cyber actor types2 in 2011, which was extended into 9 cyber actor types
in the following 2012 issue (NCSC, 2012). From 2012 until 2016 the cyber actor typology
remained basically unaltered. The 2016 cyber actor types can be seen in the 2016 CSAN
threat actor typology here reproduced as Table 1.
After having identified and articulated the intended use of the desired cyber threat actor
typology, shortly discussing CSAN’s cyber actor typology, three major shortcomings and
weaknesses of the CSAN cyber actor typologies can be identified:
2.3.1
Lack of consistent dimensions for distinguishing actors
The typology in the CSAN 2016 identifies a set of threat actors that makes intuitive sense,
but underneath the typology, a variety of dimensions are implicitly at work in an unsystematic
way (cf. CSAN, 2016). The lack of a transparent, explicit and systematic methodology can be
traced to the original typology which was “primarily distinguished based on intention”
(Govcert.nl, 2011: 17) [translated from Dutch], but also acknowledges that other threat actor
characteristics (resources, volume which is used as an indicator for the amount of attacks
and visibility) play a role in the classification process. Consequently, there is unclarity about
scientific underpinning of the choice of the dimensions, what role they play and how they
affect the classification process and thereby affect the typology.
For example, the difference between the actor groups ‘cyber vandals’ and ‘hacktivists’ in the
2016 CSAN seems to be based not on intention, but on capability: low versus high. Yet this
dimension—capability—is not applied systematically in the typology.
Furthermore, ‘professional criminals’ and ‘terrorists’ are not clearly distinguished by
capability, but rather by motive: profit versus fear. The dimension of motive is also not
systematically articulated. Certain motives seem to be missing such as individuals attacking
other individuals for personal revenge.
To make matters even more muddled, the current typology also includes ‘private
organizations’ as a threat actor type, which is a vague category that overlaps with
‘hacktivists’, ‘cyber researchers’ and ‘internal actors’.
As a final illustration of the need for a more systematic underlying framework, we point to the
paradoxical ‘no actor’ category in the typology. This category is out of place in a threat actor
typology, which is designed to classify actors, who (intend to) “adversely affect the reliability
and security of information and information systems” (NCSC, 2016:25).
2.3.2
No systematic methodology to revise actors or define new actors
Any typology should be adjusted to dynamics. After all, typologies are “historical, time-bound
mental constructions” (Clinard et al., 1994:12) and therefore need to be reviewed
periodically. Due to the lack of a systematic set of dimensions on which the typology is
based, it is also hard to put in place a systematic procedure to review and update the
identified threat actor types. This has led some threat actor types to mushroom into very
heterogeneous aggregates of actors. The 2016 CSAN typology in short shows that it is
2
These were: professional criminals, state actors, terrorists, script kiddies, hacktivists, and private
organisations.
12
primarily fed by data about (recent) events and trends rather than any threat analysis. The
most notable example is the threat actor type ‘professional criminals’, which covers a much
wider range of actors than the categories of ‘script kiddies’ or ‘cyber researchers’, for
example and does not seem to be fed on similar types of information which would allow one
to infer certain threat actors.
An even more problematic consequence is that the current typology misses threat actors that
are emerging, but which get lumped into existing categories. Consequently, over time there
is a high chance that the typology will become less and less informative. This can already be
seen with the current typology. For example, an important actor type that emerged over the
last few years are private actors that seem to be recruited for state-sponsored attacks. For
example, attacks identified by western security firms as part of Operation Pawn Storm, all
seem related to a group of hackers also known as Pawn Storm, Fancy Bear or APT28, (cf.
Kharouni et al., 2014; Hacquebord, 2017; Perlroth, 2017). The group allegedly attacked a
wide variety of economic and political targets in a rather brazen manner. Claims are made
that the group works for the Russian state or the Russian state intelligence services (cf.
Perlroth, 2017, Fox-Brewster, 2017), but the state keeps the actual attacks at a certain
distance (cf. Higgins, 2017).
Since the attackers are not associated directly with the state, they do not seem to care very
much about being discovered. In practice, this means they can work in a more overt,
standardized and efficient way than state cyber intelligence forces. Where would they fit in
the current threat actor typology? They do not fit well in the category of ‘states’, because the
attackers can be less circumspect and go after more targets against lower cost. Nor do they
easily fit in the category ‘professional criminals', because the crime itself has no monetization
strategy for the acquired information resources on the criminal market. The money is earned
because there is a client for the attack.
Pawn Storm
According to cyber security company Trend Micro, the group of threat actors known under
the heading Pawn Storm are capable of “long-term operations”, and conduct different types
of “attacks that can last for years”. In their 160 campaigns, the group is known to employ
“simple but oftentimes well-prepared credential phishing” (Hacquebord, 2017:9) as well as
spear phishing methods (Kharouni et al. 2014). Targets include US defense contractor
personnel, Russian dissidents, international media, the Organization for Security and Cooperation (OECD), the US Democratic National Committee, and the presidential campaign of
Emmanuel Macron. The group employs various tactics, displaying technical as well as social
engineering expertise in the employment of zero-days. However, at the same time the group
distinguishes itself because of its lax operation security, meaning that it does not seem to
care if their attempts are identified at some point. In fact at certain points the group “uses
mainstream media to publicize their attacks and influence public opinion” (Hacquebord,
2017:5).
Another example is the emergence of new actors that enter the cyber crime market because
of the commoditization of certain types of cyber crime. One such example comes from a
recent analysis of DDoS amplification honeypot data (Noroozian et al., 2016). The study
concluded that the so-called booter services are rarely used for large attacks on valuable
targets, like banks or governments. Instead, over 60% of the targets are regular end users.
Thus, it could be inferred that most of the attackers are also regular end users and that many
attacks take place around online gaming. These attacking end users could be lumped in with
‘cyber vandals’, but this again muddles the typology by conflating different motives. The aim
of these attackers is not to vandalize public resources, but rather to tease or harass their own
13
friends and fellow gamers. In other words: the commoditization of cyber crime leads to a
democratization of attackers and new groups enter the attack landscape around online
gaming.
2.3.3
Under-utilization of large-scale measurement data
As the previous examples already illustrate, the current typology lacks a mechanism to take
advantage of ongoing measurement data generated all over the landscape by honeypots,
sandboxes, darknets, netflow monitors, passive DNS monitors, intrusion detection systems,
et cetera. While the CSAN’s do provide information on measured trends, it is unclear how
they lead to changes in the threat actor typology. The trends that are described in the CSAN
seem to be implicitly attributed to the already identified threat actor types, reinforcing the
existing typology. This erodes the analytic power of the typology for threat assessment. A
structured process is needed to capture relevant trends observed in measurement data and
map them onto a systematic set of actor dimensions, which can distinguish new actors that
look similar on some dimensions, but are different on relevant other dimensions and thus
need to be distinguished. See the example of private attackers providing intelligence services
to state with criminal strategies and the example of regular users going after friends via
commoditized crime services.
Any new method that would result in the development of a cyber threat actor typology would
have to address and preferably solve these shortcomings.
2.4
Criteria for a good threat actor typology
After having established the goal of the project and identified shortcomings in previous threat
actor typologies for which solutions are sought, this report turns towards the identification of
a set of ‘quality indicators’ that would enable one to distinguish an improvement in the
proposed method from the previously used cyber actor typology. Literature provides some
criteria to identify a good (threat actor) typology (cf. Lindqvist & Jonsson, 1997:155; Gundel,
2005:107; Bailey, 1994:3):
1. Classes formed via the typology must be exhaustive (i.e. all potential threat actors
should be classified).
2. Classes formed must be mutually exclusive (i.e. all potential threat actors fit in just
one of the classes).
3. The threat actor typology must be relevant (i.e. the intended goal of quick, consistent
replication based on available information allows for meaningful classification of
events).
4. The threat actor typology must be pragmatic (i.e. the number of subsets should be
manageable and heterogeneity between the subsets should be ample to enable
relatively quick classification). By necessity the threat actor typology must therefore
be composed of types at a fairly high abstraction level.
Furthermore, based upon the intended goals and identified shortcomings, additional
criteria can be formulated:
5. The threat actor typology must allow for efficient classification of threat actors (section
2.2)
14
6. The threat actor typology must be based on a clear set of dimensions and the
process of classification must be transparent (section 2.3.1).
7. The threat actor typology must be dynamic. A method should be provided that allows
for the possibility to continuously update the threat actor typology based on new data
and insights (section 2.3.2 and 2.3.3).
8. Classes in the typology can be changed as a result of criterion 7. New classes can be
formed in the typology (section 2.3.2 and 2.3.3).
It should however, be noted that these quality criteria for threat actor typology designs, in
themselves are potentially conflicting. For example, a method that would be considered to
satisfy criterion 1 might yield a more complete threat actor typology, but at the risk of
violating criterion 4, the ability to enable quick classification and yield a manageable and
meaningful number of threat actor types. In short the new threat actor typology design
method would have to strike a motivated balance between these criteria. This balance and
the arguments behind the choice of the threat actor typology design will be provided in the
remainder of this report.
2.5
A method to develop a typology – building the framework
As a first step to develop a threat actor typology a systematic method to clarify, revise and
enrich the CSAN threat actor typology needs to be explained step by step.
As a starting point for the design of a systematic method, a ‘combined’, hybrid
conceptual/empirical level classification procedure can be identified (cf. Bailey, 1994:3). This
means that first, a conceptual classification of threat actors is deduced from literature and
secondly, empirical data are used to stimulate so-called induction of the threat actor
typology.
The deductive approach defines general properties or dimensions of threat actor types. The
deductive phase starts by analyzing the observed distinguishing characteristics of the threat
actors: motives, capabilities, degree of organization, et cetera. Combining these dimensions
results in a matrix of potential threat actor types who may or may not be observable in the
current threat actor landscape. To use an analogy: the dimensions would serve to identify a
set of threat actor types, like the periodic table does identify elementary particles in
chemistry. Based on a number of key characteristics elements can be ranked, grouped and
identified. Threat actors identified in practice would function like elementary particles to the
whole table of elements. Like the periodic table of elements, the conceptual threat actor
typology could take on a similar role as the table in the early 1900s when some of the
elements (i.e. certain threat actor types) were not yet identified in practice. However, all
elements eventually were identified and observed decades later. Some even because their
existence was already inferred. Unfortunately, unlike the table of elements, a generic theory
which would explain and predict cyber actor classes is (still) absent and therefore the
analogy does not hold. The typology in this report therefore ‘merely’ enables users to
systematically classify the cyber actor types.
The inductive approach forms a second additional, parallel step in the development of a
method to develop a threat actor typology. It involves the systematic process of extracting
threat actor information from available empirical data sources: specific incidents, large-scale
measurement data, victim surveys, interviews with experts, etc., to analyze developments
and trends. Behavior of threat actors and characteristics of threat actor types are identified
by analyzing data. Empirical data is thus used to feed the threat actor typology and
15
potentially yields additional information about threat actor types, enabling reflection and
improving upon the inductively deduced threat actor typology. Furthermore, the inductive
approach is necessary to accommodate for the fact that cyberspace keeps on changing and
cyber actors develop and employ new attack vectors every day. Their behavior is dynamic
and may change over time with the acquisition of new skills (Jahankhani & Al-Nemrat, 2012).
Empirical data helps to capture the dynamics. The other issue is that the data used in cyber
security assessments are based on generalizations, and the sampling leaves out a dataset of
cyber actors who avoid detection over a period of time, thereby introducing inaccuracies in
the results (Noroozian et al., 2015). Relying on inductive methods only is unsuitable for a
method that intends to produce a dynamic threat actor typology.
On the other hand, relying only on deductive profiling will leave investigators oblivious to
current trends such as popular attack methods, likely targets and victims (Tennakoon, 2011).
Therefore, a hybrid methodology is the logical remaining option to ensure the continuous
development of threat actor profiling as part of a loop (Warikoo, 2014). The method thus
assumes a cyclic character and results in a method that systematically creates a multidimensional set of characteristics of threat actors deductively and enriches this set with
empirical information that was obtained by inductively analyzing cyber security datasets and
reports.
The hybrid approach leverages a broader set of sources and methods to proactively collect
and passively detect indicators and characteristics of threat actors, thus benefitting from the
structured and continuous analysis of all potential data.
Figure 1 shows the resulting methodology that is best visualized around the cyber actor
typologies that are in use by NCSC/NCTV in the CSAN’s (cf. NCSC, 2015; 2016). The
complete method can be visualized as a sequence of at least two loops, which feed back into
the CSAN threat actor typology. The first loop deduces from existing literature key threat
actor characteristics (i.e. motives, capabilities, degree of organization, etc.). When these
characteristics are cross tabulated, a systematic and finite typology of existing and (yet) nonexisting types of threat actors can be composed.
The second loop consists of an inductive approach which utilizes the available empirical
data. Various methods such as data mining techniques can be employed to systematically
identify and observe behavior of threat actors. A complete first iteration starting with the
typology described in the CSAN 2016, followed by a loop in which a deductive approach is
applied and then a loop in which information is inductively analyzed.
This method can be divided in three subsequent steps:
2.5.1
Cycle one: deductive approach
In the first cycle, a structured model of (potential) cyber threat actors that (could) threaten
Dutch data systems is created. As a starting point, a concise literature research identifies the
dimensions that are used in (cyber) threat actor typologies. To identify and construct a new
threat actor typology, a somewhat broadened scope was chosen for the initial literature
research. Google scholar and (academic) databases Elsevier Scopus and IEEE Xplore were
searched in search of literature displaying useful methods to generate a threat actor typology
or a completed threat actor typology. The following keywords were used in various potential
combinations as outlined in Table 2.
16
Figure 1: A hybrid method to develop a new threat actor typology
Using these search terms yielded a selection of publications which could be further reduced
based on closer review and resulted in a data base of some 70 publications that seemed to
hold potentially relevant information for the development of a threat actor typology. There
exist several typologies and ways of classifying cyber actor and cyber criminals in particular
based on their motives on which was built (e.g. Johnson, 2005; Jahankhani & Al-Nemrat,
2012; Rogers 2006). Also empirical interviews were conducted to identify relevant threat
actor characteristics. In total 18, semi-structured in-depth interviews were held with security
experts that are in a privileged position with regard to knowledge about threat actors. More
details on the interviews can be read in section 3.3). The selection of respondents was based
upon a desire to achieve overall representation of stakeholders ranging from hardware
designers to software providers, IT service providers, banks, small and medium enterprises
all the way to police agencies who either work with the threat actor typology or play an
important part or are engaged in cyber security. Secondly, the classification scales are
established. This allows NCSC/NCTV staff to perform the proper threat actor classification
process themselves. To support the NCSC/NCTV staff in this task, a threat actor typology
framework is developed as part of the method. In the interview round with stakeholders, the
threat actor typology framework is validated and additional information obtained on relevant
characteristics distinguishing threat actors from those which are less relevant (for the
foreseeable future).
Cyber
Attacker
Taxonomy
Actor
Profile
Threat actor(s)
Typology
Threat agent
Hacker
Table 2: Keyword search strategy
17
2.5.2
Cycle two: inductive approach
In the second cycle – which actually was performed parallel during this research project –
different databases that contain observations of cyber incidents are analyzed in small case
studies. Four different types of empirical data are used in this report to show how this data
could feed into the treat actor typology. Data was used from honeypot data, sinkhole data,
darknet/IDS data, spam trap data, and data from cyber criminal markets. By analyzing the
data and establishing correlations between certain events and/or types of behavior, certain
characteristics common to the different threat actor characteristics resulting from the
deductive phase can be inferred allowing classification and thus yielding valuable information
about threat actor types in addition to the information obtained (through interviews) in the
deductive phase (cf. Caltagirone, Pendergast & Betz, 2013):
1. Observations of digital meeting places. Research on meeting places where various
cyber actors meet and communicate with each other, such as underground criminal
markets aid in the identification of cyber actors. Anecdotal data on the behavior in
these meeting places shed light on actors (Aston et al., 2009). Analyzing online
forums in the marketplaces provides information on how specific cyber actors meet,
how specific cyber criminal networks develop and what this means for the attack
capabilities of these networks.
2. Analyzing cyber incident datasets. Cyber incidents can be used to understand not
only the attack vector but also provide additional information on the behavior and
capabilities of cyber actors. Datasets (such as SPAMHAUS blacklists, Anti-phishing
working group phishers) and public datasets (such as Clean-MX phishers, Abuse.ch
botnet) which were used in this project contain information about phishing sites,
spam, botnet command, etc. Data mining and data warehouse techniques were used
to analyze types of cyber incidents to obtain knowledge of cyberattacks and the threat
actors involved in them.
3. Monitoring ongoing attacks. Apart from incident data, the information about threat
actors can be improved by the addition of data obtained from observations about
ongoing attacks (e.g. via honeypots and IDS logs). The information received via
DSHIELD logs were used to provide additional insights on attacker behavior.
4. Analyzing data related to victims. Additional analysis of datasets could provide
information about victims, which in turn could provide additional knowledge about
characteristics of cyber actors (type of victims chosen (MO), geographic details about
the cyber actor, information about defenses and associated skill levels of the cyber
actor). Some cyber victim analysis has already been carried out by national law
enforcement agencies. For example, London’s police created a profile of the victims
of cyber fraud over the twelve-month period of November 2014 to October 2015
(Police City of London, 2016).
2.5.3
The design cycle completed: developing a threat actor typology
The final step in the method consists of the creation of a threat actor typology making use of
the data obtained in both cycles. This then completes the method and enables NCSC/NCTV
to make use of the available information on threat actors. This cycle can be reiterated over
time. For example if new attacks or new vulnerabilities emerge, the threat actor typology
might be in need of review or reassessment. The consideration of how and when to engage
in a second threat actor development cycle forms a crucial aspect of the proposed method.
18
3 The deductive approach –
threat actor typology
framework
Chapter three describes the first phase of the deductive part of the method to develop a
typology. Literature review identified various bodies of literature and various typologies of
threat actors and dimensions to bootstrap the development of an initial typology. As a
subsequent step interview data and a workshop are used to operationalize the threat actor
dimensions and develop a threat actor typology framework.
3.1
3.1.1
Literature review: in search of threat actor dimensions
Universal cyber threat actor typologies
In literature, a number of elaborate universal classifications of cyber attackers can be
identified. One of the oldest is known as the ‘Threat Agent library’ (TAL) which can be seen
in Figure 2. This library identifies 23 threat actor types which obtain a unique score along 8
different dimensions (Casey, 2007; Casey, Koeberl & Vishik, 2011). Each threat agent is
separately and relatively extensively described, as can be seen in Figure 3.
A second, more recent generic classification scheme is developed by the European Union
Agency for Network and Information Security (ENISA) and can be seen in Figure 4. This
classification scheme, initially distinguishes seven threat actor types and is later expanded
into 15 threat actor types which are identified via three dimensions: ‘sector’, ‘capability’ and
‘motive’ (Marinos, 2013:39; 2014; 2016).
Both generic threat actor typologies show the challenges involved in establishing a threat
actor typology and the complexities of classifying threat actors. Although helpful and
elaborate, their sheer size (i.e. the number of threat actor types and/or the number of
dimensions on which they are based) raise question marks with regard to usability
requirements. However, both typologies identify the dimension ‘motivation’ as the most
relevant threat agent characteristic (cf. Pushpakumar, 2015; Van Hulst & Neve, 2008).
19
Figure 1: TAL threat actors. Source: Casey, 2007:5. Table 1: Current Library of Threat Agents
and Their Defining Attributes
Figure 2: Details on TAL’s threat agents. Source: Casey (et al.), 2011:219. Figure 2: Sample
subset of threat agents
20
Figure 3: ENISA’s threat actor characteristics (sector, capability, motive). Source: Marinos,
2013:39. Figure 20: Overview of Agents in Cyber Space
3.1.2
(Inter)national cyber threat actor typologies
A second source of knowledge (to identify dimensions) for generic threat actor typologies can
be identified in publications which identify, analyze and compare (inter)national cyber
security policies and the typologies used (cf. Burton, 2015; Luiijf et al., 2013; Robinson et al.,
2013; Canbolat & Sezgin, 2016). Interestingly, some noticeable differences exist between
various countries and their use of threat actor typologies. First of all, certain countries such
as France and Finland had not (yet) published a public version of their cyber threat actor
typology. Another distinction is the amount of threat actor types that can be observed in
various national policies. The threat actor typologies in the Dutch CSAN (NCSC, 2015; 2016)
are among the most detailed in use by nation states (cf. Robinson et al., 2013).
Other countries distinguish cyber threats from cyber actors. However, even here differences
between the various threats and threat categories exist. For example, Burton (2015:299)
identifies four cyber threats (cyber crime, cyber espionage, cyber terrorism, and cyber
warfare), whereas for example, Canada identifies three broad types of threat (cyber
espionage and military operations; terrorist use; and cyber criminal activity)(Sheldon,
2012:6). These broad threat types are further specified to produce more detailed threat actor
typologies/taxonomies. To stick with the Canadian example: the broad threats are merged
with empirically observed threat actor characteristics such as ‘motivation’, and ‘attack types’,
which produces five cyber threat actors types: nation states, terrorists, criminal organizations,
disgruntled insiders and hacktivists. Other studies (cf. Luiijf et al., 2012) identify a similar
range of threat actors: individuals, activists, criminals, terrorists, cyber spies, non-state and
state.
21
This short review of national cyber actor typologies in cyber security policies illustrates that
the typologies and methods on which nation states base their cyber security policies seem to
differ substantively. Substantial differences in granularity of identified cyber threat actor types
exist. However, all in all, nation states seem to identify “similar types of threat actor types”
(organized crime, states and terrorist networks)(Robinson et al., 2013:40).
3.1.3
Typologies focusing on specific attack types
A third group of typologies in literature distinguishes threat actor types based on the attack
type. For example, the U.S. Industrial Control Systems Cyber emergency Response Team
identifies the following cyber threat actor types (for Industrial Control Systems): national
governments, terrorists, industrial spies and organized crime groups, hacktivists and
hackers. US Congress, however, identifies another set of threat actors in cyber crime ranging
from “lone actors to expansive criminal networks or even nation states” (Finklea & Theohary,
2015:1).
Johnson (2005) and Jahankhani & Al-Nemrat (2012) argue that criminological dimensions
based on classifications of past incidents could be used to identify cyber criminals. Key
dimensions according to Johnson (2005:78) are ‘modus operandi’, “the actions taken by an
offender to perpetrate the offense successfully” and ‘signature’, “a repetitive ritualistic
behavior that the offender usually displays at every crime scene” (cf. Rogers, 2003:295,
footnote 5). However, various sources mention a persisting lack of empirical knowledge of
cyber attackers and their specific characteristics (cf. Van Hulst & Neve, 2008; Koops, 2010;
Carrapico & Lavorgna, 2015).
Table 3: Selection of hacker threat actor types. Source: Meyers (et al.), 2009:8. Table 1: A
Koops
(2010)
identifies
four key dimensions of threat actors engaged in cybercrime: ‘aims’,
Taxonomy
of Cyber
Adversaries
‘methods’, ‘skills’, and ‘motivation’. Researchers have often proposed that some cybercrimes
require more technological expertise or heavier use of digital technologies to penetrate than
others (Gordon & Ford, 2006 in: Finklea, 2015). So, implicit in the notions of growing
sophistication of attacks and capability is also the idea that these might be helping to create
such a dimensions as ‘criminal career’ but also display various levels of organization. The
dimension ‘group characteristics’ as element of cyber attackers is mentioned as an important
dimension, especially as researchers identify a trend of increasing sophistication,
22
industrialization and subsequent specialization occurring in cyber crime (cf. Koops, 2010;
Broadhurst et al., 2014). McGuire (2012) in Broadhurst et al. (2014) claims that “80% of
cyber crime could be the result of some form of organized activity”. However, much unclarity
as to the exact nature and predominance of organization in cyber crime remains (cf. Koops,
2010; Carrapico & Lavorgna, 2015). Consequently, different group characteristics (e.g. Van
Hulst & Neve, 2008) and different group types (e.g. Choo, 2008; McGuire in: Broadhurst et
al., 2014) need to be identified.
Examples of specific threat actor groups which have resulted in specific
typologies/taxonomies are: ‘insiders’ (cf. Meyers, Powers & Faissol, 2009; Nurse et al., 2014;
Nykodym, Taylor & Vilela, 2005) and ‘hackers’ (cf. McBrayer, 2013; Van Holsteijn, 2015).
One of the oldest ones is Rogers’s typology (2006; 2009) which identifies different hacker
types based on the dimensions ‘motivation’ and ‘skill level’, although others have
subsequently added more classes to the dimension ‘motivation’ (cf. Meyers, Powers &
Faissol, 2009) as can be seen in Table 3.
As one of the latest hacker typologies, Seebruck (2015) has identified a relatively simple two
dimensional (‘motivation’ and ‘sophistication of attack’) method to plot the various threat actor
types as can be seen in Figure 5.
Figure 4: Seebruck’s threat actor dimensions. Source: Seebruck, 2015:40. Figure 1: A circular
order circumplex of hacker types
These typologies again confirm that typologies in use often consist of (too) many different
threat actor types but also that dimensions such as ‘motivation’, ‘skill’ and ‘level of
sophistication of the attack’ and some aspect of organization more or less consistently
reappear.
23
3.1.4
Typologies focusing on attacks on specific targets
A fourth set of typologies in literature identifies threat actor types via typologies and
taxonomies of targets. For example, Gandhi et al. (2001) identify various important attack
dimensions such as ‘motive’, ‘victims’, ‘means of attack’ and ‘consequences’ as can be seen
in Figure 6.
Figure 5: Attack dimensions. Source: Gandhi (et al.), 2001:36-37. Figure 4: Categorization of
cyber-attack dimensions
Examples of typologies of more specific attacks on targets include cyber laundering
(Filipkowski, 2008), DDoS attacks (Mirkovic & Reiher, 2004), attacks on SCADA systems
(Zhu & Sastry, 2011), critical infrastructures (Rege-Patwardan, 2009), cloud services
(Gruschka, 2010) and high-tech crime (cf. Van Hulst & Neve, 2008). Dimensions in these
typologies are often compiled via so-called profiling studies at the classification levels of
attacks. Finally some authors discuss the term attack vectors (cf. Simmons et al., 2009;
Choo, 2011) and analyze the threat of cyberattacks but do not relate them to actors but to
the type of crime or attack.
3.1.5
Conclusion
The main finding of the literature research is that no generic concise threat actor typology
can be identified and underlying information regarding the methods used and the
construction of the typologies are often unclear. Different countries employ different methods
to identify threat actor types. Furthermore, many of the typologies in literature are either too
generic, generating unwieldly amounts of threat actor types or focused too specific on
particular attack types (e.g., DDoS attacks) or on specific classes of threat actors which
focus on specific targets (e.g., SCADA systems, critical infrastructures, etc.). The majority of
studies in which cyber threat actor types are identified or threat actor typologies are
presented fail to provide detailed information on the classification method.
24
Despite these findings, which overall indicates a disheartening picture of state-of-the-art
thinking on threat actor typologies, a certain common basis for building a cyber attacker
typology emerges. Relatively little variation exists in a number of key dimensions, which
means that the variation in the clusters of factors which describe threat actors seems fairly
low. Dimensions identified in these various literatures are highly overlapping and can be
synthesized in five dimensions:
1. target
2. expertise
3. resources
4. organization
5. motivation
While there is a lot of support in prior work for these five dimensions, there are often
inadequately conceptualized and operationalized to identify (threat actors) in the current
threat landscape. This is especially true for the dimension ‘organization’. Few prior
frameworks have explicitly conceptualized it. The frameworks that did, produced awkward
threat actor classes. For example, Broadhurst et al. (2014) identify 6 different types of cyber
criminal groups. However, as a sub-dimension, they distinguish level of online activity, and
thus identify offline and online cyber criminal groups. Such a classification clearly does not
suit the purpose of this research. It is important to develop a better understanding of this
dimension as it has become increasingly critical to our understanding of the threat
landscape. Threat actors increasingly collaborate and form larger organizations, loose
networks or flexible criminal supply chains, which makes them increasingly difficult identify as
groups (cf. Mission Support Center, 2016:17; Burton, 2015). To incorporate these insights,
the existing dimensions need to be developed beyond the state-of-the-art in the literature.
3.2
Operationalizing the dimensions: developing the framework
A second step in the deductive phase entails the conceptualization and operationalization of
the dimensions. One of the key requirements of the method to develop the threat actor
typology is that the method is replicable by security professionals. NCSC/NCTV intelligence
analysts in particular are considered to use and maintain the typology in the future (WODC,
2016). To support practitioners, an intermediate product is developed that allows
NCSC/NCTV analysts to create a manageable set of threat actor types, and a tool to support
the actor classification process and add rigor to it, which contributes to the method of the
development of a new cyber threat actor typology and its cyclic nature. The form of the tool –
a so-called cyber threat actor typology framework – is designed as a concise set of questions
that supports NCSC/NCTV staff members to quickly classify incidents or an attack (scenario)
and subsequently identify threat actor types behind security incidents. Like the typology, we
do not claim to provide the definitive cyber threat actor typology framework. The framework is
provided to illustrate the method that is developed. For example, the choice of the classes
was made by the researchers with the explicit aim to reduce the number of potential threat
actor classes where possible. Consequently, the choices made in the next subsections for
the classification scales can be criticized. We will return to this issue in the reflection in
section 5.5.
25
3.2.1
Target
The first dimension in the threat actor typology framework is the identification of the ‘target’,
i.e. victim who owns the asset that is the target of the threat actor. In Meriam-Webster, the
term target is defined as: “a place, thing, or person at which an attack is aimed.” In search for
a concise, yet useful classification the initial version of the framework yielded the classes:
‘individuals’, ‘property’, ‘organizations’. However, the classification is extended to ensure that
various target-types are identified. Various classes on a continuum from the individual citizen
to the whole of society are subsequently identified and evaluated. In the final version of the
typology framework 4 classes are identified: ‘citizen(s)’, ‘enterprise(s)’, ‘public sector’, and
‘critical infrastructure(s)’.
The second dimension was initially defined as ‘capability’, which was subdivided in three
simple classes (high, medium and low). However, to allow for a finer granularity in the
assessment of the capability of threat actors, this dimension is split up into the dimensions
‘resources’ and ‘expertise’. This is especially useful since this allows security practitioners
and particularly NCSC/NCTV staff to make full use of available incident (scenario) data.
3.2.2
Expertise
The dimension ‘expertise’ describes what knowledge and skill level the threat actor needs to
possess to plan, organize and successfully conduct the (intended) attack. Expertise is
defined as: “the level of generic knowledge of the underlying principles, product type or
attack methods (e.g. Internet protocols, Unix operating systems, buffer overflows).” (ISO/IEC
18045(2008):284). For the dimension skill, three simple values are provided: low, medium or
high (Van Holsteijn, 2015:37). When different types of expertise are required, the range of
required levels of expertise are recognized.
3.2.3
Resources
As ‘resources’, Lenin, Willemson & Sari (2014) identify such resources as ‘budget’ and
‘available time of the attacker’ (Van Holsteijn, 2015:26), which are subsequently used in the
threat actor typology framework. To further aid in the classification process, a limited number
of indicators are provided and ample examples in the illustrative text of the threat actor
typology are provided.
The combined dimensions of ‘expertise’ and ‘resources’ allow the distinction between the
various attack patterns which point towards different types of threat actors as can be seen in
Table 4. Threat actors that wage attacks with low levels of expertise and are capable to
mobilize large amounts of resources are able to mount DDoS attacks, just like Anonymous
has been generally inferred to do (Mansfield-Devine, 2011). On the other extreme a single
hacker, was thought to be singlehandedly responsible for the so-called Mirai-attacks.
Employing malware, which reportedly allowed the hacker to harvest a large a botnet
consisting of more than 500,000 IoT devices, these bots were used to conduct the largest
DDoS attacks seen so far (Krebs, 2017). The hacker had used high levels of expertise and
low levels of resources to infect millions of devices via malicious code (Pultarova, 2016).
Similarly, a low level of resource, low expertise type of attack is typically attributed to scriptkiddies who use readily available exploit kits and attack old and well-known vulnerabilities. At
the other extreme, one could identify attacks involving high levels of expertise and resources
such as the attack on the Ukrainian electricity grid (E-ISAC, 2016; Zetter, 2016a) and the
Stuxnet attack (Langner, 2011a; 2011b). It could be argued that a very different type of threat
actor would be required to undertake the complex and highly resource-intensive attack on
26
the Ukrainian electricity grid. Not only was the level of expertise high, but also the amount of
resources required for this attack could be labelled as high in that a breach was present
many months before the actual event took place, allowing the attackers access to the
Ukrainian grid operator systems.
The attack on the Ukrainian low-voltage electricity grid
For the first time in history hackers managed to gain control of the (low-voltage) power
systems in parts of the Western Ukraine in December 2015. Although the size of the impact
was small, the attack has gained notoriety as being the first physical take-over of SCADA
systems affecting a vital civilian critical infrastructure. The blackout which resulted from the
coordinated attacks on various infrastructure operators lasted between 1 and 6 hours and
affected some 230,000 people. Post-incident analysis revealed a complex and coordinated
attack-pattern, conforming an elaborate preparation, and execution of highly coordinated
attacks. Although the tools used showed high expertise (e.g. sophisticated spear phishing,
what really stuck was attacker’s “capability to perform long‐ term reconnaissance operations
required to learn the environment and execute a highly synchronized, multistage, multisite
attack” (E-ISAC, 2016:5). Zetter (2016a) quotes an expert saying: “To me what makes
sophistication is logistics and planning and operations and … what’s going on during the
length of it. And this was highly sophisticated.”
Stuxnet
Stuxnet, “was first reported in June 2010 by a security firm in Belarus, [and] appears to be
the first malicious software (malware) designed specifically to attack a particular type of
Industrial Control System (ICS)”(Kerr et al., 2010:1). It turned out to be a highly sophisticated
and aggressive worm which could spread to computers that were not connected to the
internet; it was highly targeted, yet was also specifically designed to remain undetected
(Falliere et al., 2011; Langner 2011a; 2011b). The malware was not designed to steal
information, but rather to target and disrupt control systems and disable operations. Even
more specifically, Stuxnet disrupts a Microsoft Windows-based application that is employed
by Siemens ICS’s in nuclear facilities, particularly those of centrifuges, which enrich nuclear
material. “The code’s sophistication suggests that a nation state was behind the worm’s
development, either through proxy computer specialists or a government’s own internal
government and military capabilities” (Kerr et al., 2010:1). The developer had to be
“financially well-resourced, employ a variety of skill sets (including expertise in multiple
technology areas), have an existing foreign intelligence capability in order to gain access and
knowledge of a foreign system, and be able to discretely test the worm in a laboratory
setting” (Kerr et al., 2010:2).
27
Expertise
Low
Medium
High
Resources
Low
Script-kiddie attack
Mirai-attack (2016): hacker
infects millions of machines
with malware
Anonymous mounts
DDoS attacks
Targeted attack on the
Ukrainian electricity grid
(2015), Stuxnet (2010)
Medium
High
Table 4: Expertise and resources identify different attack patterns and different threat actor
types
3.2.4
Organization
The fourth dimension, ‘level of organization’, was initially operationalized in terms provided
by McGuire (2012 in Broadhurst et al., 2014) who identified two sub-dimensions (‘level or
organization’ and ‘level of online activity’). The resulting organizational types (swarms, hubs,
clustered hybrid, extended hybrid, hierarchies and aggregate groups) however, do not seem
very informative.
To increase the conceptual rigor and analytical relevance of this dimension we turn to a wellestablished distinction from institutional economics and governance studies: hierarchy,
market, network (cf. Williamson, 1985; 1999; Bevir, 2012). This classic distinction is later
extended to expressly include more loosely organized bodies such as communities and
collectives, which also seem to play a relevant role in the current cyber threat landscape
(Tenbensel, 2005; Alexander, 1995).
Table 5 summarizes the classes of the dimension ‘organization’. On the one extreme the
collaborative form ‘hierarchy’ can be identified, which relies on “authority and centralized
control” (Bevir, 2012:16) to coordinate tasks. The assumption behind hierarchical forms of
collaboration is the existence of a unified command structure, clear purpose, and
specialization. Enforcement of authority is often “achieved by sovereignty and jurisdiction of a
nation-state, by organizational control of the firm or by contractual regime. Examples include
national laws and regulations, formal intergovernmental arrangements, organizational cyber
security policies, or ICANN and RIR contracts, etc.” (Kuerbis & Badiei, 2017). Generally,
hierarchies rely on “a rule-based approach to authority” (Bevir, 2012:16), meaning a clear
command and control structure, which emphasizes top-down control. The advantage of the
hierarchical structure is typically that it is able to take on more complex tasks that require a
lot of coordination, which is more difficult to achieve via markets or network interactions
among relatively autonomous agents.
‘Networks’ can be defined as: “multiple actors who are formally separate but depend on one
another for key resources and so build long-term relationships to exchange resources”
(Bevir, 2012:26). Network structures provide a "semi-permanent, voluntary negotiation
system…[that] allows interdependent actors to opt for collaboration or unilateral action in the
28
absence of an overarching authority” (Scharpf, 1997; Mueller, Schmidt and Kuerbis, 2013).
The rise of networks has been identified as an important trend in (cyber) criminal literature on
the attack as well as the defensive side (cf. Choo, 2008; Kshetri, 2010; Broadhurst et al.,
2014; Leukfeldt, 2016). Networks differ from hierarchies because they do not usually contain
an authoritative command and control center to resolve disputes among the actors.
Networks, instead more rely on trust across webs of associations. They differ from ‘markets’
– the next class – in that actors engage in repeated and more prolonged exchanges via
coordination methods other than bargaining. Instead, they employ mechanisms such as trust
to facilitate coordination and collaboration. Variations in network forms can occur with more
'dense' forms of networks which lean towards the more hierarchical side and 'looser'
networks in which relationships between actors are shorter and obviously closer to the
market side. Similarly interdependence in networks varies from participatory networks, where
actors have roughly equal resources to 'managed networks' where lead actors have more
resources and take on a coordinating role.
A ‘market’ is "a more or less formal arena in which goods [or services] are exchanged for
other goods and especially money” (Bevir, 2012:22). Transactions among actors are
primarily driven by information and price mechanism, and enforced by law and contract.
Examples of markets in the realm of cyber security are “the purchase of cybersecurity
consulting services, security software and equipment, zero-day markets, etc. “(Kuerbis &
Badiei, 2017). Markets for cyber crime have similarly grown quickly in complexity, size and
sophistication (cf. Holt, 2012; Ablon et al., 2014). Actors engage voluntarily in exchanging
goods at a specific price, which is determined by their interaction. In contrast to the networks,
the interactions are more “episodic” or “isolated” and “impersonal” as coordination is enabled
via mechanisms such as prices and competition (Bevir, 2012:24). Consequently, markets are
placed lower after networks on the dimension.
coordination
mechanism
basis of
relations
among
members
degree of
dependence
among
members
means of
conflict
resolution and
coordination
hierarchy
network
Market
collective
authority
trust
Price
solidarity
Jurisdiction of a
nation-state,
organizational
control of firm,
contractual
regime
exchange of
resources
contracts and
property rights
common
interest
dependent
interdependent
independent
independent
permanent
structures, rules
and commands
semi-permanent
structures,
negotiation,
diplomacy
episodic
haggling,
bargaining
all the means
of other forms,
but also voice
and exit
Table 5: Coordination mechanisms in various group settings. Based on: Bevir, 2012:17. Box 1.
A typology of organizational structure
29
Finally, as the least coordinated group, ‘collectives’ of individuals can be identified who
engage in forms of collective action, which in turn can be defined as “all activity involving two
or more individuals contributing to a collective effort on the basis of mutual interests and the
possibility of benefits from coordinated action” (Marwell & Oliver, 1993 in: Agarwal, Lim &
Wigand, 2011:226)(cf. Kumar, Raghavan, Rajagopalan & Tomkins, 1999:1481; Lee, Vogel &
Limayem, 2003).
3.2.5
Motivation
As the fifth and final dimension, the ‘motivation’ of the threat actor was identified. Van
Holsteijn (2015) identifies two main sources of motivation (internal and external) of threat
actors, resulting in a range of sub-classes: financial benefits, causing damage, knowledge
gaining, pleasure, and notoriety (cf. McBrayer, 2014). The sources of motivation were
reduced to the proposed classes: ‘personal’, ‘economic’, ‘ideological’ and ‘geo-political’ to
speed up the classification process. The ‘personal’ class contains everything a person gains
from an attack except economic gain, which includes incidents from disgruntled employees
and behavior such as cyber bullying, doxing people and cyberstalking. It should be noted that
the classes are not mutually exclusive but can be used to characterize the dominating
motivation and therefore the underlying goal of the attack of the threat actor.
3.2.6
Conclusion
After having operationalized the five dimensions of the typology design, it could be argued
that the theoretical challenge of the design of the typology is complete. With the identification
of the key threat actor dimensions and the subsequent operationalization of the dimensions a
finite range of possible cyber actor types can be identified. The sheer amount of potential
threat actor types, however, would make the typology simply unusable.
Any user of the typology design faces the daunting task to systematically cut back the
potentially vast number of options to manageable proportions. And this should be done in a
structured and controlled sense and should also be replicable over time and by different
people. In short, a second and crucial step in the design of a usable treat actor typology
design method would be a tool which users could use to quickly identify threat actor types
and aid in the classification process. The next section discusses the reaction of stakeholders
and cyber security experts on the proposed threat actor typology dimensions and classes.
This information will be used to help develop such a tool, which we call a threat actor
typology framework.
3.3
Feedback on the framework from experts and stakeholders
As part of the design of cyber threat actor typology semi-structured interviews were held with
stakeholders and potential future users about the CSAN threat actor typology. Interviews with
cyber security stakeholders such as analysts of NCSC, but also cyber security experts and
(representatives of) victims of criminal behavior and cyberattacks were conducted to validate
the deductively generated threat actor typology. In total 18, semi-structured in-depth
interviews were held with security experts that are in a privileged position with regard to
knowledge about threat actors, and that have valued perspectives on both the current CSAN
actor typology, and their preferences for certain dimensions. The selection of respondents
was based upon a desire to achieve overall representation of stakeholders ranging from
hardware designers to software providers, IT service providers, banks, small and medium
enterprises all the way to agencies engaged in cyber security who either work with the threat
actor typology or play an important part or are engaged in cyber security. The selection of
respondents was coordinated with the research committee. 4 representatives from critical
30
infrastructure industries were interviewed and a final one declined after having agreed to the
interview initially, 4 experts from (inter)national cyber security companies, 2 large
multinationals , 2 representatives from the banking industry, 2 representatives of industry
sectors, 2 cyber security researchers and 2 finally two staff members from NCTV/NCSC. The
interviews were either recorded or summarized via field notes. Respondents were provided
with short minutes of the interviews. Given the sensitivity of the research topic respondents
were promised anonymity to freely talk about threat actors and the threat actor typology. No
information will therefore be attributable to single individuals and/or organizations. The
interviews were designed in such a way that they could provide information to both the
inductive and deductive cycle. Respondents were invited to share impressions about
observed threat actor behavior or accumulated knowledge about trends or processes which
could be linked to threat actors as well as information about the design of the threat actor
typology and more specifically the threat actor typology framework. The respondents were
questioned about their opinion on three generic themes; each theme is summarized in the
following sub-sections and provides important information which for the design of threat actor
typology.
3.3.1
Dimensions of a cyber actor typology
Respondents were first asked what threat actor characteristics they considered most
relevant. Which threat actor characteristics enabled them to identify one threat actor type
from the other? Interestingly many respondents started their responses by claiming that that
their organization did not have the capability, the resources, or the time to engage in
elaborate processes of threat actor identification. Security experts added that it was almost
impossible to readily identify threat actors.
One critical infrastructure company actually declined an initial positive response to the
interview claiming that the progress towards a threat actor typology had not progressed to
the extent that a meaningful response could be provided to the interview protocol that was
sent along with a request for an interview.
However, as an important characteristic, experts from a cyber security firm, distinguished
important attacks from threat actors from less important ones based on the more ‘businessoriented’ nature of attacks and their repetitive nature. A representative of an energy network
company added that an additional important distinction to assess threat level was whether
an attack was ‘limited’ to the cyber domain or part of a much more threatening and complex
too organize mixed, coordinated physical and cyber attack. Important info the cyber security
expert needed to know about incidents is: where did the attack take place and what was hit
and what are the consequences for the primary process.
Many organizations such as NCSC, a multinational bank, as well as large international hardand software providers explained how elaborate incident monitoring and analysis were of
crucial importance to them to engage in attribution. To identify threat actor types thus
requires a good Computer Emergency Response Team (CERT) capability as well as a good
level of incident data registration. Extensive technological capabilities such as (near) real
time intrusion detection systems and elaborate procedures are used to monitor threats. The
representative from the large bank mentioned that acquiring this capability requires
substantial investments in incident registration and monitoring.
Attribution is accomplished via analysis of the detailed technical characteristics of an attack,
the so-called ‘modus operandi’. When attack patterns reappear (i.e., use of the same
infrastructure; similar attack pattern), the underlying toolbox of the different attackers is the
same. Furthermore all attackers develop unique patterns of attack, use their own toolset and
31
slightly different settings. Respondents explained how advanced analysis by forensics
specialists in special departments in large multinationals which develop hard- or software
analyze these threats, identify threat actor attack types, and develop responses as fast as
possible; for example in response to zero-day exploitations.
Representatives from various cyber security firms confirmed the limitations and approaches
mentioned by representatives of so-called target organizations and argued that threat actor
types were primarily identified and defined via analysis of their tools, techniques and
procedures (TTPs) and the consequences of the attacks. Basically, feeding this analysis is
as much information on the attacks as is possible to collect. As a consequence of this
approach one international IT security firm identified four threat actor dimensions (‘general,
capability’, ‘modus operandi’, ‘activity’). Three of these dimensions consist of 6 classes3
resulting in 11 identified threat actor types. A representative from another internationally
operating cyber security firm identified three broad threat actor types: ‘activists’, ‘criminals’
and ‘nation states’ and explained that his company specialized in cyber crime and
subsequently identified more different and specific threat actor types based on various attack
methods. The representatives of cyber security firms thus stressed the importance of a more
detailed cyber threat actor typology; this also influenced their reactions to the cyber threat
actor typology framework. Their focus seemed to lie primarily with specific threat actor
attribution rather than actor type classification.
A senior security manager at a big European bank admitted that the company had a threat
actor typology which was nearly similar to the one used in CSAN, but its role was not
formally established and consequently it was applied differently throughout the organization.
The respondent had inquired in the organization and found out that although a lot of
information was generated about aspects related to threat actor characteristics (‘modus
operandi’, ‘threat matrices’, etc.), (almost) no information was explicitly collected about cyber
threat actor characteristics.
Nearly all respondents thus employed resources and extensive processes to collect empirical
data which supported the identification of threat actor types based on incidents. In sharp
contrast, a representative of a critical information infrastructure company found an elaborate
incident reporting system largely time and resource consuming. Although the organization
recognized the importance of a CERT capability, it found elaborate incident registration too
complex and cumbersome to cope with the rapidly evolving threat landscape and the
enormous amounts of threats. Instead, the company employed a very concise typology
which consisted of three different dimensions: ‘threat vector’, ‘motivation’, and above all
‘business impact’. Furthermore, the organization only identified 4 different threat actor types.
The respondent explained that ‘business impact’ was very important as the main goal of the
typology was to inform and alert executive board members about ongoing threats and keep
their attention on these incidents. The small and concise typology, along with a ‘light’ incident
and impact registration process according to the representative, enabled the critical
information infrastructure company to quickly identify threat actors and to adhere to a
rigorous and uniform method of communication about threat actors across the organization
and especially to the executive board. Furthermore, it enabled the company to develop
additional tools such as an online threat index based on number of incidents and types of
3
General (classes: Associated events, Actor type/category, Motivation, Target sector, Target
geography, Intended effect; Impact effect); Capability (classes: Resources, Skills, Resolve, Access to
target, Risk sensitivity, Capability score); Modus Operandi (MO)(classes: Reconnaissance activities
identified, Preparation, Infiltration, Entrenchment, Compromise, Exploitation); Activity (Activity score,
Date of incident (per incident)).
32
incidents to provide the organization and its executive members with a sense of the severity
of the current situation, analogous to the public 'defcon' or terrorism alert levels.
3.3.2
Perspective on the current and proposed cyber actor dimensions
A second set of questions asked the respondents to reflect on the CSAN typology and the
dimensions which were deductively identified. First off, the CSAN typology was criticized for
a variety of reasons.
Some interviewees such as a critical infrastructure operator found the CSAN typology too
complex and too time consuming to assess incidents and identify a potential threat actor
type. A future typology would have to improve on this characteristic.
Secondly, when reviewing the list of threat actor types from the CSAN actor typology (NCSC,
2016:12, Table 1) respondents could not always explain the inclusion of threat actor types
‘no actor’ and ‘cyber researcher’ and felt these threat actor types were out-of-place in a
threat actor typology. To consider ‘no actor’ as a threat actor type was considered
paradoxical and inconsistent.
And finally, respondents responded how certain threat actor types were not visible to them.
For example, a cyber security manager of a large multinational bank acknowledged that the
treat actor type ‘cyber researcher’ was not recognized based in incident reports. A risk
manager at a critical infrastructure organization argued that certain threat actor types were
not considered in the risk analysis because the standard security norm for certain parts in her
system was the base line information security government (BIR). This standard is designed
to protect systems against threat actor types like script kiddies, hackers, etc. However, this
also means that BIR means that parts of the organization are not completely protected
against threat actor types such as highly skilled and resourced criminal groups, state actors
or terrorists. The electricity network company IT security manager confirmed the existence of
a layered defense against certain types of threat actors and argued that because of this
layered defense, CSAN did not provide enough information about whether the electricity
industry would need to (better) protect itself against certain threat actors and/or attack types.
This in turn left the security practitioners in these critical infrastructure industries wondering
when a sector or part of a sector could be considered ‘sufficiently protected’. The electricity
network company IT manager described how the perception of a reduced threat perception
resulting from incidents in less heavily protected parts of the system could be deducted from
the fact that security incidents which involved manipulation of IT and/or information in the
office environment were not immediately escalated to a crisis management level; incidents
that affected the technical systems were. It was argued that to really be (cap)able to inflict
damage in the technical network of the electricity system required fairly specific technical
expertise and knowledge of ‘technical’ software, which is often quite complex and old. This
provided additional barriers that make it difficult for certain threat actor types to actually
disrupt and damage the technical system.
The security manager of a large European bank also criticized the typology for its inability to
distinguish new threat actors such as state-affiliate hacker groups. However, overall, the
bank representative was of the opinion that the CSAN typology was quite complete regarding
the other threat actor types and that the bank used virtually the same cyber threat actor types
as the NCSC in its threat actor typology. However, he did note large differences existed
between the various threat actor types. For example, hacktivists and (cyber)criminal groups
were regularly recognized during incidents whereas other actor types such as nation states,
terrorists, and researchers were not.
33
The critique that the typology did not capture recent trends in the threat actor landscape was
shared by representatives of a cyber security company, a critical infrastructure expert at a
research institute and a representative of the internet industry.
And finally, various respondents criticized the current actor typology and the CSAN report for
being unable to aid practitioners in responding to threats and threat actors that were
identified. The CSAN reports did not enable them to fully assess the dynamics and the
magnitude of the trends. in short it did not provide them with a complete perspective on the
threat landscape. Respondents of two critical infrastructure industries, and SME and internet
industry representatives all felt the typology and CSAN reports did not provide them with the
type of information they need to organize an effective response. All these respondents
complained about the rather generic and high-level information provided by the incidents that
were described and the generic terms in which is written about trends in the threat actor
landscape. The critical infrastructure risk manager and SME representative argued for NCSC
to provide more information on the threat actors. Also unclear was whether the typology and
CSAN could be considered as input for risk assessment. Should the threat actors and
incidents mentioned in the CSAN be considered as initial risk or residual risk in the
organization's risk assessment?
However, apart from criticism, respondents also argued how despite these shortcomings, the
CSAN did provide them with useful information. The critical infrastructure risk manager
explained how the descriptions of incidents in CSAN were used as business impact
assessment tool. Also, it provides insight in trends and indications of shifting capabilities of
threat actors. However, she added, but we see this is changing very fast.
In various stages of completion, the respondents were also confronted with concepts of the
threat actor typology. Respondents recognized the proposed dimensions and could provide
examples of classifications with the help of the framework typology. However certain
responses pointed towards the need for improvements in the typology framework, its
dimensions or the classification. For example, the representative of the internet industry,
when confronted with an early classification on the dimension ‘organization’ reacted that this
dimension was perhaps not up-to-date; the dimension to him did not seem able to capture
the extremely dynamic nature of the internet in terms of organizational capability.
Furthermore the dimension motivation – which at that time was called ‘intention’, he felt,
would pose difficulties as well.
The senior security manager of a large European bank pointed out that in the version of the
threat actor typology framework ‘internal actors’ could not be identified whereas this was an
important source of threats and attacks.
A cyber security expert from an international IT security firm and the senior security manager
from a large European bank felt that the dimensions did not catch the essence of all the
important new possibilities for behavior that the internet presented for threat actors.
Additional aspects or highlights which could enhance the dimensions were the addition of
information. The bank security manager argued that information on the source of origin of the
attack would be an important source of information to classify an attack. In a similar fashion,
the cyber security expert from an international IT security firm argued that target information
such as the impact of the incident, target type/size of the intended target also yields a lot of
information about the threat actor. And so would information about the visibility of the attack
or more detailed information about the type of expertise (e.g. technical expertise, money
laundering expertise, organizational expertise or financial expertise).
34
3.3.3
Cyber security incidents and trends
The respondents were finally questioned about important incidents and trends which they felt
needed to be more accurately reflected in the new threat actor typology.
The electricity network company IT manager identified a trend in the thinking about cyber
security where protection was moving from the ‘fortress idea’ towards that of a hotel with
‘electronic locks’ which shields important parts of the building from unwanted visitors. He
and the critical infrastructure risk manager had already explained how this trend created
new challenges for the interpretation of the CSAN and the use of the threat actor typology.
A representative of a cyber security company mentioned the increasing professionalization of
cyber criminals and the speed in which this took place. This according to him required cyber
security professionals to quickly distinguish between the various forms of cyber crime to
direct resources into fighting the more dangerous and sophisticated threat actors. A security
researcher also observed this trend, identifying an increased expertise and level of
professionalism in the advanced phishing attacks (e.g. more sophisticated plan of attack,
more resources in setting up the attack). As examples of these trends, the experts mentioned
the use of personalized headings in phishing mails and the development of automated selflearning phishing mails.
Another trend that was described by the cyber security expert of international cyber security
company that the motivation of certain cyber criminals was changing. Traditionally it used to
be quite clear what the purpose of cyber criminals was for targets such as banks (i.e. to steal
money). This lead to an increasing ‘sophistication’ of the attacks on banks. But this is no
longer the case for certain cyber criminals are displaying what he considered as ‘lateral
movement’ i.e. new forms of attack and new cyber crime ‘products’ are made based for
example upon from stolen bank data. The criminals are no longer focusing on stealing the
money from the banks themselves. Instead, they create new ‘products’ which can be used in
other kill chains. For example, information of bank clients is sold to other cyber criminals to
improve their phishing attacks in order to gain access to computers of bank clients. Then
new types of attacks can be planned: for example customer credit card fraud can become a
new vulnerability. This may have consequences for the classification of the attack and threat
actor classification and also has implications for the protection of assets of these potential
targets such as banks.
Based on these interviews, continuous improvements were made in the treat actor typology,
its dimensions and the design of classes in the threat actor typology framework.
3.4
Observations and feedback from NCSC/NCTV workshop
Apart from cyber security experts and stakeholders, the threat actor typology framework was
validated via a workshop with 5 NCSC and NCTV analysts and advisors on February 23rd,
2017. The validation was used to obtain feedback on the usability of the typology framework.
Also issues that arose from using and applying the typology framework in attempts to identify
threat actor types.
To achieve this goal the group deliberations were observed and recorded. The workshop
consisted of a 2-hour session in which the NCSC and NCTV staff members were split up in
two groups and initially asked to apply the threat actor typology framework to analyze
incident descriptions which were described in the CSAN 2016 (NCSC, 2016). In a
subsequent round the workshop attendants were asked to quickly identify cyber threat actors
based on the review of headline incidents from the Security.NL-website in the period 15-02-
35
2017 until 23-02-2017). Plenary feedback rounds were held in between to collect and discuss
issues that arose from the use of the framework typology with the NCSC/NCTV staff.
Among the headlines were the following links:4
•
More smart toys made with listening function
•
Privacy regulators conduct research into Windows 10
•
Researchers infect BIOS/UEFI with ransomware
•
Ukraine target of malware that can eavesdrop on conversations
•
Germany bans smart toy because of privacy
•
Shamoon-attack which deleted thousands of pc's started with macro
•
Dozens of universities in the Unites States hacked via SQL-injection.
In total the analysts mapped 11 incidents using the threat actor typology framework.
On the whole, the workshop proved the viability and functionality of the threat actor typology.
The workshop users generally liked the set-up of the typology framework because it raised a
lot of issues about the incidents, the information provided and forced the workshop
attendants to explain their analysis on the threat actor type, which in turn raised questions on
the characteristics of the treat actor type. Use of the framework yielded substantial debates
among participants on threat actor types in the CSAN 2016, especially the cyber researcher
type (see also chapter 4).
Based upon the results of the workshop the threat actor typology framework needs to be
improved, and especially additional information is required to inform users how to use the
framework and search for an answer to the various questions (see Table 6).
Observation about threat actor typology use
Changes made to threat actor typology
Changing perspectives in analyzing incidents and
scenarios
Additional preliminary information (‘a few key
points’) to users of the typology framework provided
calling for the development of consistent
interpretation
Unclarity with regard to what constitutes a kill chain (i.e.
the sequence of events that constitutes an attack)
Additional preliminary information (‘a few key
points’) to users about how to start an analysis of
an attack scenario to identify the kill chain.
Influence of time on the classification of an incident:
Carbanak incident in CSAN 2016
Additional preliminary information (‘a few key
points’) to users about the effect of more
information in hindsight and its effect on
classification efforts
In dimension target unclarity about the class government.
How would an attack on a hospital be classified?
Change made to class from government into public
sector
4
These headlines and the underlying messages can be found via: https://www.security.nl/archive/
36
Unclarity about the meaning of various dimensions (i.e.
expertise) more clear examples of what the researchers
mean with the various dimensions
Additional explanation in the introduction of the
dimension including examples
Insufficient (detailed) information to classify incidents on all
the dimensions
Additional preliminary information (‘a few key
points’) to users about the need to answer all
questions and how to deal with insufficient
information
Use of assumptions to infer information on dimensions on
which no information exists
Additional preliminary information (‘a few key
points’) to users about the need to answer all
questions and how to deal with insufficient
information
Table 6: Issues experienced by workshop participants and remedies
The main conclusion of the workshop was that analysts and advisors felt they were incapable
of identifying a specific threat actor by filling in the cyber actor typology framework. The type
of information available to the staff members based on the CSAN-report as well as the
website yielded insufficient information. That is, based upon the information provided in the
workshop (i.e. the descriptions in the CSAN 2016 and the ‘live’ examples) staff members
found it hard to decide in which classes the incident would fit and thus allowed for multiple
classes. Information on vulnerabilities, trends and incidents as described in the CSAN
provide valuable but insufficient information to pinpoint a threat actor type and determine an
exact categorization. However, in the framework introduction (‘a few key points’) additional
information was provided to users how to deal with this perceived lack of information. The
following information was provided: “The answer categories of the questions cannot be
defined in precise detail, because of the complexity and dynamic nature of the threat
landscape. Some degree of user discretion is necessary. We suggest that different users
analyze the same threat information and then compare the outcomes, building a consistent
interpretation across the user group. This is similar to developing “inter-coder reliability” in
scientific research.”
An important topic, which evoked further discussion was the relative judgment about
incidents over time. For example, the Carbanak incident elicited discussion among analysts
about the level of expertise and resources displayed in the incident. On the one hand, the
expertise could be argued as medium to high since the attackers used sophisticated tools.
On the other hand the workshop participants argued the incident co...
Purchase answer to see full
attachment