roadmap

1. A Roadmap for Cybersecurity Research
November 2009

2. Contents
Executive Summary ................................................................................................................................................iii
Introduction ..............................................................................................................................................................v
Acknowledgements .................................................................................................................................................ix
Current Hard Problems in INFOSEC Research
1. Scalable Trustworthy Systems ...................................................................................................................1
2. Enterprise-Level Metrics (ELMs) ..........................................................................................................13
3. System Evaluation Life Cycle...................................................................................................................22
4. Combatting Insider Threats ....................................................................................................................29
5. Combatting Malware and Botnets ..........................................................................................................38
6. Global-Scale Identity Management ........................................................................................................50
7. Survivability of Time-Critical Systems ..................................................................................................57
8. Situational Understanding and Attack Attribution ..............................................................................65
9. Provenance .................................................................................................................................................76
10. Privacy-Aware Security ..........................................................................................................................83
11. Usable Security ........................................................................................................................................90
Appendices
Appendix A. Interdependencies among Topics ..............................................................................................A1
Appendix B. Technology Transfer .................................................................................................................... B1
Appendix C. List of Participants in the Roadmap Development .................................................................C1
Appendix D. Acronyms ...................................................................................................................................... D1
i

4. Executive Summary
Executive Summary
The United States is at a significant decision point. We must continue to defend our
current systems and networks and at the same time attempt to “get out in front” of
our adversaries and ensure that future generations of technology will position us to
better protect our critical infrastructures and respond to attacks from our adversaries.
The term “system” is used broadly to encompass systems of systems and networks.
This cybersecurity research roadmap is an attempt to begin to define a national R&D
agenda that is required to enable us to get ahead of our adversaries and produce
the technologies that will protect our information systems and networks into the
future. The research, development, test, evaluation, and other life cycle consider-
ations required are far reaching—from technologies that secure individuals and
their information to technologies that will ensure that our critical infrastructures
are much more resilient. The R&D investments recommended in this roadmap
must tackle the vulnerabilities of today and envision those of the future.
The intent of this document is to provide detailed research and development
agendas for the future relating to 11 hard problem areas in cybersecurity, for use
by agencies of the U.S. Government and other potential R&D funding sources.
The 11 hard problems are:
1. Scalable trustworthy systems (including system architectures and requisite
development methodology)
2. Enterprise-level metrics (including measures of overall system trustworthiness)
3. System evaluation life cycle (including approaches for sufficient assurance)
4. Combatting insider threats
5. Combatting malware and botnets
6. Global-scale identity management
7. Survivability of time-critical systems
8. Situational understanding and attack attribution
9. Provenance (relating to information, systems, and hardware)
10. Privacy-aware security
11. Usable security
For each of these hard problems, the roadmap identifies critical needs, gaps in
research, and research agenda appropriate for near, medium, and long term
attention.
DHS S&T assembled a large team of subject matter experts who provided input
into the development of this research roadmap. The content was developed over
the course of 15 months that included three regional multi-day workshops, two
virtual workshops for each topic, and numerous editing activities by the participants.
iii

6. Introduction
Introduction
Information technology has become pervasive in every way—from our phones and
other small devices to our enterprise networks to the infrastructure that runs our
economy. Improvements to the security of this information technology are essential
for our future. As the critical infrastructures of the United States have become more
and more dependent on public and private networks, the potential for widespread
national impact resulting from disruption or failure of these networks has also
increased. Securing the nation’s critical infrastructures requires protecting not only
their physical systems but, just as important, the cyber portions of the systems on
which they rely. The most significant cyber threats to the nation are fundamentally
different from those posed by the “script kiddies” or virus writers who tradition-
ally have plagued users of the Internet. Today, the Internet has a significant role
in enabling the communications, monitoring, operations, and business systems
underlying many of the nation’s critical infrastructures. Cyberattacks are increas-
ing in frequency and impact. Adversaries seeking to disrupt the nation’s critical
infrastructures are driven by different motives and view cyberspace as a possible
means to have much greater impact, such as causing harm to people or widespread
economic damage. Although to date no cyberattack has had a significant impact on
our nation’s critical infrastructures, previous attacks have demonstrated that exten-
sive vulnerabilities exist in information systems and networks, with the potential for
serious damage. The effects of a successful attack might include serious economic
consequences through impacts on major economic and industrial sectors, threats
to infrastructure elements such as electric power, and disruptions that impede the
response and communication capabilities of first responders in crisis situations.
The United States is at a significant decision point. We must continue to defend our
current systems and networks and at the same time attempt to “get out in front”
of our adversaries and ensure that future generations of technology will position
us to better protect our critical infrastructures and respond to attacks from our
adversaries. It is the opinion of those involved in creating this research roadmap that
government-funded research and development (R&D) must play an increasing role
to enable us to accomplish this goal of national and economic security. The research
topics in this roadmap, however, are relevant not only to the federal government
but also to the private sector and others who are interested in securing the future.
This cybersecurity research roadmap is an attempt to begin to define a national R&D
agenda that is required to enable us to get ahead of our adversaries and produce
the technologies that will protect our information systems and networks into the
future. The research, development, test, evaluation, and other life cycle consider-
ations required are far reaching—from technologies that secure individuals and
their information to technologies that will ensure that our critical infrastructures
“The time is now near at hand...” are much more resilient. These investments must tackle the vulnerabilities of today
— George Washington, July 2, 1776 and envision those of the future.
v

7. Historical background research programs. The original list has mixes of legacy systems), and the pres-
proven useful in guiding INFOSEC ence of significant, asymmetric threats.
The INFOSEC Research Council (IRC) research, and policy makers and planners
is an informal organization of govern- may find the document useful in evalu- The area of cybersecurity and the associ-
ment program managers who sponsor ating the contributions of ongoing and ated research and development activities
information security research within the proposed INFOSEC research programs. have been written about frequently over
U.S. Government. Many organizations However, the significant evolution of the past decade. In addition to both
have representatives as regular members technology and threats between 1999 the original IRC HPL in 1999 and the
of the IRC: Central Intelligence Agency, and 2005 required an update to the list. revision in 2005, the following reports
Department of Defense (including the Therefore, an updated version of the have discussed the need for investment
Air Force, Army, Defense Advanced HPL was published in November 2005. in this critical area:
Research Projects Agency, National This updated document included the
ƒ Toward a Safer and More Secure
Reconnaissance Office, National Secu- following technical hard problems from
rity Agency, Navy, and Office of the the information security perspective: Cyberspace
Secretary of Defense), Department ƒ Federal Plan for Cyber Security
1. Global-Scale Identity Management and Information Assurance
of Energy, Department of Homeland
Security, Federal Aviation Administra- 2. Insider Threat Research and Development
tion, Intelligence Advanced Research 3. Availability of Time-Critical ƒ Cyber Security: A Crisis of
Projects Activity, National Aeronautics Systems
Prioritization
and Space Administration, National 4. Building Scalable Secure Systems
ƒ Hardening the Internet
Institutes of Health, National Institute 5. Situational Understanding and
of Standards and Technology, National Attack Attribution ƒ Information Security
Science Foundation, and the Technical 6. Information Provenance Governance: A Call to Action
Support Working Group. In addition, ƒ The National Strategy to Secure
7. Security with Privacy
the IRC is regularly attended by partner Cyberspace
organizations from Canada and the 8. Enterprise-Level Security Metrics
ƒ Cyber Security Research and
United Kingdom.
Development Agenda
These eight problems were selected
The IRC developed the original Hard as the hardest and most critical chal-
Problem List (HPL), which was com- lenges that must be addressed by the These reports can be found at http://
posed in 1997 and published in draft INFOSEC research community if trust- www.cyber.st.dhs.gov/documents.html
form in 1999. The HPL defines desir- worthy systems envisioned by the U.S.
able research topics by identifying a set Government are to be built. INFOSEC Current context
of key problems from the U.S. Govern- problems may be characterized as “hard”
ment perspective and in the context of for several reasons. Some problems are On January 8, 2008, the President
IRC member missions. Solutions to hard because of the fundamental techni- issued National Security Presiden-
these problems would remove major cal challenges of building secure systems, tial Directive 54/Homeland Security
barriers to effective information secu- others because of the complexity of Presidential Directive 23, which for-
rity (INFOSEC). The Hard Problem information technology (IT) system malized the Comprehensive National
List was intended to help guide the applications. Contributing to these Cybersecurity Initiative (CNCI) and a
research program planning of the IRC problems are conflicting regulatory and series of continuous efforts designed to
member organizations. It was also hoped policy goals, poor understanding of establish a frontline defense (reducing
that nonmember organizations and operational needs and user interfaces, current vulnerabilities and preventing
industrial partners would consider these rapid changes in technology, large het- intrusions), defending against the full
problems in the development of their erogeneous environments (including spectrum of threats by using intelligence
vi

8. and strengthening supply chain security, influence in networking and IT systems, interagency coordination to ensure cov-
and shaping the future environment by components, and standards among U.S. erage of all the topics.
enhancing our research, development, competitors. Federal agencies with
and education, as well as investing in mission-critical needs for increased Each of the following topic areas is
“leap-ahead” technologies. cybersecurity, which includes informa- treated in detail in a subsequent section
tion assurance as well as network and of its own, from Section 1 to Section 11.
The vision of the CNCI research com- system security, can play a direct role
1. Scalable trustworthy systems
munity over the next 10 years is to in determining research priorities and
(including system architectures and
“transform the cyber-infrastructure so assessing emerging technology proto-
requisite development methodol-
that critical national interests are pro- types. Moreover, through technology
ogy)
tected from catastrophic damage and transfer efforts, the federal government
2. Enterprise-level metrics (including
our society can confidently adopt new can encourage rapid adoption of the
measures of overall system trust-
technological advances.” results of leap-ahead research. Technol-
worthiness)
ogy breakthroughs that can curb or
Two components of the CNCI deal break the resource-draining cycle of 3. System evaluation life cycle (in-
cluding approaches for sufficient
with cybersecurity research and develop- security patching will have a high likeli-
assurance)
ment—one focused on the coordination hood of marketplace implementation.
of federal R&D and the other on the 4. Combatting insider threats
development of leap-ahead technologies. As stated previously, this Cybersecu- 5. Combatting malware and botnets
rity Research Roadmap is an attempt 6. Global-scale identity management
No single federal agency “owns” the to begin to address a national R&D 7. Survivability of time-critical
issue of cybersecurity. In fact, the agenda that is required to enable us to systems
federal government does not uniquely get ahead of our adversaries and produce 8. Situational understanding and
own cybersecurity. It is a national and the technologies that will protect our attack attribution
global challenge with far-reaching con- information systems and networks into 9. Provenance (relating to informa-
sequences that requires a cooperative, the future. The topics contained in this tion, systems, and hardware)
comprehensive effort across the public roadmap and the research and develop-
10. Privacy-aware security
and private sectors. However, as it has ment that would be accomplished if the
11. Usable security
done historically, U.S. Government roadmap were implemented are, in fact,
R&D in key technologies working in leap-ahead in nature and address many
close cooperation with private-sector of the topics that have been identified Eight of these topics (1, 2, 4, 6, 7, 8,
partners can jump-start the necessary in the CNCI activities 9, 10) are adopted from the November
fundamental technical transformation. 2005 IRC Hard Problem List [IRC05]
Document format and are still of vital relevance. The
The leap-ahead strategy aligns with the other three topics (3, 5, 11) represent
consensus of the nation’s networking The intent of this document is to additional areas considered to be of
and cybersecurity research communi- provide detailed research and develop- particular importance for the future.
ties that the only long-term solution to ment agendas for the future relating to
the vulnerabilities of today’s network- 11 hard problem areas in cybersecurity, The order in which the 11 topics are
ing and information technologies is to for use by agencies of the U.S. Govern- presented reflects some structural simi-
ensure that future generations of these ment and anyone else that is funding larities among subgroups of the topics
technologies are designed with secu- or doing R&D. It is expected that each and exhibits clearly some of their major
rity built in from the ground up. The agency will find certain parts of the interdependencies. The order proceeds
leap-ahead strategy will help extend document resonant with its own needs roughly from overarching system con-
U.S. leadership at a time of growing and will proceed accordingly with some cepts to more detailed issues—except
vii

9. for the last topic—and has the following Background ƒ What R&D is evolutionary and
structure: what is more basic, higher risk,
ƒ What is the problem being game changing?
a. Topics 1–3 frame the overarching addressed?
ƒ Resources
problems. ƒ What are the potential threats?
ƒ Measures of success
b. Topics 4–5 relate to specific major ƒ Who are the potential
beneficiaries? What are their ƒ What needs to be in place for test
threats and needs.
respective needs? and evaluation?
c. Topics 6–10 relate to some of the
ƒ What is the current state of the ƒ To what extent can we test real
“ilities” and to system concepts
practice? systems?
required for implementing the
previous topics. ƒ What is the status of current Following the 11 sections are three
research? appendices:
Topic 11, usable security, is different
from the others in its cross-cutting Future Directions Appendix A: Interdependencies among
nature. If taken seriously enough, it Topics
ƒ On what categories can we
can influence the success of almost all
the other topics. However, some sort subdivide the topics? Appendix B: Technology Transfer
of transcendent usability requirements ƒ What are the major research
need to be embedded pervasively in all gaps? Appendix C: List of Participants in the
the other topics. ƒ What are some exemplary Roadmap Development
problems for R&D on this topic?
Each of the 11 sections follows a
ƒ What are the challenges that
similar format. To get a full picture of
must be addressed?
the problem, where we are, and where
we need to go, we ask the following ƒ What approaches might be
questions: desirable?
References
[IRC2005] INFOSEC Research Council Hard Problem List, November 2005
http://www.cyber.st.dhs.gov/docs/IRC_Hard_Problem_List.pdf.
[USAF-SAB07] United States Air Force Scientific Advisory Board, Report on Implications of Cyber Warfare. Volume 1:
Executive Summary and Annotated Brief; Volume 2: Final Report, August 2007. For Official Use Only.
Additional background documents (including the two most recent National Research Council study reports on cybersecurity)
can be found online. (http://www.cyber.st.dhs.gov/documents.html).
viii

10. Acknowledgements
Acknowledgements
The content of this research roadmap was developed over the course of 15 months
that included three workshops, two phone sessions for each topic, and numer-
ous editing activities by the participants. Appendix C lists all the participants.
The Cyber Security program of the Department of Homeland Security (DHS)
Science and Technology (S&T) Directorate would like to express its appre-
ciation for the considerable amount of time they dedicated to this effort.
DHS S&T would also like to acknowledge the support provided by the staff of SRI
International in Menlo Park, CA, and Washington, DC. SRI is under contract with
DHS S&T to provide technical, management, and subject matter expert support for
the DHS S&T Cyber Security program. Those involved in this effort include Gary
Bridges, Steve Dawson, Drew Dean, Jeremy Epstein, Pat Lincoln, Ulf Lindqvist,
Jenny McNeill, Peter Neumann, Robin Roy, Zach Tudor, and Alfonso Valdes.
Of particular note is the work of Jenny McNeill and Peter Neumann. Jenny
has been responsible for the organization of each of the workshops and phone
sessions and has worked with SRI staff members Klaus Krause, Roxanne Jones,
and Ascencion Villanueva to produce the final document. Peter Neumann
has been relentless in his efforts to ensure that this research roadmap rep-
resents the real needs of the community and has worked with roadmap
participants and government sponsors to produce a high-quality product.
ix

12. Current Hard Problems in INFOSEC Research
1. Scalable Trustworthy Systems
BACKGROUND
What is the problem being addressed?
Trustworthiness is a multidimensional measure of the extent to which a system is
likely to satisfy each of multiple aspects of each stated requirement for some desired
combination of system integrity, system availability and survivability, data confi-
dentiality, guaranteed real-time performance, accountability, attribution, usability,
and other critical needs. Precise definitions of what trustworthiness means for these
requirements and well-defined measures against which trustworthiness can be evalu-
ated are fundamental precursors to developing and operating trustworthy systems.
These precursors cut across everything related to scalable trustworthy systems. If
what must be depended on does not perform according to its expectations, then
whatever must depend on it may itself not be trustworthy. A trusted system is
one that must be assumed to satisfy its requirements—whether or not it is actu-
ally trustworthy; indeed, it is a system whose failure in any way may compromise
those requirements. Unfortunately, today’s systems are typically not well suited for
applications with critical trustworthiness requirements.
Scalability is the ability to satisfy given requirements as systems, networks, and
systems of systems expand in functionality, capacity, complexity, and scope of trust-
worthiness requirements security, reliability, survivability, and improved real-time
performance. Scalability must typically be addressed from the outset; experience
shows that scalability usually cannot be retrofitted into systems for which it was
not an original design goal. Scalable trustworthiness will be essential for many
national- and world-scale systems, including those supporting critical infrastructures.
Current methodologies for creating high-assurance systems do not scale to the size
of today’s—let alone tomorrow’s—critical systems.
Composability is the ability to create systems and applications with predictably
satisfactory behavior from components, subsystems, and other systems. To enhance
scalability in complex distributed applications that must be trustworthy, high-
assurance systems should be developed from a set of composable components and
subsystems, each of which is itself suitably trustworthy, within a system architecture
that inherently supports facile composability. Composition includes the ability to
run software compatibly on different hardware, aided considerably by abstraction,
operating systems, and suitable programming languages. However, we do not yet
have a suitable set of trustworthy building blocks, composition methodologies,
and analytic tools that would ensure that trustworthy systems could be developed
as systems of other systems. In addition, requirements and evaluations should
also compose accordingly. In the future, it will be vital that new systems can be
incrementally added to a system of systems with some predictable confidence that
the trustworthiness of the resulting systems of systems will not be weakened—or
indeed that it may be strengthened.
1

13. Growing interconnectedness among follows: (1) trustworthiness, (2) com- computing base that would provide a
existing systems results, in effect, in posability, and (3) scalability. Thus, the suitable foundation for such computing.
new composite systems at increasingly challenge addressed here is threefold: However, this assumption has not been
large scales. Existing hardware, operat- (a) to provide a sound basis for compos- justified. In the future, we must be able
ing system, networking, and application ability that can scale to the development to develop scalable trustworthy systems
architectures do not adequately account of large and complex trustworthy effectively.
for combined requirements for security, systems; (b) to stimulate the develop-
performance, and usability—confound- ment of the components, analysis tools, Who are the potential
ing attempts to build trustworthy and testbeds required for that effort;
beneficiaries? What are their
systems on them. As a result, today the and (c) to ensure that trustworthiness
respective needs?
security of a system of systems may be evaluations themselves can be composed.
drastically less than that of most of its Large organizations in all sectors—for
components. What are the potential example, government, military, com-
mercial, financial, and energy—suffer
threats?
In certain cases, it may be possible the consequences of using large-scale
to build systems that are more trust- Threats to a system in operation include computing systems whose trustworthi-
worthy than some (or even most) everything that can prevent critical appli- ness either is not assured or is potentially
of their components—for example, cations from satisfying their intended compromised because of costs that
through constructive system design and requirements, including insider and out- outweigh the perceived benefits. All
meticulous attention to good software sider misuse, malware and other system stakeholders have requirements for
engineering practices. Techniques for subversions, software flaws, hardware confidentiality, integrity, and availabil-
building more trustworthy systems out malfunctions, human failures, physical ity in their computing infrastructures,
of less trustworthy components have damage, and environmental disruptions. although the relative importance of
long been known and used in practice Indeed, systems sometimes fail without these requirements varies by application.
(e.g., summarized in [Neu2004], in the any external provocation, as a result Achieving scalability and evolvability of
context of composability). For example, of design flaws, implementation bugs, systems without compromising trust-
error-correcting codes can overcome misconfiguration, and system aging. worthiness is a major need. Typical
unreliable communications and storage Additional threats arise in the system customers include the following:
media, and encryption can be used to acquisition and code distribution pro-
ƒƒ Large-system developers (e.g., of
increase confidentiality and integrity cesses. Serious security problems have
despite insecure communication chan- also resulted from discarded or stolen operating systems, database
nels. These techniques are incomplete by systems. For large-scale systems consist- management systems, national
themselves and generally ignore many ing of many independent installations infrastructures such as the power
security threats. They typically depend (such as the Domain Name System, grid)
on the existence of some combination DNS), security updates must reach and ƒƒ Application developers
of trustworthy developers, trustwor- be installed in all relevant components ƒƒ Microelectronics developers
thy systems, trustworthy users, and throughout the entire life cycle of the
ƒƒ System integrators
trustworthy administrators, and their systems. This scope of updating has proven
trustworthy embedding in those systems. to be difficult to achieve. ƒƒ Large- and small-scale users
ƒƒ Purveyors of potential exemplar
The primary focus of this topic area is Critical systems and their operating envi- applications for scalable
scalability that preserves and enhances ronments must be trustworthy despite a trustworthiness
trustworthiness in real systems. The per- very wide range of adversities and adver-
ceived order of importance for research saries. Historically, many system uses Several types of systems suggest the
and development in this topic area is as assumed the existence of a trustworthy importance of being able to develop
2 SCALABLE TRUSTWORTHY SYSTEMS

14. scalable trustworthy systems. Examples What is the current state of insufficient in the long run. Research
include the following: the practice? is needed to establish the repertoire of
architected hardware protections that
ƒƒ Air traffic control systems Hardware developers have recently made are essential for system trustworthiness.
ƒƒ Power grids significant investments in specification, It is unlikely that software alone can ever
ƒƒ Worldwide funds transfer systems formal methods, configuration control, compensate fully for the lack of such
modeling, and prediction, partly in hardware protections.
ƒƒ Cellphone networks
response to recognized problems, such
Such systems need to be robust and as the Intel floating point flaw, and A possible implication is that the com-
capable of satisfying the perceived trust- partly as a result of increased demon- mercial off-the-shelf (COTS) systems in
worthiness requirements. Outages in strations of the effectiveness of those pervasive use today will never become
these systems can be extremely costly techniques. sufficiently trustworthy. If that is indeed
and dangerous. However, the extent to true, testing that implication should be
which the underlying concepts used to The foundation for trustworthy scalable identified as an activity and milestone
build these existing systems can continue systems is established by the underly- in the recommended research agenda.
to scale and also be responsive to more ing hardware architecture. Adequate
exacting trustworthiness requirements hardware protections are essential, and Convincing hardware manufacturers
is unknown—especially in the face of nearly all extant hardware architectures and software developers to provide and
increasing cyberthreats. The R&D must lack needed capabilities. Examples support needed hardware capabilities,
provide convincing arguments that they include fine-grain memory protec- of course, is a fundamental obstacle.
will scale appropriately. Exemplars of tion, inaccessible program control state, The manufacturers’ main motivations
potential component systems might unmodifiable executable codes, fully are least change and time to market.
include the following: granular access protections, and virtu- Until compelling research findings, legal
ally mapped bus access by I/O and consequences (e.g., financial liability
ƒƒ Trustworthy handheld other adapter boards. for customer damages), and economic
multipurpose devices and other forces (e.g., purchase policies mandat-
end-user devices Although it might be appealing to try ing the needed capabilities) are brought
ƒƒ Trustworthy special-purpose to apply those approaches to software, to bear, it seems unlikely that goals for
servers the issues of scalability suggest that the securing COTS and open source
ƒƒ Embedded control systems additional approaches may be necessary. products can be realized.
that can be composed and used Numerous software-related failures
effectively have occurred (e.g., see [Neu1995]). What is the status of current
In addition, techniques are needed to
ƒƒ Trustworthy networks research?
address how software/hardware inter-
ƒƒ Navigation systems, such as actions affect the overall trust level. Over the past decade, significant com-
the Global Positioning Systems Unfortunately, there is no existing puter security investments have been
(GPS) mandate for significant investment made in attempts to create greater
during software system development to assurance for existing applications
One or more such systems should be ensure scalable trustworthiness. Conse- and computer-based enterprises that
chosen for deeper study to develop quently, such efforts are generally not are based predominantly on COTS
better understanding of the approaches adequately addressed. components. Despite some progress,
to scalable security developed in this there are severe limits to this approach,
program. In turn, the results of ongoing Diagnostic tools to detect software and success has been meager at best,
work on scalable trustworthiness should flaws on today’s hardware architectures particularly with respect to trustwor-
be applied to those and other exemplars. may be useful in the short run but are thiness, composability, and scalability.
SCALABLE TRUSTWORTHY SYSTEMS 3

15. The assurance attainable by incremental example of how trustworthy com- In recent years, research has advanced
improvements on COTS products is puting systems can be designed and significantly in formal methods appli-
fundamentally inadequate for critical built. It will make all elements of the cable to software trustworthiness. That
applications. constructive security process openly research is generally more applicable to
available. Recent advances in cryptog- new systems rather than to being retro-
Various research projects over the past raphy can also help, although some fitted into existing systems. However, it
half-century have been aimed at the composability issues remain to be needs to focus on attributes and subsys-
challenge of designing and evaluating resolved as to how to embed those tems for which it can be most effective,
scalable trustworthy systems and net- advances securely into marginally and must deal with complexity, scal-
works, with some important research secure computer systems. Also, public ability, hardware and software, and
contributions with respect to both key infrastructures (PKIs) are becom- practical issues such as device drivers
hardware and software. Some of these ing more widely used and embedded and excessive root privileges.
date back to the 1960s and 1970s, such in applications. However, many gaps
as Multics, PSOS (the Provably Secure remain in reusable requirements for
Operating System) and its formally trustworthiness, system architectures, FUTURE DIRECTIONS
based Hierarchical Development Meth- software engineering practices, sound
odology (HDM), the Blacker system programming languages that avoid On what categories can we
as an early example of a virtual private many of the characteristic flaws, and
subdivide this topic?
network, the CLInc (Computational analysis tools that scale up to entire
Logic, Inc.) stack, Gypsy, InaJo, Euclid, systems. Thoroughly worked examples For present purposes, different
ML and other functional programming of trustworthy systems are needed that approaches to development of trustwor-
languages, and the verifying compiler, can clearly demonstrate that well-con- thy scalable systems are associated with
to name just a few. However, very few ceived composability can enhance both the following three roadmap categories.
systems available today have taken trustworthiness and scalability. For These categories are distinguished from
serious advantage of such potentially example, each of the exemplars noted one another roughly based on the extent
far-reaching research efforts, or even above would benefit greatly from the to which they are able to reuse existing
the rather minimal guidance of Security incorporation of scalable trustworthy components.
Level 4 in FIPS 140-1. Also, the valued systems.
but inadequately observed 1975 secu- 1. Improving trustworthiness in
rity principles of Saltzer and Schroeder At present, even for small systems, there existing systems. This incremental
have recently been updated by Saltzer exist very few examples of requirements, approach could entail augmenting rela-
and Kaashoek [Sal+2009]. trustworthiness metrics, and opera- tively untrustworthy systems with some
tional systems that encompass a broad trustworthy components and enforcing
Some more recent efforts can also be spectrum of trustworthiness with any operational constraints in attempts to
cited here. For example, architectures generality. Furthermore, such require- achieve either trustworthy functions or
exist or are contemplated for robust ments, metrics, and systems need to systems with more clearly understood
hardware that would inherently be composable and scalable into trust- trust properties. Can we make existing
increase system trustworthiness worthy systems of systems. However, systems significantly more trustworthy
by avoiding common vulnerabilities, a few examples exist for dedicated without wholesale replacement?
including modernized capability- special-purpose systems, such as data
based architectures. In addition, the diodes enforcing one-way communi- 2. Clean-slate approaches. This entails
Trusted Computing Exemplar Project cation paths and the Naval Research building trustworthy primitives, com-
at the Naval Postgraduate School Laboratory Pump enabling trustworthy posing them into trustworthy functions,
(http://cisr.nps.edu/projects/tcx.html) reading of information at lower levels and then verifying the overall trust level
is intended to provide a working of multilevel security. of the composite system. How much
4 SCALABLE TRUSTWORTHY SYSTEMS

16. better would this be? Would this enable controlled. A clean-slate approach tol- practices that can yield greater trust-
solutions of problems that cannot be erating an ongoing level of continuous worthiness. See also [Can2001], which
adequately addressed today, and for compromise in its system components represents the beginning of work on
what requirements? Under what circum- might also be viewed as a hybrid of the notion of universal composability
stances and for what requirements might categories 2 and 3. Further R&D is applied to cryptography.
this be possible? What new technologies, clearly required to determine the trade-
system architectures, and tools might offs in cost-effectiveness, practicality, However, there are gaps in our under-
be needed? performance, usability, and relative standing of composability as it relates
trustworthiness attainable for any par- to security, and to trustworthiness more
3. Operating successfully for given ticular set of requirements. DARPA’s generally, primarily because we lack
requirements despite the presence IAMANET is a step in that direction. precise specifications of most of the
of partially untrusted environments. important requirements and desired
For example, existing computing An urgent need exists for R&D on properties. For example, we are often
systems might be viewed as “enemy incremental, clean-slate, and hybrids good at developing specific solutions
territory” because they have been approaches. Trustworthiness issues may to specific security problems, but we
subject to unknown influences within affect the development process and the do not understand how to apply and
the commercial supply chain and the resulting system performance. Adding combine these specific solutions to
overall life cycle (design, implementa-functionality and concomitant com- produce trustworthy systems. We lack
tion, operations, maintenance, and plexity to achieve trustworthiness may methods for analyzing how even small
decommissioning). be counterproductive, if not done con- changes to systems affect their trust-
structively; it typically merely introduces worthiness. More broadly, we lack a
It is inherently impossible to control new vulnerabilities. Trustworthiness good understanding of how to develop
every aspect of the entire life cycle must be designed in from the outset and maintain trustworthy systems com-
and the surrounding operational envi- with complete specified requirements. prehensively throughout the entire life
ronments. For example, end-to-end Functionality and trustworthiness are cycle. We lack methods and tools for
cryptography enables communications inherently in conflict in the design decomposing high-level trustworthiness
over untrustworthy media—but does process, and this conflict must be goals into specific design requirements,
not address denial-of-service attacks resolved before any implementation. capturing and specifying security require-
en route or insider subversion at the ments, analyzing security requirements,
endpoints. What are the major research mapping higher-layer requirements into
lower-layer ones, and verifying system
gaps?
The three categories are not intended trustworthiness properties. We do not
to be mutually exclusive. For example, Research relating to composability has understand how to combine systems in
hybrid approaches can combine legacy addressed some of the fundamental ways that ensure that the combination is
systems from category 1 with incremen- problems and underlying theory. For more, rather than less, secure and resil-
tal changes and significant advances example, see [Neu2004] for a recent ient than its weakest components. We
from category 2. Indeed, hybrids among consideration of past work, current prac- lack a detailed case history of past suc-
these three categories are not merely tice, and R&D directions that might be cesses and failures in the development
possible but quite likely. For example, useful in the future. It contains numer- of trustworthy systems that could help
approaches that begin with a clean-slate ous references to papers and reports us to elucidate principles and properties
architecture could also incorporate some on composability. It also considers a of trustworthy systems, both in an over-
improvements of existing systems, and variety of techniques for compositions arching sense and in specific application
even allow some operations to take place of subsystems that can increase trustwor- areas. We lack development tools and
in untrusted environments—if suitably thiness, as well as system and network languages that could enable separation
encapsulated, confined, or otherwise architectures and system development of functionality and trustworthiness
SCALABLE TRUSTWORTHY SYSTEMS 5

17. concerns for developers. For small for composing trustworthy ƒƒ More extensive detailed worked
systems, ad hoc solutions seldom suffice systems examples.
if they do not reflect such fundamental
ƒƒ Well-defined composable
understanding of the problems. For the Several threads could run through this
specifications for requirements
large-scale, highly complex systems of timeline—for example, R&D relating
and components
the future, we cannot expect to achieve to trustworthy isolation, separation,
adequate trustworthiness without ƒƒ Realistic provable security and virtualization in hardware and
deeper understanding, better tools, and properties for small-scale systems software; composability of designs and
more reliable evaluation methods—as ƒƒ Urgent need for detailed worked implementations; analyses that could
well as composable building blocks and examples greatly simplify evaluation of trustwor-
well-documented, worked examples of thiness before putting applications into
less complex systems. ƒƒ Better understanding of the operation; robust architectures that
security properties of existing provide self-testing, self-diagnosing,
The research directions can be parti- major components. self-reconfiguring, compromise resil-
tioned into near-term, medium-term, ient, and automated remediation; and
Medium term
and long-term opportunities. In general, architectures that break the current
ƒƒ New hardware with well-
the near-term approaches fall into the asymmetric advantage for attackers
understood trustworthiness
incremental category, and the long- (offense is cheaper than defense, at
properties
term approaches fall into clean-slate present). The emphasis needs to be
and hybrid categories. However, the ƒƒ Better operating systems and on realistic, practical approaches to
long-term approaches often have staged networking developing systems that are scalable,
efforts that begin with near-term efforts. ƒƒ Better application architectures
composable, and trustworthy.
Also, the hybrid efforts tend to require
for trustworthy systems
longer-term schedules because some of The gaps in practice and R&D,
them rely on near- and medium-term ƒƒ Isolation of legacy systems approaches, and potential benefits are
efforts. in trustworthy virtualization summarized in Table 1.1. The research
environments directions for scalable trustworthy
Near term ƒƒ Continued research in systems are intended to address these
ƒƒ Development of prototype composability, techniques for gaps. Table 1.2 also provides a summary
trustworthy systems in selected verifying the security properties of this section.
application and infrastructure of composed systems in terms of
This topic area interacts strongly with
domains their specifications
enterprise-level metrics (Section 2) and
ƒƒ Exploitation of cloud ƒƒ Urgent need for detailed realistic evaluation methodology (Section 3) to
architectures and web-based and practical worked examples. provide assurance of trustworthiness.
applications Long term In the absence of such metrics and suit-
ƒƒ Development of simulation ƒƒ Tools for verifying able evaluation methodologies, security
environments for testing would be difficult to comprehend, and
trustworthiness of composite
approaches to development of the cost-benefit trade-offs would be
systems
scalable trustworthy systems difficult to evaluate. In addition, all the
ƒƒ Techniques and tools for other topic areas can benefit from scal-
ƒƒ Intensive further research in developing and maintaining able trustworthy systems, as discussed
composability trustworthy systems throughout in Appendix A.
ƒƒ Development of building blocks the life cycle
6 SCALABLE TRUSTWORTHY SYSTEMS

18. TABLE 1.1: Summary of Gaps, Approaches, and Benefits
Concept GapsƒinƒPractice GapsƒinƒR&D Approaches PotentialƒBenefits
Requirements Nonexistent, inconsistent, Orange Book/Common Canonical, composable, Relevant developments;
incomplete nonscalable Criteria have inherent scalable trustworthiness Simplified procurement
requirements limitations requirements process
System Inflexibility; Constraints of Evolvable architectures, Scalably composable Long-term scalable
architectures flawed legacy systems scalable theory of components and evolvability maintaining
composability are needed trustworthy architectures trustworthy operation
Development Unprincipled systems, Principles not Built-in assured Fewer flaws and risks;
methodologies unsafe languages, sloppy experientially scalably composable Simplified evaluations
and software programming practices demonstrated; Good trustworthiness
engineering programming language
theory widely ignored
Analytic tools Ad-hoc, piecemeal tools with Tools need sounder bases Rigorously based Eliminating many flaws
limited usefulness composable tools
Whole-system Impossible today for large Top-to-bottom, end-to- Formal methods, Scalable incremental
evaluations systems end analyses needed hierarchical staged evaluations
reasoning
Operational Enormous burdens on User and administrator Dynamic self-diagnosis Simplified, scalable
practices administrators usability are often ignored and self-healing operational management
What are the challenges that of high assurance information technol- could compromise the trustworthi-
must be addressed? ogy. Time-consuming evaluations of ness of the entire system. Designing
trustworthy systems today create long complex secure systems from the ground
The absence of sound systemwide delays when compared with conven- up is an exceptionally hard problem,
architectures designed for trustworthi- tional system developments with weaker particularly since large systems may
ness and the relatively large costs of evaluations. Consequently, development have catastrophic flaws in their design
full verification and validation (V&V) of trustworthy systems can be expected and implementation that are not dis-
have kept any secure computing base to take longer than is typically planned covered until late in development, or
from economically providing the req- for COTS systems. In addition, the even after deployment. Catastrophic
uisite assurance and functionality. (The performance of trustworthy systems software flaws may occur even in just
sole exception is provided by “high- typically lags the performance of COTS a few lines of mission-critical code,
consequence” government applications, systems with comparable functions. and are almost inevitable in the tens
in which cost is a secondary concern of millions of lines of code in today’s
to national security.) This situation is One of the most pressing challenges systems. Given the relatively minuscule
exacerbated by the scale and complexity involves designing system architectures size of programs and systems that have
often needed to provide required func- that minimize how much of the system been extensively verified and the huge
tionality. In addition, the length of must be trustworthy—i.e., minimiz- size of modern systems and applica-
the evaluation process can exceed the ing the size and extent of the trusted tions, scaling up formal approaches to
time available for patches and system computing base (TCB). In contrast, for production and verification of bug-free
upgrades and retarded the incorporation a poorly designed system, any failure systems seems like a Herculean task. Yet,
SCALABLE TRUSTWORTHY SYSTEMS 7

19. TABLE 1.2: Scalable Trustworthy Systems Overview
Vision:ƒMake the development of trustworthy systems of systems (TSoS) practical; ensure that even very large and complex systems
can be built with predictable scalability and demonstrable trustworthiness, using well-understood composable architectures and well-
designed, soundly developed, assuredly trustworthy components.
Challenges: Most of today’s systems are built out of untrustworthy legacy systems using inadequate architectures, development
practices, and tools. We lack appropriate theory, metrics of trustworthiness and scalability, sound composable architectures, synthesis and
analysis tools, and trustworthy building blocks.
Goals: Sound foundations and supporting tools that can relate mechanisms to policies, attacks to mechanisms, and systems to
requirements, enabling facile development of composable TSoS systematically enhancing trustworthiness (i.e., making them more
trustworthy than their weakest components); documented TSoS developments, from specifications to prototypes to deployed systems.
MILESTONES
IncrementalƒSystems Clean-SlateƒSystems HybridƒSystems
Near-termƒmilestones: Near-termƒmilestones: Near-termƒmilestones:
Sound analytic tools Alternative architectures Mix-and-match systems
Secure bootloading Well-specified requirements Integration tools
Trusted platforms Sound kernels/VMMs Evaluation strategies
Medium-termƒmilestones: Medium-termƒmilestones: Medium-termƒmilestones:
Systematic use of tools Provably sound prototypes Use in infrastructures
More tool development Proven architectures Integration experiments
Long-termƒmilestones: Long-termƒmilestones: Long-termƒmilestones:
Extensively evaluated systems Top-to-bottom formal evaluations Seamless integration of COTS/open-source
components
Test/evaluation: Identify measures of trustworthiness, composability, and scalability, and apply them to real systems.
Techƒtransfer: Publish composition methodologies for developing TSoS with mix-and-match components. Release open-source tools
for creating, configuring, and maintaining TSoS. Release open-source composable, trustworthy components. Publish successful, well-
documented TSoS developments. Develop profitable business models for public-private TSoS development partnerships for critical
applications, and pursue them in selected areas.
formally inspired approaches may be components is almost certainly an even of the executable code has not been
more promising than any of the less harder problem. compromised and (b) that the code
formal approaches attempted to date. resides in memory in a manner that it
In addition, considerable progress is As one example, securing the bootload can be neither read nor altered, but only
being made in analyzing system behav- process would be very valuable, but the executed. Firmware residing in ROM,
ior across multiple layers of abstraction. underlying general principle is that every when ROM updating is cryptographi-
On the other hand, designing complex module of executable software within cally protected for integrity, meets these
trustworthy systems and “compromise- a system should be backed by a chain criteria. Software that is cryptographi-
resilient” systems on top of insecure of trust, assuring (a) that the integrity cally protected for integrity, validated
8 SCALABLE TRUSTWORTHY SYSTEMS

20. when loaded, and protected by hardware there are no accepted methodologies for languages; and corresponding analysis
so it can only be executed also meets design, implementation, operation, and techniques. System design and analysis,
these criteria. evaluation that adequately characterize of course, must also anticipate desired
the trade-offs among trustworthiness, operational practice and human usabil-
One of the most relevant challenges for functionality, cost, and so on. ity. It must also encompass the entire
this topic area is how to achieve highly system life cycle and consider both
principled system development pro- What approaches might be environmental adversaries and other
cesses based on detailed and farsighted desirable? adverse influences.
requirements and sound architectures Currently, searching for flaws in micro-
that can be composed out of demon- processor design makes effective use Recent years have seen considerable
strably trustworthy components and of formal verification tools to evaluate progress in model checking and theorem
subsystems, and subjected to rigor- a chip’s logic design, in addition to proving. In particular, significant prog-
ous software, hardware, and system other forms of testing and simulation. ress has been made in the past decade
engineering disciplines for its imple- This technology is now becoming very on static and dynamic analysis of source
mentation. The tools currently being cost-effective. However, it is not likely code. This progress needs to be extended,
used do not even ensure that a com- to scale up by itself to the evaluation with particular emphasis on realistic
posed system is at least as trustworthy of entire hardware/software systems, scalability that would be applicable to
as its components. including their applications. Also, it large-scale systems and their applications.
is unclear whether existing hardware
Measuring confidentiality and integrity verification tools are robust against Verification of a poorly built system after
flaws in trustworthy system construc- nation-state types of adversaries. Formal the fact has never been accomplished,
tion requires the ability to identify and verification and other analytic tools that and is never likely to work. However,
measure the channels through which can scale will be critical to building because we cannot afford to scrap our
information can leak out of a system. systems with significantly higher assur- existing systems, we must seek an evo-
Covert channels have been well studied ance than today’s systems. Better tools lutionary strategy that composes new
in the constrained, older, local sense are needed for incorporating assurance systems out of combinations of old and
of the term. In an increasingly con- in the development process and for auto- new subsystems, while minimizing the
nected world of cross-domain traffic, mating formal verification. These tools risks from the old systems. A first step
distributed covert channels become may provide the functionality to build might involve a more formal under-
increasingly available. For more distrib- a secure computing base to meet many standing of the security limitations
uted forms of covert channels or other of users’ needs for assurance and func- and deficiencies of important exist-
out-of-band signaling channels, we lack tionality. They should be available for ing components, which would at least
the science, mathematics, fundamental pervasive use in military systems, as well allow us to know the risks being taken
theory, tools for risk assessment, and the as to commercial providers of process by using such components in trustwor-
ability to seal off such adverse channels. control systems, real-time operating thy composable systems. The ultimate
systems, and application environments. goal is to replace old systems gradually
Legacy constraints on COTS soft- Tools that can scale up to entire systems and piecewise over time, to increase
ware, lack of networking support, and (such as national-scale infrastructures) trustworthiness for progressively more
serious interoperability constraints have will require rethinking how we design, complex systems.
retarded progress. Meaningful security build, analyze, operate, and maintain
has not been seen as a competitive systems; addressing requirements; Verification is expensive. Most COTS
advantage in the mainstream. Even if system architectures; software engi- systems are built around functional-
trustworthiness were seen in that light, neering; programming and specification ity rather than trustworthiness, and
SCALABLE TRUSTWORTHY SYSTEMS 9

21. are optimized on cost of development forever remain stuck in the intractable composability of function enables scal-
and time to deployment—generally to position of starting from scratch each ability of system development today).
the detriment of trustworthiness and time. This foundation must include Fundamental research in writing security
often resulting in undetected vulner- verified and validated hardware, soft- specifications that are precise enough to
abilities. An alternative approach is to ware, compilers, and libraries with be verified, strict enough to be trusted,
start from a specification and check the easily composable models that include and flexible enough to be implemented
soundness of the system as it is being responses to environmental stimuli, will be crucial to major advances in
built. The success of such an approach misconfigurations and other human this area.
would depend on new languages, envi- errors, and adversarial influences, as
ronments that enable piecewise formal well as means of verifying composi- Resources
verification, and more scalable proof- tions of those components.
generation technology that requires As noted above, this topic is absolutely
less user input for proof-carrying code. What R&D is evolutionary and fundamental to the other topics. The
A computer automated secure software what is more basic, higher costs of not being able to develop scal-
engineering environment could greatly able trustworthy systems have already
risk, game changing?
facilitate the construction of secure proven to be enormous and will con-
systems. Better yet, it should encompass Evolutionary R&D might include incre- tinue to escalate. Unfortunately, the
hardware and total system trustworthi- mental improvements of large-scale costs of developing high-assurance
ness as well. systems for certain critical national systems in the past have been consider-
infrastructures and specific applica- able. Thus, we must reduce those costs
Another critical element is the creation tion domains, such as DNS and without compromising the effective-
of comprehensible models of logic and DNSSEC, routing and securing the ness of the development and evaluation
behavior, with comprehensible inter- Border Gateway Protocol (BGP), vir- processes and the trustworthiness of
faces so that developers can maintain tualization and hypervisors, network the resulting systems. Although it is
an understanding of systems even as file systems and other dedicated servers, difficult to assess the costs of develop-
they increase in size and scale. Such exploitation of multicore architectures, ing trustworthy systems in the absence
models and interfaces should help and web environments (e.g., browsers, of soundly conceived building blocks,
developers avoid situations where cata- web servers, and application servers we are concerned here with the costs
strophic bugs lurk in the complexity such as WebSphere and WebLogic). of the research and prototype devel-
of incomprehensible systems or in the However, approaches such as harden- opments that would demonstrate the
complexity of the interactions among ing particularly vulnerable components efficacy and scalability of the desired
systems. Creation of a language for or starkly subsetting functionality are approaches. This may seem to be a
effectively specifying a policy involving inherently limited, and belief in their rather open-ended challenge. However,
many components is a hard problem. effectiveness is full of risks. Goals of incisive approaches that can increase
Problems that emerge from interac- this line of R&D include identifying composability, scalability, and trust-
tions between components underscore needs, principles, methodologies, tools, worthiness are urgently needed, and
the need for verifying behavior not and reusable building blocks for scalable even relatively small steps forward can
only in the lab, but in the field as well. trustworthy systems development. have significant benefits.
Finally, efficiently creating provably More basic, higher-risk, game-changing To this end, many resources will be
trustworthy systems will require R&D broadly includes various topics essential. The most precious resource is
creation of secure but flexible com- under the umbrella of composability, undoubtedly the diverse collection of
ponents, and theories and tools for because it is believed that only effec- people who could contribute. Also vital
combining them. Without a secure tive composability for trustworthiness are suitable languages for requirements,
computing foundation, developers will can achieve true scalability (just as specification, programming, and so on,
10 SCALABLE TRUSTWORTHY SYSTEMS

22. along with suitable development tools. computer automated secure software could proceed for any systems in the
In particular, theories are needed to engineering environment (including its context of the exemplars noted above,
support analytic tools that can facili- generalization to hardware and systems) initially with respect to prototypes and
tate the prediction of trustworthiness, should be measured in the reduction of potentially scaling upward to enterprises.
inclusion modeling, simulation, and person-hours required to construct and
formal methods. verify systems of comparable assurance To what extent can we test
levels and security. The reuse and size
real systems?
Measures of success of components being reused should be
measured, since the most commonly In general, it may be more cost-effective
Overall, the most important measure used components in mission-critical to carry out R&D on components, com-
of success would be the demonstrable systems should be verified components. posability, and scalability in trustworthy
avoidance of the characteristic system Evaluation methodologies need to be environments at the subsystem level
failures that have been so common in developed to systematically exploit the than in general system environments.
the past (e.g., see [Neu1995]), just a few metrics. The measures of success for scal-However, composition still requires test
of which are noted earlier in this section. able trustworthy systems also themselves and evaluation of the entire system. In
need to be composable into enterprise- that it is clearly undesirable to experi-
Properties that are important to the level measures of success, along with the ment with critical systems such as
designers of systems should be measured measures contained in the sections on power grids, although owners of these
in terms of the scale of systems that can the other topic areas that follow. systems have realistic but limited-scale
be shown to have achieved a specified test environments. There is consider-
level of trustworthiness. As noted at the What needs to be in place for able need for better analytic tools and
beginning of this section, trustworthi- testbeds that closely represent reality.
test and evaluation?
ness typically encompasses requirements Furthermore, if applicable principles,
for security, reliability, survivability, and Significant improvements are necessary techniques, and system architectures
many other system properties. Each in system architectures, development can be demonstrated for less critical
system will need to have its own set methodologies, evaluation methodolo- systems, successful system developments
of metrics for evaluation of trustwor- gies, composable subsystems, scalability, would give insights and inspiration that
thiness, composability, and scalability. and carefully documented, successful would be applicable to the more critical
Those metrics should mirror generic worked examples of scalable prototypes. systems without having to test them
requirements, as well as any require- Production of a reasonable number of initially in more difficult environments.
ments that are specific to the intended examples will typically require that will
applications. The effectiveness of any not all succeed. Test and evaluation
References
[Can2001] Ran Canetti. Universally composable security: A new paradigm for cryptographic protocols
(http://eprint.iacr.org/2000/067), 2005. An extended version of the paper from the 42nd
Symposium on Foundations of Computer Science (FOCS’01) began a series of papers
applying the notion of universal composability to cryptography. Much can be learned
from this work regarding the more general problems of system composability.
[Neu1995] Peter G. Neumann. Computer-Related Risks, Addison-Wesley/ACM Press, New York, 1995. See also an
annotated index to online sources for the incidents noted here, as well as many more recent cases
(http://www.csl.sri.com/neumann/illustrative.html).
SCALABLE TRUSTWORTHY SYSTEMS 11

23. [Neu2004] Peter G. Neumann. Principled assuredly trustworthy composable architectures. DARPA-CHATS Final
Report, SRI International, Menlo Park, California, December 2004
(http://www.csl.sri.com/neumann/chats4.html). This report characterizes many of the
obstacles that must be overcome in achieving composability with predictable results.
[Sal+2009] J.H. Saltzer and F. Kaashoek. Principles of computer design. Morgan Kauffman, 2009. (Chapters
1-6; Chapters 7-11 are online at: http://ocw.mit.edu/ans7870/resources/system/index.htm).
12 SCALABLE TRUSTWORTHY SYSTEMS

24. Current Hard Problems in INFOSEC Research
2. Enterprise-Level Metrics (ELMs)
BACKGROUND
What is the problem being addressed?
Defining effective metrics for information security (and for trustworthiness more
generally) has proven very difficult, even though there is general agreement that such
metrics could allow measurement of progress in security measures and at least rough
comparisons between systems for security. Metrics underlie and quantify progress
in all other roadmap topic areas. We cannot manage what we cannot measure, as
the saying goes. However, general community agreement on meaningful metrics
has been hard to achieve, partly because of the rapid evolution of information
technology (IT), as well as the shifting locus of adversarial action.
Along with the systems- and component-level metrics that are discussed elsewhere
in this document and the technology-specific metrics that are continuing to emerge
with new technologies year after year, it is essential to have a macro-level view of
security within an organization. A successful research program in metrics should
define a security-relevant science of measurement. The goals should be to develop
metrics to allow us to answer questions such as the following:
ƒƒ How secure is my organization?
ƒƒ Has our security posture improved over the last year?
ƒƒ To what degree has security improved in response to changing threats and
technology?
ƒƒ How do we compare with our peers with respect to security?
ƒƒ How secure is this product or software that we are purchasing or deploying?
ƒƒ How does that product or software fit into the existing systems and
networks?
ƒƒ What is the marginal change in our security (for better or for worse), given
the use of a new tool or practice?
ƒƒ How should we invest our resources to maximize security and minimize
risks?
ƒƒ What combination of requirement specification, up-front architecture,
formal modeling, detailed analysis, tool building, code reviews, programmer
training, and so on, would be most effective for a given situation?
ƒƒ How much security is enough, given the current and projected threats?
ƒƒ How robust are our systems against cyber threats, misconfiguration,
environmental effects, and other problems? This question is especially
important for critical infrastructures, national security, and many other
large-scale computer-related applications.
13

25. Enterprise-level metrics (ELMs) address environment. Note that this definition quantifiable, feasible to measure, and
the security posture of an organization incorporates a specification of system repeatable. They provide relevant trends
and complement the component-level objectives and a specification of the over time and are useful in tracking
metrics examined elsewhere in the system environment, which would performance and directing resources
roadmap topics. “Enterprise” is a term include some notion of a threat model. to initiate performance improvement
that encompasses a wide range. It could Although this type of probability metric actions.” [http://www.itl.nist.gov/lab/
in principle apply to the Internet as a has been computed for system reliability bulletns/bltnaug03.htm]
whole, but realistically it is intended and for certain system risk assessments,
here to scale in scope from a large cor- the potential accuracy of such assess- Most organizations view the answers to
poration or department of the federal ments with respect to security seems the questions listed above in the short
government down to the small office/ to be extremely questionable, given the term from a financial mind-set and
home office (SOHO). For our purposes, rapidly changing threat environment for attempt to make cost-benefit trade-
an enterprise has a centralized decision IT systems. For example, a presumed off analyses. However, in the absence
making authority to ensure the use of high probability of meeting security of good metrics, it is unclear whether
ELMs to rationally select among alterna- objectives essentially goes to zero at the those analyses are addressing the right
tives to improve the security posture of instant security exploits are announced problems. Decisions resulting from
that enterprise. ELMs can support deci- and immediately perpetrated. such analyses will frequently be detri-
sions such as whether adoption of one mental to making significant security
technology or another might improve Security metrics are difficult to develop improvements in the long term and
enterprise security. ELMs also provide because they typically try to measure thus eventually require costly new
the basis for accurate situational aware- the absence of something negative (e.g., developments.
ness of the enterprise’s security posture. lack of any unknown vulnerabilities in
systems and lack of adversary capabilities What are the potential
In this discussion, we define metrics rel- to exploit both known and unknown
threats?
evant to systems and networking within vulnerabilities). This task is difficult
an enterprise, and consider composing because there are always unknowns in Lack of effective ELMs leaves one in the
host-level and other lower-layer mea- the system and the landscape is dynamic dark about cyberthreats in general. With
surements up to an enterprise level. In and adversarial. We need better defini- respect to enterprises as a whole, cyber-
other words, the goals of ELMs are to tions of the environment and attacker security has been without meaningful
understand the security of a large-scale models to guide risk-based determi- measurements and metrics throughout
system—enabling us to understand nation. These are difficult areas, but the history of information technol-
enterprise security as a whole, with a progress is achievable. ogy. (Some success has been achieved
goal of using these measurements to with specific attributes at the compo-
guide rational investments in security. The following definition from NIST nent level.) This lack seriously impedes
If these ELM goals are met, then exten- may provide useful insights. the ability to make enterprise-wide
sions to other related cases, such as informed decisions of how to effectively
Internet service providers (ISPs) and “IT security metrics provide a practical avoid or control innumerable known
their customers, should be feasible. approach to measuring information and unknown threats and risks at every
security. Evaluating security at the system stage of development and operation.
Security itself is typically poorly defined level, IT security metrics are tools that
in real systems, or is merely implicit. facilitate decision making and account- Who are the potential
One view might be to define it as the ability through collection, analysis, and
beneficiaries? What are their
probability that a system under attack reporting of relevant performance data.
respective needs?
will meet its specified objectives for a Based on IT security performance goals
specified period of time in a specified and objectives, IT security metrics are In short, everyone who is affected by an
14 ENTERPRISE-LEVEL METRICS

26. automated IT system has the potential caused by cyber attacks, which might short-term economic losses caused by
to benefit from better security metrics, be enhanced with the existence of mean- system outages. Potential beneficiaries,
especially at the enterprise level. Spon- ingful metrics. However, that market challenges, and needs are summarized
sors of security R&D require such is perhaps undercut not by the lack in Table 2.1.
metrics to measure progress. With such of suitable metrics, but more by the
metrics, decision makers, acquisition prevalence of insecure systems and their What is the current state of
managers and investors in security tech- exploitations and by a historical lack of
the practice?
nology could make a better business case consistent actuarial data.
for such technology, and guide intel- At present, the practice of measuring
ligent investment in such technology. Metrics defined relative to a mission security is very ad hoc. Many of the
This demand of course would guide threat model are necessary to understand processes for measurement and metric
the market for development of mea- the components of risk, to make risk selection are mostly or completely sub-
surably more secure systems. Metrics calculations, and to improve decision jective or procedural, as in evaluation
can be applied not just to technol- making in response to perceived risk. of compliance with Sarbanes-Oxley,
ogy, but to practices as well, and can A risk model must incorporate threat HIPAA, and so on. New approaches
provide management with an incentive information, the value of the enterprise are introduced continually as the old
structure oriented toward security per- information being protected, poten- approaches prove to be ineffective. There
formance improvement. Robust metrics tial consequences of system failure, are measurements such as size and scope
would enhance the certification and operational practices, and technology. of botnets, number of infections in a
accreditation process, moving toward More specifically, risk assessment needs a set of networks, number of break-ins,
quantitative rather than qualitative pro- threat model (encompassing intent and antivirus detection rates over time, and
cesses. Metrics also can be used to assess capabilities), a model of actual protective numbers of warrants served, crimi-
the relative security implications of measures, a model of the probability that nal convictions obtained, and national
alternative security measures, practices, the adversary will defeat those protective security letters issued (enforcement).
or policies. measures, and identification of the con- These are not related to fundamental
sequences of concern or adversary goals. characteristics of systems, but are more
Administrators require metrics to guide These consequences of concern are typi- about what can be measured about
the development of optimal network cally specific to each enterprise, although adversaries. Examples include websites
configurations that explicitly consider many commonalities exist. For critical that attempt to categorize the current
security, usability, cost, and perfor- infrastructures, loss of system availability state of the Internet’s health, the current
mance. There seems to be a potential may be the key concern. For commercial state of virus infections world wide, or
market in insurance and underwriting enterprises, loss of proprietary infor- the number and sizes of botnets cur-
for predicting and reducing damages mation may be a greater concern than rently active.
TABLE 2.1: Beneficiaries, Challenges, and Needs
Beneficiaries Challenges Needs
Developers Establishing meaningful ELMs Specification languages, analysis tools
(comprehensive, feasibly implementable, for feasibility, hierarchical evaluation,
realistic) and incremental change
System procurers Insisting on the use of meaningful ELMs Certified evaluations
User communities Having access to the evaluations of Detailed evaluations spanning all
meaningful ELMs relevant aspects of trustworthiness
ENTERPRISE-LEVEL METRICS 15