Cybersecurity Experimentation of the Future (CEF)
Cyberspace is rapidly evolving with nearly every aspect of society moving toward pervasive computing and networking. These changes bring real and wide-ranging cybersecurity threats and challenges that require new solutions based on sound scientific principles. The scale and complexity of the challenges require that researchers employ experimentation infrastructure to enable discovery, validation, and ongoing analysis. The NSF-funded Cybersecurity Experimentation of the Future (CEF) project explored current and expected experimentation infrastructure needs in the context of current and future cybersecurity research challenges and produced a roadmap for developing an accessible, broad, and multi-organizational cybersecurity experimentation capability that supports tomorrow’s research. The collaborative, community-based effort included three main thrusts: (1) investigate and assess existing experimentation infrastructure and user community experience; (2) identify future cybersecurity experimentation infrastructure needs; and (3) organize the requirements and needed capabilities into a strategic plan and roadmap for future cybersecurity infrastructure development. The future needs were identified through a series of community-based study groups to understand hard cybersecurity problems and use cases that can benefit from experiment-driven research, identify the experimentation infrastructure needed to facilitate research, and identify gaps between needed and current capabilities and prioritize capabilities based on domain needs. The strategic plan will help shape the development of sustainable experimentation infrastructure to support evolving cybersecurity research needs. The resulting experimentation capabilities will provide essential means for research communities to develop a fundamental understanding of and to reason about cybersecurity solutions and ultimately provide for secure and trustworthy cyberspace.
NSF Award Abstract ACI-1346277
CEF Final Report
The CEF Final report was released on July 31, 2015 and is available at www.cyberexperimentation.org.
This report presents a strategic plan and enabling roadmap intended to catalyze generational advances in the field of experimental cybersecurity research. These results represent the conclusions of a study conducted under NSF auspices by SRI International and USC Information Sciences Institute throughout calendar year 2014. The study had broad participation by stakeholders representing the cybersecurity research, research sponsor, and customer communities. The report outlines the process and methodology of the project, presents key inputs, supporting evidence developed through the course of the study, and synthesized results, and then presents our final conclusions.
Our overarching finding is that transformational progress in three distinct, yet synergistic, areas is required to achieve the desired objectives:
1. Fundamental and broad intellectual advances in the field of experimental methodologies and techniques, with particular focus on complex systems and human-computer interactions.
2. New approaches to rapid and effective sharing of data and knowledge and information synthesis that accelerate multi-discipline and cross-organizational knowledge generation and community building.
3. Advanced, accessible experimentation infrastructure capabilities.
The central result of our study is a roadmap that presents requirements, objectives and goals in each of the areas outlined above over three, five and ten year phases. In some cases, the phases build upon each other, and in other cases, new fundamental research is required over a longer period of time to satisfy the objectives of the roadmap.
Taken together, these areas, as embodied in the roadmap, paint a vision for a new generation of experimental cybersecurity research – one that offers powerful assistance towards helping researchers shift the asymmetric cyberspace context to one of greater planning, preparedness, and higher assurance fielded solutions.
The capabilities identified in the roadmap take into account the current state of the art in experimental cybersecurity research and its supporting infrastructure, other types of research facilities, and existing cyber-domain “test and evaluation” capabilities. In addition to leveraging current and expected capabilities in cybersecurity and adjacent areas, the roadmap presumes advances in key computer science disciplines such as ontologies, metadata, libraries, and corresponding resource discovery.
We emphasize that while this type of study would typically focus heavily on experimentation infrastructure (i.e., tools and testbeds), and while we did pay significant attention to this topic, our fundamental conclusion is that an emphasis on infrastructure alone will fall far short of achieving the transformational shift in the research, community, and experimentation required to address cybersecurity in the rapidly changing cyber environment.
Our conclusion is that strong, coupled, and synergistic advances across each of the areas outlined above – fundamental methodological development, fostering and leveraging communities of researchers, and in the capabilities of the infrastructure supporting that research – will transform the field of cybersecurity.