SRI International's Computer Science Lab has been actively involved in intrusion-detection research since 1983. The original groundwork for SRI's intrusion-detection research explored statistical techniques for audit-trail reduction and analysis. The first-generation statistics component was used to analyze System Management Facility (SMF) records from an IBM mainframe system in the first half of the 1980s. Later, this research examined the use of a rule-based expert system to detect known malicious activity This early research led to the development of a prototype Intrusion-Detection Expert System (IDES), capable of providing real-time detection of security violations on single-target host systems. IDES was a critical first step toward the development of real-time dual-analysis (signature analysis and anomaly-detection) intrusion-detection technology for monitoring security-critical government computing environment. By 1990, efforts began to integrate the IDES (later NIDES) prototype into a real-world computing environment (see the FBI FOIMS project).
With the maturity of the analysis methodologies developed under IDES, SRI began a comprehensive effort to enhance, optimize, and re-engineer the earlier IDES prototype into a production-quality intrusion-detection system called Next-Generation Intrusion Detection Expert System. NIDES introduced a results-fusion component called the Resolver to integrate its response logic with the results produced by the statistical anomaly-detection subsystem and PBEST signature analysis tool. The NIDES statistical subsystem (NIDES/Stats) employs a wide range of multivariate statistical measures to profile the behavior of individual users or other computational entities. Analysis is profile-based, where a statistical score is assigned to each session representing how closely currently observed usage corresponds to the established patterns of usage for that individual. NIDES/Stats produces a separate usage profile for each user or other entity, and updates individual profiles as their corresponding audit records are encountered NIDES also included a signature analysis component, developed using PBEST, to characterize known intrusive activity through rule encodings Lastly, NIDES added an X/Motif-based graphical user interface facility to provide location-independent configuration and monitoring of NIDES operation and greatly increase usability.
The IDES/NIDES work pioneered the field of intrusion-detection, and sought to solve a difficult problem with a general and flexible approach, with no inherent restrictions on target systems, type of audit data to be analyzed, and techniques to be used. IDES/NIDES sought to address the need for user-oriented monitoring and profiling with a general and flexible approach, with no inherent restrictions on target systems, type of audit data to be analyzed, and techniques to be used. These efforts did, however, have some inherent limitations in scalability, applicability to network environments by their focus on users as the analysis targets, and lack of features to support interoperability. In addition, IDES/NIDES did not include features to address the more global threats from multi-domain coordinated attacks. CSL's Safeguard effort subsequently overcame profile explosion and scalability problems by profiling the activities of subsystems and commands rather than of individual users.
Our current research focuses on EMERALD: Event Monitoring Enabling Responses to Anomalous Live Disturbances, with Phillip A. Porras as principal investigator. In our current DARPA project (Contract F30602-96-C-0294, Analysis and Response for Intrusion Detection in Large Networks), we are developing a successor system to NIDES that will considerably extend the NIDES concept to accommodate network-based analyses and dramatically increase interoperability and ease of integration into distributed computing environments. This effort will include extending components for profile-based analysis, signature-based analysis, and localized results fusion with automated response capability. In addition, we are considerably extending our results analysis capability to facilitate hierarchical interpretations of our distributed monitoring units, which will enable cross-platform analysis at various layers of abstraction, and successive refinement of the resulting analyses within increasingly broader scopes. We are also developing an accompanying set of exportable API that will permit interoperability between EMERALD components and network monitoring facilities.
Summary of Intrusion-Detection Research at SRI's Computer Science Laboratory:
Analysis and Response for Misuse Detection
in Large Networks.
[SRI Project 1494, Contract Number F30602-96-C-0294, DARPA ITO Order No. E302, 28 August 1996 through 27 August 1999]. Phillip Porras and Peter Neumann are leading a project to develop EMERALD (Event Monitoring Enabling Response to Anomalous Live Disturbances), a distributed scalable tool suite for tracking malicious activity through and across large networks. EMERALD introduces a highly distributed, building-block approach to network surveillance, attack isolation, and automated response. The approach is novel in its use of highly distributed, independently tunable, surveillance and response monitors that are deployable polymorphically at various abstract layers in a large network. These monitors demonstrate a streamlined intrusion-detection design that combines signature analysis with statistical profiling to provide localized real-time protection of the most widely used network services on the Internet. Equally important, EMERALD introduces a framework for coordinating the dissemination of analyses from the distributed monitors to provide a global detection and response capability to counter attacks occurring across an entire network enterprise. Also, EMERALD introduces a versatile application-programmers' interface that enhances its ability to integrate with the target hosts and provides a high degree of interoperability with third-party tool suites. See the EMERALD Home Page for details, postscript documents, and future availability of prototype releases.
Safeguard: Detecting Unusual Program Behavior
Using the NIDES Statistical Component.
[SRI Project 2596, Contract Number 910097C (Trusted Information Systems) under F30602-91-C-0067 (Rome Labs), 1995]. Debra Anderson led a project to adapt the NIDES statistical anomaly-detection subsystem to profile the behavior of individual applications. Statistical measures were customized to measure and differentiate the proper operation of an application from operation that may indicate Trojan horse substitution. Under the Safeguard model, analysis is application-based, where a statistical score is assigned to the operation of applications and represents the degree to which current behavior of the application corresponds to its established patterns of operation. The Safeguard effort demonstrated the ability of statistical profiling tools to clearly differentiate the scope of execution among general-purpose applications. It also showed that statistical analyses can be very effective in analyzing activities other than individual users; by instead monitoring applications, the Safeguard analysis greatly reduced the required number of profiles and computational requirements, and also decreased the typical false-positive and false-negative ratios. These results suggest the possible utility of performing statistical analyses on activities at higher layers of abstraction.
Next-Generation IDES (NIDES).
[SRI Project 3131, Contract Number N00039-92-C-0015, 1992-1994]. Teresa Lunt and R. Jagannathan led an extensive effort to rearchitect and consolidate earlier IDES research results and prototypes into a production-quality tool suite. Most notably, NIDES incorporated distributed audit collection and consolidation mechanisms to address the need for multi-host intrusion-detection coverage. It also provided significant enhancement to the statistical analysis algorithms and rule-based expert system, as well as introducing an X-Window GUI for administrative control and monitoring. In February 1993, CSL released the alpha-version of NIDES, and the final NIDES Beta2 Release was completed in September 1994 for Sun Microsystems SunOS 4.1.4 for Sun and SPARC workstations. See the NIDES Home Page for details, postscript documents, and availability of NIDES Software.
IDES for a Network of Workstations.
[SRI Project 6784, Contract Number N00039-89-C-0050, ending 1992]. Teresa Lunt led a project to extend CSL's prototype Intrusion Detection Expert System (IDES) to be able to simultaneously monitor users on a network of Sun workstations and a DEC machine at SRI. The prototype IDES runs on several Sun 3 Workstations.
FOIMS-IDES, for the FBI Field Office Information
[SRI Project 6768, Contract J-FBI-88-171, 1991-93]. FOIMS is a classified IBM mainframe-based system used by FBI field offices throughout the U.S. to manage their cases. Following a previous one-year study that established the feasibility of applying IDES to the FOIMS environment, this contract implemented a version of IDES for FOIMS -- although it was not deployed in other than test environments. (Cleared insiders tend to be trusted, even if not trustworthy.)
The Enhanced IDES Prototype.
[SRI Project 4185, Contract Number 9-X5H-4074J-1, Los Alamos National Laboratory, Government Prime Contarct No. W-7405-ENG-36, SPAWAR, ending 1988]. Teresa Lunt led a project to enhance CSL's prototype Intrusion Detection Expert System (IDES). The prototype IDES is based on the IDES model developed at SRI. The prototype IDES runs on a Sun 3 Workstation and is able to monitor, in real time, all users from an SRI target system, to adaptively learn user behavior patterns, and to detect abnormal behavior on the target system. This project also added an expert system component to IDES. Other enhancements included adding additional intrusion-detection measures, improving the statistical algorithms, monitoring more users and more event types, improving performance, and improving the user interface. Under this contract, SRI also performed a feasibility study for the FBI for implementing an IDES for their nationwide information system FOIMS.
Real-Time Intrusion Detection Expert System (IDES) Prototype.
[SRI Project 7508-200, U.S. Government Contract N66001-84-D-0077, Delivery Order 0019, for the Space and Naval Warfare Command (SPAWAR), ending 1985]. SRI developed a prototype Intrusion Detection Expert System (IDES) to demonstrate proof-of-concept. The initial prototype ran on a SUN/3 workstation and could monitor, in real time, some users and some event types from an SRI target system, adaptively learn user behavior patterns, and detect some types of abnormal behavior on the target system. This project demonstrated that departures from normal user behavior can be detected in real-time.
Audit Trail Analysis and Usage Data Collection and Processing.
[SRI Project 5910, Defense Communications Agency Contract DCA 200-83-C-0025, ending 1984]. Peter Neumann led the design and development of an audit-trail analyzer for TAC logins on MILNET/ARPANET, providing both live detection and after-the-intrusion analysis. This work was also applicable to the auditing of classified networks.
Intrusion Detection Expert System (IDES Model).
[SRI Project 6169-70, Amendment 5 to U.S. Government Contract 83F83-01-00 for SPAWAR, 15 July 1984 to 16 September 1985]. Dorothy Denning and Peter Neumann developed a model for a real-time Intrusion-Detection System (IDES). This model forms the basis for the prototype IDES.
Statistical Techniques Development for
an Audit Trail System.
[SRI Project 6169, U.S. Government Contract 83F83-01-00, 15 July 1983 to 30 November 1986]. In this study, an extensive statistical analysis was performed on Government-furnished audit data from IBM systems running MVS and VM. A high-speed algorithm was developed that could accurately discriminate between users based on their behavior profiles. The project demonstrated that users can be distinguished from one another by their behavior profiles. These statistical procedures are potentially capable of reducing the audit trail by a factor of 100 while demonstrating a high degree of accuracy in detecting intrusion attempts. Harold Javitz led the project, assisted by Dorothy Denning, Al Valdes, and Peter Neumann.