IMPROVING USEFUL DATA EXTRACTION FROM CYBERSECURITY INCIDENT REPORTS
No ratings
Presented at First 2017 by
One of the central services of Computer Security Incident Response Teams (CSIRTs) and Security Operation Centers (SOCs) is the receipt of incident reports from their constituency. Some teams have large constituencies and receive tens of thousands of incident reports per year. Some teams have turned to automation, when dealing with large volumes of incident reports, to assist analysts with incident prioritization, workflow assignment, and more. This is a complex process because incident reports often have a unique 'form.' This ‘Form” is typically a mixture of header/identifying information, structured information, free text narrative, cybersecurity jargon terminology, and uniquely transformed information such as 'defanging' of potentially dangerous information. Over the past few years we have worked in collaboration with US-CERT on exploratory analysis of incident data and reports. The focus has been on improving the quality and amount of useful information that can be extracted from incident reports and used for correlation, trending, situational awareness, and eventually predictive analysis. A recent part of this work has focused on developing a method for improving the use of regular expression searching for the extraction of indicators and other useful information. In our method, we first assemble a set of ground truth incident reports with the information manually extracted. We then identify false positives and indicators that were not being extracted. We also developed a framework to measure the improvement that a given extraction method has had using the ground truth data. This allows us to monitor the affect a change in the regular expression has had upon extractions of the larger corpus. This presentation will discuss our method, challenges teams can expect to encounter when automating extraction from incident reports, our lessons learned during the creation of the ground truth data (such as other useful types of information we noticed in the incident reports and future ideas for extracting it), our testing process, and some initial observations related to our results.