Using Anthropology to Study Security Incident ResponsReturn to TOC
No ratings
Presented at FIRST 2014 by
The most critical assets in guarding the nation from cyber terrorists are our cyber defenders -- these are the security analysts, security center operators, incident investigators, etc. who are at the front line of our national defense, working ceaselessly in commercial, academic and government security operations centers (SOCs) to detect, repel, and prevent cyber intrusions into the sprawling cyber infrastructure. Like in any war, our assets need to have adequate arsenal, and have to be trained and re-trained to keep up with the enemy. Both have been extremely challenging. It is now well accepted that training cyber analysts is a long and hard process, and something that keeps us from expanding our defensive operations to match the enormity of the task. Government agencies, commercial SOCs, private consulting companies, universities are all suffering from a severe shortage of trained personnel in this area and yet there is no clear large-scale program in place to train security analysts. Our current approach to training is in an individual one-on-one mode that does not scale. Indeed, as we discovered in many SOCs, no systematic training is taking place at all and this is even culturally accepted as a fact of life for working in those organizations. Adding to this is the lack of effective tools, both commercially and from the research community, to help analysts do their job more efficiently. Without adequate arsenal and training, one is not expected to win in any war. We took the novel approach of trying to study the problem in its native environment, i.e. rather than study the problem of training as an abstract concept in a lab, we aim to observe and study the difficult art of analysts' job in the security operations center itself. Furthermore, in order to study the rather human problem of learning and teaching cyber analysis, we are applying tools and techniques from the humanities, in particular socio-cultural anthropology. Anthropologists use the term "tacit knowledge" to capture the idea that a lot of the knowledge that people use in their jobs is inside their heads and not written down or documented anywhere. This is especially true in SOCs; indeed many times the analysts who possess the knowledge do not even know how to express it. Added to that is the natural censure against sharing of sensitive information and the cultural belief that one has to learn his/her way through the roughs, and we have a "tribal" knowledge regime wherein critical knowledge is only transferred from human to human through long apprenticeships and inter-personal/trust relationships. Working with a professional anthropologist, our strategy is to have student interns embedded in various security operations centers with the goal of "learning by participation." Our embeds participate in the SOC just like regular employee trainees and keep detailed field notes on what they observe in their day to day interactions. These analysts process large amounts of data under time-stress conditions when handling cyber threats. The job requires intelligence and high levels of skills but has many mundane/repetitive aspects as well. Adequate tool support is largely lacking and many of the skills and procedures involved are un-codified and undocumented, resulting in a large body of tacit knowledge. We place computer science and anthropology researchers and graduate students trained in both fields into SOCs, working side by side with the analysts. This "participant observation" approach developed in socio-cultural anthropology provides a method and means to access the tacit knowledge of the analysts and to convert it into more explicit knowledge, leading to the development of algorithms that can help automate the tasks. More importantly, the ethnographic fieldwork provides an opportunity to observe real security operation centers' work processes and identify factors that influence the effectiveness and efficiency with which cybersecurity incidents are handled. This may help explain why some cybersecurity problems are hard to address in practice, what roles humans and organizational structures play, and where procedures might be inefficient or completely fail for non-technical reasons. The research is carried out through a collaborative effort involving researchers from Kansas State University and two companies, Honeywell and RedJack, LLC. Results from the research will not only create practical tools that leverage tacit knowledge in security analytics and automate/aid tasks in incident response and forensic analysis, but also informs the training of cybersecurity professionals by making explicit the tacit knowledge of effective security analytics acquired during participant observation. Our belief is that, rather than using the traditional approach of trying to build tools to address the problems that appear on the surface, we want to study the deeper relationship between the requirements of the job, the incentives and disincentives in the work environment, and the unique attributes of cyber security operations. Thus far we have conducted the fieldwork at Kansas State University's Office of Information Security and Compliance for about a year, where multiple PhD students work side by side with the security analysts in their daily duties. We have identified significant gaps in the tool support that no existing commercial or open-source solutions address, and we have been building tools to help the analysts' job and using the tool building as a means to open up the discussion on the technical details of their job that constitute the tacit knowledge. This has been a highly beneficial experience for both the researchers and the analysts. The tools that resulted from the discussion have tremendously improved the analysts' work performance (by their evaluation) and reduced the amount of labor they would have to put in to perform the repetitive, low-level, mundane tasks. This has also allowed the analysts to focus more of their effort to investigate sophisticated attacks, which in turn fosters more discussion with the research team and thenceforth to more tacit-to-explicit knowledge conversion. This will be extremely helpful in informing what types of training are most effective for new analysts so that they can more quickly acquire the deep analytical skills and not be overwhelmed by the low-level repetitive processes that can be automated. We are now extending and expanding this effort to studying more SOCs, including the commercial SOC at Honeywell. We would like to find more partners to work with us, so that our study can be more representative and valid. In particular, we will need help to access more SOCs to conduct the fieldwork. What we will need from our collaborators is to dedicate some human resources for doing this fieldwork. For academic collaborators, this could mean sending some students to SOC(s) and having regular meeting with the whole research team to exchange the findings produced from the fieldwork. For industry collaborators, this could mean having an analyst working with the student fieldworkers (apprentices) to train them in doing the job. The collaborating organizations will benefit from a third-party perspective of operational effectiveness, intra-team interactions, and other organizational attributes in the context of cyber security operations. They may also benefit from any tools that the fieldworkers build or help build specifically for the organization. At the end of the project, we expect to write a training manual with do's and don'ts for organizations employing cyber security operations personnel that are common to such organizations across businesses, academia, and governmental agencies. Collaborating organizations can also benefit by contributing to the framing and prioritization of issues to be addressed in such a manual as well as early access to learning. Any study such as this can only improve with more participation. We hope to invite the FIRST community of SOC analysts and managers alike to participate in our study and making the derived outcomes truly global. This work is supported by the National Science Foundation under Grant No. 1314925. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.