IRonMAN: InterpRetable Incident Inspector Based ON Large-Scale Language Model and Association miNing
No ratings
Presented at BlackHat USA 2023 by
Contextual incident investigation and incident similarity assessment are crucial components of modern IR and proactive threat hunting strategies. However, current automated systems often rely on pattern- and heuristic-based approaches due to their reliability and competitive performance. These approaches lack the ability to correlate events with contextual information and are susceptible to evasion through slight variations, resulting in false alerts. Recent advances in large-scale language models (LLMs) have shown promising results in language representation. By adopting LLM embedding strategies for security incidents, contextual relationships and similarities of events can be modeled, leading to a reduced false alert rate. However, LLM-based approaches often lack interpretability, which is essential for security analysts.In this work, we propose the first explainable LLM-based incident inspector. We combine a large-scale language embedding model with a frequent association algorithm to extract significant tokens, providing strong interpretability for incident similarity in feature space representation. Moreover, the contextual comprehension capabilities of the LLM ensure robustness against input variations. We demonstrate the practicality of our method in real-world incidents by applying it to our global visibility platform (200M+ events per day). The significant tokens generated by our model clearly identify the reasons why incidents are believed to stem from the same APT groups. Additionally, compare the results generated by our method to feedback from security analysts and thus provide different analytical perspectives for incident analysis.In conclusion, our method seamlessly merges traditional incident response strategies with advanced data science techniques, enriching the information available to security analysts. Moreover, our method can be applied to incident similarity, attribution and archiving. Our work, along with the comparative analysis, serves as a catalyst for the development of even more robust and interpretable methods for incident analysis.