Network Volatility Analysis for Threat Detection

No ratings

Presented at flocon 2018 by

Network usage patterns can vary throughout the day but abrupt and unexpected changes in behavior can be a leading indicator of potentially malicious activity. Pinpointing these unanticipated, temporal events can be problematic, especially when normal, daily fluctuations can be expected. Traditional SQL based queries for these volatile spikes in activity are challenging across large amounts of data. Data resolution, time frame, and various permutations can impact the compute time required and may destroy any early warning advantage that this detection mechanism can provide. Adopting methods from high-frequency, stock trading analysis, we can define appropriate highs and lows that adapt to data as it changes over time. We will demonstrate how to quickly and efficiently detect these volatility spikes for various data inputs. Temporally examining port/protocol usage for volatility lends itself well to help detect and visualize erratic changes in unexpected places. This can be activity that can go unnoticed from new malicious software or unwanted applications. These volatility measures can also be used with other fields as well, such as using IP addresses paired with unsuccessful connection states and can help uncover potentially loud scanning that does not occur over sustained periods of time. Our discussion will focus on approaches and strategies for exploring this flow volatility. We will recount our experiences executing these metrics on real-world, multi-billion record Bro and NetFlow datasets and approaches for dealing with this data at scale. We will also discuss ways analysts can use these metrics and approaches for threat detection, analysis validation, and response. Attendees will learn:This talk focuses on the real-world impact of using analytical methods not traditionally reserved for security operations. Using real-world Bro and NetFlow datasets, we demonstrate how mechanisms from high-frequency stock trading analysis lend themselves to detecting potential and adapting network security events at scale. Attendees will learn how these mechanisms can best be applied to network data to detect not only past, but live threats and the methods used for acting on these threats.