InSight2: An Interactive Web-Based Platform for Modeling and Analysis of Large-Scale Argus Network Flow Data
No ratings
Presented at flocon 2018 by
Network monitoring systems are paramount to the proactive detection and mitigation of problems in computer networks related to performance and security. Degraded performance of network equipment and compromised end-nodes can cost computer networks downtime, data loss, and reputation. InSight2 is a web-based platform developed for the purpose of proactive and predictive monitoring of network performance and security aspects and providing intuitive visualizations thereof in organized dashboards in near real time. InSight2 models and analyzes network transactions to provide insight in to the network performance such as current bandwidth utilization, packet rate, packets dropped and the number of nodes online. InSight2 also uses up-to-date emerging threat lists and data analytics to identify denial of service attacks, botnets, ransomware servers, bogons, compromised hosts, spammers, scanners and a host of other types of malicious agents in the network. All data is automatically tagged with geographical, organizational, and other related information for identification and further investigation. InSight2 processes Argus flow records which provide information such as number of bytes and packets transmitted, number of packets lost and retransmitted, jitter, and inter-packet delay for each flow. Emerging threats are extracted from multiple up-to-date repositories to build a threats database which is used to enrich each flow by adding one or more searchable tags. InSight2 utilizes MaxMind GeoIP to add geographical information such as country and city information as well as latitude-longitude coordinates which are used to plot the source and destination nodes in interactive global maps. The Global Science Registry from the GLORIAD project is used to enrich network flows with organizational information. Elasticsearch serves as the back-end database and search engine. An associated Kibana module handles the data visualization. Markov Chains are used to predict network activity based on past behavior. InSight2’s front-end incorporates user authentication, SSL encryption, and isolation of the dashboard controls from the end user by displaying the dashboards in a modern and unified web-interface that allows the network administrator to show customized information based on user privileges. InSight2 runs under any Linux operating system as a system service. An installer is provided that requires minimal user interaction. InSight2 includes a user guide and a video tutorial to get the users up to speed with installation and usage quickly. Development of InSight2 is supported by the National Science Foundation under Grant No. IRNC-1450959.