Network Flow Query Language—Design, Implementation, Performance, and Applications

Cisco's NetFlow protocol and Internet engineering task force's Internet protocol flow information export open standard are widely deployed protocols for collecting network flow statistics. Understanding intricate traffic patterns in these network statistics requires sophisticated flow analysis tools that can efficiently mine network flow records. We present a network flow query language (NFQL), which can be used to write expressive queries to process flow records, aggregate them into groups, apply absolute or relative filters, and invoke Allen interval algebra rules to merge group records. We demonstrate nfql, an implementation of the language that has comparable execution times to SiLK and flow-tools with absolute filters. However, it trades performance when grouping and merging flows in favor of more operational capabilities that help increase the expressiveness of NFQL. We present two applications to demonstrate richer capabilities of the language. We show queries to identify flow signatures of popular applications and behavioural signatures to identify SSH compromise detection attacks.