Title | Large-Scale Network Traffic Monitoring with DBStream, a System for Rolling Big Data Analysis |
Publication Type | Conference Paper |
Year of Publication | 2014 |
Authors | Bär, A., A. Finamore, P. Casas, L. Golab, and M. Mellia |
Conference Name | International Conference on Big Data, IEEE BigData |
Date Published | 11/2014 |
Publisher | IEEE |
Conference Location | Washington D.C., USA |
Keywords | Big Data Analysis, Data Stream Processing, network data analysis, System Performance |
Abstract | The complexity of the Internet has rapidly increased, making it more important and challenging to design scalable network monitoring tools. Network monitoring typically requires rolling data analysis, i.e., continuously and incrementally updating (rolling-over) various reports and statistics over high-volume data streams. In this paper, we describe DBStream, which is an SQL-based system that explicitly supports incremental queries for rolling data analysis. We also present a performance comparison of DBStream with a parallel data processing engine (Spark), showing that, in some scenarios, a single DBStream node can outperform a cluster of ten Spark nodes on rolling network monitoring workloads. Although our performance evaluation is based on network monitoring data, our results can be generalized to other big data problems with high volume and velocity. |
Citation Key | Bae2014a |
Refereed Designation | Refereed |