You are here

Large-Scale Network Traffic Monitoring with DBStream, a System for Rolling Big Data Analysis

TitleLarge-Scale Network Traffic Monitoring with DBStream, a System for Rolling Big Data Analysis
Publication TypeConference Paper
Year of Publication2014
AuthorsBär, A., A. Finamore, P. Casas, L. Golab, and M. Mellia
Conference NameInternational Conference on Big Data, IEEE BigData
Date Published11/2014
Conference LocationWashington D.C., USA
KeywordsBig Data Analysis, Data Stream Processing, network data analysis, System Performance

The complexity of the Internet has rapidly increased, making it more important and challenging to design scalable network monitoring tools. Network monitoring typically requires rolling data analysis, i.e., continuously and incrementally updating (rolling-over) various reports and statistics over high-volume data streams. In this paper, we describe DBStream, which is an SQL-based system that explicitly supports incremental queries for rolling data analysis. We also present a performance comparison of DBStream with a parallel data processing engine (Spark), showing that, in some scenarios, a single DBStream node can outperform a cluster of ten Spark nodes on rolling network monitoring workloads. Although our performance evaluation is based on network monitoring data, our results can be generalized to other big data problems with high volume and velocity.

Citation KeyBae2014a
Refereed DesignationRefereed
Project year: 
Second year
WP(s) associated with the paper: 
WP3 - Large-scale data analysis
Partner(s) associated with the paper's author(s): 
Politecnico di Torino
Forschungszentrum Telekommunikation Wien Gmbh
Is this an OFFICIALLY supported mPlane paper?: