Work Package 1 - Use Cases, Requirements and Architecture
This document specifies the mPlane architecture and protocol: in this executive summary, we attempt at providing a concise yet complete view, as a useful starting point for a walk-through of the mPlane architecture, additionally providing hyper-links to the specific sections of this document treating a selection of the most relevant items in the mPlane architecture.
This deliverable first provides a top-down introduction (Ch. 1.1) to all core mPlane concepts, providing insights about the main decisions to achieve flexibility and extensibility. This includes the definition of a schema-centric protocol, capable of iterative measurement and pledging for weak imperativeness to avoid deadlock situations. The chapter also introduces the different entities such as components, clients, probes, repositories supervisors, and reasoners. It finally describes relationships among entities, illustrating e.g., the interfaces to these entities, as well as the message types and typical exchange sequences.
The deliverable then delves into the details, building the mPlane protcol from bottom-up across three layers. The first layer consists of an information model for mPlane messages (Ch. 2). The information model includes an element registry (defining, e.g., name structure as well as several primitive types) that are expressed in several message of different types, such as capability specification result and event notification (including receipt, redemption, indirection and exception). Details of the messages are thoroughly illustrated -- including both low-level details e.g., temporal scope, parameters and metadata, as well as high-level principles such as message uniqueness and idempotence
The next layer is then represented by the representation and session protocols (Ch. 3), where details of the serialization of the mPlane information model using a JSON representation are illustrated, and examples of textual representations of element values are given. Session protocol description includes secure access control schemes over HTTPS, over WebSockets and TLS or over SSH; only the first of these is presently implemented.
The deliverable then illustrates several example workflows (Ch. 4), including client-initiated processes, capability discovery phases, component-initiated worflow, callback control and indirect export, also covering how errors are handles in mPlane.
The remaining chapters (Ch. 5--8) are dedicated to regaining a complete vision of the architecture, so to fit detailed knowledge acquired so far (Ch. 2--4). Chapters covering for instance the role of the supervisor, the status of interoperable implementations, and data protection aspects, providing finally the core registry for completeness.
D1.4 v1 - Nov 2014: available here
This document defines the mPlane architecture and the interface provided between mPlane clients and components. The protocol is divided into layers: an information model for mPlane messages - measurement capabilities, specifications, results, and event notifications; serialization representations using JSON, YAML, and XML, and session protocol bindings using SSH and HTTP over TLS.
This document primarily defines security specifications for the mPlane architecture (in terms of authentication, access control and safe communications), on the basis of what specified in the D1.1. Also, it provides a description of the measures that can be adopted in order to guarantee the privacy of the data gathered through the probes. This aspect of the mPlane infrastructure must not be neglected, since from a legal point of view the users' right to privacy must be protected in any case. The techniques to be adopted are anonymization and aggregation, but utility of data decreases as the level of privacy increases, hence it is necessary to find a good trade-off. Two protocols are proposed for secure communications among components: TLS and SSH, which adopt respectively X.509 certificates and RSA keys for identity management. As the access control policy that will be adopted depends mostly on the mPlane administrators' choices, this document provides a survey of several approaches. The cross-domain and the mobile scenarios are also analyzed, providing solutions that can guarantee access control, security
The document defines the requirements for the mPlane architecture on the background of a set of scenarios explored by the consortium, a survey of existing comparable measurement systems and platforms and applicable standards therefore, and a set of architectural first principles drawn from the description of work and the consortium's experience. As mPlane is intended to be a fully flexible measurement platform, freely integrating existing probes and repositories with ones to be developed in the project, this document is primarily concerned with the definition of interfaces among mPlane components. While it does enumerate capabilities to be provided by these components, these are primarily intended to ensure the platform has the flexibility required to meet all the scenarios envisioned; the enumerations of measurements, metrics, data types, and other component capabilities are therefore not to be construed to limit the scope of work on components within the project to just those scenarios treated in this document; nor do the scenarios enumerated here define the capabilities to be demonstrated in the project's integrated trial.
Work Package 2 - Programmable Probes
This deliverable collects the software released by the mPlane Consortium at month 32. This is a software deliverable, so this document briefly describes the software by collecting the information that is present on the website page at the time of writing. Software and instruction on how to access it is available at http://www.ict-mplane.eu/public/software.
This deliverable collects the software released by the mPlane Consortium at month 18. This is a software deliverable, so this document briefly describes the software by collecting the information that is present on the website page at the time of writing. Software and instruction on how to access it must be accessed from http://www.ict-mplane.eu/public/software.
Work Package 3 - Large-scale data analysis
This deliverable collects the software released by the mPlane Consortium at month 32 within Work Package 3. This is a software deliverable, so this document briefly describes the software by collecting the information that is present on the website page at the time of writing. Software and instruction on how to access it must be accessed from http://www.ict-mplane.eu/public/software, and in particular under the Repository Tools section.
- D3.3 v2 - Apr 2015: Algorithm and Scheduler Design and Implementation
This deliverable describes the data-analysis algorithms implemented at the repository for each use case. Furthermore, it contains the algorithms and tools to support the parallel computation of the algorithms by using a distributed cluster of resources. These tools are publicly released by the mPlane Consortium, and include:
• Hadoop Fair Sojourn Protocol, a scheduler for Apache Hadoop;
• Schedule, a tool for cache-oblivious scheduling of shared workloads;
• RepoSim, a ns2 based simulator to fine-tune the mPlane repository performance;
• Blockmon-controller, the controller to run distributed tasks over a set of Blockmon nodes.
Software and instruction on how to access and use them at http://www.ict-mplane.eu/public/software. To highlight the strong collaboration with other EU-projects, the Hadoop Fair Sojourn Protocol has been developed jointly with partners from the BigFoot project.
D3.3 v1 - Sep 2014: available here.
This document describes the design of the database layer used in the mPlane project. Starting from the per use case algorithms defined in D3.1 we infer the types and format of data stored in mPlane repositories. In addition, we give a description of external data sources, which might either be mirrored inside an mPlane repository or accessed via the provided API. By the term mPlane repository we understand a logical or physical instance, providing data access, storage or both to other mPlane components. We continue by giving a description of data indexing and in-repository data processing. The in-repository data processing focuses on handling data streams of very high volume, continuously arriving at the repository. The document is completed, by a detailed description of the integration of repositories into the general mPlane architecture by giving an overview of the exposed capabilities. This part focuses on how mPlane repositories are used by the supervisors and reasoners from WP4.
This document describes the requirements, input, output for the algorithms needed to perform analytic tasks on a large amount of data, in the context of WP3. Starting from the use cases defined in WP1, we identify the algorithms needed to address the various scenario requirements. Operating on a large amount of data, these algorithms strive for parallel and scalable approaches; the designing and implementation of the algorithm itself can be a challenging research task since today very little is known concerning how to develop efficient and scalable algorithms that runs on parallel processing frameworks.
The algorithm in the storage layer are characterized by the fact that they operate on a large amount of data, and produce a concise representation of it, extracting features and aggregating it, so that the produced output is easier to handle and understand. Depending on the amount of data produced, on the scenario characteristics and on the time constraints, algorithms can require a real time (or near real time) or a batch processing.
For each algorithm and use case, the input data and the initial state, the computation to run and the output produced are described.
Work Package 4 - mPlane Supervisor: Iterative and Adaptive Analysis
This software deliverable presents the final release of the software developed within WP4, including a complete description of the multiple mPlane Reasoners and Analysis Modules. The deliverable additionally describes how to run the corresponding mPlane components together to tackle each of the use-cases addressed by mPlane. This is a software deliverable, thus the document briefly describes the software by collecting the information that is present on the mPlane website page at the time of writing it. Software and instructions on how to access the provided tools are available in the mPlane website at https://www.ict-mplane.eu/public/reasoners-and-analysis-modules.
This deliverable presents an extended set of Analysis Modules, including both the improvements done to those presented in deliverable D4.1 as well as the new analysis algorithms designed and developed to address use-cases. The deliverable also describes a complete workflow description for the different use-cases, including both stream processing for real-time monitoring applications as well as batch processing for “off-line” analysis. This workflow description specifies the iterative interaction loop between WP2, WP3, T4.1, and T4.2, thereby allowing for a cross-checking of the analysis modules and the reasoner interactions.
D4.3 v1 - Nov 2014: available here.
This deliverable describes the design and specification of the Reasoner system with a limited set of analysis/diagnosis rules as knowledge structure, and also evaluates the possible extensions to be included into the knowledge structure regarding learning of new rules. The deliverable presents a per-use case definition and instantiation of the Reasoner, including a first set of domain-knowledge-based analysis rules as well as the associated workflow of the iterative analysis. Some first evaluation results of the iterative process are reported. Finally, different learning techniques for extending and/or generating the knowledge structure of the Reasoner are overviewed.
D4.2 v1 - Jun 2014: available here.
This public deliverable describes the design and specification of a first set of basic analysis modules for addressing the use cases identified in WP1. The document focuses on the required algorithms, which use as input the measurements and analysis provided by the lower layers (WP2 and WP3) of the mPlane architecture to provide more advanced analysis and answers towards the resolution of the problem addressed by the use case. These analysis modules include both stream and batch processing algorithms and address issues such as classifications, estimations, predictions, detections, correlations and diagnosis.
Work Package 5 - Integration, Deployment, Data Collection, Evaluation
This deliverable reports the outcome of assessment and evaluation activities, including tests and experiments performed on selected use cases defined in WP1.
This document is organized in two main parts: Results of Use Cases evaluation activity, and Experiments and collected data analysis.
For a better understanding of the activities and tests performed, general information regarding the use cases is taken from Deliverable 5.5. In particular for each use case a short description and objectives are reported before showing final tests outcome. Links to previous documents and to web pages are preferred to simplify the sharing of information, and to avoid repetition of information.
Tests have been performed in partner premises and replicated in the FastWeb testplant when possible. In most cases, the actual deployment is up and running in real time, with a simple GUI that have been developed to showcase the results. All use cases have been successfully demonstrated during the final mPlane Workshop, held in Heidelberg, whose results are collected in D6.3 and D 7.6 in details.
This deliverable assesses the completion of the integrated prototypes of the mPlane pla tform (including the results of integration tests). In addition to this documentation, the final release of the mPlane library and software are also part of this deliverable.The details of the system architecture, element functions and interactions, use cases descriptions and integration plans can be found in previous mPlane deliverables available at the official mPlane website http://www.ict-mplane.eu/.
As a prelude to this deliverable, D5.1 contained a report about the project's data collection track record, D5.2 reported on the mPlane components to be included in the integration test plants, including a description of the mPlane SDK and the use cases to be realized in an integrated manner, and D5.3 described the requirements of the integrated mPlane prototype and test beds.
This document describes the requirements of the integrated mPlane prototype and test beds and it reports on the status of the deployment at the time of this writing.
In particular, details about the design and implementation of the target deployment scenario are described along with the progress of the integration activities. Details about the system architecture, element functions and interactions, use cases description and integration plans can be found in other mPlane deliverables available at the official mPlane website (http://www.ict-mplane.eu/).
In contrast to this deliverable, D5.1 contained a report about the project’s data collection track record and D5.2 reported on the mPlance components to be included in the integration test plants, including a description of the mPlane SDK and the use cases to be realized in an integrated manner.
Work Package 6 - Demonstration
This public deliverable provides a summary of all the demonstration events in which mPlane was demonstrated, outlining the impact obtained, and the feedback received from the respective communities and venues.
D6.3 v1 - Dec 2015: available here
This deliverable provides information to setup and demonstrate the different use cases. Its aim is not to faithfully reproduce all the fully-fledged mPlane demonstrations (shown at conferences, YouTube videos, etc.) but to provide a quick yet complete bootstrap guide to start replicating such demos.
Specifically, for each use case the deliverable provides guidelines in required hardware, mPlane software and their configuration, as well as instructions to setup and interact with the mPlane software.
This deliverable describes the design specification of the demo architecture and the organization of the demonstration for the specific use cases that are defined in WP1. The document also provides a complete description for the selected use-cases that includes the demo strategy, mapping of components and impairments into the demo architecture. Finally, for each use case involved, the demo execution flowchart specifies the process that would be used as a baseline for the demo.
Work Package 7 - Dissemination, Exploitation and Standardization
This deliverable reports the record of the final mPlane workshop held in Heidelberg, Germany. This deliverables also reports the final antology of articles and the patent proposals published within the project.
This deliverable describes the dissemination activities and standardization achievements of mPlane during the third year. The document provides detailed dissemination activities undertaken by the project, with the publication of a total of over 110 research papers to journals and various conferences and workshops since the start of the project. Furthermore, the document provides mPlane's standardization efforts, the establishment of links to various working groups and a tutorial to the international delegates. Finally, the document provides a detail report on the software open source release and covers the exploitation activities from all the partners.
D7.5 v1 - Dec 2015: available here
The mPlane project - just like any other applied research project - depends on a community, on people outside the project to help establish the technologies built and developed as part of the project. A key to building a community is dissemination, education and training. The main dissemination outlet of the project is scientific publications. In terms of education and training, the mPlane consortium has made use of a number of activities to foster awareness of mPlane concepts and technologies and to train people in the use of these technologies. This document summarizes these activities.
This deliverable describes the dissemination activities and standardization achievements of mPlane during the second year. The document provides detailed dissemination activities undertaken by the project, with the publication of a total ofover eighty research papers to journals and various conferences and workshops since the start of the project. Furthermore, the document provides mPlane's standardization efforts, the establishment of links to various working groups and a tutorial to the international delegates. Finally, the document provides a detail report on the software open source release and covers the exploitation activities from all the partners.
D7.3 v1 - Nov 2014: available here.
This deliverable outlines the plans for disseminating mPlane knowledge and describes the dissemination activities and standardization achievements of mPlane after the first year. The document provides detailed dissemination activities undertaken by the project through press release, fact sheet and posters, project website, and the publication of thirty research papers to journals and various conferences and workshops. Furthermore, the document provides mPlane's standardization efforts, with the publication of four IETF RFCs, the establishment of links to various working groups and a tutorial to the international delegates. Finally, the document provides a detail report on the software open source release and covers the exploitation planned in an early phase from all the partners.