OTT probe

Developers, contributors

Developer: NETVISOR, H-1119 Budapest, Petzvál József utca 56., Hungary

Contributing partner(s): FW, Via Caracciolo 51, 20155 Milano – Italy

Description

OTT Probe is an active probe to verify access "over-the-top" media content, i.e. various forms of live and recorded video transport over the Internet. It is designed to run from typical customer premises, like residential and business locations with broadband Internet access.

OTT Probe can access content offered in various popular adaptive streaming formats like Apple HLS, Microsoft Smooth Streaming and MPEG-DASH. Also OTT probe will access content in major proprietary formats like Youtube videos.

When accessing content OTT Probe basically emulates the behaviour of a standard media player application, i.e. it downloads content progressively, in synch with the relative presentation time. However, compared to media players the probe uses much less resources (no decoding in most cases), and measures a a number of quality parameters and attributes on the download process, which are offered as mPlane capabilities to its clients.

OTT Probe provides various strategies to access content which is available in multiple qualities for adaptive download: parallel download of all qualities, sequential shuffling between qualities, or the truly adaptive download.

The protocols used by the probe are listed below.

Transport protocols

• HTTP and HTTPS (ports depending on content server configuration)

Streaming formats over transport

• HLS (Apple), MSS (Microsoft), MPEG-DASH, Youtube/MP4 (Q2 2015)

Usage scenarios

OTT Probes are to be used in either of two typical locations

• At some subscriber premises, where the probe downloads only one or a few streams at a time. This scenario is used for "End user experience" testing, where access issues or bottlenecks are a primary reason for non-perfect results.
• In some provider's premises, where a probe of larger capacity downloads dozens or hundreds of streams in parallel. If the provider is the service provider, the probes are simply used to verify that the components of the content are available and downloadable form the server. If the provider is an ISP, the focus may be on checking the availability of the content CDN from the ISP network.

Metrics and Capabilities

Metrics

The OTT probe offers a single capability, which measures access to some content specified by source.url parameter. The other parameter, source.ip4is used to define the source address OTT probe measures from. The maximum frequency of downloads is 10s, i.e. this is the minimum length of a measurement cycle that produces results.

Return values:

• time: time of measurement
• bandwidth.nominal.kbps: the highest bandwidth of the best quality accessed during the period
• http.code.max: the maximum value of the http response codes
• http.redirectcount.max: the maximum value of the redirect counts measured during the period
• qos.manifest: a 0-100 value representing the relative quality of the manifest downloads and processing
• qos.content: a 0-100 value representing the relative quality of the content download
• qos.aggregate: a 0-100 value representing the average of relative quality of the manifest downloads and processing, and the content download, respectively
• qos.level: overall service level, based on the above metrics

Capabilities

    {
      "capability": "measure",
      "label": "ott-download",
      "parameters": {
        "content.url": "*",
        "source.ip4": "192.168.96.1"
      },
      "registry": "http://ict-mplane.eu/registry/core",
      "results": [
        "time",
        "bandwidth.nominal.kbps",
        "http.code.max",
        "http.redirectcount.max",
        "qos.manifest",
        "qos.content",
        "qos.aggregate",
        "qos.level"
      ],
      "token": "5915d71a91e46b7070298bc23e813b80",
      "version": 1,
      "when": "now ... future / 10s"
    }

Probe execution environment

OTT probes require a Python3 environment (to run the proxy derived from RI/SDK), and a binary component (aka standalone probe) that actually executes the downloads.

The binary is compiled for x86 Linux machines (with glibc), and for MiniProbes (MIPS32/uclibc). Should compile on other architectures as well. libcurl, libpthread and boost/program­_options are required for compilation and execution.

Quick start

Installation of standalone probe

Prerequisites

Required libraries for the C++ module:

• libcurl.so.4
• boost_program_options
• libpthread
• libz
• libssl
• libreactor - published by NETvisor Ltd.
• probe-ott - published by NETvisor Ltd.

The published modules can be downloaded from ftp://ftp.netvisor.hu/outgoing/mplane/mplane-ottmodule.tar.gz. To speed up testing there is no need to compile it from source as it is already avaible for various platforms:

• ar71xx (tested on OpenWRT 12.04)
• x86_64 (tested on Ubuntu 14.04)
• i386 (tested on CentOS release 6.5)

If there is any problem (or a new platform is requested) please contact tufa@netvisor.hu.

Installation

Copy probe-ott to your PATH ( /usr/bin ) and add libreactor to LD_LIBRARY_PATH. The easiest way to check that all the libraries are installed is to run the object file:

$ probe-ott
the option '--slot' is required but missing

If it fails with the aforementioned error, the measurement module is configured well.

Usage of standalone probe

probe-ott --slot <n> --url <url> [ --buffLen <n> --httpTimeout <n> 
    --protocol < HLS | IIS | MPEG-DASH > --downloadMode < 0 | 1 | 2 > --qualityLevel < n >

where

• slot - the slot used to send the measurements to
• url - manifest to download
• buffLen - buffer length in seconds
• httpTimeout - transaction timeout for the full HTTP transaction in millisecs
• protocol - protocol to use. Could be HLS, IIS or MPEG-DASH. If not defined, it tries to guess
• downloadMode - possible values are 0 (default): for adaptive download, 1: for fixed level download, 2: parallel download
• qualityLevel - when downloadMode is 1, qualityLevel defines the download level

Integration into an MPlane environment

The probe implements the Component Initiated mPlane workflow. The proxy is developed from the mPlane RI and supports the mPlane HTTP API with or without https. It is tested with the mPlane RI standard supervisor (from the "master" branch).

In the following guide we use the following legend:

<ott-probe_installdir> the directory where the OTT-probe binary (probe-ott) has been installed (eg. /usr/bin)

<protocol-ri_dir> the directory where the GitHub repository of the mPlane protocol reference implementation has been cloned to (eg./home/<mplane_user>/protocol-ri)

<components_dir> the directory where the GitHub repository of the components (this repository) has been cloned to (eg./home/<mplane_user>/components).

Put <ott-probe_installdir> into the PATH variable, eg.

export PATH=$PATH:<ott-probe_installdir>

Put <protocol-ri_dir> into the PYTHONPATH variable, eg.

export PYTHONPATH=$PYTHONPATH:<protocol-ri_dir>

Copy the ott-probe interface stuff (<components_dir>\ott-probe folder) into <protocol-ri_dir>/mplane/components. This directory should contains the followings:

• ott-registry.json The registry.json file containing the needed extensions for the OTT-probe (core registry is included within). It can be also downloaded from http://tufaweb.netvisor.hu/mplane/ott-probe/ott-registry.json. This is an extended version of the core JSON file, with all the needed definitions for ott-probe.
• ott.py The Python interface
• supervisor.conf,client.conf, component.conf The config files for running OTT in the component framework
• ott-capabilities The capabilities file, does not needed for installation
• README.md The installation guide.

Adjust the parameters in the *.conf files if needed (e.g. path to certificates, supervisor address, client port and address, roles, etc).

In a terminal window start supervisor:

root@h139-40:~/protocol-ri# test2@h139-40:~/protocol-ri$ scripts/mpsup --config mplane/components/ott-probe/supervisor.conf
ListenerHttpComponent running on port 8890

In another terminal start the OTT probe as a component and check if communication is established and the probe capabilities are registered with the supervisor:

test2@h139-40:~/protocol-ri$ scripts/mpcom --config mplane/components/ott-probe/component.conf
Added <Service for <capability: measure (ott-download) when now ... future / 10s token b77698ec schema 33a0f637 p/m/r 2/0/8>>
Added <Service for <capability: measure (ping-average-ip4) when now ... future / 1s token a74fabd1 schema e2ca42e6 p/m/r 2/0/4>>
Added <Service for <capability: measure (ping-detail-ip4) when now ... future / 1s token 75cd8c84 schema db8ef547 p/m/r 2/0/2>>

Capability registration outcome:
ping-detail-ip4: Ok
ping-average-ip4: Ok
ott-download: Ok
callback: Ok

Checking for Specifications...

Now we can start a clientshell in a third window to test the measurement functionalities:

test2@h139-40:~/protocol-ri$ scripts/mpcli --config mplane/components/ott-probe/client.conf
ok
mPlane client shell (rev 20.1.2015, sdk branch)
Type help or ? to list commands. ^D to exit.

|mplane|

Now check that capabilities are registered and run a measurement:

|mplane| listcap
Capability ott-download (token b77698ecef311f599940612f51ac7e27)
Capability ping-average-ip4 (token a74fabd15ef8bbaaf68eb106a6c1c54e)
Capability ping-detail-ip4 (token 75cd8c844dce30a09c579d9fc89caa3d)
|mplane| runcap ott-download
|when| = now + 30s / 10s
content.url = http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8
source.ip4 = 127.0.0.1
ok
|mplane| listmeas
Receipt ott-download-0 (token b5133e57c985a7194cb51caebf552bc9): now + 30s / 10s
|mplane|

During this we should see something similar in the component window, showing the launched application and than the receipt:

<Service for <capability: measure (ott-download) when now ... future / 10s token 0a01b50e schema 503f24c6 p/m/r 2/0/8>> matches <specification: measure (ott-download-0) when now + 30s / 10s token 0a4f3eac schema 503f24c6 p(v)/m/r 2(2)/0/8>
Will interrupt <Job for <specification: measure (ott-download-0) when now + 30s / 10s token 0a4f3eac schema 503f24c6 p(v)/m/r 2(2)/0/8>> after 30.0 sec
Scheduling <Job for <specification: measure (ott-download-0) when now + 30s / 10s token 0a4f3eac schema 503f24c6 p(v)/m/r 2(2)/0/8>> immediately

2015-06-10 10:58:33.906406: running probe-ott --slot -1 --mplane 10 --url http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8Returning <receipt:  (ott-download-0)0a4f3eacabd93030d32da1ede87875f4>

After a while we can notice the result notification in the component window as the measurement has been finished:

<result: measure (ott-download-0) when 2015-06-10 10:58:34.969026 token 0a4f3eac schema 503f24c6 p/m/r(r) 2/0/8(1)>
Result for ott-download-0 successfully returned!

We can now check the results in the client window:

|mplane| listmeas
Receipt ott-download-0 (token b5133e57c985a7194cb51caebf552bc9): now + 30s / 10s
|mplane| listmeas
Result  ott-download-0 (token b5133e57c985a7194cb51caebf552bc9): 2015-06-10 10:58:34.969026
|mplane| showmeas ott-download-0
result: measure
    label       : ott-download-0
    token       : b5133e57c985a7194cb51caebf552bc9
    when        : 2015-06-10 10:58:34.969026
    parameters  ( 2):
                             content.url: http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8
                              source.ip4: 127.0.0.1
    resultvalues( 1):
          result 0:
                                        time: 2015-06-10 10:58:34.969026
                      bandwidth.nominal.kbps: 720
                               http.code.max: 200
                      http.redirectcount.max: 0
                                qos.manifest: 100
                                 qos.content: 100
                               qos.aggregate: 100
                                   qos.level: 100

|mplane|

Official version

• June 15th, 2015 - frozen release for D2.3 on mPlane SVN or in the tarball file.

• Latest version is available on GitHub.
  

New features supported by the mPlane project

• Python based proxy for the mPlane RI SDK to enable its usage in mPlane Reference Implementation's distributed measurement environment.

Changes since D2.2

• Ported to new reference implementation version (master branch)
• Minor bugfixes

References

Links to sources, binaries

• Official link on mPlane website  
• Description on compilation and usage  
• Standalone probe binaries  
• Source code for mPlane proxy ('master' branch at 'components' repository) 

Links to additional documentation

• iTVSense (OTT probe product line) - Solution Brief
• "iTVSense Probes Technologies" - Product Brief 

Dissemination

• OTT probe demo video