youtube-probe

The probe module downloads a YouTube video (specified by the 11-character video_id), performs a lightweight playback emulation (frame header parsing), and calculates the following metrics:

 The probe terminates playback emulation when the download is complete, i.e. usually it finishes sooner than the video duration (the probe supports bandwidth throttling via the underlying CURL library).

The probe queries the FLV (YouTube format code 5) version version of the video, as it is the only format all YouTube videos are available in.  Extending the probe to support the new DASH format promoted by YouTube is underway. Table Comparison of YouTube media encoding options on Wikipedia describes more details on the FLV  and other YouTube video formats.

The software contains an FLV processing module that originates from the public domain flvlib, ported to python3 and enhanced by NETvisor.

youtube-probe is available on github, temporarily as a branch of the mPlane protocol Reference Implementation project.

Quick start:

First, check and satisfy the following software dependencies youtube-probe relies on:

• pyCURL module (ver. 7.19.3.1 tested) is used to download the video content
• youtube-dl (ver. 2014.04.04.7 tested) python module and CLI is used to obtain the URL for the YouTube video id
• all dependencies required by the mPlane Reference Implementation framework
  • pyton3.3 with the following modules:
    • pyyaml 
    • tornado

Testing the probe natively (not via the mPlane interface)

The yp-test utility is provided for testing the probe directly, providing a very simple CLI:

Usage: yp-test.py [<video-id>=riyXuGoqJlY]

YouTube Download Test Client

Options:
 -h, --help show this help message and exit
 -b BWLIMIT, --bwlimit=BWLIMIT
 limit download bandwidth to BW kBps
 -v, --verbose be more verbose, each -v increases verbosity

A test run for a sample video with bandwidth throttling:

(mplane)tiv@mplane:~/youtube-probe$ python yp-test.py -b 25000 EJQsBL6vEk4
[INFO] 0.007 Limiting pycurl bandwidth to 25000
[INFO] 0.007 YouTubeClient of video_id EJQsBL6vEk4, metrics: {'delay.srvresponse.ms': 0, 'octets.layer7': 0, 
'bandwidth.min.bps': 0, 'rebuffer.events': 0, 'bandwidth.avg.bps': 0, 'bandwidth.max.bps': 0, 'delay.buffering.ms': 0, 
'delay.urlresolve.ms': 0, 'delay.download.ms': 0}
[INFO] 0.008 Query for media URL
[INFO] 1.572 URL extacted, starting download
[INFO] 1.829 Player starts BUFFERING
[INFO] 3.867 3 seconds of media buffered, starting playout
[INFO] 3.867 Player starts PLAYING, buffered: 3.971 secs
[INFO] 62.092 Player stalled at 58.225 secs of media
[INFO] 62.092 Player starts REBUFFERING
[INFO] 63.227 3 seconds of media buffered, starting playout
[INFO] 63.227 Player resumes PLAYING, buffered: 3.058 secs
[INFO] 102.467 Player stalled at 97.464 secs of media
[INFO] 102.467 Player starts REBUFFERING
[INFO] 104.815 3 seconds of media buffered, starting playout
[INFO] 104.815 Player resumes PLAYING, buffered: 3.937 secs
[INFO] 158.336 Player stalled at 150.984 secs of media
[INFO] 158.336 Player starts REBUFFERING
[INFO] 159.608 3 seconds of media buffered, starting playout
[INFO] 159.608 Player resumes PLAYING, buffered: 3.165 secs
[INFO] 196.646 Player stalled at 188.021 secs of media
[INFO] 196.646 Player starts REBUFFERING
[INFO] 199.165 3 seconds of media buffered, starting playout
[INFO] 199.165 Player resumes PLAYING, buffered: 3.938 secs
[INFO] 224.015 Download finished
Done: YouTubeClient of video_id EJQsBL6vEk4, metrics: {'delay.srvresponse.ms': 256.65926933288574, 'octets.layer7': 5542407, 
'bandwidth.min.bps': 0.0, 'rebuffer.events': 4, 'bandwidth.avg.bps': 199328.9885519995, 'bandwidth.max.bps': 393216.0, 
'delay.buffering.ms': 9.3124361038208, 'delay.urlresolve.ms': 1564.445972442627, 'delay.download.ms': 222442.58761405945}

New features supported by the mPlane project

The probe was developed entirely within the mPlane project.

mPlane proxy interface

youtube-probe supports the native mPlane probe interface

Official version

• May 15th, 2014, frozen release for D2.2 [tarball] 
• Development version available on github at https://github.com/fp7mplane/protocol-ri/tree/youtube-probe