The mobile probe stores data in a mongoDB database in json format.
For this purpose a mongoDB to mPlane reference implemenation proxy has been created.
The proxy is not specific to the mobile proxy and, therefore, it can be configureed to extract data from any mongoDB schema.
The repository consists of a MongoDB database and a node.js interface. The Mobile Probe sends HTTP requests from the device to the server with the aggregated measurements. The node.js script is in turn responsible for receiving, parsing and storing the incoming data to the database. Each measurement is inserted with fields such as timestamp, the device's unique ID and IP address. In this way, the repository can handle concurrent reports from multiple devices. As can be seen in the provided sample below, a report contains the required performance metrics from the different layers that the probe is monitoring.
{
"report":{
"measurementsArray":[
{
"timestamp":"2015-03-31 13:19:57.608",
"LOCATION":{
"timeStamp":1427800797607,
"provider":"NO_LOCATION",
"measurementType":"location",
"contentType":"Measurement"
},
"extras":"mplane",
"deviceID":"352605059221028",
"ip":"83.53.30.1",
"version":"2"
},
{
"CELL_INFO":{
"measurementReason":"onSignalStrengthsChanged",
"networkOperatorName":"",
"currentCellLocationCID":125067479,
"timeStamp":1427800798013,
"measurementType":"cellInfo",
"contentType":"Measurement",
"neighborCellInfo":[],
"currentCellLocationLAC":864,
"dataActivity":,
"networkOperatorCountryCode":"es",
"latestGSMBitError":99,
"currentCellLocation":"[864,125067479,-1]",
"getNetworkType":3,
"currentCellLocationPSC":-1,
"dataState":,
"latestGSMSignalStrength":30,
"networkOperatorCode":"21407"
},
"timestamp":"2015-03-31 13:19:58.050",
"extras":"mplane",
"deviceID":"352605059221028",
"ip":"83.53.30.1",
"version":"2"
},
{
"timestamp":"2015-03-31 13:19:58.851",
"HW":{
"network_type":"WiFi",
"wifi_RSSI":-37,
"Cpu Tot: ":1,
"MemFree":"20908"
},
"extras":"mplane",
"deviceID":"352605059221028",
"ip":"83.53.30.1",
"version":"2"
},
{
"timestamp":"2015-03-31 14:54:55.716",
"LogCat":"I/AwesomePlayer( 1854): setDataSource_l(URL suppressed)",
"version":"2",
"deviceID":"358848043406974",
"ip":"83.53.30.1",
"extras":"mplane"
},
{
"timestamp":"2015-03-31 14:54:56.546",
"LogCat":"I/AwesomePlayer( 1854): Could not offload audio decode, try pcm offload",
"version":"2",
"deviceID":"358848043406974",
"ip":"83.53.30.1",
"extras":"mplane"
},
{
"tcp_complete":{
"c_sack_opt":"1",
"s_rtt_avg":"0.914270",
"s_cwin_max":"59856",
"c_win_min":"14600",
"s_last":"11838.654000",
"s_pkts_dup":"0",
"s_rtt_cnt":"965",
"s_rtt_max":"36.791000",
"c_pkts_unk":"0",
"c_bytes_all":"16916",
"s_pkts_data":"2522",
"s_pkts_unrto":"0",
"c_pkts_reor":"0",
"s_syn_retx":"0",
"c_win_max":"522880",
"p2p_st":"0",
"ed2k_c2s":"0",
"c_pkts_push":"17",
"c_ssl":"r3---sn-h5q7ened.googlevideo.com",
"c_pkts_rto":"0",
"c_rtt_cnt":"18",
"s_sack_opt":"1",
"ed2k_c2c":"0",
"c_bytes_retx":"0",
"c_mss":"1460",
"c_bytes_uniq":"16916",
"ed2k_sig":"0",
"s_bytes_all":"3481036",
"c_pkts_data":"17",
"c_cwin_min":"235",
"s_pkts_ooo":"19",
"c_pkts_retx":"0",
"c_rtt_std":"60.187121",
"durat":"41905.741000",
"c_win_scl":"6",
"c_rtt_min":"21.642000",
"s_pkts_fc":"0",
"http_t":"0",
"c_f1323_opt":"1",
"c_rtt_max":"187.524000",
"s_ssl":"*.c.docs.google.com",
"s_first_ack":"218.297000",
"ed2k_chat":"0",
"s_rst_cnt":"0",
"c_ack_cnt_p":"1195",
"first_abs":"1427836180809.872070",
"s_win_0":"0",
"s_pkts_fs":"17",
"s_rtt_min":"0.051000",
"c_cwin_max":"1093",
"s_cwin_ini":"1440",
"c_rtt_avg":"73.632758",
"s_bytes_retx":"0",
"s_mss":"1460",
"c_ip":"192.168.1.243",
"c_pkts_unfs":"0",
"s_win_scl":"7",
"last":"249797.323000",
"c_pkts_dup":"0",
"c_pkts_unrto":"0",
"s_fin_cnt":"1",
"s_pkts_unfs":"0",
"c_sack_cnt":"235",
"s_rtt_std":"2.552807",
"s_win_max":"63872",
"s_ack_cnt":"2529",
"s_pkts_retx":"0",
"s_win_min":"28960",
"c_mss_min":"235",
"s_port":"443",
"s_pkts_unk":"0",
"s_ttl_max":"57",
"s_pkts_rto":"2",
"s_ack_cnt_p":"5",
"c_port":"40886",
"c_isint":"1",
"s_sack_cnt":"0",
"c_ttl_max":"64",
"c_tm_opt":"1",
"s_pkts_all":"2529",
"s_pkts_push":"44",
"c_ttl_min":"64",
"c_pkts_all":"1213",
"c_mss_max":"1093",
"s_f1323_opt":"1",
"c_syn_retx":"0",
"c_last":"11217.761000",
"s_isint":"0",
"c_pkts_fs":"0",
"s_mss_max":"1440",
"s_mss_min":"51",
"c_first_ack":"22.055000",
"s_pkts_reor":"0",
"ed2k_data":"0",
"c_pkts_fc":"0",
"c_syn_cnt":"1",
"c_fin_cnt":"0",
"c_cwin_ini":"235",
"s_syn_cnt":"1",
"c_pkts_ooo":"0",
"c_first":"30.773000",
"s_first":"218.529000",
"con_t":"8192",
"s_ip":"74.125.168.8",
"p2p_t":"0",
"c_rst_cnt":"0",
"c_win_0":"0",
"s_tm_opt":"1",
"s_cwin_min":"51",
"s_ttl_min":"53",
"first":"207891.582000",
"c_ack_cnt":"1212",
"s_bytes_uniq":"3481036"
},
"timestamp":"2015-03-31 22:09:40.809",
"version":"2",
"deviceID":"358848047597851",
"ip":"83.53.30.1",
"extras":"mplane"
}
]
}
}
The proxy can be found here: https://github.com/fp7mplane/components/blob/master/mongoDB-mobileProbe/(link is external)
The proxy Interface is written in python and it sits on top of MongoDB. It exports the collections of mongoDB as capabilities using the Reference Implementation. Afterwards, the mPlane clients can pull the requested data.
The mongoDB proxy can be used by any component that has data stored in a mongoDB database. For details on how to install and run the probe visit the github help file:
https://github.com/fp7mplane/components/blob/master/mongoDB-mobileProbe/(link is external)
Copy files from the mPlane interface (from the components GitHub) into protocol-ri/: - mobile_probe_settings.json
The registry.json file, copy it into protocol-ri/mplane/components/. - mongo.py
The Python interface, copy it into protocol-ri/mplane/components/. - supervisor.conf
, component.conf
and client.conf
The configuration files, copy them into protocol-ri/conf/.
The mongoDB proxy can be used by any component that has data stored in a mongoDB database. The configuration is given at a json file of the following format.
{
"measurements":{
"mobile_probe_hardware":{
"collection":"mobileMeasurements",
"search":"HW",
"return":{
"observer.link":"HW.network_type",
"snr":"HW.wifi_RSSI",
"cpuload":"HW.Cpu Tot: ",
"memload":"HW.MemFree"
}
},
"mobile_probe_cellInfo":{
"collection":"mobileMeasurements",
"search":"CELL_INFO",
"return":{
"snr":"lCELL_INFO.atestGSMSignalStrength",
"observer.link":"CELL_INFO.currentCellLocation"
}
},
"mobile_probe_videoLog":{
"collection":"mobileMeasurements",
"capability_name":"mobile_probe_videoLog",
"search":"LogCat",
"return":{
"measurement.identifier":"LogCat"
}
}
},
"collection":"mobileMeasurements",
"db":"mPlane",
"timestamp_field":"date",
"source_filed":"deviceID"
}
The settings above will pull data from db mplane and collection mobileMeasurements. The field "devicID" will be used as the source field when making a query for a specific device. Furthermore, the timestamp field is "date"
The measurements dictionary contains information about the offered capabilities and the association of the return values between the mongoDB database and the mPlane reference implemenation. For instance, the mongoDB entryHW.wifi_RSSI
will be returned as the mPlane's RI snr
field.
The settings above will generate three capabilities.
query (mobile_probe_hardware)
query (mobile_probe_videoLog)
query (mobile_probe_cellInfo)
After installing the RI, in a terminal window start supervisor:
cd ~/protocol-ri
python3 -m scripts/mpsup --config ./conf/supervisor.conf
alternatively:
cd ~/protocol-ri
python3 -m mplane.supervisor --config ./conf/supervisor.conf
In another terminal start the mobile probe as a component:
cd ~/protocol-ri
python3 -m scripts/mpcom --config ./conf/component.conf
The expected output for this example should be:
...
Capability registration outcome:
mobile_probe_hardware: Ok
mobile_probe_videoLog: Ok
mobile_probe_cellInfo: Ok
Checking for Specifications...
You need to have mongoDB setup and running on your machine. You should also have the settings.json configured in order to access and translate the mongoDB field into the mPlanes RI entries (see above).
Start a client to test the component:
mpcli --config ./conf/client.conf
The expected output should be:
ok
mPlane client shell (rev 20.1.2015, sdk branch)
Type help or ? to list commands. ^D to exit.
Now check that the mongoDB capabilities is registered:
|mplane| listcap
Capability mobile_probe_cellInfo (token cd6b5fa2dc4ddfd3d7c4f16898bf1b60)
Capability mobile_probe_hardware (token f3a12891623783d20d4f63f01e86104d)
Capability mobile_probe_videoLog (token 57a0a9484788520090bf3e1c97aa6df8)
And run a test capability. In this example we ask for any mongoDB hardware entries for mobile
device with IMEI 352605059221028 between 2013-09-20 and 2015-5-5:
|mplane| runcap mobile_probe_hardware
|when| = 2013-09-20 ... 2015-5-5
source.device = 352605059221028
ok
If everything works ok the data should be pulled from the mongoDB database and transformed into an mPlane result:
|mplane| listmeas
Result mobile_probe_hardware-0 (token 6ed23ada2c8921f7cf9de917402a4283):
2015-03-31 11:19:58.851000 ... 2015-03-31 11:20:04.377000
You can now access the measurments:
|mplane| showmeas 6ed23ada2c8921f7cf9de917402a4283
result: query
label : mobile_probe_hardware-0
token : 6ed23ada2c8921f7cf9de917402a4283
when : 2015-03-31 11:19:58.851000 ... 2015-03-31 11:20:04.377000
registry : http://ict-mplane.eu/registry/demo
parameters ( 1):
source.device: 352605059221028
metadata ( 1):
System_version: 1.0
resultvalues( 2):
result 0:
cpuload: 1.0
memload: 20908.0
observer.link: WiFi
snr: -37.0
result 1:
cpuload: 0.6805555820465088
memload: 44084.0
observer.link: WiFi
snr: -39.0