Running a Video Streaming Performance Test with Blazemeter and UbikLoadPack

In this blog, we will see how to run a Video Streaming Performance Test on a server delivering one of the following formats:

Apple HLS
Mpeg-Dash
Microsoft Smooth (HSS)
Adobe HDS with

We’ll use the following tools:

Pre-requisites

To run what is described below:

Install Java 8
Request a trial version of UbikLoadPack Video Streaming Plugin, you’ll receive a mail with an attached trial license and a link to download the bundle.
Clone our repository:
- git clone https://github.com/ubikloadpack/videostreaming_taurus_demo.git
cd into the folder videostreaming_taurus_demo/blazemeter

The JMeter test plan

We’ll use a very simple test plan :

Note that we don’t configure anything in Thread Group regarding Users Arrival and schedule as it will be done in Taurus YAML file

In UbikLoadPack streaming sampler we just set

Playlist URL to the HLS manifest URL (we use a property MANIFEST_URL to be able to modify it from outside of JMeter, it defaults to our Mock server “http://localhost:8080/hls/live/test.m3u8“)
the duration we want to read (we also use a property DURATION_WATCHED with default value of 20 seconds here)

Finally note we setup a Simple Data Writer (right click on Test Plan > Add > Listener > Simple Data Writer) to output UbikLoadPack additional metrics :

Taurus Setup

The setup is done in YAML syntax, we’ll call the file bzm-ubikloadpack-vs.yml

We first configure how test will be executed, here we’ll run:

the existing JMX file called ulp_video_streaming_demo.jmx
with 3 Virtual Users (as trial is limited to this number of threads) but you can run as many virtual users as your license allows it
ramping up from 1 to 3 in 30 seconds
holding the load for 5 minutes
running from us-east-1

execution:
 scenario: UbikLoadPack-Demo
 concurrency: 3
 hold-for: 5m
 ramp-up: 30s
 locations:
 us-east-1: 1 

scenarios:
   UbikLoadPack-Demo:
     script: ulp_video_streaming_demo.jmx

We then tell Taurus which JMeter version to use, note we’ll use JMeter 5.0 as we want to use new sexy features mainly the ability to graph over time additional metric, we need this feature to graph UbikLoadPack custom video metrics

We configure it this way:

settings:
   env:
      # A Version higher or equal to 5.0 is required
      JMETER_VERSION: "5.0"

Finally we set some properties specific to our test:

modules:
   jmeter:
     detect-plugins: false
     properties:
       # Change this for the URL of your manifest
       #MANIFEST_URL: 
       # Duration of video watched in seconds
       DURATION_WATCHED: 10
       jmeter.reportgenerator.report_title: UbikLoadPack Video Streaming Demo
       jmeter.reportgenerator.exporter.html.series_filter: ^(ULP_PLAYER)(-success|-failure)?$
       jmeter.reportgenerator.overall_granularity: 1000
       ULP.videostreaming.licensepath: ubik-streaming-plugin.license
       sample_variables: ulp_buffer_fill,ulp_lag_time,ulp_play_time,ulp_lag_ratio,ulp_dwn_time,ulp_hits,ulp_avg_chunk_time,ulp_avg_manifest_time
       output: ${TAURUS_ARTIFACTS_DIR}/

The whole YAML file can be downloaded from github project.

Note that in folder blazemeter, we have:

ulp_video_streaming_demo.jmx : The JMeter Test plan
bzm-ubikloadpack-vs.yml : The Taurus YAML file
saveservice.properties : This file is modified locally by the plugin

Testing your own Video Streaming servers

If you want to test your own servers, just replace in YAML file the MANIFEST_URL property by your manifest URL, for example:

MANIFEST_URL: https://bitmovin-a.akamaihd.net/content/sintel/sintel.mpd

Creation Steps on Blazemeter

Select Performance Test:

Click on Upload Script:

Upload the files in blazemeter folder that we showed previously and add the 2 below files that are available when you request a trial:

ubik-streaming-plugin.license (attached to the trial mail)
ubik-jmeter-videostreaming-plugin-X.Y.Z.jar (you’ll receive in trial mail the link to download the zip bundle, the jar is located in lib/ext folder)

You end up with this:

Then click on Run Test:

A popup opens, click on Launch Servers:

You’ll then have to wait for test to start:

The test then starts and you can see graphs appear to monitor your test:

You can select the KPI you want to watch:

Once test is finished, go to Logs tab where you can download artifacts by clicking on artifacts.zip:

Note that the HTML report of JMeter has also been generated as part of those artifacts, and contains the UbikLoadPack custom metrics graphed over time:

Report:

At the end, we have a JMeter HTML report generated with custom UbikLoadPack report in the report folder.

Average Chunk Response time

Average Manifest Response time

Lag Ratio over time:

Buffer Fill over time

Hits over time

Try it:

Request a trial version HERE.
Ask for a turnkey or consulting service at ubik-load-pack-sales@ubik-ingenierie.com.

L’article Running a Video Streaming Performance Test with Blazemeter and UbikLoadPack est apparu en premier sur Ubik Ingénierie.

Running a Video Streaming Performance Test with Blazemeter and UbikLoadPack