Benchmark Transcoding

This guide provides steps to test the performance of your GPU(s) with the livepeer_bench benchmark transcoding tool provided with the Livepeer install.

The benchmarking tool is designed to:

• Simulate live-mode to measure transcoding as if connected to the network
• Gauge local transcoding capacity
• Provide output statistics to show real-time segment ratio and stream duration ratio
• Provide for adjusting configurations according to GPU capacity

​

Transcoding Performance

Overall transcoding performance on the network depends on the speed video is transcoded and uploaded/downloaded from an orchestrator.

The transcode time should be as low as possible. At the least, it should be lower than the total segment duration which is faster than real-time.

If you want to get an approximation of how many streams you can transcode simultaneously, increase the number of concurrent sessions with the -concurrentSessions flag to assess the total transcoding time.

​

Start Benchmarking

1. Download the test stream we provide for benchmarking transcoding:

wget -c https://storage.googleapis.com/lp_testharness_assets/bbb_1080p_30fps_1min_2sec_hls.tar.gz
tar -xvf bbb_1080p_30fps_1min_2sec_hls.tar.gz
ls bbb/   # Should print the stream *.ts segments and source.m3u8 manifest

2. Download the common output rendition configuration .json file that you can copy and save in the Livepeer folder as transcodingOptions.json.

3. Run livepeer_bench

The number and type of output renditions will impact benchmark results.

livepeer_bench \
    -in bbb/source.m3u8 \
    -transcodingOptions transcodingOptions.json \
    -nvidia <NVIDIA_GPU_IDs> # Only required for transcoding with GPUs
    -concurrentSessions <CONCURRENT_SESSIONS>

​

Flags

• The -nvidia flag is only required when transcoding with Nvidia GPUs. It is used to specify a comma-delimited string of Nvidia GPU IDs.

• The -concurrentSessions flag is used to specify the number of concurrent transcoding sessions. The default value is 1.

​

Livestreams

Livestreams are the most common type of workload on the network. By default, the flag is set to live mode -live=*true* so that the segments of the stream will be queued one-by-one as they arrive in a livestream.

If you want to queue all segments at once without any gaps in time, thus replicating a video-on-demand scenario, you can switch off live mode by changing the ‘live’ flag to =false.

For example:

livepeer_bench \
  -in bbb/source.m3u8 \
  -live=false

The first few lines of the output should display the source file, the output rendition profiles, and the number of concurrent sessions.

For example:

*---------------------*----------------------------------------------*
| Source File         | .../go-livepeer/bbb/source.m3u8              |
| Transcoding Options | P240p30fps16x9,P360p30fps16x9,P720p30fps16x9 |
| Concurrent Sessions | 1                                            |
| Live Mode           | true                                         |
*---------------------*----------------------------------------------*

The transcoding metrics will be output in CSV format per segment as each segment gets transcoded.

For example:

timestamp,session,segment,seg_dur,transcode_time
2021-03-12 00:21:29.1412,0,0,2,0.2545
2021-03-12 00:21:30.998,0,1,2,0.1107
2021-03-12 00:21:32.9816,0,2,2,0.09381
2021-03-12 00:21:34.9786,0,3,2,0.09031
2021-03-12 00:21:36.9806,0,4,2,0.09178
2021-03-12 00:21:38.9811,0,5,2,0.09216
2021-03-12 00:21:40.9831,0,6,2,0.09363
2021-03-12 00:21:42.9874,0,7,2,0.09746
2021-03-12 00:21:44.9885,0,8,2,0.09811
2021-03-12 00:21:46.9851,0,9,2,0.09412

When all the transcoding sessions end, metrics output will be generated.

• Real-Time Segs Ratio captures the number of segments transcoded in real-time.

• Real-Time Duration Ratio captures the total time taken to transcode all segments relative to the total duration of all source segments.

*------------------------------*---------------------*
| Concurrent Sessions          | 1                   |
| Total Segs Transcoded        | 10                  |
| Real-Time Segs Transcoded    | 10                  |
| * Real-Time Segs Ratio *     | 1                   |
| Total Source Duration        | 20s                 |
| Total Transcoding Duration   | 1.1165546499999999s |
| * Real-Time Duration Ratio * | 0.05583             |
*------------------------------*---------------------*

You can export the per-segment CSV data to an output.csv file to analyze it with other tools:

livepeer_bench \
    -in bbb/source.m3u8 \
    -transcodingOptions transcodingOptions.json \
    -nvidia <NVIDIA_GPU_IDs> # Only required for transcoding with GPUs
    -concurrentSessions <CONCURRENT_SESSIONS> > output.csv