HRD Emulator in HTML5

Just like MPEG-2 video uses VBV (Video Buffer Verifier), H.264 standard uses HRD (Hypothetical Reference Decoder) to define correctness of streams.

One of the verification provided by HRD is CPB buffer fullness. The idea is simple; feed H.264 stream to HRD and monitor the CPB fullness. Make sure the fullness never goes negative nor above the CPB buffer size. The importance of this buffer fullness requirement was explained here.

HRD defines the CPB fullness in somewhat counter intuitive way. To make it easy, I wrote an interactive HRD emulator in HTML5.

HRD Emulator (CBR)

The blue line the number of bits entered to HRD over time. The input rate is constant in CBR mode, i.e. the blue line is straight. The red line is the number of bits removed (= Decoded) from CPB. Thus, the CPB fullness is the blue line minus the red line for a given time.

Let’s select the VBR in Rate Control Mode.

HRD Emulator (VBR)

HRD Emulator (VBR)

The blue line now has some periods where the input rate is zero. H.264 standard uses init_cpb_removal_delay to control the period; Bits for a picture N can enter HRD only after the picture decode time minus init_cpb_removal_delay.

To see this rule more clearly, check “Show init_cpb_removal_delay” option. When the last bit of picture N – 1 arrives before the picture N decoding time minus init_cpb_removal_delay, the input rate goes down to zero.

HRD (VBR) with init_cpb_removal_delay

HRD (VBR) with init_cpb_removal_delay

The value of init_cpb_removal_delay also determines the initial decoding delay. It’s currently set to 50 ms. It means the first picture is decoded 50 ms after the first bit of the picture enters HRD.

The initial decoding delay determines the channel switching delay as discussed here. Can we reduce it to make channel switching faster?

HRD underflow due to the short initial delay

Not really. As shown above, reducing the value of init_cpb_removal_delay resulted in the underflow error (the red line crosses the blue line).

H.264 standard has introduced another parameter to allow the reduction of initial decoding delay at the expense of increased average bit rate. Here we have init_cpb_removal_delay_offset parameter. It’s added to init_cpb_removal_delay for every picture except the first one.

HRD with non-zero init_cpb_removal_delay_offset

H.264 HRD provides these parameters to encoders so that they can choose optimal bit stream configuration for each target devices.

Here are HTML and Java Script of the HRD emulator above.

<html>
  <head>
    <title>HRD Emulator</title>
    <script src="./script.js" type="text/javascript"></script>
    
    <style type="text/css">
      #plot {
        float : left
      }
      #control-container {
        float : left
      }
      #widget {
        float : right;
        width : 10em
      }
      #heading {
        float : left
      }
    </style>    
  </head>
  <body onLoad="init()">
    <div id="plot">
      <canvas id="canvas" width="800" height="480"></canvas>
    </div>

    <div id="control-container">
      <div class="control">
        <div class="heading">HRD bit_rate [<span id="showBitRate"> </span> kbps]</div>
        <div class="widget">
          <input id="bitRate" onChange="update()" type="range" min="0" max="2000" value="500"/>
        </div>
      </div>
      <div class="control">
        <div class="heading">initial_cpb_removal_delay [<span id="showICRD"> </span> ms]</div>
        <div>
          <input id="initial_cpb_removal_delay" onChange="update()" type="range" min="0" max="100" value="50"/>
        </div>
      </div>
      <div class="control">
        <div class="heading">initial_cpb_removal_delay_offset [<span id="showICRDO"> </span> ms]</div>
        <div>
          <input id="initial_cpb_removal_delay_offset" onChange="update()" type="range" min="0" max="100" value="0"/>
        </div>
      </div>
      <div class="control">
        <div>Rate Control Mode</div>
        <div>
          <input type="radio" name="mode" value="cbr" onClick="setMode('cbr'); update()" checked="checked" />CBR
        </div>
        <div>
          <input type="radio" name="mode" value="vbr" onClick="setMode('vbr');  update()" />VBR
        </div>
      </div>
      <div class="control">
        <div>
          <input type="checkbox" id="isICRDVisible" onClick="update()"/>Show init_cpb_removal_delay
        </div>
      </div>
      <div class="control">
        <div>
          <input type="checkbox" id="isICRDOVisible" onClick="update()"/>Show init_cpb_removal_delay_offset
        </div>
      </div>
      <div class="control">
        <input id="reset" type="button" value="Redraw Graph" onClick="resetStream(); update();"/>
      </div>
    </div>
  </body>
</html>
// Save this as "script.js"
var canvas;
var picture = new Array();
var pictureDuration                  = 900900;
var initial_cpb_removal_delay        = 900900;
var initial_cpb_removal_delay_offset = 90090;
var cbr_flag = true;

function init()
{
    canvas = document.getElementById('canvas');
    resetStream();
    update();
}

function resetStream()
{
    var bit_rate = document.getElementById('bitRate').value * 1000;
    var averagePicSize = pictureDuration / 27000000 * bit_rate / 8;
    for(var i = 0; i < 10; i++){
        picture[i] = averagePicSize + (Math.random() - 0.5) * averagePicSize;
    }
}

function tickToX(tick)
{
    return tick / 900900 * 64;
}

function levelToY(level)
{
    return canvas.height - level / 50;
}

function drawDecodePictures(ctx)
{
    ctx.strokeStyle = 'rgba(255, 0, 0, 255)';

    ctx.beginPath();
    ctx.moveTo(0, levelToY(0));
    ctx.lineTo(tickToX(initial_cpb_removal_delay), levelToY(0));

    var prevX = tickToX(initial_cpb_removal_delay)
    var prevLevel = 0;
    for(var i = 0; i < 50; i++){
        var level = prevLevel + picture[i];
        var x1 = prevX;
        var x2 = prevX + tickToX(pictureDuration)
        var y  = levelToY(level);

        ctx.lineTo(x1, y);
        ctx.lineTo(x2, y);
        prevLevel = level;
        prevX     = x2
    }
    ctx.stroke();
}

function levelFromPeriod(period, bitRate)
{
    return period / 27000000.0 * bitRate / 8;
}

function drawInputData(ctx)
{
    var t_af          = new Array();
    var t_ai          = new Array();
    var t_ai_earliest = new Array();
    var t_rn          = new Array();

    var bit_rate = document.getElementById('bitRate').value * 1000;

    t_ai[0]   = 0;
    t_af[0]   = picture[0] * 8 * 27000000 / bit_rate;
    t_rn[0]   = initial_cpb_removal_delay;

    for(var n = 1; n < picture.length; n++)
    {
        t_rn[n] = t_rn[n - 1] + pictureDuration;
        if(cbr_flag == true)
        {
            t_ai[n] = t_af[n - 1];
        }
        else
        {
            t_ai_earliest = t_rn[n] - (initial_cpb_removal_delay + initial_cpb_removal_delay_offset);
            t_ai[n] = Math.max(t_af[n - 1], t_ai_earliest);
        }
        t_af[n] = t_ai[n] + picture[n] * 8 * 27000000 / bit_rate;
    }

    ctx.strokeStyle = 'rgba(0, 0, 255, 255)';
    ctx.beginPath();
    ctx.moveTo(tickToX(0), levelToY(0));

    var level = 0;
    for(var n = 0; n < picture.length; n++)
    {
        ctx.lineTo(tickToX(t_ai[n]), levelToY(level));
        level += picture[n];
        ctx.lineTo(tickToX(t_af[n]), levelToY(level));
    }
    ctx.stroke();

    level = 0;
    for(var n = 0; n < picture.length; n++)
    {
        var isICRDVisible = document.getElementById("isICRDVisible").checked;
        if(isICRDVisible){
            // draw initial_cpb_removal_delay period
            var icrd_x1 = tickToX(t_rn[n] - initial_cpb_removal_delay)
            var icrd_x2 = tickToX(t_rn[n])
            var icrd_y  = levelToY(level) - 4

            ctx.strokeStyle = 'rgba(0, 0, 0, 255)';
            ctx.beginPath();
            ctx.moveTo(icrd_x1, icrd_y);
            ctx.lineTo(icrd_x2, icrd_y);
            ctx.stroke();

            ctx.beginPath();
            ctx.moveTo(icrd_x1 + 2, icrd_y - 2);
            ctx.lineTo(icrd_x1, icrd_y);
            ctx.lineTo(icrd_x1 + 2, icrd_y + 2);
            ctx.stroke();

            ctx.beginPath();
            ctx.moveTo(icrd_x2 - 2, icrd_y - 2);
            ctx.lineTo(icrd_x2, icrd_y);
            ctx.lineTo(icrd_x2 - 2, icrd_y + 2);
            ctx.stroke();
        }

        var isICRDOVisible = document.getElementById("isICRDOVisible").checked;
        if(isICRDOVisible){
            // draw initial_cpb_removal_delay period
            var icrdo_x1 = tickToX(t_rn[n] - initial_cpb_removal_delay - initial_cpb_removal_delay_offset)
            var icrdo_x2 = icrd_x1
            var icrdo_y  = levelToY(level) - 4

            ctx.strokeStyle = 'rgba(0, 0, 0, 255)';
            ctx.beginPath();
            ctx.moveTo(icrdo_x1, icrdo_y);
            ctx.lineTo(icrdo_x2, icrdo_y);
            ctx.stroke();

            ctx.beginPath();
            ctx.moveTo(icrdo_x1 + 2, icrdo_y - 2);
            ctx.lineTo(icrdo_x1, icrdo_y);
            ctx.lineTo(icrdo_x1 + 2, icrdo_y + 2);
            ctx.stroke();

            ctx.beginPath();
            ctx.moveTo(icrdo_x2 - 2, icrdo_y - 2);
            ctx.lineTo(icrdo_x2, icrdo_y);
            ctx.lineTo(icrdo_x2 - 2, icrdo_y + 2);
            ctx.stroke();
        }
        level += picture[n];
    }
}

function setMode(mode)
{
  cbr_flag = mode == "cbr";
}

function updateUI(uiName, valueName)
{
    document.getElementById(uiName).innerHTML = document.getElementById(valueName).value
}

function update()
{
    var ctx    = canvas.getContext('2d');
    ctx.fillStyle = 'rgba(255, 255, 255, 255)';
    ctx.fillRect(0, 0, canvas.width, canvas.height);

    updateUI("showBitRate", "bitRate");
    updateUI("showICRD", "initial_cpb_removal_delay");
    updateUI("showICRDO", "initial_cpb_removal_delay_offset");

    initial_cpb_removal_delay = document.getElementById('initial_cpb_removal_delay').value * 27000;
    initial_cpb_removal_delay_offset = document.getElementById('initial_cpb_removal_delay_offset').value * 27000;

    drawDecodePictures(ctx);
    drawInputData(ctx);
};
Advertisements

About Moto

Engineer who likes coding
This entry was posted in Video and tagged , , , , . Bookmark the permalink.

One Response to HRD Emulator in HTML5

  1. pewpew says:

    Great post, finally got what HRD is 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s