VP8 and VP9

View Source

VP8 is the video codec every WebRTC peer has; VP9 gives the same quality at a lower bitrate for more CPU. Use this plugin to get the video of a peer as pictures, or to send pictures you produce. It runs on libvpx, so the library has to be there when you build.

Build with libvpx

brew install libvpx cmake pkg-config        # macOS
sudo apt-get install libvpx-dev cmake pkg-config   # Debian, Ubuntu
rebar3 compile

Set up on your system has the other systems.

Check that the codecs are there:

[<<"VP8">>, <<"VP9">>] = [N || #{name := N, kind := video} <- webrtc_codecs:available()].

Receive

{ok, Pc} = webrtc:new_peer_connection(#{
    frames => true,
    codec_plugins => #{video => {webrtc_codecs, #{}}}
}),

receive
    {webrtc, Pc, {decoded, Mid, #{format := i420, width := W, height := H} = Picture}} ->
        show(Picture);
    {webrtc, Pc, {decoded, Mid, #{dropped := _Reason}}} ->
        webrtc:request_keyframe(Pc, Mid)
end.

Send

Give one picture at a time, with its timestamp in 90 kHz units:

{ok, _} = webrtc:add_transceiver(Pc, video, #{frames => true, codec_plugin => {webrtc_codecs, #{}}}),
ok = webrtc:send_media(Pc, Mid, #{
    format => i420,
    width => 640,
    height => 480,
    y => Y,
    u => U,
    v => V,
    timestamp => Ts
}).

Follow the network

receive
    {webrtc, Pc, {bandwidth_estimate, Bps}} ->
        ok = webrtc:send_media(Pc, Mid, #{bitrate => Bps});
    {webrtc, Pc, {keyframe_request, Mid}} ->
        ok = webrtc:send_media(Pc, Mid, #{keyframe => true})
end.

Both apply to the next picture. You can also put bitrate and keyframe in the picture itself.

Options

OptionDefaultMeaning
bitratefrom the picture sizeBitrate of the encoder at start, in bits per second
min_bitrate30000Lowest bitrate the encoder is given
max_bitratenoneHighest bitrate the encoder is given
keyframe_interval3000Most frames between two key frames
threadsfrom the picture size (encoder), 1 (decoder)Threads libvpx uses
cpu_used-6 (VP8), 7 (VP9)libvpx speed setting: higher is faster and coarser
max_pixels8294400 (3840 x 2160)Largest picture the decoder accepts
codec_plugin => {webrtc_codecs, #{bitrate => 800000, max_bitrate => 2500000}}

Notes

  • The encoder works in real time at a constant bitrate and holds no frame back: one picture in, one frame out.
  • The default bitrate is 0.1 bit per pixel at 30 pictures per second: about 230 kbit/s at 320 x 240, 920 kbit/s at 640 x 480.
  • Pictures of another size start a new stream with a key frame.
  • Width and height may be odd; the chroma planes are then rounded up.
  • A frame that cannot be decoded gives #{dropped => Reason, timestamp => Ts}, and so does every frame after it until a key frame (waiting_for_keyframe). Ask for one with webrtc:request_keyframe/2.
  • A frame that declares a picture larger than max_pixels is dropped (too_large) before any memory is taken for it. For VP9 that is every frame of a superframe, the ones not shown included; a superframe index that does not fit its data is dropped as invalid_frame.
  • VP9 is profile 0 only (8 bit, 4:2:0), which is what erlang_webrtc and browsers negotiate by default. Another profile-id fails init/3, and a frame of another profile is dropped (unsupported_profile).
  • qpSum counts the quantizer index of each frame: 0 to 127 for VP8, 0 to 255 for VP9.