H.264 is the codec to use with peers that have no VP8, such as some phones, SIP gateways and hardware. Use this plugin to get the H.264 video of a peer as pictures, or to send pictures you produce. It runs on OpenH264, so the library has to be there when you build.

Build with OpenH264

brew install openh264 cmake pkg-config        # macOS
sudo apt-get install libopenh264-dev cmake pkg-config   # Debian 13, Ubuntu 24.04 or later
pkg install openh264 cmake pkgconf            # FreeBSD
rebar3 compile

Set up on your system has the other systems.

Check that the codec is there:

[#{implementation := <<"OpenH264 ", _/binary>>}] = [C || #{name := <<"H264">>} = C <- webrtc_codecs:available()].

Receive

{ok, Pc} = webrtc:new_peer_connection(#{
    frames => true,
    codec_plugins => #{video => {webrtc_codecs, #{}}}
}),

receive
    {webrtc, Pc, {decoded, Mid, #{format := i420, width := W, height := H} = Picture}} ->
        show(Picture);
    {webrtc, Pc, {decoded, Mid, #{dropped := _Reason}}} ->
        webrtc:request_keyframe(Pc, Mid)
end.

Send

erlang_webrtc offers VP8 first. To send H.264, put it first in the codecs you offer, with its retransmission format:

Video = webrtc_jsep_caps:default_codecs(video),
[#{pt := Pt} = H264 | _] = [C || #{name := <<"H264">>} = C <- Video],
Rtx = [C || #{name := <<"rtx">>, fmtp := [{<<"apt">>, Apt}]} = C <- Video, Apt =:= integer_to_binary(Pt)],
{ok, Pc} = webrtc:new_peer_connection(#{codecs => #{video => [H264 | Rtx]}}),
{ok, _} = webrtc:add_transceiver(Pc, video, #{frames => true, codec_plugin => {webrtc_codecs, #{}}}),

ok = webrtc:send_media(Pc, Mid, #{
    format => i420,
    width => 640,
    height => 480,
    y => Y,
    u => U,
    v => V,
    timestamp => Ts
}).

Follow the network

receive
    {webrtc, Pc, {bandwidth_estimate, Bps}} ->
        ok = webrtc:send_media(Pc, Mid, #{bitrate => Bps});
    {webrtc, Pc, {keyframe_request, Mid}} ->
        ok = webrtc:send_media(Pc, Mid, #{keyframe => true})
end.

Options

OptionDefaultMeaning
bitratefrom the picture sizeBitrate of the encoder at start, in bits per second
min_bitrate30000Lowest bitrate the encoder is given
max_bitratenoneHighest bitrate the encoder is given
keyframe_interval3000Most frames between two key frames
threadsfrom the picture sizeThreads of the encoder
max_pixels8294400 (3840 x 2160)Largest picture the decoder accepts

Notes

  • The encoder writes constrained baseline, one slice per picture, which a peer that negotiated baseline, main or high can decode. The level in profile-level-id is not enforced: keep the picture size and the bitrate within what the peer announced.
  • The decoder takes baseline, main and high, 8 bit 4:2:0.
  • A frame is an access unit in Annex B. A key frame is one with an IDR slice, and the encoder puts the parameter sets in front of each.
  • Width and height have to be even when you encode: OpenH264 would cut the last column or line, so an odd size is an error.
  • A frame that cannot be decoded gives #{dropped => Reason, timestamp => Ts}, and so does every frame after it until a key frame (waiting_for_keyframe). Errors are not concealed.
  • Every sequence parameter set a peer sends is read before OpenH264 sees it, in a key frame or not. One that declares more than max_pixels, on the size as coded, makes the frame dropped (too_large). One that cannot be read, of a profile other than baseline, main, extended or high, or a subset set, makes it dropped as unsupported_parameter_set.
  • OpenH264 does not tell the quantizer, so qpSum is not reported.
  • Keep the rtx format next to H.264 when you narrow the codecs, see "Issues met in erlang_webrtc" in What the codec behaviour lacks.
  • OpenH264 is BSD licensed. The H.264 patent licence Cisco pays for covers the binaries Cisco distributes, not one you or your distribution built.