H.264
View SourceH.264 is the codec to use with peers that have no VP8, such as some phones, SIP gateways and hardware. Use this plugin to get the H.264 video of a peer as pictures, or to send pictures you produce. It runs on OpenH264, so the library has to be there when you build.
Build with OpenH264
brew install openh264 cmake pkg-config # macOS
sudo apt-get install libopenh264-dev cmake pkg-config # Debian 13, Ubuntu 24.04 or later
pkg install openh264 cmake pkgconf # FreeBSD
rebar3 compile
Set up on your system has the other systems.
Check that the codec is there:
[#{implementation := <<"OpenH264 ", _/binary>>}] = [C || #{name := <<"H264">>} = C <- webrtc_codecs:available()].Receive
{ok, Pc} = webrtc:new_peer_connection(#{
frames => true,
codec_plugins => #{video => {webrtc_codecs, #{}}}
}),
receive
{webrtc, Pc, {decoded, Mid, #{format := i420, width := W, height := H} = Picture}} ->
show(Picture);
{webrtc, Pc, {decoded, Mid, #{dropped := _Reason}}} ->
webrtc:request_keyframe(Pc, Mid)
end.Send
erlang_webrtc offers VP8 first. To send H.264, put it first in the codecs you offer, with its retransmission format:
Video = webrtc_jsep_caps:default_codecs(video),
[#{pt := Pt} = H264 | _] = [C || #{name := <<"H264">>} = C <- Video],
Rtx = [C || #{name := <<"rtx">>, fmtp := [{<<"apt">>, Apt}]} = C <- Video, Apt =:= integer_to_binary(Pt)],
{ok, Pc} = webrtc:new_peer_connection(#{codecs => #{video => [H264 | Rtx]}}),
{ok, _} = webrtc:add_transceiver(Pc, video, #{frames => true, codec_plugin => {webrtc_codecs, #{}}}),
ok = webrtc:send_media(Pc, Mid, #{
format => i420,
width => 640,
height => 480,
y => Y,
u => U,
v => V,
timestamp => Ts
}).Follow the network
receive
{webrtc, Pc, {bandwidth_estimate, Bps}} ->
ok = webrtc:send_media(Pc, Mid, #{bitrate => Bps});
{webrtc, Pc, {keyframe_request, Mid}} ->
ok = webrtc:send_media(Pc, Mid, #{keyframe => true})
end.Options
| Option | Default | Meaning |
|---|---|---|
bitrate | from the picture size | Bitrate of the encoder at start, in bits per second |
min_bitrate | 30000 | Lowest bitrate the encoder is given |
max_bitrate | none | Highest bitrate the encoder is given |
keyframe_interval | 3000 | Most frames between two key frames |
threads | from the picture size | Threads of the encoder |
max_pixels | 8294400 (3840 x 2160) | Largest picture the decoder accepts |
Notes
- The encoder writes constrained baseline, one slice per picture, which a
peer that negotiated baseline, main or high can decode. The level in
profile-level-idis not enforced: keep the picture size and the bitrate within what the peer announced. - The decoder takes baseline, main and high, 8 bit 4:2:0.
- A frame is an access unit in Annex B. A key frame is one with an IDR slice, and the encoder puts the parameter sets in front of each.
- Width and height have to be even when you encode: OpenH264 would cut the last column or line, so an odd size is an error.
- A frame that cannot be decoded gives
#{dropped => Reason, timestamp => Ts}, and so does every frame after it until a key frame (waiting_for_keyframe). Errors are not concealed. - Every sequence parameter set a peer sends is read before OpenH264 sees
it, in a key frame or not. One that declares more than
max_pixels, on the size as coded, makes the frame dropped (too_large). One that cannot be read, of a profile other than baseline, main, extended or high, or a subset set, makes it dropped asunsupported_parameter_set. - OpenH264 does not tell the quantizer, so
qpSumis not reported. - Keep the
rtxformat next to H.264 when you narrow the codecs, see "Issues met in erlang_webrtc" in What the codec behaviour lacks. - OpenH264 is BSD licensed. The H.264 patent licence Cisco pays for covers the binaries Cisco distributes, not one you or your distribution built.