Opus is the audio codec every browser sends and the one erlang_webrtc offers first. Use this plugin to get the audio of a peer as samples, or to send audio you produce. It runs on libopus, so the library has to be there when you build.

Build with libopus

brew install opus cmake pkg-config        # macOS
sudo apt-get install libopus-dev cmake pkg-config   # Debian, Ubuntu
rebar3 compile

Set up on your system has the other systems.

Check that the codec is there:

[#{name := <<"opus">>, implementation := <<"libopus ", _/binary>>} | _] = webrtc_codecs:available().

To make the build fail when libopus is missing, rather than leave Opus out:

WEBRTC_CODECS_REQUIRE=opus rebar3 compile

Receive

{ok, Pc} = webrtc:new_peer_connection(#{
    frames => true,
    codec_plugins => #{audio => {webrtc_codecs, #{}}}
}),

receive
    {webrtc, Pc, {decoded, Mid, #{sample_rate := 48000, channels := Channels, samples := Pcm}}} ->
        play(Pcm, Channels)
end.

Decoded audio is at 48 kHz, in stereo when stereo=1 was negotiated and mono otherwise. Ask for another format with options:

codec_plugins => #{audio => {webrtc_codecs, #{sample_rate => 16000, channels => 1}}}

Send

Give 20 ms at a time, at 8, 12, 16, 24 or 48 kHz, one or two channels:

ok = webrtc:send_media(Pc, Mid, #{
    format => s16le,
    sample_rate => 48000,
    channels => 1,
    samples => Pcm
}).

Follow the network

Pass the audio share of the bandwidth estimate, and the loss the peer reports, in a raw map. Both apply from the next frame.

ok = webrtc:send_media(Pc, Mid, #{bitrate => 24000}),
ok = webrtc:send_media(Pc, Mid, #{packet_loss => 10}).

packet_loss is a percentage. With in-band FEC negotiated (useinbandfec=1, which erlang_webrtc offers), the encoder adds redundancy only when it is above 0.

Options

Each defaults to what the negotiated fmtp says.

OptionDefaultMeaning
sample_rate48000Rate of decoded audio
channels2 with stereo=1, else 1Channels of decoded audio
bitratechosen by libopusBitrate of the encoder at start, in bits per second
fecuseinbandfecAdd redundancy for the previous frame
dtxusedtxSend nothing during silence
packet_loss0Loss expected at start, in percent
applicationvoipvoip, audio (music) or lowdelay
complexitychosen by libopus0 (fastest) to 10 (best)

Notes

  • When frames were lost, the next frame first gives the audio that fills the hole, up to 120 ms: the last lost frame recovered from the redundancy in the packet, the earlier ones synthesized. concealedSamples counts the synthesized ones. A packet without redundancy gets synthesized audio too, which is then not counted.
  • A longer hole is taken as a pause and nothing is made up for it.
  • A packet that is not valid Opus gives #{dropped => invalid_packet, timestamp => Ts}.
  • The bitrate is kept between 6000 and 510000, and under maxaveragebitrate when the peer gave one.
  • Frames of 2.5, 5, 10, 20, 40 and 60 ms are accepted; another duration or sample rate is an error, which turns the plugin off for the transceiver.
  • With dtx, a call may send nothing; timestamps keep counting.
  • RTP timestamps are at 48 kHz whatever the rate of your samples.