webrtc_codecs_capture (webrtc_codecs v0.1.0)
View SourceCapture raw pictures or audio from a camera, a microphone, a file or a
test source, through the ffmpeg executable running as a port.
{ok, Cam} = webrtc_codecs_capture:start({camera, 0}, #{width => 640, height => 480, frame_rate => 30}),
receive
{webrtc_capture, Cam, #{format := i420} = Picture} -> webrtc:send_media(Pc, Mid, Picture)
end.The capture process owns the device. It sends each picture
(webrtc_codecs:picture/0) or chunk of audio (webrtc_codecs:audio/0,
without a timestamp) to its owner as {webrtc_capture, Pid, Raw}, then
{webrtc_capture, Pid, eof} or {webrtc_capture, Pid, {error, Reason}}
when the source ends, and stops. It stops with its owner.
When the owner has more than max_queue messages waiting, what is
captured is dropped and counted, so a slow owner costs frames, not
memory.
With the encode option FFmpeg also encodes the video, and the owner
gets encoded frames to pass to webrtc:send_frame/4, with no codec
plugin:
{ok, Cam} = webrtc_codecs_capture:start({camera, 0}, #{encode => #{codec => vp8, bitrate => 1000000}}),
receive
{webrtc_capture, Cam, #{data := Frame, timestamp := Ts, keyframe := Key}} ->
webrtc:send_frame(Pc, Mid, Frame, #{timestamp => Ts, keyframe => Key})
end.
Summary
Types
An encoded frame, as webrtc:send_frame/4 takes it: timestamp is in 90 kHz units.
Functions
The cameras and microphones FFmpeg sees. On macOS the terminal or the
application needs the camera and microphone permissions to open them; on
Linux the user has to be in the video and audio groups.
Start capturing. The process is linked to the caller.
Pictures, chunks of audio or encoded frames captured so far, and how many of them were dropped.
Stop capturing and release the device.
Types
-type device() :: #{index := non_neg_integer(), name := binary()}.
-type encode() :: #{codec := vp8 | vp9 | av1 | h264, bitrate => pos_integer(), keyframe_interval => pos_integer(), encoder => string(), encoder_options => [string()]}.
codec:vp8,vp9,av1orh264bitrate: in bits per second, default 1000000keyframe_interval: frames between two key frames, default 60. FFmpeg cannot be asked for a key frame while it runs, so a peer that loses a frame waits for the next one.encoder: the FFmpeg encoder, defaultlibvpx,libvpx-vp9,libsvtav1, and for H.264h264_videotoolboxon macOS andlibopenh264elsewhereencoder_options: FFmpeg options for the encoder, replacing the real time ones this module sets for the default encoders
-type encoded_frame() :: #{codec := vp8 | vp9 | av1 | h264, data := binary(), timestamp := non_neg_integer(), keyframe := boolean()}.
An encoded frame, as webrtc:send_frame/4 takes it: timestamp is in 90 kHz units.
-type options() :: #{owner => pid(), width => pos_integer(), height => pos_integer(), frame_rate => pos_integer(), sample_rate => pos_integer(), channels => pos_integer(), frame_ms => pos_integer(), max_queue => non_neg_integer(), realtime => boolean(), encode => encode(), input_options => [string()], ffmpeg => file:filename_all()}.
owner: the process that gets the media, default the callerwidth,height,frame_rate: of pictures, default 640, 480, 30; width and height are evensample_rate,channels,frame_ms: of audio, default 48000, 1, 20max_queue: messages the owner may have waiting before capture drops, default 10realtime: pace a file or a test source at its own rate, defaulttrue; withfalseFFmpeg reads as fast as it can, and what the owner does not take in time is droppedencode: have FFmpeg encode the video and give encoded frames, seeencode/0input_options: FFmpeg options placed before the input, to replace the ones this module derives for a deviceffmpeg: path of the executable, default the one inPATH
-type source() :: {camera, non_neg_integer() | binary()} | {microphone, non_neg_integer() | binary()} | {file, video | audio, file:filename_all()} | {test, video | audio}.
What to capture: a device by its index or its name in devices/0, a
file FFmpeg can read, or a source FFmpeg generates (a moving test
picture, a 440 Hz tone).
Functions
The cameras and microphones FFmpeg sees. On macOS the terminal or the
application needs the camera and microphone permissions to open them; on
Linux the user has to be in the video and audio groups.
Start capturing. The process is linked to the caller.
-spec stats(pid()) -> #{captured := non_neg_integer(), dropped := non_neg_integer()}.
Pictures, chunks of audio or encoded frames captured so far, and how many of them were dropped.
-spec stop(pid()) -> ok.
Stop capturing and release the device.