This is my FFmpeg API notes when I was working on audio and video encoding/decoding projects. This note isn’t complete and you may first want to take a look at An ffmpeg and SDL Tutorial.

Initialization

FFmpeg library must be initialized before using any of the encoder, decoder, muxers, demuxers, and filters.

• avcodec_register_all. For libavcodec, register all codecs, parsers, and bitstream filters which were enabled at configuration time.
• av_register_all. For libavformat, initialize and register all the muxers, demuxers, and protocols.
• avfilter_register_all. For libavfilter, initialize FFmpeg graph-based frame editing library.

Encoding, Decoding, and Filtering

A context structure should be initialized and opened for encoding, decoding, and filtering routines. FFmpeg developers recommend using AVOptions to access context fields from user application.

AVCodecContext

Always avcodec_open2 the AVCodecContext to use the given AVCodec before using encoding and decoding routines. The context has to be allocated with avcodec_alloc_context3 and sizeof(AVCodecContext) must not be used outside FFmpeg library.

// 0. libavcodec initialization
avcodec_register_all();
// 1. find a registered codec
av_dict_set(&opts, "b", "2.5M", 0);
codec = avcodec_find_decoder(AV_CODEC_ID_H264);
if (!codec)
  return false;
// 2. allocate an codec context
ctx = avcodec_alloc_context3(codec);
// ...
// 3. set necessary fileds in the context
// ...
// 4. initialize the context to use the given codec
if (avcodec_open2(ctx, codec, opts) < 0)
  return false;
else
  return true;
// 5. loop to encode or decode
// ...

AVFilterGraph

An AVFilterGraph is a directed graph of connected filters. It can contain cycles and there can be multiple links between a pair of filters. An AVFilterContext represents an instance of an AVFilter class registered in the library (enabled at FFmpeg configuration time). We can avfilter_get_by_name a filter defined by the name. avfilter_graph_create_filter creates and adds a filter context instance into an existing filter graph. After linking all filter contexts, the filter graph should be configured by avfilter_graph_config. Raw data are fed into filter graph using av_buffersrc_write_frame or av_buffersrc_add_frame and fetched by av_buffersink_get_frame.

// 0. libavfilter initialization
avfilter_register_all();
// 1. create a filter graph
graph_ctx = avfilter_graph_alloc();
if (!graph_ctx)
  return false;
// 2. find a filter
filt = avfilter_get_by_name("scale");
if (!filt)
  return false;
// 3. create a filter context
if (avfilter_graph_create_filter(filt_ctx, filt, "name", args, 0, graph_ctx) < 0)
  return false;
// ...
// 4. more filt_ctx initializations and link all filt_ctx
// ...
// 5. configure the filter graph
if (avfilter_graph_config(graph_ctx, nullptr) < 0)
  return false;
else
  return true;

Ownership

Normally, FFmpeg library will take the ownership of the allocated buffer, i.e., the encoder/decoder will allocate memory for compressed/decompressed data and release them afterwards. In order to avoid unnecessary data copying, we have to:

• Decoder. When AVCodecContext.refcounted_frames is set to 1, the frame is reference counted and the returned reference belongs to the caller. The caller must release the frame using av_frame_unref when the frame is no longer needed.
• Encoder. The caller should supply an output buffer by setting AVPacket.data and AVPacket.size prior to calling the encoding function.
• AVFrame and AVPacket are all reference-­counted data buffers by FFmpeg AVBuffer API. When the buf field is nullptr, data are not referenced­-counted.