From 7193f740cad0d6bb35778b419a8656ae8126ddf7 Mon Sep 17 00:00:00 2001 From: "hunter@kvog.sh" Date: Wed, 17 Jun 2026 21:40:13 -0500 Subject: --- content/notes/ffmpeg.md | 209 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 209 insertions(+) create mode 100644 content/notes/ffmpeg.md (limited to 'content/notes') diff --git a/content/notes/ffmpeg.md b/content/notes/ffmpeg.md new file mode 100644 index 0000000..afc47e4 --- /dev/null +++ b/content/notes/ffmpeg.md @@ -0,0 +1,209 @@ ++++ +date = '2026-06-17T17:50:30-05:00' +title = 'FFmpeg' ++++ + +## Websites + +* [ffmpeg.org](https://ffmpeg.org/) +* [code.ffmpeg.org](https://code.ffmpeg.org/FFmpeg/FFmpeg) - code forge +* [trac.ffmpeg.org](https://trac.ffmpeg.org/query?status=!closed&type=defect&order=id&desc=1) - bug tracker, mostly abandoned in favor of forgejo +* [ffmpeg-devel mailing list](https://lists.ffmpeg.org/mailman3/lists/ffmpeg-devel.ffmpeg.org/) - development mailing list, mostly abandoned in favor of forgejo +* [ffmpeg-devel IRC](https://libera.catirclogs.org/ffmpeg-devel/2026-06) - development chat, very active +* [patchwork.ffmpeg.org](https://patchwork.ffmpeg.org/project/ffmpeg/list/) - archive of patches sent to ffmpeg-devel + +## Documentation + +* Command-line tools, filters, etc: https://ffmpeg.org/ffmpeg-all.html +* Component libraries: https://ffmpeg.org/doxygen/trunk/index.html + +## Building + +List all compile options: +``` +$ ./configure --help | vim - +``` + +Quick development build; takes about 90 seconds on an M1 MacBook Air: +``` +$ ./configure --disable-optimizations --disable-asm +$ make -j $(nproc) +$ make -j $(sysctl -n hw.logicalcpu) +``` + +Troubleshooting build: +``` +$ ./configure --disable-optimizations --assert-level=2 --toolchain=clang-asan +$ ./configure --disable-optimizations --assert-level=2 --toolchain=clang-asan-ubsan +``` + +`--enable-memory-poisoning` makes `av_malloc` and friends basically just do `memset(ptr, 0x2a, size)`. Does not add any asan-style bounds checking. + +## Regression testing + +FATE documentation: https://ffmpeg.org/fate.html, TLDR: +``` +$ ./configure --disable-optimizations --assert-level=2 --toolchain=clang-asan --samples=$HOME/Proj/FFmpeg-FATE +$ make +$ make fate-rsync +$ make fate +``` + +## Benchmarking + +checkasm docs: https://checkasm.videolan.me/ + +Build within FFmpeg: +``` +$ make checkasm +``` + +List tests: +``` +$ ./tests/checkasm/checkasm --list-tests | sort | vim - + +``` + +The easiest way to see what test calls what functions is to check the source code: +
+[`-> tests/checkasm/checkasm.c`](https://code.ffmpeg.org/FFmpeg/FFmpeg/src/branch/master/tests/checkasm/checkasm.c). + +Example: +``` +$ ./tests/checkasm/checkasm --bench --test=vf_bwdif +``` +
+Output + +``` +checkasm: + - CPU: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz (000906E9) + - Timing source: x86 (rdtsc) + - Bench duration: 1000 µs per function (4003730 cycles) + - Random seed: 2684063072 +C: + - vf_bwdif.bwdif8 [OK] + - vf_bwdif.bwdif10 [OK] + - vf_bwdif.bwdif8.line3 [OK] + - vf_bwdif.bwdif8.edge [OK] + - vf_bwdif.bwdif8.intra [OK] +SSE2: + - vf_bwdif.bwdif8 [OK] + - vf_bwdif.bwdif10 [OK] +SSSE3: + - vf_bwdif.bwdif8 [OK] + - vf_bwdif.bwdif10 [OK] +AVX2: + - vf_bwdif.bwdif8 [OK] + - vf_bwdif.bwdif10 [OK] +checkasm: all 6 tests passed +Benchmark results: + name cycles (vs ref) + bwdif8_c: 28876.2 + bwdif8_sse2: 1056.1 (25.12x) + bwdif8_ssse3: 870.5 (22.92x) + bwdif8_avx2: 550.6 (40.40x) + bwdif8.edge.s0.p0_c: 1373.2 + bwdif8.edge.s0.p1_c: 1310.2 + bwdif8.edge.s1.p0_c: 5368.5 + bwdif8.edge.s1.p1_c: 5824.8 + bwdif8.intra_c: 584.1 + bwdif8.line3.rnd.p0_c: 40020.9 + bwdif8.line3.rnd.p1_c: 30286.6 + bwdif10_c: 27988.9 + bwdif10_sse2: 935.6 (27.51x) + bwdif10_ssse3: 864.2 (27.04x) + bwdif10_avx2: 594.3 (36.53x) +``` + +
+ +## libavformat architecture + +Note: This is all based off of FFmpeg 8.1.2 (commit `38b88335f99e76ed89ff3c93f877fdefce736c13`) + +Stepping through [`doc/examples/remux.c`](https://code.ffmpeg.org/FFmpeg/FFmpeg/src/commit/38b88335f99e76ed89ff3c93f877fdefce736c13/doc/examples/remux.c): + + + +
+ +[`doc/examples/remux.c:75`](https://code.ffmpeg.org/FFmpeg/FFmpeg/src/commit/38b88335f99e76ed89ff3c93f877fdefce736c13/doc/examples/remux.c#L75): +`main` calls `avformat_open_input`. + + +
+ +[`libavformat/demux.c:241`](https://code.ffmpeg.org/FFmpeg/FFmpeg/src/commit/38b88335f99e76ed89ff3c93f877fdefce736c13/libavformat/demux.c#L241): +`avformat_open_input` calls `avformat_alloc_context` to allocate memory. + +[`libavformat/demux.c:266`](https://code.ffmpeg.org/FFmpeg/FFmpeg/src/commit/38b88335f99e76ed89ff3c93f877fdefce736c13/libavformat/demux.c#L266): +`avformat_open_input` calls `init_input`. + +
+ +[`libavformat/demux.c:185`](https://code.ffmpeg.org/FFmpeg/FFmpeg/src/commit/38b88335f99e76ed89ff3c93f877fdefce736c13/libavformat/demux.c#L185): +`init_input` calls `av_probe_input_buffer2`, which reads the first 2048 bytes of the input file +and passes it to `av_probe_input_format2`, which calls `avprobe_input_format3`. + +
+ +[`libavformat/format.c:175`](https://code.ffmpeg.org/FFmpeg/FFmpeg/src/commit/38b88335f99e76ed89ff3c93f877fdefce736c13/libavformat/format.c#L175): +`av_probe_input_format3` attemptes to locate id3 tags within the buffer + +[`libavformat/format.c:191`](https://code.ffmpeg.org/FFmpeg/FFmpeg/src/commit/38b88335f99e76ed89ff3c93f877fdefce736c13/libavformat/format.c#L191): +`av_probe_input_format3` iterates over each registered `FFInputFormat`, and calls the `probe` +function pointer, if present. `probe` takes the buffer and returns a score that represents how +likely the data matches that format. Some other factors are taken into account, such as the +extension of the file and what id3 tags were found. + +The most likely `AVInputFormat *` (composite class in `FFInputFormat`) is returned if found, +otherwise `NULL`. + +
+ +[`libavformat/demux.c:305`](https://code.ffmpeg.org/FFmpeg/FFmpeg/src/commit/38b88335f99e76ed89ff3c93f877fdefce736c13/libavformat/demux.c#L305): +`init_input` allocates any private data required by the `FFInputFormat`. + +[`libavformat/demux.c:322`](https://code.ffmpeg.org/FFmpeg/FFmpeg/src/commit/38b88335f99e76ed89ff3c93f877fdefce736c13/libavformat/demux.c#L322): +`init_input` calls the `read_header` function pointer, if present. + +
+ +`read_header` will typically call `avformat_new_stream` to allocate streams. + +
+ +[`libavformat/demux.c:355`](https://code.ffmpeg.org/FFmpeg/FFmpeg/src/commit/38b88335f99e76ed89ff3c93f877fdefce736c13/libavformat/demux.c#L355): +`init_input` calls the `update_stream_avctx`, which updates contexts for each stream. I am not sure +why this is needed since no codecs should be open at this point, only demuxing. + +
+ +[`libavformat/demux.c:194`](https://code.ffmpeg.org/FFmpeg/FFmpeg/src/commit/38b88335f99e76ed89ff3c93f877fdefce736c13/libavformat/demux.c#L194): +`update_stream_avctx` iterates over each detected stream and calls `avcodec_parameters_to_context` +if `FFStream->need_context_update == 1`. + +
+
+ +To summarize, `avformat_open_input` probes the file to detect the container, initializes the +demuxer instance, and reads the file header. + +
+ +[`doc/examples/remux.c:80`](https://code.ffmpeg.org/FFmpeg/FFmpeg/src/commit/38b88335f99e76ed89ff3c93f877fdefce736c13/doc/examples/remux.c#L80): +`main` calls `avformat_find_stream_info`. + +
-- cgit v1.2.3