Re: Burn caption into video

On Tue, 10/15/2024 7:53 PM, Oliver wrote:

On Tue, 15 Oct 2024 13:24:16 -0400, Paul <nospam@needed.invalid> wrote

ffmpeg -i sample_video_ffmpeg.mp4 -vf subtitles=sample_video_subtitle_ffmpeg.srt output_srt.mp4

That seems to be missing a -c copy, but maybe all the tracks will be there.

Using something like ffprobe.exe afterwards on output_srt.mp4 , should
indicate whether an additional new track is present.

Using the simple ten second video clip shot from my phone and this simple hand-crafted SRT file, the suggested command worked (almost) perfectly.  ====================
1
00:00:05,000 --> 00:00:07,000

This is line 1.
This is line 2.
 ====================
ffmpeg -i test.mp4 -vf subtitles=test.srt merged.mp4
ffprobe merged.mp4
move test.srt test.bak
 ====================
Thanks to your suggested ffmpeg melding of files command,
(most of) the SRT was hard coded merged into the video.

The only thing that didn't work is line 2 never showed up.
But I don't care about that as a line is enough to prove it works.

BTW, I have no idea what you mean by "all the tracks" but I did run the suggested ffprobe command which output what you see below.

Did the ffprobe output indicate to you any possible missing tracks?

 ====================
ffmpeg -i test.mp4 -vf subtitles=test.srt merged.mp4
 ====================
ffmpeg version N-114515-gfa110c32b5-20240329 Copyright (c) 2000-2024 the FFmpeg developers
 built with gcc 13.2.0 (crosstool-NG 1.26.0.65_ecc5e41)
 configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32-   --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --enable-shared --disable-static --disable-w32threads
  --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma   --enable-fontconfig --enable-libharfbuzz --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb   --disable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-chromaprint --enable-libdav1d   --enable-libdavs2 --enable-libdvdread --enable-libdvdnav --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r   --enable-libgme --enable-libkvazaar --enable-libaribcaption --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame   --enable-libopus --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libvpl   --enable-openal --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt   --enable-librav1e --enable-librubberband --enable-schannel
--enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1   --enable-libtwolame --enable-libuavs3d --disable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc   --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi   --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-libs=-lgomp --extra-ldflags=-pthread --extra-ldexeflags=   --cc=x86_64-w64-mingw32-gcc --cxx=x86_64-w64-mingw32-g++ --ar=x86_64-w64-mingw32-gcc-ar --ranlib=x86_64-w64-mingw32-gcc-ranlib   --nm=x86_64-w64-mingw32-gcc-nm --extra-version=20240329  libavutil      59. 10.100 / 59. 10.100
 libavcodec     61.  4.100 / 61.  4.100
 libavformat    61.  2.100 / 61.  2.100
 libavdevice    61.  2.100 / 61.  2.100
 libavfilter    10.  2.100 / 10.  2.100
 libswscale      8.  2.100 /  8.  2.100
 libswresample   5.  2.100 /  5.  2.100
 libpostproc    58.  2.100 / 58.  2.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test.mp4':
 Metadata:
   major_brand     : isom
   minor_version   : 0
   compatible_brands: mp41avc1
   creation_time   : 2024-10-15T06:44:53.000000Z
   encoder         : vlc 3.0.20 stream output
   encoder-eng     : vlc 3.0.20 stream output
 Duration: 00:00:06.23, start: 0.000000, bitrate: 14964 kb/s
 Stream #0:0[0x1](eng): Audio: mp3 (mp3float) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
     Metadata:
       creation_time   : 2024-10-15T06:44:53.000000Z
       handler_name    : SoundHandler
       vendor_id       : [0][0][0][0]
 Stream #0:1[0x2](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 14827 kb/s, 30.17 fps, 30 tbr, 1000k tbn (default)
     Metadata:
       creation_time   : 2024-10-15T06:44:53.000000Z
       handler_name    : VideoHandler
       vendor_id       : [0][0][0][0]
     Side data:
       displaymatrix: rotation of -90.00 degrees
[Parsed_subtitles_0 @ 0000025597957c00] libass API version: 0x1701000
[Parsed_subtitles_0 @ 0000025597957c00] libass source: commit: 649a7c2e1fc6f4188ea1a89968560715800b883d-dirty
[Parsed_subtitles_0 @ 0000025597957c00] Shaper: FriBidi 1.0.13 (SIMPLE) HarfBuzz-ng 8.3.1 (COMPLEX)
[Parsed_subtitles_0 @ 0000025597957c00] Using font provider directwrite (with GDI)
[Parsed_subtitles_0 @ 0000025597957c00] libass wasn't built with ASS_FEATURE_WRAP_UNICODE support
Stream mapping:
 Stream #0:1 -> #0:0 (h264 (native) -> h264 (libx264))
 Stream #0:0 -> #0:1 (mp3 (mp3float) -> aac (native))
Press [q] to stop, [?] for help
[Parsed_subtitles_0 @ 0000025597955800] libass API version: 0x1701000
[Parsed_subtitles_0 @ 0000025597955800] libass source: commit: 649a7c2e1fc6f4188ea1a89968560715800b883d-dirty
[Parsed_subtitles_0 @ 0000025597955800] Shaper: FriBidi 1.0.13 (SIMPLE) HarfBuzz-ng 8.3.1 (COMPLEX)
[Parsed_subtitles_0 @ 0000025597955800] Using font provider directwrite (with GDI)
[Parsed_subtitles_0 @ 0000025597955800] libass wasn't built with ASS_FEATURE_WRAP_UNICODE support
[libx264 @ 0000025597949c00] using SAR=1/1
[libx264 @ 0000025597949c00] using cpu capabilities: MMX2 SSE2Fast LZCNT
[libx264 @ 0000025597949c00] profile High, level 4.0, 4:2:0, 8-bit
[libx264 @ 0000025597949c00] 264 - core 164 - H.264/MPEG-4 AVC codec - Copyleft 2003-2024 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'merged.mp4':
 Metadata:
   major_brand     : isom
   minor_version   : 0
   compatible_brands: mp41avc1
   encoder         : Lavf61.2.100
 Stream #0:0(eng): Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1080x1920 [SAR 1:1 DAR 9:16], q=2-31, 30 fps, 15360 tbn (default)
     Metadata:
       creation_time   : 2024-10-15T06:44:53.000000Z
       handler_name    : VideoHandler
       vendor_id       : [0][0][0][0]
       encoder         : Lavc61.4.100 libx264
     Side data:
       cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
 Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
     Metadata:
       creation_time   : 2024-10-15T06:44:53.000000Z
       handler_name    : SoundHandler
       vendor_id       : [0][0][0][0]
       encoder         : Lavc61.4.100 aac
[Parsed_subtitles_0 @ 0000025597955800] fontselect: (Arial, 400, 0) -> ArialMT, 0, ArialMTdrop=0 speed=0.171x
[out#0/mp4 @ 00000255978eed00] video:11219KiB audio:102KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.081056%
frame=  189 fps=6.0 q=-1.0 Lsize=   11329KiB time=00:00:06.20 bitrate=14960.4kbits/s dup=1 drop=0 speed=0.196x
[libx264 @ 0000025597949c00] frame I:5     Avg QP:22.96  size: 94847
[libx264 @ 0000025597949c00] frame P:52    Avg QP:25.76  size: 67137
[libx264 @ 0000025597949c00] frame B:132   Avg QP:26.43  size: 56983
[libx264 @ 0000025597949c00] consecutive B-frames:  6.3%  1.1%  1.6% 91.0%
[libx264 @ 0000025597949c00] mb I  I16..4: 17.9% 66.7% 15.4%
[libx264 @ 0000025597949c00] mb P  I16..4: 10.8% 57.5%  8.8%  P16..4: 13.8%  3.5%  1.1%  0.0%  0.0%    skip: 4.5%
[libx264 @ 0000025597949c00] mb B  I16..4:  5.7% 31.9%  6.5%  B16..8: 26.3%  9.0%  1.6%  direct: 3.6%  skip:15.5%  L0:53.8% L1:40.2% BI: 6.1%
[libx264 @ 0000025597949c00] 8x8 transform intra:72.9% inter:83.4%
[libx264 @ 0000025597949c00] coded y,uvDC,uvAC intra: 60.5% 46.3% 8.0% inter: 26.1% 30.2% 0.9%
[libx264 @ 0000025597949c00] i16 v,h,dc,p: 30% 28% 13% 29%
[libx264 @ 0000025597949c00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 18% 19%  6%  8%  8%  8%  8%  9%
[libx264 @ 0000025597949c00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 19% 13%  6% 11%  9% 10%  6%  7%
[libx264 @ 0000025597949c00] i8c dc,h,v,p: 55% 20% 22%  4%
[libx264 @ 0000025597949c00] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0000025597949c00] ref P L0: 56.6%  7.6% 20.0% 15.8%
[libx264 @ 0000025597949c00] ref B L0: 86.1% 10.1%  3.9%
[libx264 @ 0000025597949c00] ref B L1: 94.3%  5.7%
[libx264 @ 0000025597949c00] kb/s:14586.81
[aac @ 0000025599caa040] Qavg: 1098.128
 ====================
ffprobe.exe merged.mp4
 ====================
ffprobe version N-114515-gfa110c32b5-20240329 Copyright (c) 2007-2024 the FFmpeg developers
 built with gcc 13.2.0 (crosstool-NG 1.26.0.65_ecc5e41)
 configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32-   --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --enable-shared --disable-static --disable-w32threads    --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma   --enable-fontconfig --enable-libharfbuzz --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb   --disable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-chromaprint --enable-libdav1d   --enable-libdavs2 --enable-libdvdread --enable-libdvdnav --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r   --enable-libgme --enable-libkvazaar --enable-libaribcaption --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame   --enable-libopus --enable-librist --enable-libssh
--enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libvpl   --enable-openal --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt   --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1   --enable-libtwolame --enable-libuavs3d --disable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc   --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi   --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-libs=-lgomp --extra-ldflags=-pthread --extra-ldexeflags=   --cc=x86_64-w64-mingw32-gcc --cxx=x86_64-w64-mingw32-g++ --ar=x86_64-w64-mingw32-gcc-ar --ranlib=x86_64-w64-mingw32-gcc-ranlib   --nm=x86_64-w64-mingw32-gcc-nm --extra-version=20240329
 libavutil      59. 10.100 / 59. 10.100
 libavcodec     61.  4.100 / 61.  4.100
 libavformat    61.  2.100 / 61.  2.100
 libavdevice    61.  2.100 / 61.  2.100
 libavfilter    10.  2.100 / 10.  2.100
 libswscale      8.  2.100 /  8.  2.100
 libswresample   5.  2.100 /  5.  2.100
 libpostproc    58.  2.100 / 58.  2.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'merged.mp4':
 Metadata:
   major_brand     : isom
   minor_version   : 512
   compatible_brands: isomiso2avc1mp41
   encoder         : Lavf61.2.100
 Duration: 00:00:06.30, start: 0.000000, bitrate: 14731 kb/s
 Stream #0:0[0x1](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1080x1920 [SAR 1:1 DAR 9:16], 14587 kb/s, 30 fps, 30 tbr, 15360 tbn (default)
     Metadata:
       handler_name    : VideoHandler
       vendor_id       : [0][0][0][0]
       encoder         : Lavc61.4.100 libx264
 Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 134 kb/s (default)
     Metadata:
       handler_name    : SoundHandler
       vendor_id       : [0][0][0][0]
 ====================

My guess is, that's burned in now, and not a separate track.

I recollect on other videos, you have

    Stream n:m  Video
    Stream p:q  Audio
    Stream x:y  Subtitles

and then a recipient can strip Subtitles with either
their player settings or with an editor.

When you burn subtitles in, the subtitle is in front
of the video content, so some of the video content
would be obscured if you tried to remove the text the
hard way.

I don't think I've ever added subtitles to video content,
but I've seen references to it in passing, on editing
discussions. They can either be permanently burned in,
or made available as a stream that can be removed.
The subtitles can even have languages, there can be
more than one, and the player selects the English
subtitle for you (using localization declaration the
OS has already got). Maybe a Spanish subtitle, if the
Windows was a Spanish version.

Just as a video can have more than one sound track.
One of the sound tracks should be a "lowest common denominator"
and be playable on any equipment. The others could be
Dolby or whatever.

Maybe you should get some recordings off a TV tuner
card and see how those are constructed. There may be
an option on your TV remote, to turn on subtitles.

In this example, nothing has been able to access the "Thumbnail".

  Stream #0:0[0x2f](eng): Audio: ac3, 48000 Hz, 5.1(side), fltp, 384 kb/s
  Stream #0:1[0x30](enm): Audio: ac3, 48000 Hz, stereo, fltp, 192 kb/s (visual impaired)
  Stream #0:2[0x31](eng): Audio: ac3, 48000 Hz, stereo, fltp, 192 kb/s
  Stream #0:3[0x32]: Video: mpeg2video (Main), yuv420p(tv, bt709, top first),
                     1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, 16999 kb/s, 29.97 fps, 29.97 tbr, 10000k tbn, 59.94 tbc
                     Side data:
                     cpb: bitrate max/min/avg: 16999200/0/0 buffer size: 5652480 vbv_delay: N/A
  Stream #0:4[0x33]: Subtitle: eia_608
  Stream #0:5[0xffffffff]: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 200x113 [SAR 96:96 DAR 200:113], 90k tbr, 90k tbn, 90k tbc (attached pic)
    Metadata:
      title           : TV Thumbnail

A second sample. The second line here must be some kind of joke.
It has one fewer audio tracks. Every program at the TV station,
can be encoded slightly differently if they want.

  Stream #0:0[0xc](eng): Audio: ac3, 48000 Hz, 5.1(side), fltp, 384 kb/s
  Stream #0:1[0xd](enm): Audio: ac3, 48000 Hz, stereo, fltp, 192 kb/s (hearing impaired)
  Stream #0:2[0xe]: Video: mpeg2video (Main), yuv420p(tv, top first),
                    1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, 29.97 fps, 29.97 tbr, 10000k tbn, 59.94 tbc
                    Side data:
                    cpb: bitrate max/min/avg: 24000000/0/0 buffer size: 7995392 vbv_delay: N/A
  Stream #0:3[0xffffffff]: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 200x113 [SAR 96:96 DAR 200:113], 90k tbr, 90k tbn, 90k tbc (attached pic)
    Metadata:
      title           : TV Thumbnail
  Stream #0:4[0xf]: Subtitle: eia_608

Whatever that EIA_608 Subtitle is, it is selectable in VLC as Track 1 in
the Subtitle menu. I can switch the subtitle on and off.

   Paul