Complex filtergraphs in ffmpeg may seem complicated at first, but it makes sense once you try it a few times. You need to be familiar with the filtergraph syntax. Start by reading Filtering Introduction and Filtergraph Description. You do not have to understand it completely but it will help you understand the following example.
Example
Use the scale
video filter to scale (resize) the inputs to a specific size, and then use the overlay
video filter to place the videos over the static images.
ffmpeg -loop 1 -i background.png -i video1.mp4 -i video2.mp4 -filter_complex \
"[1:v]scale=(iw/2)-20:-1[a]; \
[2:v]scale=(iw/2)-20:-1[b]; \
[0:v][a]overlay=10:(main_h/2)-(overlay_h/2):shortest=1[c]; \
[c][b]overlay=main_w-overlay_w-10:(main_h/2)-(overlay_h/2)[video]" \
-map "[video]" output.mkv
What this means
Non-filtering options:
-loop 1
Continuously loop the next input which isbackground.png
.background.png
The background image. The stream specifier is[0:v]
It is sized 1280x720.video1.mp4
This first video input (Big Buck Bunny in the example image). The stream specifier is[1:v]
. It is sized 640x360.video2.mp4
This second video input (the varmints in the example image). The stream specifier is[2:v]
. It is sized 640x360.
Filtering options
-filter_complex
The option to start the complex filtergraph.[1:v]scale=(iw/2)-20:-1[a]
This is takingvideo1.mp4
, referred to as[1:v]
, and scaling it.iw
is an alias for input width, and in this case it is a value of 640. We divide than in half and subtract an additional 20 pixels as padding so there will be space around each video when it is overlaid.-1
means to automatically calculate a value that will preserve aspect. If course you can omit the fanciness and manually provide values such asscale=320:240
. Then use an output link label named[a]
so we can refer to this output later.[2:v]scale=(iw/2)-20:-1[b]
Same as above, but usevideo2.mp4
as the input and name the output link label as[b]
.[0:v][a]overlay=10:(main_h/2)-(overlay_h/2):shortest=1[c]
Usebackground.png
as first overlay input, and use the results of our first scale filter, referred to as[a]
, as the second overlay input. Place[a]
over[0:v]
.main_h
is an alias for main height which refers to the background input ([0:v]
) height.overlay_h
is an alias for overlay height and refers to the height of the foreground ([a]
). This example will place Big Buck Bunny on the left side.shortest=1
will force the output to terminate when the shortest input terminates; otherwise it will loop forever sincebackground.png
is looping. Name the results of this filter[c]
.[c][b]overlay=overlay_w*2:overlay_h:shortest=1[video]
Use[c]
as the first overlay input and[b]
as the second overlay input. Using overlay parametersoverlay_w
andoverlay_h
(overlay input width and height). This example will place the verminy varmints on the right side. Label the output as[video]
.-map "[video]"
map the output from the filter to the output file. The[video]
link label at the end of the filtergraph is not necessarily required but it is recommended to be explicit with mapping.
Audio
Have two separate audio streams
By default only the first input audio channel encountered will be used in the output as defined in Stream Selection. You can use the -map
option to add an additional audio track from the second video input (the output will have two audio streams). This example will stream copy the audio instead of re-encoding:
ffmpeg -loop 1 -i background.png -i video1.mp4 -i video2.mp4 -filter_complex \
"[1:v]scale=(iw/2)-20:-1[a]; \
[2:v]scale=(iw/2)-20:-1[b]; \
[0:v][a]overlay=10:(main_h/2)-(overlay_h/2):shortest=1[c]; \
[c][b]overlay=main_w-overlay_w-10:(main_h/2)-(overlay_h/2)[video]" \
-map "[video]" -map 1:a -map 2:a -codec:a copy output.mkv
Combine both audio streams
Or combine both audio inputs into one using the amerge
and pan
audio filters (assuming both inputs are stereo and you want stereo output):
ffmpeg -loop 1 -i background.png -i video1.mp4 -i video2.mp4 -filter_complex \
"[1:v]scale=(iw/2)-20:-1[a]; \
[2:v]scale=(iw/2)-20:-1[b]; \
[0:v][a]overlay=10:(main_h/2)-(overlay_h/2):shortest=1[c]; \
[c][b]overlay=main_w-overlay_w-10:(main_h/2)-(overlay_h/2)[video]" \
[1:a][2:a]amerge,pan=stereo:c0<c0+c2:c1<c1+c3[audio]" \
-map "[video]" -map "[audio]" output.mkv