Editing videos with ffmpeg

Want to show off your awesome game and a plain video recording won't do? Maybe you don't have a video editor or can't use one, but ffmpeg will run just fine? If so, here's a list of common tasks and how to achieve them using ffmpeg.

Converting a video to MP4

Most sites these days want videos in MP4 (or support it as one of their formats). In theory, converting a video to MP4 is as simple as this (where the -i option is the input file, and the one without an option is the output): 

ffmpeg -i filename.avi filename.mp4

The problem is that in practice it doesn't work: ffmpeg prefers to default to YUV 4:4:4 because it's higher quality, but many sites only accept YUV 4:2:0. To work around this, we need to pass the -pix_fmt option to switch to the correct format: 

ffmpeg -i filename.avi -pix_fmt yuv420p filename.mp4

Resizing the picture

The video we recorded may be in the original resolution, which for many retro systems is low and to make it worse will become extremely blurred after encoding to a lossy format like MP4. To compensate for this we have to resize the video using the -s option: 

-s widthxheight

So e.g. if the video was 320×240 and we want to make it double the size: 

-s 640x480

Note that by default ffmpeg will use bilinear filtering when resizing. This is OK for most stuff, but when dealing with pixelart we prefer to keep the pixels sharp (the video encoding itself will soften out the borders anyway), so we can use the -sws_flags option to change this: 

-s 640x480 -sws_flags neighbor

Cutting a chunk of the video

Maybe you only want to keep part of the video instead of all of it. You do it as follows: 

-ss start -t length

The timestamps are written as hh:mm:ss.ss, i.e. hours, minutes and seconds (and centiseconds). Being able to specify fractions of a second is useful since sometimes things go really fast and the part you want to keep starts halfway through a second or something like that.

For example, if the part we want to keep begins halfway through 1:23 and we want to keep half a minute: 

-ss 00:01:23.50 -t 00:00:30.00

Cropping the picture

Sometimes you don't want to keep the entire picture but only crop a small portion of it. You can use the crop video filter to do this (note that dimensions come before the position): 

-vf "crop=width:height:x:y"

Take into account that cropping is done before resizing, so size and position need to be passed based on the original resolution and you'll need to adjust the resizing accordingly as well.

For example, to only show the center of the screen in a 320×240 video: 

-vf "crop=160:120:80:60"

Putting multiple videos together

You want to make a trailer and it shows different parts of your game, which you've recorded each into their own video. Now you need to put them together to upload as a single video (or add your own track on top).

For this we need to use filters. It looks kind of a mess but it's easy to understand what's going on once you pay attention. The following has been split into multiple lines for clarity but you must write this into a single line (or whatever your shell allows you to do). 

ffmpeg -i video1.avi -i video2.avi -i video3.avi
       -filter_complex "[0:v][1:v][2:v]concat=n=3:v=1:a=1[out]"
       -map "[out]" output.mp4

What's going on here:

The concat filter is used to string together multiple videos in a row. The [0:v], [1:v], etc. before the concat word is each of the original videos (make sure there's one for each video), while the [out] after it is the name of the resulting video it makes (in this case, what we pass to -map later).

The n=3 argument to concat indicates how many videos are being concatenated, make sure it matches the number of videos. The v=1 and a=1 tell it that we want one video and one audio stream, if you want to drop the audio you can pass a=0 instead.

Trim and put together at the same time

Since we're using -filter_complex that can take much more complex operations, we can go ahead and trim the videos at the same time, in case we only need a portion of each video: 

ffmpeg -i video1.avi -i video2.avi -i video3.avi
       -filter_complex "[0:v]trim=0:5[v0];
                        [1:v]trim=60:65[v1];
                        [2:v]trim=0:120[v2];
                        [v0][v1][v2]concat=n=3:v=1:a=1[out]"
       -map "[out]" output.mp4

The trim filter works similar to the -ss and -t options from earlier, it just happens to work in the intermediate streams used by filters so you don't need to make separate files for each video you need to cut down.

The trim takes an argument in the form start:end, which are the beginning and end of the section to keep (in seconds). In the example above, it uses 0:00 to 0:05 from the first video, 1:10 to 1:15 from the second video, and 0:00 to 2:00 from the third video. Then all this is passed to the concat filter to merge them together into a single stream.

The trim filter also can take other arguments, and there are many other filters you can try, so consider taking a look at the ffmpeg filter documentation when you can.

Mixing video and audio

Continuing from the above, let's say you're making the trailer for your game and already put together all videos ­— now you need the trailer's music to play alongside them. To do this you need to grab the video from one file, the music from another and put those two together into the final movie.

It's… kinda tricky but the basic idea is as follows (note that the command has been split into multiple lines for clarity here, but you must do it all in a single line): 

ffmpeg -i video.avi -i music.wav
       -map 0:v:0 -map 1:a:0
       output.mp4

Explanation of what's going on:

In practice however there's a high chance that the source videos were already edited and you're just trying to merge the final result. Trying to reencode them would result in a waste of time and worse quality, so we can instead use the -c option to tell ffmpeg what codecs to use: 

ffmpeg -i video.mp4 -i music.wav
       -c:v copy -c:a aac
       -map 0:v:0 -map 1:a:0
       output.mp4