Edit HD video on a slow computer with an intermediate codec
We have reached a point in time that consumer cameras can capture HD video. However, editing HD video it is still a source of much confusion and frustration for many people who want to ejoy the benefits of HD but run into problems with large files that their computer can't seem to play or edit.
My new camera shoots video at full HD, which means an image resolution of 1920 by 1080 pixels at 60 frames per second. Using uncompressed video would cause the memory card to fill very fast, so fast in fact that even the fastest memory cards would not be able to keep up.
To compensate for the limited amount of storage speed and space the video (and audio) is compressed. There are many different video compression algorithms in existence, they all have different properties.
The H.264 codec
My camera compresses video using a codec called H.264 which is one of the most powerful codecs in existence today. It results in relatively small files, additionally it has the benefit of being rather universal because it is part of the MPEG4 standard there are decoders available for most modern operating systems and there are even hardware decoders in many mobile devices as well.
There is however one major drawback of H.264: it takes a lot of processing power, for both encoding (making) and decoding (play back). This is not much of a problem for small video clips with a limited resolution and frame rate, but playing back HD video is a problem for many computers that don't have hardware acceleration on the video card.
My Macbook has a Core 2 duo 2 Ghz processor and cannot playback 1080p or even 720p video encoded with H.264 without stuttering, editing it is even worse.
So what can I do with a camera of which the video I cannot play on my computer?
Although it may not seem like it the computer is not incapable of displaying HD video, in fact I have a 24" screen with an even higher resolution hooked up to my Macbook all the time. The only thing required for smooth playback of the video is that the data in the frame buffer (inside the video memory) is updated at least 30 times each second.
Intermediate video codecs
Because the processor can't keep up we need to make it less intensive, for example by using uncompressed video, then the processor would have nothing more to do than move the data from the hard disk to the frame buffer. However, this would require a lot of disk space and a very fast hard disk (or more than one in RAID) to read all this data from, which is something you don't find on most computers.
The solution for this is using another codec, one that instead of trying to create the smallest file size is designed to be the fastest for play back and editing.
There is a number of codecs which will do this. They are called intermediate codecs, designed not for recording or distribution, but right there in between. Examples are DNxHD (Avid), AVC-Intra (Panasonic), DVCPRO-HD (Panasonic), XDCAM (Sony), and AIC (Apple).
The Apple ProRes 422 codec
If you are working on Mac OS X the best choice at the moment is ProRes 422 which is a relatively new codec developed by Apple. Using this codec even an old Macbook like mine can play back 1080p video at 30 fps easily, you can jump to any point in the timeline without any delay, play the video at other speeds, or in reverse without stuttering, hiccups or re-rendering it.
Unlike some of the other codecs ProRes 422 is a full raster codec, which means that the resolution and frame rate is maintained. Other codecs scale down the vertical resolution (for example from 1920 x 1080 to 1280 x 1080 or 1440 x 1080) to save some space which means there is less detail. Other codecs like H.264 or MPEG-2 also have the problem that they are lossy, every time you re-render it there is some loss of detail and shift in colors, that is another reason why they are not very well suited for editing.
Technically ProRes 422 is also lossy, but compression is done in such a way that it is nearly impossible to see with the human eye. Additionally the image quality only degrades the first time the video is transcoded to ProRes 422, and will not degrade further while editing. This last attribute is not unique for ProRes 422 but is the case for for every intermediate video codec.
ProRes is also optimized for processing by multiple processor cores, in my case I only have 2, but if your computer has more cores they will all be put to use. Audio is "passed through" which means that it won't be touched during the conversion process, so there weill be no loss of quality there.
There is a number of variants of the ProRes codec. There is ProRes 4444 for high quality work, and ProRes Proxy for editing on systems with limited disk space and a number of other ones in between. Apple has made a nice graph about this which illustrates the differences.
Please note that this graph is based on only 30 fps, so in my case I get double the data rate with 60 fps!
What codec you should use depends on your camera, the system you are editing with, and the trade off between speed and quality that you want to make yourself.
When importing a video clip from the camera the first time I found out that my Macbook can't play 1080p @ 60 fps. Switching back to 720p@30fps is better but still unusable. The only thing my computer can smoothly play with the H.264 codec is VGA video (640x480@30fps) which is not much better than the old standard definition camera I had before.
It's obvious that I need to convert the video to an intermediate codec. I converted one clip to both ProRes 422 and ProRes 422 Proxy, and played it back several times with the following results:
The standard ProRes 422 files are too big for my liking. The hard disk quickly fills up and can barely keep up causing a drop in the frame rate when the hard disk is busy. The following image is a screenshot taken from the Activity Monitor which shows the data rate of the hard disk while playing the 3 clips:
I have done some frame by frame comparision and could not see any difference between the two codecs, while viewing a single frame at 100% and alternating between them. Even though I cannot see the difference they are different, they have slightly different file sizes which means they are not exactly the same.
The reason that the difference is not noticable is likely due to the composition of the image and the fact that it has already been heavily compressed in the camera, and so it may be different with a different scene, different settings or another camera.
The difference between H.264 and ProRes is just barely visible in a frame by frame comparision, and only if you know where to look. There is very little loss of detail, and nobody is ever going to notice it in moving video, certainly not after transcoding the video again for exporting.
The same frame compressed with different codecs. To allow a lossless image to be uploaded the left side of the image has been cropped off. The difference in brightness is because of automatic gamma correction while transcoding.
Tools for transcoding
Now that we know what we want, how do we get video clips converted to the right format? There are various applications available depedining on what operating system you use. On Mac OS X there is Final Cut Studio with the Compressor application, but also Quicktime (pro) can be used.
However, my favorite tool for this happens to be MPEG Streamclip, which is free and available for Mac OS X and Windows.
One of the nice features is that you can add files to a batch list and leave the computer for a few hours (or days!) while your video is being transcoded.
In the menu select [List] -> [Batch List] A window opens, press the button [Add Files...] select all the files that you want to transcode and click the button [To Batch]. Select the task [Export to Other Formats] and uncheck the other options. Then you can select the destination folder, don't use the folder where the originals are in, you want to keep these files separated.
In the "Other Formats" window select [QuickTime movie] and press [Options]. In the Movie Settings window choose [Settings...] and select your desired codec, for example [Apple ProRes 422 Proxy]. You might want turn off automatic gamma correction here, depending on your camera and how you like the colors. Press [OK] a few times and then [To Batch] when you are ready.
You might want to save all these settings if you plan on doing this again later, just press the button [Presets...] and then [New...] then name it and save it. Select one from the list to load it.
That's all for now, the next time I'll describe how you how to make the best use of these intermediate video files in your editing workflow.
Their, to consumers, most interesting products are Neoscene and Neohd, both are available for OSX and Windows. They claim a tremendous decrease in CPU load.
(If you've got plenty of money; Their, more expensive products (Prospect), offers feature which is, in theory, amazing: the realtime for Adobe Premiere. But perhaps CS5s performance, because of the GPU, already increased that much you won't need the products of Cineform anymore.)
[Comment edited on Tuesday 4 May 2010 23:33]
But it gave me quite some headaches, and still takes a lot of time. The time you win because of copying from a flash-card instead of having to record it from DV-tape, you lose with having to convert it again.
MPEG Streamclip is great for Mac, on the pc it somehow doesn't work that well, for me at least. And it doesn't read MTS, at least not when I tried just now.
I will check out the intermediate codecs. For pc, which works well in Adobe Premiere?
I am not very familiar with the capabilities of the Intel GMA 950 chipset but I do know that it's not very powerful. I seriously doubt if this computer would be fast enough to decode H.264 1080p video at 60fps in real time no matter what the drivers. If there was an update coming I would have expected it years ago. But even if the computer could play the files H.264 would still not be very good for editing.guanche wrote on Wednesday 05 May 2010 @ 00:50:
The problem is not so much your processor capabilities, but more a driver issue on the mac for your videocard. Nice in-depth article, but i think your problems will be resolved woth an upcomming driver update (?)
MJPEG does not have all the features of (the more high end variants of) ProRes. One of the major drawbacks of MJPEG for me would be that it is lossy every time you save/export your video.Skinkie wrote on Wednesday 05 May 2010 @ 01:03:
Whats wrong with the good old jpeg standard for this? Just a frame-by-frame rendering, works as a charm and is easily used in any environment. This HD editing was long implemented products link Cinelerra by just applying an EDL to a different filename. So from mymovie-hd.mov to mymovie-sd.mov.
But as I wrote in the article, you can use any codec you want, ProRes is just an example which works for me and meets my needs in my workflow with my camera, operating system and applications, if you found a workflow that works for you please share it here.
I looked for your next blog where you promised to discuss how to setup your editing software but I cannot find it anywhere.
I just switched over to Final Cut Pro X. Many people hate it, but for what I do it is wonderful. I particularly want to know how to set up the preferences panel. i already turned off the processing in the background part. I edit it to what I want then render it when finished. I also have a copy of Premeire Pro 5.5. If you have good reasons for chosing that platform over another then please explain why. I don't have much use for third party plug-ins, which currently not available for FCPx, and even though some of the controls for FCPx are different I watched the ripple training videos and feel very comfortable with the program now. I can edit very quickly, add color correction, and end up with a great result. I am a little unsure as to what options I should choose in the import panel of the preferences, to use proxy media. create optimize media, or chose nothing. I am going with your advice to encode my original files into ProRes 422 Proxy.
I have been using a canon 600D SLR and filming with 1920 x 1080 at 60fps. I needed to buy an SD Card that was a grade 10 in order to not have dropped frames. But if I am not on a tripod, the camera is very very shaky and slow to set up. I have been editing out the shakes using Motion 5.1 which does a very nice job, I find that using FCPx's analyze for shake not the way to go. I don't feel that comfortable with how the settings work and often end up with a fuzzy image. Just recently I found out about the camera Sony DSC-HX9v which has been blogged as beating out the Canon 600D t3i. I went to the store and tried one out. It was amazing! It is only $300 and so easy to use. The autofocus feature is great if you are filming a person,, it does a fantastic job of tracking them, and its anti-shake is really amazing. The end result looked I had a boom and was so very smooth. But there also is a lot of talk about a whole line of new mirror-less cameras coming out the end of 2011, maybe I should wait.
My biggest question is in the editing preferences and what format I should save the final product as. I primarily use my videos for online marketing and post them on youtube.
Even though Final Cut Pro X and Compressor have a setting that lets you send the final product to youtube. I find that it is unreliable and slower then just saving the final product to my computer and then uploading them directly to youtube through their uploader.
Another question is whether or not to render the project at all. I have read that it isn't necessary. It is slower processing but it also says that rendering first may give you a slightly inferior result. How do you feel about this?
I know I am asking a lot of questions but I am sure there are many people just like me and you that can benefit from knowing these things.
One last thing. I would like to know how to keep this program from building up a lot of memory eating files. I found out quickly that Final Cut Pro X is really bad about connecting to media files. If one is moved, there simply isn't an option for reconnecting the media to the library like I can do in Premiere.