(Note that since I first published this post I have found an alternative, direct way to embed video into blogs where the video quality is as high as I wish. Details are here.)

Scenario

Input: The video to be encoded is an avi file in DV format, such as you might capture from a camcorder using Windows Movie Maker or other video editing application. The video may have been edited, so long as the edited footage has then been saved in DV format in an avi file. Audio content is assumed to be in PCM (uncompressed) format, typically with a 48,000 Hz sampling rate.

Output: The encoding process results in an mp4 file containing H.264 encoded video and AAC audio. The output file is suitable for uploading to on-line video hosting services such as youTube or vimeo.

Defaults: The default settings are video resolution of 448 x 336 and H.264 encoding quality of 26. These can be overridden by changing the relevant lines in the video.avs and video.bat files (see below).

Preliminary

It makes sense to create a folder on your PC where you will keep the relevant programs and working files. For the purposes of the post I will assume this is c:\videoenc

What you need to download and install

AviSynth

AviSynth 2.5 is a frameserver which we use here to perform the preliminary processing such as cropping and deinterlacing. Download the file Avisynth_257.exe from here. Run it to install AviSynth. Accept all defaults.

Smooth Deinterlacer

Get it from here and save the file smoothdeinterlacer.dll to c:\videoenc. Be sure to select the version designed to work with AviSynth 2.5x not the older version for AviSynth 2.0x.

x264

This is the H.264 encoder program itself. You can get it from here*. Click on any of the mirror links, e.g. mirror 01, next to x264.exe in the table of encoder download options. Save the x264.exe file to c:\videoenc. You don’t need to install anything. The file just needs to be in that folder.

[*Update 5 Dec 07: The link above appears to be down or unreliable. I have found another source for the x264.exe file here but have not yet had the opportunity to check its origin, version or to test it]

MP4Box

MP4Box takes the raw compressed video file and places it in an mp4 container file. The program is available from here. I downloaded version 0.4.4 compiled 3 June 2007. Again, you just need MP4Box.exe to be in the folder c:\videoenc.

MPlayer

You can get this from here. Download “MPlayer 1.0rc1 Windows” (not the GUI version) from any of the mirror site links. You need MPlayer to extract the uncompressed audio from the DV video file, so it can then be compressed as AAC audio. You only need the mplayer.exe file. Put it in c:\videoenc.

faac

The utility that compresses your extracted audio in AAC format. Available from here. Download the faac-1.26.1.zip file. Place the faac.exe file in c:\videoenc.

Creating the text files that automate the process

Within the c:\videoenc folder create a text file called video.txt. Open it and copy the following text into it:

LoadPlugin(“SmoothDeinterlacer.dll”)
DirectShowSource(“c:\videoenc\video.avi”)
SmoothDeinterlace(tff=false, doublerate=false)
LanczosResize(448,336)
converttoyv12()

Close it, saving the changes. Change the filename to video.avs.

Next create another file called video.txt. Open it and copy the following text into it:

@ECHO OFF
x264.exe –fps 25 –qp 26 –progress –output video.264 video.avs
MP4Box -flat -add video.264:fps=25 -v -new video.mp4
mplayer -vc null -vo null -ao pcm:fast video.avi
faac -b 128 –mpeg-vers 4 audiodump.wav
MP4box -add audiodump.aac video.mp4
PAUSE
CLS
EXIT

Close it, saving the changes. Change the filename to video.bat.

NOTE: The commands in video.bat as shown above assume a PAL DV source hence a frame rate of 25 fps. If your source video is in NTSC format you need to modify lines 2 and 3 to refer to a frame rate of 30 fps.

Encoding a DV avi file

Just save your raw or edited DV file to the c:\videoenc folder. It has to be called video.avi so save it under that name or rename it as applicable.

Double click the video.bat file in the same folder.

That’s it. The whole encoding process will now run from beginning to end under control of the commands in that file. You can go and make a cup of coffee.

Meantime a DOS box will open and display lots of stuff. Eventually, the display stops changing and the bottom line reads “Press any key to continue …” When you do that the DOS box disappears.

You can then find your encoded file, containing H.264 encoded video and AAC encoded audio, in c:\videoenc. It is called video.mp4. You can now upload it directly to youTube, vimeo, veoh, whatever.

Changing settings

The default resolution is 448 x 336 pixels. If you want something different just change line 4 of the video.avs file. You can just open the file with Notepad to make the changes. You must though ensure that both the horizontal and vertical resolutions are multiples of 16.

The default H.264 encoding quality is 26, on a scale from 1 to 51, using single-pass encoding. You can modify the quality setting by changing the “qp” parameter in line 2 of video.bat. Lower values for qp improve quality but increase the bitrate and filesize, vice versa for higher values.

In principle it should be possible to obtain better quality for the same filesize by using 2-pass or 3-pass encoding. So far I haven’t seen much improvement in my experiments with multi-pass encoding but it’s early days. I’ll report on my discoveries in a future exciting episode of Vista’s Video Nasties.

Credits

I found this very helpful when I was getting started with a command line solution for use of x264. It seems to be part of an encoding guide that looked highly promising but was abandoned years ago. Shame.

The rest of it was down to trial, error and slog.

I have my solution sorted and it will appear shortly as #4 in the series.

What gets me is that I have had to resort to DOS-style batch files, having had no joy with Windows user interfaces for x264 such as meGUI.

The batch file approach is not so bad. Once all the right command line instructions are in place you can double-click your batch file and the entire encoding process is pretty much automatic from there.

It does though seem ironic that despite two decades of successive versions of Windows, each supposedly a step forward from its predecessor, the technology is still so rickety that you can find yourself having to resort to the world of DOS when things get too complicated.

As mentioned in earlier posts in the Vista’s Video Nasties series, it has not exactly been plain sailing.

I abandoned the XviD codec because it could be relied upon to crash Vista’s COM Surrogate process, and accessing an XviD file in a Windows folder could bring down and force a restart of Windows Explorer.

For a while I switched to DivX, which did not seem to upset Vista’s delicate constitution as much. This at least allowed me to bring the COM Surrogate back within DEP‘s protective fold.

I then turned my attention to H.264 in the search for better quality at lower file sizes, having decided that Vimeo, rather than youTube, was the way to go. Now for Mac users, producing H.264 encoded video is trivial. It drops out of iMovie without a second thought. For Windows users, the options are far more limited. That probably explains why Vimeo’s guidance on preparing video for upload is so Apple-centric.

I wanted to experiment with H.264, but there is only one free option, namely x264. This is a command line program for encoding avi files into H.264 format. The encoding options are numerous and baffling, but at least there is a GUI program for it called meGUI which handles all the details. This includes the automatic creation of an “avs” file whose job it is to control a pre-processing tool called AviSynth which I am familiar with from my old DVD encoding experiences. AviSynth handles the resizing, deinterlacing and so forth which is required before passing the video to x264 for compression.

It seemed too good to be true, and was too good to be true. I could open meGUI but whenever I tried to open up a video file for processing I was greeted with a Windows dialog box announcing the fact that “meGUI had stopped working”, whereupon meGUI closed.

I spent a while looking for answers on the Internet, but was not able to find any. I could run meGUI on XP which is how I produced the H.264 encoded clip of Sunlight Rock (Gulangyu Island, China) which I then uploaded to Vimeo. But nothing could induce meGUI to run on Vista.

The only advice I was able to find on the web was that it might start working if I reinstalled Vista from scratch. Really useful. Thanks, pal!

But then I decided to start dabbling in video work, for the first time in earnest since acquiring a Vista PC. I’m now in a world where the COM Surrogate crashes with alarming regularity, the video tools I try to use (eg meGUI) stop working and even Windows Explorer keeps going down. To add to this disconcerting mix, a couple of days ago Outlook 2007 crashed while downloading mail (it looked like a buffer overrun stopped by DEP) but I couldn’t really tell whether that was related to all the other crashes or if it was something genuinely up with Microsoft Office. In any event, Vista invited me to install the latest Office updates and Outlook seems to have been fine since.

The Explorer crash seems to be associated with video files that have been consigned to the Recycle Bin, in particular videos encoded with codecs that Vista or possibly DEP don’t like. I would open the Recycle Bin and select all the files, to see how much space the Bin was taking up. That would be enough to crash Explorer. Selecting the one problem file would also do it. I could not delete it singly – any attempt to highlight it for deletion would bring on the Explorer crash – so I had to empty the whole Bin.

I also tried running DEP at its less intrusive level, where it only guards against exploits in Windows components, but that did not stop the flurry of crashes so I turned it back up to full coverage again.

My confidence in Vista has been shaken again. I’m well out of comfort land and have had to return to XP for recent video encoding work. Not satisfied that this is sustainable in the long run I’ve started the slow painful process of finding workarounds so I can encode my video reliably in Vista.

I will continue to document my experiences on this blog, in future posts under the Vista’s Video Nasties series. The aim is to arrive at robust methodology for creating high quality video that I can upload to one of the better on-line video hosts such as Vimeo. I will document the definitive process for anyone who is interested.

Wish me luck and patience.

This time I made the cardinal error of installing the XviD codec so I could compress some video taken on my travels for upload to YouTube. A new error message started to appear:

“The COM Surrogate has stopped working”

Huh? What the hell is the COM Surrogate, thought I? Well, it turns out this is the new Vista name for the dll host service (dllhost.exe). This is the dummy process that acts as a host to “unattached” dlls, dynamic link libraries that are normally called by applications to do common tasks for them. Sometimes these dlls are required to run “on their own”, rather than being loaded up by a specific program, and Windows provides a host process, dllhost.exe, to load them up and run them.

OK, dllhost is not exactly an inspired name but COM Surrogate is hardly clearer. It just puts me in mind of stand-in mothers and biological engineering. Horrid name.

Anyway, certain video codecs seem to fall foul of Data Execution Prevention (DEP), XviD being one. Vista itself may or may not be implicated, I’m not sure. Whichever way, DEP kills the host process, resulting in the error message.

The only cure I have to date is to add dllhost.exe to the list of applications whitelisted for DEP purposes. It is not dreadfully satisfactory. What if some real malware exploited a buffer overrun vulnerability in a dll hosted by the COM Surrogate? I’ve now exempted all dlls running under the COM Surrogate from DEP checks, not just XviD.

I may look for a less troublesome codec. I only picked on XviD because of the recommendations on the youTube website.

PS (added 7 Nov 07):  With hindsight this post is really a precursor to the Vista’s Video Nasties series of posts, so I’m giving it the honorary subtitle of Vista’s Video Nasties #0.