Definitions & Limitations

First, I should provide some definitions and/or limitations; in order to make any claims we first need to lay down some commonly accepted premises. I’m not going to argue that the following definitions are universally correct or even that they represent any general consensus; this is just how I define the following terms and what these definitions mean — subjectively — to me.

1080p

This one is going to be controversial. Technically speaking, a video is defined as within the 1080p standard if (a) its picture is progressive (non-interlaced) and (b) the picture is made up of 1,080 rows of pixels. What is implied but not specifically stated in this definition is that the aspect ratio of a 1080p picture is 16:9, yielding a resolution of 1920×1080. However, since many movies are filmed in wider aspect ratios, commercial 1080p media is often letterboxed in order to avoid cropping. As such, for my purposes, I define 1080p to be any picture with 1,920 columns of pixels.

H.264

The H.264 codec is an ugly multi-headed beast. The specification for this particular video codec provides many different profiles which all require different amounts of processing power to encode or decode. There are all sorts of available ‘options’ within the standard and many of them probably require a supercomputer just to decode (in 1080p) in realtime. For the purposes of this discussion, I’ll say that we’re dealing with the baseline or main H.264 profiles, using a reasonable amount of b-frames and reference frames — say, one to three — at a reasonable bitrate of about 10mbit (average) for 1080p films, using CABAC. Most of my test clips were encoded using the wonderful x264 open-source H.264 encoder.

Hardware

When I set out to build a Home Theater PC (HTPC), I wanted to keep 1080p H.264 playback in mind, and knowing that this problem is computationally difficult, I bought fairly high-end hardware with budget somewhat in mind. I won’t argue for or against my hardware, since that’s a whole different debate better suited to people more so inclined. I will say that my hardware works me.

Processor

Whilst not getting into the AMD vs. Intel debate, I will say this: I feel that going with AMD is more cost-effective. At time of purchase, the chip I bought had just had a huge price drop and as such became quite attractive. Once I got the components together and verified that everything worked, I overclocked this bad-boy from its stock 3.0Ghz (200Mhz x 15) to 3.15Ghz (210Mhz x 15) just to squeeze a bit (5% to be precise) out of it. Whilst I could get it to run at 3.3Ghz, it wasn’t stable and I wasn’t willing to bump the voltages up too much on the stock cooling I have. Currently this processor runs at around 35°C idle and around 65°C under load.

Motherboard

This is where I spent a lot of time researching. I wanted a motherboard that had as many integrated components as possible. Most motherboards these days have integrated sound but not all have integrated sound with optical out, which was an absolute necessity for me. I was also looking for a motherboard with integrated video. I found this one with the added bonus of having integrated HDMI. I couldn’t get the sound pass-through over HDMI working under Linux, but since I have an optical connection to the receiver anyways I’m not too fettered about it.

RAM

Quite simply, just whatever cheap dual-channel 800Mhz RAM compatible with my motherboard I happened to find on newegg.com.

Video

As I found out just after putting all of the hardware together, ATI’s drivers for Linux do not support XVideo (hardware accelerated video scaling, color-space conversions, and display), rendering the integrated video card on the motherboard less useful. I decided to buy the cheapest PCI-Express NVIDIA card I could find, making sure that it was fully supported with the latest NVIDIA Linux drivers. More on this in the software section below.

Display

Most new HDTVs now have some sort of direct-from-computer input — mine has a DVI port right on the back. Some HDTVs, however, do some overscanning and other image processing on the signal received, even for digital signals. Depending on the source, this can be good or bad, but in general when attaching a computer to a display you want a 1:1 (no overscan) pixel mapping. Fortunately my display supports this via its DVI port.

Software

There was no doubt in my mind that I’d be running Linux, so hardware acceleration of H.264 playback was immediately a no-go: the driver support just isn’t there yet. It turns out that I’m pretty much stuck with ffmpeg’s H.264 decoder for playback (I won’t get into the controversial CoreAVC-on-Linux fiasco here other than to say I’m not about to go hacking up mplayer code to get CoreAVC on Linux under 64bit). But there’s more to playback than just decoding the input video stream; once the stream is decoded, there is an often necessary colorspace conversion to do, there may or may not be some scaling involved, and of course you have to get the decoded image to the display somehow, which can take significant amounts of compute time.

Decoding

The ffmpeg H.264 decoder is really great with one major caveat: it’s not multithreaded. This means that no matter how many cores you have, H.264 decoding will use exactly one. I hear there’s some work being done on parallelizing said decoder, but as of yet I’ve heard of no working prototypes let alone a release. Knowing this is mostly why I wanted such a high clock speed for my main processor, regardless of how many cores it has — for sure I would have bought a cheaper, slower dual-core processor had the ffmpeg H.264 decoder been capable of taking advantage of two cores simultaneously.

That having been said, there are some decoding options that you can pass to ffmpeg’s H.264 decoder via mplayer that will improve performance at the cost of quality. Specifically, there’s the ability so skip all of the in-loop filters (deblocking, mainly), which should yield a performance increase of ranging from zero to three hundred precent or more depending on the source material and encoding options. Simply invoke mplayer with the option -lavdopts fast=1:skiploopfilter=all — you can read more about these in your mplayer manpage. If your processor just isn’t capable of handling H.264 videos you’re trying to play, a good first pass is to try disabling in-loop filters. If your processor still can’t handle it, it’s time for you to upgrade — there’s pretty much nothing else you can do.

Displaying

Once a video stream has been decoded, it is typically sent to a display for viewing. Often this involves colorspace conversions and/or scaling. In my case, with 1080p playback, there is typically no scaling necessary as my display is exactly as large (at least in the limiting dimension) as the image being displayed. In the case of 720p on a 1080p display, however, some upscaling needs to be done. Some upscaling and colorspace conversion computations can be done in hardware on your graphics card if your graphics drivers are capable and set up for it. One such technology under Linux is XVideo, or XV for short.

XV can handle colorspace conversions and upscaling natively in graphics hardware so that your CPU doesn’t have to burn cycles doing so. In the absence of XV, I have found mplayer’s X11 output driver to be the best on my setup. Even though the X11 processing seemingly went on in the X11 server (according to top), which of course does not run on the same core as mplayer so as to maximize performance, I found that using the X11 display driver cause a significant decrease in video playback performance — enough to drop a few frames in some test clips. Since the ATI drivers for the integrated graphics chipset on my motherboard do not provide XV capabilities at time of writing (note that this is a specific case for my video chipset, the X1250 — most other ATI chipsets boast XV support in the proprietary ATI drivers), I bought a cheap NVIDIA card that I knew would work with XV. Once I had XV working, I was able to play back clips that had previously stuttered due to dropped frames.

It is my preference, however, to not upscale videos using XV, as generally I find the results to be much better when using a decent software upscaler. This is especially visible when viewing, for example, NTSC television captures on a 1080p display. There are a number of software upscale implementations bundled with mplayer, and for most people the default will suffice. If this is a bit too CPU-intensive, though, I find that the fast bilinear algorithm does a fairly good job and is a little bit less CPU intensive than the default bicubic algorithm. To select this, simply invoke mplayer with the -zoom -sws 0 arguments. Of course, the quality of the upscale will depend on the source and the algorithm of choice, so as with anything else, try a few of the different options and go with what you prefer.

A Note On Sound

I run an optical line between my HTPC and my sound receiver and use it for both PCM and surround (Dolby or DTS) sound. While this article is, as the title suggests, about video, I feel compelled here to mention that decoding surround sound in software does take a nontrivial amount of CPU time. You’ll be much better off — for a number of reasons, compute time inclusive — sending the audio stream to a hardware decoder (read: sound receiver) for playback there. Sometimes the little bit of difference that decoding sound can make is what you’ll need to avoid a few dropped frames here and there.