The chip has a hardware video decoder. It is described in the "i.MX5x VPU Application Programming Interface Linux Reference Manual" PDF. It requires libraries, which are in the imx-lib-11.01.00.tar.gz package (source, apparently), binary firmware, which I believe is probably in one of the other packages in the nested tarball but which I've copied from /lib/firmware/vpu/vpu_fw_imx53.bin on the Freescale Ubuntu filesystem, and a kernel driver, which is present in the Freescale kernel patch.
A separate hardware block is the Image Processing Unit (IPU). This is described in the "Image Processing Unit V3 (IPUV3) Library User's Guide" PDF. This does image scaling, roation, cropping, colour space conversion, and similar functions. Unlike the VPU it doesn't require binary firmware; it does require a kernel driver and a user-space library, which are also in the Freescale kernel and the same libary source package.
The library seems to depend on linux/videodev.h, which is a v4l version 1 thing that is now obsolete. It also uses some kernel headers added by its own kernel drivers. I hacked this with a symlink from /usr/local/include/linux to /usr/local/src/linux-2.6.35.3/include/linux. Then the libraries can be built:
$ cd /usr/local/src/imx-lib-11.01.00/ipu $ make PLATFORM=IMX53 $ cp mxc_ipu_hl_lib.h /usr/local/include $ cp libipu.so /usr/local/lib/ $ cd ../vpu $ make PLATFORM=IMX53 $ cp vpu_lib.h vpu_io.h /usr/local/include $ cp libvpu.so /usr/local/lib
(I don't recommend "make install" unless you want things in /usr.)
There is a test program in the imx-test-11.01.00 package, in the test/mxc_vpu_test directory. It builds with just "make" (but fails immediately after creating the executable). This can then be run on any bit of MPEG video you happen to have:
# ./mxc_vpu_test.out -D "-i /tmp/vid.mpeg2 -f 4 -o /tmp/dec -c 5"
That decodes the first 5 frames of the MPEG-2 video into a file. Note that this MPEG-2 input seems to want just a video stream, not a "TS" (transport stream) which is what tzap gets from the DVB device. The raw output format seems to be 720x576 luminance followed by quarter-size chroma images, i.e. "display -size 720x864 -depth 8 GRAY:/tmp/dec" shows this:
That's called a "planar" image. It's possible to send the video to the screen like this:
# ./mxc_vpu_test.out -D "-i /tmp/vid.mpeg2 -f 4 -x 0"
"-x 0" means to output using the v4l API. 1 means using the IPU (Image Processing Unit), and that also works.
I've written a test program that uses the VPU and IPU to decode an MPEG-2 file and display it on the framebuffer. This is a learning exercise to understand the VPU and IPU. Essentially, it just creates a VPU MPEG-2 decoder instance, feeds it input via a circular buffer, allocates some number of framebuffers, and retrieves decoded frames. These are fed to the IPU for YCrCb to RGB conversion and scaling and and to be displayed on the screen.
The VPU driver needs physically and virtually contiguous buffers, and I have seen its memory allocations functions fail (though I can't be certain that this is due to fragmentation). As it happens, LWN has an article about this; the driver currently calls dma_alloc_coherent(), and the proposed new feature will apparently make this more likely to succeed by shuffling physical pages around as necessary. I'll investigate using this once it's available in a kernel that I can use.
This seems to need only about 1% of the CPU, which is good.
The main challenge has been the documentation. It is far from bad - many things are documented much less thoroughly than this stuff - but it could be so much better with a bit of effort. For example, it would benefit greatly from a glossary of acronyms; maybe things like VDI, VF, and PP all mean something to video experts, but I can only use trial-and-error!
I have attempted to enable a de-interlacing filter in the IPU, but I can only make it work in one of its modes. I really don't understand what is going on with interlacing; can anyone help?
The raw data from the DVB device (/dev/dvb/*/dvr*) is an MPEG "Transport stream" (TS) which wraps a "Packetised Elementary Stream" (PES); the data from a DVD (.VOB file) is a "Program Stream". The VPU needs a "elementary" video stream. So I've written a couple of utilities to do the necessary unpacking, and I can now view video from both sources (though not in real time; only via a temporary file at present).
I've now found that the DVB driver also provides /dev/dvb/*/demux* which do TS decoding, and it has a usable API in libdvbapi. It also seems that the hardware MPEG-2 decoder will accept a PES, as long as it only contains the video data; helpfully the DVB API can also do PES filtering. Software decoding of one or other wrapper will still be needed for e.g. audio, DVD streams, etc.