Home

Building a Gentoo Linux-based DVB PVR

Introduction

Choosing the hardware

Software installation

Installing Gentoo Linux

Selecting the USE flags

Installing and configuring DirectFB

Installing latest Linux DVB tree from Mercurial

Setting console resolution

DVB tuning

Installing and configuring VDR

Postprocessing the recordings

Preparing the recordings for editing

Recordings with DVB/Teletext subtitles

Recordings without separate subtitles

About transcoding the recordings

Getting help

Troubleshooting

Restarting VDR remotely

Introduction

There are hundreds of DVB set-top boxes available. Cheapest ones cost around 35 euros, and even the full feature models with two tuners and recording capability are sold for 300 euros or less. So why should you want to build your own personal DVB video recorder? A couple of reasons come to mind:

One final note before we begin... don't expect all to go smoothly. Most likely some things just won't work, and you don't have a clue why. But you can rest assured that you can make your PVR work, eventually. I'd suggest reserving a week just for this project. You'll probably need much less, hopefully thanks to this HOWTO. As I learned most of this the hard way, it is quite possible that I've unintentionally left out something. In that case feel free to send me an email.

Choosing the hardware

Choosing the right hardware is for your PVR is essential if you want to make it feel like an appliance, instead of a computer. Right hardware does not mean expensive hardware, however. Quite on the contrary, actually. Here are some guidelines:

Case:Anything you see fit. I made my own from pinewood :).
Motherboard:Anything well supported under Linux.
Processor:Any Intel compatible CPU at 700Mhz+ will be enough with DirectFB + xineliboutput.
Memory:128MB should be enough.
Graphics card: DirectFB capable, preferably a Matrox G200/G400/G450 (see http://www.directfb.org).
Soundcard:Anything well supported under Linux (see http://www.alsa-project.org).
Ethernet card:Anything well supported under Linux. Preferably 100Mbps or higher.
DVB card:Anything that well supported under Linux (see http://www.linuxtv.org).
Harddrive:Anything big enough (40+ GB).

If you don't need the PVR to feel like a consumer device, you could use VDR (Video Disk Recorder) on top of X instead of DirectFB. For some video cards that's the only option. Although pretty much all video cards support Framebuffer mode, not that many are well supported in the accelerated DirectFB mode. You will need lots and lots of CPU if you use plain Framebuffer, so don't do it unless you really know what you're doing. I tried running VDR on top of X briefly but I could not get it to work properly. If you have to run VDR on top of X, you will probably need more memory and possibly a more powerful CPU. You also need to do some work to automate starting X and VDR. With DirectFB you can simply add "vdr" service to the default runlevel and be done with it. No logging in or anything. DirectFB is actually quite elegant solution so if you can, go with it.

Software installation

Installing Gentoo Linux

If you're new to Gentoo, make sure that you browse through their excellent documents, or you will run into lots of trouble. During install follow the install guide closely and you can't fail. A DVB PVR is no different from any other computer, so just install a minimal base system and start customizing after that. After base install make sure that you have OpenSSH server (sshd) running, so that you can access the PVR via network. That way you don't need any input or output devices (keyboard, display) after the initial install.

When you're partitioning your harddrive, allocate a partition just for the recordings. You can then use a special filesystem on that partition, for example XFS. This is useful because Ext3 is quite slow when deleting large files, and DVB recordings tend to be quite large. XFS on the other hand handles deleting big files really well.

Depending on your DVB card you might need to get a recent vanilla kernel from kernel.org. If you're using an older DVB card, gentoo's own kernel sources (gentoo-sources) should be sufficient. You don't necessarily have to reconfigure the kernel - try genkernel first, and if it does not work, then reconfigure and compile the kernel manually. I've never used genkernels so I don't know how well they support various hardware devices. If you want framebuffer support from the start (=beginning of the boot), you probably want to build framebuffer support into the kernel, instead of using modules. Alternatively you could create an initial ramdisk and include the framebuffer modules in it, but it's a bit harder than building the support into the kernel. Here's my kernel config for kernel.org's 2.6.20.7 kernel for reference.

Selecting the USE flags

As everyone knows, having appropriate USE flags is essential in a Gentoo system. I tried to keep my use flags minimal and add functionality when needed. The flags that are related to VDR specifically are in bold:

/etc/make.conf

USE="-ipv6 jpeg png tiff imagemagick logrotate mmx sysfs X acpi alsa apache2
dbus directfb dvb fbcon ffmpeg hal lm_sensors matrox mp3 nls ogg pam samba
theora truetype unicode usb vorbis xv xvid mpeg win32codecs mad"

VIDEO_CARDS="mga matrox"
LINGUAS="fi"

/etc/portage/package.use

media-video/vdr dvbplayer jumpplay debug cmdsubmenu setup-plugin submenu subtitles aio
media-video/mplayer X xv
media-libs/xine-lib win32codecs mad oss
sys-apps/lm_sensors sensord
dev-libs/DirectFB fusion jpeg mpeg png fbcon
media-libs/xine-lib v4l
media-video/xine-ui vdr

The USE flag subtitles should take care of both DVB- and Teletext subtitles. Both are VDR plugins, which you'll have to activate at VDR startup. In a Gentoo system there shouldn't have to do anything else to get them working: if the USE flag is activated, VDR should get patched automatically to include subtitle support.

NOTE: xine-lib needs mad-support to be able to decode mp2 audio from DVB streams. This is essential because we're going to use vdr's xineliboutput to play back the DVB transmissions. After you've updated your use flags, it might be a good idea to do a emerge --sync && emerge --newuse --deep world. After that's done, you'd better do a backup of the system, in case you hose it. Easiest way to do it is by piping tar's (TApe Archiver) output to SSH to a remote machine:

digikuutio / # ls
bin   dev  home  lost+found  opt   root   sbin  tmp  var    video.old
boot  etc  lib   mnt         proc  samba  sys   usr  video
digikuutio / # tar -cf - /bin /boot /dev /etc /home /lib /opt /root /sbin /usr /var /video|ssh username@server "cat > /path/to/backup_filename.tar"

Do NOT include /proc or /sys in the backup or it will fail.

Installing and configuring DirectFB

After doing the emerge --newuse --deep world you should install relevant Framebuffer packages if they are not already installed. You probably need only these:

digikuutio / # qlist --installed|grep -i fb
dev-libs/DFB++
dev-libs/DirectFB
sys-apps/fbset

Because I don't want to remove any packages to see if something break, here's full list of packages installed on my system for reference. Note that there are lots of X window system-related packages which are not necessary. When the necessary software is installed, make sure that the relevant modules are autoloaded at boot:

digikuutio / # cat /etc/modules.autoload.d/kernel-2.6 
# /etc/modules.autoload.d/kernel-2.6:  kernel modules to load when system boots.

# Matrox framebuffer acceleration 
matroxfb_accel

# DirectFB sound system (fusion). Not required if you use basic alsa.
fusion


# Various i2c stuff. Possibly not necessary at all unless you're using Matrox
# G400+

i2c-dev
i2c-algo-bit
i2c-algo-pca
i2c-algo-pcf
i2c-matroxfb

# ACPI stuff. Module "button" allows you to receive ACPI button events which can
# then be intercepted by ACPI daemon (acpid). This allows you to safely shutdown
# the PVR just by pressing the power button.
button
thermal

# Evdev is needed if you have a remote which functions as a typical controller,
# similar to a keyboard or a mouse. Usbhid is required for USB keyboards and
# mice.
evdev
usbhid

# Various other modules which might or might not be relevant for you.
e100
snd_pcm_oss
snd_mixer_oss
smbfs

After a reboot or a modprobe session you can try out the framebuffer support by installing mplayer:

digikuutio / # emerge -va mplayer 

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R   ] media-video/mplayer-1.0_rc1-r2  USE="X alsa directfb dvb
fbcon iconv jpeg* mad matrox mmx png* samba theora truetype unicode vorbis
win32codecs xv xvid -3dfx -3dnow -3dnowext -aac -aalib (-altivec) -amr
-arts -bidi -bindist -bl -cdparanoia -cpudetection -custom-cflags -debug
-dga -doc -dts -dv -dvd -dvdread -enca -encode -esd -ggi -gif -gtk -ipv6
-jack -joystick -libcaca -lirc -live -livecd -lzo -mmxext -musepack -nas
-openal -opengl -oss -real -rtc -sdl -speex -sse -sse2 -svga -tga -v4l
-v4l2 -x264 -xanim -xinerama -xvmc" LINGUAS="-bg -cs -da -de -el -en -es
-fr -hu -ja -ko -mk -nl -no -pl -pt_BR -ro -ru -sk -tr -uk -zh_CN -zh_TW"
0 kB 

Total: 1 package (1 reinstall), Size of downloads: 0 kB

Would you like to merge these packages? [Yes/No] No

Quitting.

If the USE flags were set correctly, you can answer Yes instead and install mplayer. After the install finishes, try playing a really basic video file (mpeg2 for example) with mplayer -vo directfb myvideofile.mpg. You can also try dfbmga, fbdev and fbdev2 as video out devices. If directfb and dfbmga do not work, but fbdev and/or fbdev2 do work, there's something wrong with your DirectFB installation. If none of those work, check that you have one or more framebuffer devices:

digikuutio / # ls /dev/fb? 
/dev/fb0

If you don't have any /dev/fbX devices, your kernel configuration does not work or the relevant modules are not loaded. Fix the problem and then try again. I have to admit that I never got mplayer to play files using directfb (just fb). My DirectFB was still disfunctional during this phase.

Installing the latest Linux DVB tree from Mercurial

If you have a relatively old DVB card, have an already supported remote, or you don't want to use a remote control at all, you might be able to skip this step. I had to get the latest Linux DVB tree from Mercurial to use the remote bundled with my Technotrend C-1500. Gentoo-patched kernels did have some support for it as some buttons worked. Vanilla kernels on the other hand did support it at all. Finally I ended up trying the Mercurial DVB tree, which worked 100% with my remote.

The latest development versions Linux DVB tree can be obtained from the Mercurial SVN repository (or was it CVS). This repository contains all the latest patches which will eventually end up into the mainstream kernel. When that happens, you can safely skip this step. Building the Mercurial DVB tree is quite easily, just follow the instructions in the various README's. You can either build all modules, or use the "menuconfig" program to select only those DVB cards and options you need. In most cases you can simply install the Mercurial DVB tree over the in-kernel tree. This has so far worked perfectly for me, but I guess you could simply leave the DVB things out in the main kernel configuration and just install this new Mercurial tree afterwards.

Setting console resolution

If you want to make your console use higher resolution than the standard, you can do so with gtf and fbset. One thing you should know is that manpage of fbset is hopelessly outdated, so it's not of much use. The same goes for other fbset documentation. Note that gtf is included in the x11base/xorg-server package which you will have to install. Of course you can use gtf from another computer also. See man gtf for more info.

digikuutio tmp # gtf 1024 768 60 -f

mode "1024x768 60.00Hz 32bit (GTF)"
    # PCLK: 64.11 MHz, H: 47.70 kHz, V: 60.00 Hz
    geometry 1024 768 1024 768 32
    timings 15598 160 56 23 1 104 3
    hsync low
    vsync high
endmode

You can add these to your /etc/fb.modes -file. If you want to make 16-bit modes instead, just replace the 32 with 16, as highlighted below:

digikuutio tmp # cat /etc/fb.modes
#mode "768x576 80.00Hz 32bit (GTF)"
mode "80low"
    # PCLK: 48.71 MHz, H: 48.32 kHz, V: 80.00 Hz
    geometry 768 576 768 576 16
    timings 20531 120 40 24 1 80 3
    hsync low
    vsync high
endmode

#mode "1024x768 70.00Hz 32bit (GTF)"
mode "70hz"
    # PCLK: 76.16 MHz, H: 56.00 kHz, V: 70.00 Hz
    geometry 1024 768 1024 768 16
    timings 13130 168 56 28 1 112 3
    hsync low
    vsync high
endmode

#mode "1024x768 80.00Hz 32bit (GTF)"
mode "80hz"
    # PCLK: 88.50 MHz, H: 64.32 kHz, V: 80.00 Hz
    geometry 1024 768 1024 768 16
    timings 11299 176 64 32 1 112 3
    hsync low
    vsync high
endmode

Testing these modes is simple (but undocumented):

digikuutio tmp # fbset 80hz

If your monitor goes crazy, do a fbset another_mode by typing blindly and hope it works :). If it doesn't, try changing to another virtual terminal (VT) by pressing Ctrl-Alt-Fx where x is 1-7. When you find a good mode, add it to the local startup file /etc/conf.d/local.start:

# /etc/conf.d/local.start
# This is a good place to load any misc programs
# on startup (use &>/dev/null to hide output)

# Set the correct framebuffer mode
/usr/bin/fbset 80low

# Restore sound card settings (volume levels and such)
/usr/sbin/alsactl restore

DVB tuning

Just like analog TV's, Digital TV receivers have to be tuned. For this you need a tuning file for your region, which you can find with Google. You also need a tool called dvbscan which is included in the ebuild media-tv/linuxtv-dvb-apps. Using dvbscan is simple:

digikuutio ~ # dvbscan -o vdr fi-Turku > /tmp/channels.conf

This will create a channels.conf -file suited for vdr (the -o vdr). See dvbscan --help for more info. Put this file aside for later use.

Installing and configuring VDR

Unless you have bought a full-feature DVB card with hardware MPEG2 decoder chip you will need a software device to decode the DVB transmission. I tried the softdevice plugin, but never got it working correctly (colors messed up, among other things). Therefore I use xineliboutput, which has worked like a charm and can do streaming too. In addition to the software device you might want to install some VDR extras. Here's a list the packages I have installed that are directly related to VDR:

digikuutio ~ # qlist --installed|grep vdr
media-plugins/vdr-femon
media-plugins/vdr-mp3ng
media-plugins/vdr-mplayer
media-plugins/vdr-remote
media-plugins/vdr-softdevice
media-plugins/vdr-subtitles
media-plugins/vdr-ttxtsubs
media-plugins/vdr-xineliboutput
media-tv/gentoo-vdr-scripts
media-tv/vdrplugin-rebuild
media-video/vdr

You will probably want at least the vdr-xineliboutput, vdr-remote, vdr-subtitles, gentoo-vdr-scripts and of course vdr. There's nothing special about installing vdr, just do a emerge and check if it's using correct USE flags and then proceed. Configuring it is a bit trickier.

Here are my config files for reference:

Postprocessing the recordings

Preparing the recordings for editing and transcoding

When I first tried to edit my recordings I ran into irritating problems. Whatever I did all edited material lost audio/video sync to the point of being unusable. I initially thought that this had something to do with transcoding, but the problem persisted even if the audio and video tracks were just copied. So it became obvious that it was the cutting that caused all these problems. I still did not know what the actual problem was. Then I stumbled across an article in Avidemux wiki: Editing MPEG capture (DVB or IVTV). What the article explains is that all mpeg streams - including dvb - are full of errors and thus they need to be fixed prior to editing. Otherwise audio and video will not keep in sync after editing. Video players such as mplayer do a good job of hiding various errors and play the files just fine. In the following chapters I'll cover the possible scenarios for postprocessing and (possibly) transcoding the recordings.

Recordings with DVB/Teletext subtitles

If you have recordings which have separate subtitle streams, you will face difficulties when trying to get them to archivable form and to keep the subtitles at the same time. I'll explain a couple of different approaches which I tested and which seemed to work just fine. In all cases you need to get a recent ProjectX from CVS in order to get the subtitles out in correct (vobsub) format. As of now (July 2007) there is no vobsub export in the released versions of ProjectX. You also need to change some configuration settings in ProjectX to extract the correct subtitles.

Approach 1: Keep subtitles in separate stream. No transcoding. Store in Matroska container

Pros: Fastest way to edit and archive the recordings. It is possible to include several different subtitle streams.
Cons: Files take lots of space and are not playable on most consumer Xvid/DivX players.

This approach is relatively straightforward:

  1. Load the recording into ProjectX. Edit it any way you wish - this is probably your last chance.
  2. Run ProjectX to split the recorded MPEG2 stream into elementary streams and fix any discontinuities.
  3. Load the separate streams (audio, video and subtitles) with Matroska tools GUI and create the Matroska container.

Approach 2: Keep subtitles in separate stream. Transcode video and audio. Store in Matroska container

Pros: Recordings take only a little space. It is possible to include several different subtitle streams.
Cons: Quite a few extra steps needed. Takes lots of processing time. Not playable on most consumer Xvid/DivX players.

  1. Edit the recording with ProjectX, as above.
  2. Split the recording to elementary streams, as above.
  3. Multiplex the video and audio streams together with mplex.
  4. Transcode the resulting (audio+video) MPEG file to whatever format you want. I use Xvid for video and mp3 for audio in AVI container. It is possible that you could transcode the audio and video separately, but I haven't experimented with it.
  5. Add the resulting AVI file to Matroska GUI tool. Next add the vobsub subtitle file to the same container. Then create the Matroska container.

Approach 3: Embed the subtitles into the video stream. Transcode video (to Xvid) and audio (to mp3). Store in AVI container.

Pros: Playable on most consumer Xvid/DivX players.
Cons: No way to turn off the subtitles or store several different subtitles in the same file.

  1. Edit the recording with ProjectX, as above.
  2. Split the recording to elementary streams, as above.
  3. Multiplex the recording with mplex, as above
  4. Open the resulting MPEG file with Avidemux. Use the Avidemux Vobsub filter to "burn" the subtitles into the video stream.
  5. Transcode video to Xvid and audio to mp3 and use AVI as the container. It's easiest to do this all from within avidemux.

Recordings without separate subtitles

NOTE: You can always follow the procedure above, except that you don't have to use a CVS version of ProjectX as you don't have any DVB subtitles to worry about. Therefore, in any case, things will be quite a bit easier. This is also the path I followed before I had to deal with recording that had subtitles in them.

  1. Split the recording to elementary streams with ProjectX, as above. No editing is necessary yet.
  2. Multiplex the elementary streams back together with mplex, as above.
  3. Open the resulting MPEG file with Avidemux, edit it and transcode it, like above.

To make this whole process easier, I created a couple of scripts that do all the work for me:

The mkidx script is not strictly necessary, but if you use avidemux for editing & transcoding, it will save you a lot of trouble. If you don't run it, avidemux creates the index on the fly, which takes a long time: around 3 minutes for a 30 minute recording on a recent 64-bit AMD Sempron. That's why I prefer do these things uninteractively with scripts.

About transcoding the recordings

Some tips regarding the transcoding process are probably in place:

The Avidemux Wiki is a good source of information regarding transcoding, and you should definitely take a look at it.

Getting help

It always best to help yourself, if possible. These are your best friends:

You will find lots of resources from the Internet:

Troubleshooting

Restarting VDR remotely

NOTE: VDR has it's internal watchdog which will restart it if it hangs. In that case you should not have to do any restarting manually.

Sometime VDR just hangs. The watchdog (see above) should take care of these but in the case it doesn't, you can just log in remotely and issue openvt -f /etc/init.d/vdr restart. The openvt -f is required, because otherwise vdr refuses to claim a VT that is already in use. This is not a problem the first time you start it if you use a console that has not been grabbed already (say VT8). If VDR is stopped then this becomes a problem, as the VT is already in use. That's where openvt comes in.


Author:

Samuli Seppanen, University of Turku, Finland
http://users.utu.fi/sjsepp


Registered Linux user #392780