PCI passthrough for paravirtualized guests on Citrix XenServer without an IOMMU

I installed Citrix XenServer 6.0 on an AMD Phenom box. The mainboard does not have an IOMMU which is required to pass PCI devices to non paravirtualized or HVM based guests. The onboard network card is an Atheros AT8131 which XenServer does not support. To pass that network card to a virtual machine, you have to know its device id. Run lspci on the XenServer once (you can ssh into that box or run it on the console):

[root@phenom bus]# lspci
(several lines deleted)
06:00.0 Ethernet controller: Atheros Communications AR8131 Gigabit Ethernet (rev c0)
Still on the XenServer, edit /boot/extlinux.conf and append pciback.hide=(06:00.0) to the Linux commandline:
label xe
  # XenServer
  kernel mboot.c32
  append /boot/xen.gz dom0_mem=752M lowmem_emergency_pool=1M crashkernel=64M@32M console= vga=mode-0x0311 --- \
         /boot/vmlinuz-2.6-xen root=LABEL=root-iongqteq ro xencons=hvc console=hvc0 console=tty0 quiet \
         vga=785 splash pciback.hide=(06:00.0) --- /boot/initrd-2.6-xen.img
Do a extlinux -i /boot. Reboot the XenServer. The guest operating system has to support the Xen PCI frontend. I installed CentOS 6.0 x86_64 and downloaded the vanilla kernel source code from ftp.kernel.org, eg. linux-2.6.39.4.tar.bz2 (kernels shipped with CentOS don't have PCI frontend support). During kernel configuration, make sure that Processor type and features -> Paravirtualized guest support -> Xen guest support and Bus options -> Xen PCI Frontend are enabled. Build and install both the kernel and modules (make bzImage modules modules_install install). Edit /etc/grub.conf and make sure that your freshly installed kernel will be booted by default. Add iommu=soft swiotlb=force to the Linux commandline:
title CentOS Linux (2.6.39.4)
 root (hd0,0)
 kernel /vmlinuz-2.6.39.4 ro root=/dev/mapper/VolGroup-lv_root rd_LVM_LV=VolGroup/lv_root \
       rd_LVM_LV=VolGroup/lv_swap rd_NO_LUKS rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 \
       SYSFONT=latarcyrheb-sun16 KEYTABLE=us console=hvc0 crashkernel=auto rhgb quiet \
       iommu=soft swiotlb=force
Shutdown the virtual machine. Get a list of all virtual machines that are currently installed on the XenServer:
[root@phenom ~]# xe vm-list
uuid ( RO)           : eed0f177-d635-4354-a99e-b21a2886a2c1
     name-label ( RW): Control domain on host: phenom.intra.ogris.net
    power-state ( RO): running


uuid ( RO)           : db4c64e1-44ce-f9f3-3236-0d86df260249
     name-label ( RW): CentOS 6.0 (64-bit) (experimental) (1)
    power-state ( RO): running
Add the network card's device id to the PCI configuration of your CentOS VM:
xe vm-param-set other-config:pci=0/0000:06:00.0 uuid=db4c64e1-44ce-f9f3-3236-0d86df260249
Boot the virtual machine. lspci on the virtual machine shows the network card, presumably with a (virtual) device id like 00:00.0.
To pass multiple devices to a virtual machine, add their device ids to the pciback.hide parameter of the XenServer's kernel commandline:
...pciback.hide=(06:00.0)(01:00.0)...
Add those ids to the configuration of the VM, too:
xe vm-param-set other-config:pci=0/0000:06:00.0,0/0000:01:00.0 uuid=...
Note that due to some VGA bios legacy stuff, you can not pass the primary graphics card of the physical host to any virtual machine.