These notes are now OUT OF DATE

Please see the cleaned up HowTo documents at PeterMaydell/KVM/HowTo instead. I'll delete the rest of this text in the not too distant future.

This is work in progress notes on how to build an A15 kernel and get it running on an ARM Fast Model. Note that to do anything useful with KVM under the model you'll need a host system with a decent amount of RAM. 2GB is definitely not enough; 6GB seems OK.

This page deals primarily with:

  • building a kernel and turning it into an ELF .axf file to pass to the model
  • creating a directory tree on the host system that is suitable for passing as an NFS root filesystem (including a cross-compiled qemu binary)

See Loic's notes for how to get and install a Fast Model to run things on. Note that you should now get a version 7.0 Fast Model, and make sure you build the Release version of the RTSM_VE Cortex-A15x1 model. (Unless you want to play with the not-yet-functional KVM SMP support, in which case try x2.) 32 bit models (ie "Linux-Release-GCC-4.1" config) are a bit faster than 64 bit ones (ie "Linux64-Release-GCC-4.1") but have the obvious maximum memory overhead imposed by the 32 bit address space.

The current missing piece of documentation is how to set up networking for the model.

NB: these are put together based on my initial run through. I haven't done a complete retest so there may be minor errors. Please correct them if so!

Install a cross compiler

In oneiric the packages you need should all be in the (universe) archive. Install these:

gcc-4.6-arm-linux-gnueabi binutils-arm-linux-gnueabi libc6-armel-cross linux-libc-dev-armel-cross gcc-arm-linux-gnueabi libc6-dev-armel-cross cpp-arm-linux-gnueabi

Set up a guest filesystem

You'll need at least the qemu-user-static package installed for this.

$ cd /srv
$ sudo mkdir arm-oneiric-root
$ sudo qemu-debootstrap --arch=armel oneiric ./arm-oneiric-root/

When it's done, you'll need to tweak some files in the chroot:

  • add 'ttyAMA0' to etc/securetty
  • copy etc/init/tty1.conf to etc/init/ttyAMA0.conf and edit it to change 'tty1' to 'ttyAMA0' everywhere
  • create an etc/apt/sources.list:

deb oneiric main restricted universe
deb-src oneiric main restricted universe

Note that since this is a qemu-debootstrap-created directory tree you can run "sudo chroot /srv/arm-oneiric-root" to chroot into it using qemu's linux-user emulation. This is particularly useful for installing random extra packages. (If you try to do anything too exciting in the chroot you may run into issues with QEMU. Sorry...)

Get and build the kernel

We're using Marc Zyngier's kernel tree here. WARNING: this tree is temporary and unstable and rebases -- deal with it accordingly.

$ git clone git:// linux-kvm-arm
$ cd linux-kvm-arm && git checkout kvm

Get the a15-config and save it as .config.

$ make oldconfig ARCH=arm CROSS_COMPILE=arm-linux-gnueabi-

(just hit return if it prompts for anything). MAKE SURE TO ENABLE 10/100 Ethernet :)

Edit arch/arm/boot/compressed/Makefile and comment out the line:

asflags-y := -Wa,-march=all

(this is working around a bug building kernels in Thumb mode (fixed upstream, just not in this tree yet).

$ make -j4 ARCH=arm CROSS_COMPILE=arm-linux-gnueabi-
$ sudo make modules_install ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- INSTALL_MOD_PATH=/srv/arm-oneiric-root

Get the boot wrapper

$ cd ..
$ git clone git://
$ cd boot-wrapper

You need to fiddle with the config here. You can do that by copying to and editing it.

  • set CROSS_COMPILE to arm-linux-gnueabi-
  • set KERNEL_SRC if you put the kernel somewhere odd
  • set NFS_ROOT to /srv/arm-oneiric-root
  • if your NFS server is not the local machine, set NFS_SERVER

Then you should just be able to run make to build the kernel and produce a linux-system.axf which you can pass to the model.

The boot-wrapper makefile defaults to passing targets it doesn't know about through to the kernel makefile, so you can rebuild and reinstall modules with:

make -j4 modules
sudo make modules_install INSTALL_MOD_PATH=/srv/arm-oneiric-root

Cross compile QEMU

Install some native build dependencies:

$ sudo apt-get install build-dep qemu-linaro

(This pulls in more than necessary but is an easy way to do it).

We'll use xapt to create and install cross versions of libz and glib, which is the minimal set of dependencies QEMU need (assuming you already have qemu-user-static installed):

$ sudo apt-get install xapt

First create a new file /etc/apt/sources.list.d/armel-oneiric.list with the following contents:

deb [arch=armel] oneiric main restricted universe multiverse
deb-src [arch=armel] oneiric main restricted universe multiverse

Then run

$ sudo xapt -a armel -m -b zlib1g-dev libglib2.0-dev

This will build a pile of _all debs in /var/lib/xapt/output/; we install only the minimum set we need:

$ sudo dpkg -i /var/lib/xapt/output/multiarch-support-armel-cross_2.13-20ubuntu5_all.deb \
 /var/lib/xapt/output/zlib1g-armel-cross_1.2.3.4.dfsg-3ubuntu3_all.deb \
 /var/lib/xapt/output/libffi6-armel-cross_3.0.11~rc1-2_all.deb \
 /var/lib/xapt/output/libpcre3-armel-cross_8.12-3ubuntu2_all.deb \
 /var/lib/xapt/output/libselinux1-armel-cross_2.0.98-1.1_all.deb \
 /var/lib/xapt/output/libglib2.0-0-armel-cross_2.30.0-0ubuntu4_all.deb \
 /var/lib/xapt/output/libpopt0-armel-cross_1.16-1_all.deb \
 /var/lib/xapt/output/pkg-config-armel-cross_0.26-1ubuntu1_all.deb \
 /var/lib/xapt/output/libglib2.0-data-armel-cross_2.30.0-0ubuntu4_all.deb \
 /var/lib/xapt/output/libglib2.0-bin-armel-cross_2.30.0-0ubuntu4_all.deb \

Put this arm-linux-gnueabi-pkg-config script on your PATH somewhere, and make it executable, eg:

$ chmod a+x /usr/local/bin/arm-linux-gnueabi-pkg-config

Get and build QEMU:

$ git clone git:// qemu
$ cd qemu
$ mkdir arm-build
$ cd arm-build
$ ../configure --cross-prefix=arm-linux-gnueabi- --target-list=arm-softmmu --enable-kvm
$ make
$ sudo make install DESTDIR=/srv/arm-oneiric-root

This will build QEMU with the cross compiler and install it into the nfs root's /usr/local/bin.

Run the model

$ ~/ARM/FastModelsTools_7.0/bin/model_shell64 ~/ARM/FastModelsPortfolio_7.0/examples/RTSM_VE/Build_Cortex-A15/Linux64-Release-GCC-4.1/ linux-system.axf -C motherboard.smsc_91c111.enabled=1 -C motherboard.hostbridge.userNetworking=1 -C motherboard.vis.rate_limit-enable=0

(Note that this uses the "user networking" feature of Fast Models 7.0. Paths may differ slightly if you built a 32 bit model.) If this fails to mount its root fs then you probably forgot to set the NFS server IP address in the boot wrapper, or perhaps misconfigured your host /etc/exports.

The above command line disables rate limiting. Rate limiting keeps the model running at no greater than a particular 'real time' rate. Disabling this means it will always run as fast as it possibly can; this is good for being a bit faster at certain things, but on the other hand also means it will use 100% of your host CPU even when the guest is in an idle/halt loop. You can also toggle this by clicking the red square below the "Rate limit ON/OFF" label in the visualization window.

I'm going to assume you know how to run a guest kernel under QEMU. By default this will use TCG, so use '-enable-kvm' on the qemu-system-arm command line to select KVM.

PeterMaydell/A15OnFastModels (last modified 2012-01-30 19:34:14)