This page is not a howto but rather tries to give some background on how the SOCs targeted by Linaro boot along with some history to help make sense of the way things are.

Table of contents:

History

See http://en.wikipedia.org/wiki/Booting for more back ground.

At startup any CPU needs a place to start executing instructions. The place in a CPUs memory map that it starts executing is typically called the reset vector.

NOR

Until a few years ago, a typical embedded device booted from NOR flash. The great thing about NOR flash is that it is randomly accessible like RAM so one can wire it up to a CPU with whatever logic needed to make it appear in the CPU's reset vector. NOR flash is expensive compared to NAND so vendors wanted to be able to boot from NAND.

NAND

NAND is cheap compared to NOR but the problem is that NAND flash is completely different than NOR in how it is accessed. It is not a random access device, it behaves more like a storage device. Commands and addresses of a requested page are fed to the NAND chip through a one word port then the data is read back through the same port. This one word window into the data is pretty useless for holding instructions for a CPU. For that we need a NAND flash controller NFC.

An NFC is typically a state machine that is only able to read one or two pages of the NAND into a RAM buffer. Once the data is there and some magic has been done to either change the reset vector to point at the RAM or remap the RAM to the reset vector, the cpu is allowed to run.

A page of NAND is typically 2KB these days but not too many years ago was only 512 bytes. If the state machine only reads one or two pages there may be as little as 1K of code loaded by the NFC. This small memory constrained code is typically called the secondary program loader or SPL. The job of the SPL is to load a larger bit of code into a larger RAM buffer and jump to it. This larger hunk of code is typically the full fledged boot loader like u-boot.

MMC

Just as NAND is an attractive alternative to NOR, in recent years MMC (including eMMC and SD) has become an attactive alternative to NAND. However again there is a downside. Reading some blocks from an MMC card is more complicated than reading from NAND so a more sophisticated state machine is needed for that. Also if one wants to read from files on the MMC card instead of hardcoded blocks then things are harder still. There must be a better way.

NOR Again?

If one wanted the primary storage for a device to be MMC and the chosen SOC had no capability of booting from MMC what could be done? One possibility is to add NOR flash to the board. To keep it as inexpensive as possible it would be very small, only large enough to hold enough code to boot from MMC. Thinking of this hybrid solution something else comes to mind what if the code to boot from MMC was on the SOC itself in mask ROM. This on chip read only memory is called mask rom because it is hard coded with the chip's mask. Since it is readonly it it relatively small and cheap.

The modern ARM SOCs targeted by Linaro all have mask rom containing boot code. The advantages are obvious in that the complicated NFC state machine can be eliminated. With code instead of hardware running there is more flexibility. Typically these mask rom boot loaders (RBL from now on) provide multiple boot options, MMC, NAND, USB (gadget not host), Serial, Network. are common options. And one can even ignore the mask rom completely and boot from external NOR.

With the mask rom we have gone back to the "easy" way of booting however since it is mask rom internal to the SOC the board vendor has no way to change it. The SOC vendor must get it right at the factory.

All the SOCs targeted by Linaro have some form of RBL. History lesson over now let's look at the features of these RBLs.

RBL

The Ideal RBL

In a perfect world the RBL would: Initialize enough of the board to run the kernel which would do the rest. Copy all the files necessary for booting Linux (kernel, devicetree, initrd see some section later) from any file system on any storage mediam into memory (initialized in previous step) and jump to it. No additional boot loader needed, kernel does all the rest, fantastic!

Reality Check

RBL knows nothing (almost, see boot option pins) about the board the SOC is on. RBL lives in a mask rom which while not constrained like older generation NAND SPL boot loaders is still finite. RBL cannot know about filesystem ext99 invented three months after this SOC went to market. RBL cannot know about the new version of SD-XXXHC with multi-terabyte capacity. RBL does not know what it is running on and thus cannont tell Linux which wants to know. RBL can not do weird workaround 42 on the Penguin board that is required to run the old supported kernel that does not deal with this issue itself.

Ok, so there are limitations to what RBL can do. Lets look at issues/characteristics.

RAM and ROM Size

The RBL runs in place in SOC mask rom. The size of this rom will be limited so of course there are limits to the feature set the RBL can support.

Likewise the SOC internal SRAM has a limited size. Before DRAM on a board is intialized the only ram available is the internal SRAM. The main consequence of this limit is the size of image RBL can load since it must fit in SRAM.

Boot Media

Supported boot media is typically determined by what market the SOC is targeted for. For example an SOC designed to go into a settop box with PVR cabability may be able to boot from SATA or USB mass storage. An SOC with on chip ethernet may support Network boot while it would make no sense to do so in a mobile device with no wired network support.

MMC boot support is universal accross the SOCs currently targeted by Linaro. However even here there can be differences. While most support booting from and external SD card the UX500 based Snowball platform only boots from the on board eMMC.

At this point the default boot medium for Linaro images is eMMC and MMC.

Boot Media Format

There are a few options here. All platforms assume a formated MMC card. Some then read the boot image from a fixed location in a particular place on disk or partition. Others expect a file with a certain name in a certain filesystyem type on a certain partition. One platform has its own index called a TOC that is read to find the boot files.

There are gotcha's for every approach. For the case of the FAT filesystem on a formated MMC card there are sometimes unspecified constraints on the alignment of the partitions and on the specific version of the filesystem.

When developing tools for creating a boot image that uses a FAT or VFAT filesystem it is likely that one can reuse tools that already exist. Also having the images in a filesystem means updating them later should be easier.

Chosing to use a fixed location in a particular partition type means the RBL does not need code to read from a FAT filesystem so it will likely be smaller and less likely to have hidden bugs.

Updating an image at a fixed location might be easy for someone comfortable with dd but a little scary for a novice.

Chosing a proprietary format means one can not leverage any existing tools and one has to write the code that goes into RBL from scratch. Likewise the tools for updating the image are custom as well.

Boot Image Format

The binary boot image will typically have a header describing it. The header can be very simple containing only a length and address of where to copy the binary before jumping to it. The header can be more complicated containing checksums and signatures. Some SOCs have extended header formats that allow for some table driven functionality to take place before jumping to the binary. This is typically a list of reads and writes. In some cases one can embed in this table dram init so the copy of the binary can be straight to dram avoiding the SPL phase.

Boot Image Size

The boot image size is constrained by the size of on chip SRAM. This constraint can be lifted in the case where table driven init starts up DRAM before copying the image.

Board Initialization

As stated before, the RBL knows very little about the board so does not do much here. There are a few things that can be configured. Typically SOCs have some bootstrap config pins that are pull high or low to select a boot configuration. RBL can use this boot info to decide what port to attempt to boot from and in what order. Since attempts to boot on ports that have no media waste time, having the boot strap pins set for the correct config will speed up boot order.

Is Code the Only Way to Do Configuration

As stated above some (maybe all?) SOCs have the ability to encode hw init instructions in the the form of reads and write to specified instructions to allow a limited amount of board init to happen before loading a binary. The most common use of this is to initialize DRAM.

Specific RBLs

Here are descriptions of some real instances of RBLs in terms of the above properties.

Freescale i.mx5

Texas Instruments OMAP[34]

Samsung XXXXXXX

TBD

ST Ericson XXXXXX

TBD

SPL

Not to be confused with the previously mentioned tiny NAND spl. This is the code that the RBL loads and runs. I usually runs in on chip SRAM so it does have that size constraint.

SPL typically deals with board specifics like pin muxing and config and dram setup. SPL may also deal with booting from media that is not directly supported by RBL.

Once board init is complete SPL loads and jumps to the next step in the boot process. This is typically u-boot but could be something else. On particularly useful variation is booting a kernel directly from SPL.

U-Boot

U-Boot SPL usually boots u-boot (it can boot linux directly as of upstream v2012.4 on some platforms). What does u-boot do that could not be done by U-Boot SPL? Since u-boot runs in DRAM is is not constrained by the size of on chip SRAM. U-Boot can therefore many more features.

Some features that U-Boot has that are missing include

  • U-Boot has a command line while SPL has a compiled in behaviour with no ability to modify it from the console.
  • U-Boot can have write access to things like saved environment while SPL has read only access.
  • In some configurations, U-Boot will have the necessary functionality to update itself. That could include the ability download a new u-boot and write it to boot media.
  • U-Boot will typically have diagnostic features enabled while SPL will only have the minimal feature necessary to boot. These extra U-Boot features might include
    • Memory dumping and test
    • I2C access
    • Network protocols like tftp
    • Simple shell scripting functionality at the command line

Linaroisms

Sharing

Linux

What we really want in this is to get a kernel running. We have a proper bootloader running what does it need to do to boot a kernel.

Kernel

Device Tree

Initrd or Initramfs

Commandline Parameters

Other Issues

What Would Make Things Easier

If ARM and the SOC vendors agreed to a standard boot binary format and exposed a common user blowable fuse interface one could imagine a boot binary that could run on multiple SOCs.

At manufacturing time one would blow some fuses to write a unique board ID into the SOC.

At boot time after RBL has loaded and jumped to a binary from the boot media, that boot code could look in a standard place for the id register written at board manufacturing time. It would then use that id to select what code to run next.

Why Can't I?

A common request is for the ability to install an os image like Ubuntu, Android or something else onto an MMC card and be able to stick it into any ARMv7 based board and have it boot. This works with bootable CDs on PCs, what's the deal?

There are a number of reasons, some being worked on and others insurmountable with current generation SOCs.

There are two main reasons this works on PCs. First the Intel architecture is less fragmented so it is relatively easy to produce a kernel that works on multple platforms. This issue is being addressed by Linaro and progress has been made.

The second issue is that PCs basically boot from NOR as described at the beginning. It is EEPROM not NOR but the concept is the same. The BIOS EEPROM can contain all the nasty board and cpu specific ugliness to load and jump to a standard boot loader like GRUB.

Imagine if the SOCs targeted by Linaro had their RBL in an external board specific NOR flash chip. Everything above that could be the same assuming the kernel folks get to the unified kernel sometime.

UEFI

There is not problem a new industry standard cannot make worse. Thus UEFI is here to fix everything. Actually if it was in a separate soldered on but updatable chip on a board (aka NOR boot) maybe this could fix things. Your author knows next to nothig about UEFI so this section is as they say a STUB.

GRUB

Ask a PC Linux booting guru what they want for a boot loader on some ARM widget and they will say "GRUB of course". This could be helpful because all the infrastructure and conventions in a distro that have been built up over the years to update bootloaders and kernels could be shared accross Intel and ARM if GRUB was under Linux. It is not that easy. The issues here are the same as above in "Why can't I....?"

Fastboot

Contrary to the what is impied by the name, Fastboot is a protocol over USB for booting and updating Android devices. There is overlap here with an industry standard call DFU. There are implementations of DFU and Fastboot in various out of mainline u-boot trees. There are also rumors and have been rfc patch submissions to the u-boot mailing list but nothing has made it to mainline.

Linaro's u-boot-usbspl uses a stripped down version of the fastboot protocol to download the u-boot payload. See OMAP above.

JohnRigby/LinaroBootingBasics (last modified 2012-04-05 22:35:42)