Summary

Design and implement a prototype for a network boot image, either for running a server workload or for starting an automated installer.

Rationale

At the time of writing, Linaro doesn't have a Server Working Group, yet running server workloads on ARM is getting increasingly interesting. To prepare the ground for a future Server WG, the Office of the CTO team is researching the state of server use cases on ARM. The goal is to identify future areas of improvement while fixing trivial issues along the way.

Large server farms, ARM or not, are typically installed automatically over the network. The goal of this spec is to deliver a network boot prototype to automatically install a couple of different ARM boards as to serve as servers.

User stories

Irina maintains a grid of 20 000 ARM boards and would like to install them automatically as she likes.

Assumptions

Debian-style installer/userspace is deployed. Work would likely be reusable with other userspaces e.g. Fedora or OpenEmbedded, but that's not demonstrated here.

Installer can be loaded from the network through either U-Boot TFTP support, or through kexec. Good U-Boot network support is typically expected on server-class SoCs, but kexec might have to be used instead for the prototype as it is being demonstrated on community boards.

Boot media is created manually and is separate from target rootfs media.

Design

Debian Installer (d-i) will be extended to build server images for the SoCs we care to demonstrate.

Two SoCs will be demonstrated:

  • Pandaboard as to demonstrate multiple cores (SMP)
  • MX53 Quickstart as to demonstrate native SATA

Optionally, a QEMU-supported SoC will be demonstrated (either Versatile Express or OMAP3).

Local bootable SD card images will be created as a kernel-version agnostic loader and will contain:

  • either a version of U-Boot specially configured to PXE-boot or TFTP-boot the installer
  • or a version of linux with a custom initrd which will kexec-boot the installer

Cobbler will be used to serve the images over TFTP/PXE.

The whole process will be documented and missing features and bugs will be tracked along the way.

Implementation

First, "netboot" debian-installer images will be generated for target SoCs. These images will be manually tested by writing them on SD and starting an installation.

Next, U-Boot network support will be tested on the target SoCs. On OMAP, some early patches are available to enable networking from U-Boot, these patches will be tested and applied to an u-boot-linaro based tree and package and tested.

If U-Boot PXE patches are available from Calxeda or others, these will also be applied to an u-boot-linaro based tree and package and tested.

For each board, a minimal SD card image will be created to boot from the network. If possible, the image should be created with Linaro Image Tools and be automatically generated from the d-i build process.

Depending on the capabilities of the SoC, the boot SD card will either PXE-boot or TFTP-boot or kexec-boot.

Cobbler will be setup to serve the generated images and start an automatic installation of a LAMP server with automatic partitioning of an externally connect target storage on each board (SATA or USB, but not SD).

Unresolved issues

TBD

BoF agenda and discussion

Notes from Linaro@UDS-O Budapest session, also at http://summit.ubuntu.com/uds-o/meeting/linaro-platforms-o-network-boot/:

No server SoC would connect Ethernet over USB

What do you with the PXE-loaded payload?
- either install images
- or nodes that have no local storage


U-Boot loaded from Flash then loads PXE

Would it make sense to have some U-Boot image implementation of PXE; but it's ea
sier to have a single image to load (just u-boot with PXE support)

What about Kexec-loading the PXE-payload
=> could work for some SoCs


how to do you reboot into the installed system?  Probably by changing the u-boot config, in flash or SD
- might be best to switch the boot script on the server side

pxelinux is based on syslinux; it can offer a menu over serial port to chose which image to boot from.  That can't work on ARM because there is on standard way to read/write from the network or talk over serial.

Some x86 systems have key sequences that you can press to boot of the network; that requires a human, so not suited for mass deployment or validation.



NFS does not really need to be part of the prototype; might be nice to use every file over the network.

What image types are to be supported as an installable payload?  Anything linux ought to be supported, but specific implementations like d-i or kickstart are ok for now; should eventually be abstracted away.

When doing installation to local node, what do we want to log and keep?  Do we want to provide input/output?
- would probably be done out of band

Does UEFI play a role in network boot standards?  Would we want an implementation based on UEFI in the future.

Might want to host many different types of systems; would it make sense to send information from the board to the PXE server?  e.g. the local systems might be of different hardware types
- in the intiial bootp packets there are some strings which you could dynamically change; could use node name to select different configs

MAC address: not always unique in pools of boards -- e.g. pandas have no ROMs
- needs to be valid at boot time any way to fetch PXE config

This project would apply to diskless workstations as well

Could we use HTTP instead of TFTP?  TFTP is standard, but HTTP is more flexible.

gpxe new name for etherboot


2 prototypes:
- boot of network and run some network payload of RAM, e.g. http server with some page; needs linux + initrd
- load installer of network and deploy onto local storage; install some extra packages with the installer and reboot into the installed system


CategorySpec CategoryTemplate

OfficeofCTO/Specs/NetworkBootPrototype (last modified 2011-05-30 15:59:11)