Docker[1] is an open platform for developers and sysadmins to build, ship, and run distributed applications, but it's very immature for ARM64 architecture, this wiki is to address to estimate the latest status of Docker on ARM64 platform, and try to get some temporary workaround for Docker on ARM64 platform too.

Quick Start

For Ubuntu vivid, there is already an available docker application for ARM64 platform, you can install it by following commands directly.

sudo apt-get update && apt-get install docker.io
sudo service docker restart

You can get basic docker information with following commands.

docker version
sudo docker info

So far, there is not any officially available Ubuntu image for ARM64 in Docker image hub[2], so we can try it with following special Ubuntu image, and show "hello, world" by docker.

sudo docker pull justinzh/arm64-vivid
sudo docker run justinzh/arm64-vivid echo "hello, world"

Preparation for Host

Before you start any docker testing or building docker source code, please make some basic preparation for your ARM64 host machine as follows.

sudo apt-get update && sudo apt-get install -y \
       apparmor \
       aufs-tools \
       automake \
       bash-completion \
       btrfs-tools \
       build-essential \
       curl \
       dpkg-sig \
       git \
       iptables \
       libapparmor-dev \
       libcap-dev \
       libsqlite3-dev \
       mercurial \
       parallel \
       python-mock \
       python-pip \
       python-websocket \
       reprepro \
       ruby \
       ruby-dev \
       lxc \
       gcc \
       gccgo \
       --no-install-recommends

Because golang is not ready for arm64 architecture yet, we have to install gccgo instead of golang as above. Apart from above these, you still need to install a device mapper on host machine as follows firstly.

sudo su 
git clone https://github.com/hisilicon/patches.git ~/patches
git clone https://github.com/Distrotech/LVM2.git /usr/local/lvm2
cd /usr/local/lvm2
cp ~/patches/config.* ./autoconf/
./configure --enable-static_link
make device-mapper
make install_device-mapper

Now you are ready to build the docker from docker source code on you arm64 host.

Build Your Own Docker Image

Following command is to get a whole distribution rootfs from Ubuntu official website.

sudo debootstrap --arch=arm64 --variant=minbase vivid ubuntu-vivid-arm64 http://ports.ubuntu.com

After validating with above commands, it's confirmed that the image gotten by above command is not available for many commands, e.g. apt-get install, so you can create your own Docker image according to following commands

cd ~
wget -c https://cloud-images.ubuntu.com/releases/vivid/release/ubuntu-15.04-server-cloudimg-arm64-root.tar.gz
mkdir arm64-vivid
cd arm64-vivid
tar -xzf ~/ubuntu-15.04-server-cloudimg-arm64-root.tar.gz
tar -c . | docker import - <your docker account>/arm64-vivid

Now, run

sudo docker images

You will get

REPOSITORY                             TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
<your docker account>/arm64-vivid      latest              9c83367cd13b        2 hours ago         565.3 MB

You can also push your own image into Docker hub by your own account as follows

sudo docker logout
sudo docker push <your docker account>/arm64-vivid

Rebuild Docker with Source Code

Because the Docker is not mature for ARM64 architecture, to build the binary targets or all test targets, some patches must be applied firstly before you try to rebuild them. Now one patch[3] has been provided for building binary and cross targets on ARM64 architecture with gccgo. Firstly, you need to get the patches.

git clone https://github.com/hisilicon/patches.git ~/patches

These patches[3] are based on Docker 1.7.0 and gccgo, resolved all issues for build\binary\cross\test-unit\test-integration-cali\test-docker-py targets on ARM64 platform, you can try them according following commands.

git clone https://github.com/docker/docker.git ~/docker
cd ~/docker
git checkout 2b27fe1
git am ~/patches/000?-Resolved-*
sudo make test

From the test result, we can get:

 *1. binary target will be built successfully, right docker binary can be gotten.
 *2. cross target will fail, it means you can't build cross targets in aarch64 architecture board.
 *3. for test-unit target, 65 cases will pass with 4 cases failed: ./pkg/archive ./pkg/ioutils ./pkg/plugins ./runconfig. More detail, please to refer to unit test log[4].
 *4. for test-integration-cli and test-docker-py targets, all test cases will fail because they all base on "busybox" image, which is an x86\amd64-supported-only image. More detail, please to refer to integration test log[5].

Some Conclusions

Some conclusions can be gotten now, maybe need update later.

  • 1. Basic docker application\service can be installed successfully in Ubuntu vivid on APM ARM64 board by apt-get, and seems they can work well for some basic features.
  • 2. Golang is not ready for ARM64 architecture yet.
  • 3. Docker v1.5.0 gotten by apt-get source docker.io is worse than latest version v1.7.0, it only provides a gccgo target to build binary, all other targets can not be built yet after a lot of efforts.
  • 5. With patches[3], all targets can be built from Docker's source code with gccgo on APM ARM64 board successfully.
  • 6. For unit test on arm64 board, totally 65 cases can pass with 4 cases failed: ./pkg/archive ./pkg/ioutils ./pkg/plugins ./runconfig.
  • 7. For integration-cli test and docker-py on arm64 board, all test cases failed because they all run in a default image "busybox", which is x86\amd64 format docker image.
  • 8. The whole Docker source code is not friendly for gccgo, apart from binary target, all other targets can't be built by gccgo. If you want to build other different targets, you must apply some special patches[3] firstly.
  • 9. The whole Docker source code entirely doesn't consider multi-architecture host, it defaults only to support x86\amd64 architecture. e.g.: there are so many hard code with "busybox" image, which is an x86\amd64-supported-only image. Even if you can resolved all issues about gccgo with patches[3], many test cases will still fail because of x86\amd64-supported-only "busybox" image.

Reference

LEG/Engineering/DockerOnARM64 (last modified 2015-06-19 07:48:32)