Contents

  1. Overview
  2. Process Information
    1. IT Services Communication
  3. Specs, HowTo's & Setup Information
    1. Configuration Management - SCM - Ansible
    2. Linaro Image Tools
      1. Hardware Packs
    3. EC2 Usage and Best Practices
    4. Continuous Integration Service
      1. Common to all CI services
      2. ci.linaro.org
      3. CI Tools
    5. Android-related Infrastructure
      1. Android Build System - Android CI Loop
      2. Android Build Downloads
      3. "Seeded" builds approach
      4. Linaro Android Gerrit
      5. Android Gerrit Continuous Integration
      6. Android Gerrit Mirroring System
    6. patches.linaro.org (patchmetrics)
    7. releases.linaro.org , snapshots.linaro.org and License Protection Handling
      1. Publishing new builds
      2. Accessing license-protected files in automated environments
    8. repo.linaro.org
    9. Jenkins server Local setup HowTo
    10. Setting up Jenkins development environment
    11. Deploying Jenkins to ci.l.o
    12. CBuild/LAVA integration
    13. Git & Gerrit servers and instructions
    14. Toolchain Mirror
    15. OpenJDK Mirror
    16. bugs.linaro.org
    17. Austin Colo Facility
    18. Hetzner

Overview

This page is a one stop place which hosts all the specs, manuals, howto's and other project related information handled by infrastructure team. Please note that some specs which referenced here are pre-implementation documentation they may be or may be not updated for changes during initial implementation phase, and for sure not updated during further development and maintenance phases. Thus, they are provided for completeness and historical purposes (but they also may include discussion of choices, decisions, and implementation details not available elsewhere). On the other hand, manual and howto's are expected to be updated throughout project's lifetime.

Process Information

IT Services Communication

To submit a request, go to https://servicedesk.linaro.org or send email to it-support@linaro.org.

Specs, HowTo's & Setup Information

Configuration Management - SCM - Ansible

Linaro Image Tools

Tools and Scripts to manipulate and install Linaro Images/Releases on media.

Hardware Packs

EC2 Usage and Best Practices

Few of our systems are based on/interact with EC2. We maintain list of best practices, tools, and known issues for such systems.

Continuous Integration Service

Common to all CI services

ci.linaro.org

The Continuous Integration Jenkins Build/Test Service is an instance of Jenkins that uses slaves provisioned from EC2. The following link is mostly a guide on how the service was set up and how to create a new job on ci.linaro.org.

CI Tools

Android Build System - Android CI Loop

Cloud Buildd is a generic platform to perform dynamic, parametrized builds in the Cloud (EC2), consisting of Jenkins, frontend for it, build scripts and "repo" tool mirror service. The Android Buildd is specific installation instance of Cloud Buildd, used to perform Linaro Android builds.

Android Build Downloads

All downloads are now available from http://snapshots.linaro.org/android/ . Direct downloads from Jenkins (starting with https://android-build.linaro.org/jenkins/ are internal-only and deprecated for general usage). For the cases which involve "permalinks" like lastSuccessful, links to http://android-build.linaro.org/download/ should be used for now (note http://)

"Seeded" builds approach

Linaro Android Gerrit

Gerrit is used for git code hosting and review by Linaro Android team.

Android Gerrit Continuous Integration

Joint project with Android team (led by them).

Android Gerrit Mirroring System

patches.linaro.org (patchmetrics)

releases.linaro.org , snapshots.linaro.org and License Protection Handling

  • They provide click-through licensing which is part of lp:linaro-license-protection project.

  • Details on the set-up are in:

    • README #1 (includes specification for BUILD-INFO.txt)

    • README #2 (includes information about staging.releases.linaro.org and staging.snapshots.linaro.org)

  • We used to have integration tests to run after each rollout, but they were not updated for latest changes (TODO: fix them): testr run testplans.test_suite

Publishing new builds

Please read Publishing page for adding new builds (includes intro on BUILD-INFO.txt).

Accessing license-protected files in automated environments

The latest and greatest tool to handle downloading of license-protected files: lp:linaro-license-protection/download. Requirements on which it is based: lp:1130810.

repo.linaro.org

This is deployed on an EC2 instance. (ec2-54-224-40-137.compute-1.amazonaws.com)

It contains reprepro repositories in /srv/repos/<subject>

Currently one repo: ubuntu/linaro-overlay, which contains two suites

  • quantal (for armhf and arm64 overaly packages)
  • quantal-tools (for tools/packages to be using in builds, not yet upstream (amd64, armhf)

The repo key for apt authentication is http://repo.linaro.org/ubuntu/linarorepo.key

Jenkins server Local setup HowTo

  • Get the source code here.

  • For building, follow this link. If the build fails with this exception download and use Sun Java.

  • To configure environment in eclipse run: 

    mvn -DdownloadSources=true eclipse:eclipse

    and then follow this set of instructions.

  • For deployment you can use any of the existing web containers (instructions listed here)

  • If you're trying to deploy it under tomcat and get this error, switch to running Jenkins by itself (which uses Winstone.): 

    java -jar jenkins.war

Setting up Jenkins development environment

  • The codebase for Jenkins for Linaro is located here. At the moment of writing the Jenkins version we use is 1.456

  • After branching this code, one should do: 

    mvn -Plight-test clean install
mvn eclipse:eclipse
    if the aim is to develop code under Eclipse (recommended). Importing project to Eclipse should be straight forward.

  • If jenkins-remoting library is marked for particular bug-fixing/development, the code should be used from here: https://code.launchpad.net/~linaro-infrastructure/jenkins-remoting/linaro-trunk

  • The process for import is pretty much the same except for the first time maven build when this library source should be moved to jenkins code base due to some strange pom dependencies.
  • After building jenkins-remoting or some other library used by jenkins, be sure to build jenkins code as well, since the jenkins war has to pick up the new version of the lib from the maven repository.

Deploying Jenkins to ci.l.o

  • Get you jenkins.war file from war/target dir in your local jenkins build directory and copy it to ci.l.o.

  • Login to ci.l.o shell and execute following steps: 

    sudo service jenkins stop
sudo rm -rf /var/run/jenkins/war/*
sudo cp jenkins.war /usr/share/jenkins/
sudo service jenkins start
  • Optionally you can create a backup of old jenkins.war beforehand in case something goes wrong and you want to revert quickly.
  • After that check the /var/run/jenkins/war/ folder, it should contain unpacked jenkins war file.

CBuild/LAVA integration

Git & Gerrit servers and instructions

User help pages:

Toolchain Mirror

  • Toolchains Mirror: instructions on how the gcc mirror has been setup and how to add new branches

OpenJDK Mirror

  • OpenJDK Mirror: instruction on how the OpenJDK mirrors have been setup

bugs.linaro.org

Austin Colo Facility

Hetzner

Platform/Systems/KnowledgeBase (last modified 2016-06-20 19:59:20)