Random Notes That Might Help CI Developers

Once you have got sufficient permission to create a job, the following information should take few minutes to get a gcc build and testing started.

setting up new kernel build job

Once you have got sufficient permission to create a job, the following information should take few minutes to get a kernel build and testing started

For quick reference go to <<Go To Existing Kernel jobs on ci.linaro.org section and use any one of the jobs as a reference to setup a new job>>

Here are the steps to create a new kernel job. This will cross compile the kernel and create a new debian package. At the end of this execution you will get a new hwpack created which will have your newly created deian package.

1) Login to Jenkins and click on the “New Job” icon listed on the top left of the window.

2) Give an meaningful name which describes the purpose of the Job

3) Select the “Build a free-style software project” option.

4) You can either select the “Copy existing job” if there is a job which you want to copy and modify later or you can press ok to go to next step of the setup.[ You may want to copy the linux-linaro-3.0_panda-omap2plus which explains how to build linux-linaro-3.0 for panda board using the omap2plus defconfig on a ec2 slave provisioned dynamically. All the kernel related jobs existing on the jenkins use the scripts sitting under the branch lp:~linaro-infrastructure/linaro-ci/lci-build-tools]

5) Click on the “Restrict where this project can be run” option

6) Under the Source Code Management select the git option [if your code is hosted on git repository] and provide the link to the git tree for the Repository URL. For example for linux_linaro-3.0 the git repo would be git://git.linaro.org/kernel/linux-linaro-3.0.git.

7) You can specify the build to be executed at some frequency, for example to execute the job once in a day you could enter @daily under the Build Periodically option.

8) Under the “Build” click on the drop down menu and click on the “Execute Shell” option. Add the commands that you would want to execute for the build and testing.

For example to build the linux-linaro 3.0 for panda board with omap2 defconfig the following steps are needed.
rm -rf lci-build-tools
export hwpack_type='panda'
bzr branch lp:~linaro-infrastructure/linaro-ci/lci-build-tools
./lci-build-tools/jenkins_kernel_build_inst

[NOTE: the upstream builddeb available in the kernel git repository fails for cross build option. Hence we would need to copy the modified builddeb from the jenkins master until the upstream builddeb is fixed. To do this you need to specify the following in the "Copy files into the job's workspace before building " option.]

[PS: If you selected “Copy existing job” while creating a new job then this field might already contain some information, you might want to add/modify or leave this steps as is.]

9) If you want the build failures to be notified then you can specify the email information in the whitespace separated formation by checking the “E-mail Notification” option.

10) click save, you can try to verify the instructions work correctly by clicking on the “Build Now” option available on the left side of the window. Sit back and relax or go for a cup of tea while your build is done.

This completes the kernel build steps, but if your are interested in testing the newly built hwpack which includes the newly built kernel debian package

1) You can either download the hwpack that was created and burn it on SD card and test it .

NOTE: The new hwpack name can be obtained by going to the https://ci.linaro.org/jenkins/view/<viewname>job/<job_name>/<build_number>/consoleText
For example for if the 

viewname="Linux%20Linaro%20Tracking"
job_name=linux-linaro-tracking-llct-branch_panda-omap2plus
build_number=1 then, 

https://ci.linaro.org/jenkins/view/Linux%20Linaro%20Tracking/job/linux-linaro-tracking-llct-branch_panda-omap2plus/1/consoleText 

will give you the console information, scroll to the end of the page and check for use_hwpack_name variable. The value assigned to it will give you the newly created hwapck name.

The hwpacks will be stored on https://snapshots.linaro.org/kernel-hwpack/<job_name without board name and defconfig>/<job_name>
ex: https://snapshots.linaro.org/kernel-hwpack/linux-linaro-tracking/linux-linaro-tracking_panda-omap2plus

2) You can submit the newly built hwpack information to LAVA to test. We can come up with a new job which is purely meant for testing and then call this job as a downstream job from the main job.

This can be done by enabling the 
"Trigger parameterized build on other projects" 

Entering the test job name under "Projects to build", 

Select the "Stable" under "Trigger when build is" option for successful build and "Failed" for the unsuccessful build.

Select the appropriate "Add Parameter" option, for example select File Parameter if you are passing the parameters from the kernel main build job to test job by storing the hwpack information into a file.  

To create a test job use the following steps:

For example to test the newly created hwpack out of linux-linaro 3.0 for panda board with omap2 defconfig, we could use the following option:

1) click on the “New Job” icon listed on the top left of the window 

2) Give an meaningful name which describes the purpose of the Job.

   [ PS: Be sure to include the name of the test job 

3) Select the “Build a free-style software project” option.

4) Select "This build is parameterized " so that this allows us to pass parameters if needed. For example we would need to send the hwpack name to be used for testing. 

5) Then click on the "Add Parameter" option and select the appropriate way to send the build parameters 
   
{{{
 
For example if we can select "File Parameter" to pass a file name containing the hwpackname and other useful information stored by the main upstream ( kernel build ) job.

6) Then select "Restrict where this project can be run" option and enter master. [PS: We select master because the master is already configured for LAVA job submission]

7) Then Go to the Build section and click on the drop down menu and click on the “Execute Shell” option. Add the commands that you would want to execute for the build and testing.

For example you can use the lava submission script present under the lci-build-tools 

rm -rf lci-build-tools
bzr branch lp:~linaro-infrastructure/linaro-ci/lci-build-tools
export BUILD_STATUS='pass'
sh lci-build-tools/jenkins_lava_job_submissions

The script constructs the build status bundle information, creates a kernel testing json containing the tests following metadata information 

Build Details:
"build.id": "117" [[ job id in jenkins ]]
"kernel.build_url": "https://ci.linaro.org/jenkins/job/linux-arm-soc-for-next_beagle-omap2plus/197/" [[ link to the build url ]]
"kernel.git_branch_name": "master" [[ current branch name ]]
"kernel.config": "omap2plus_defconfig" [[ defconfig name ]]
"packages" like gcc and gcc-arm-linux-gnueabi along with their version
"branch_revision": "f392cbf75615e9d8cb90ef5ffb8d4e752b7be3c7" [[ git commit id ]]
"branch_url": "git://git.linaro.org/kernel/linux-linaro-3.0.git", [[ tree built ]]
"branch_vcs": "git", [[ CVS name ]]
"commit_timestamp": "2011-08-30T15:55:01Z", [[ commit timestamp ]]
"project_name": "Linux Linaro"
"test_case_id": "build-status"
"public_url": "https://ci.linaro.org/jenkins/job/linux-arm-soc-for-next_beagle-omap2plus/197/consoleText" [[ external link to the build console ]]

Kernel Test Run Details:
"hwpack": "https://snapshots.linaro.org/kernel-hwpack/linux-arm-soc-for-next/linux-arm-soc-for-next_panda-omap2plus/hwpack_linaro-lt-panda_20121015-0733_b502_armhf_supported.tar.gz", [[ newly created hwpack to the tested that was built on ci.linaro.org ]]
"rootfs": "https://snapshots.linaro.org/precise/images/nano/latest/linaro-precise-nano-20121025-479.tar.gz" [[ latest rootfs to be used for testing ]]
"kernel_version": "v2.6.35-rc5-62853-ga63d2e6" [[ kernel version built ]]
 "hwpack.date": "20110919" [[ hwpack generation date ]]
"hwpack.type": "omap3" [[ hwpack type ]]
"rootfs.date": "20110919" [[ rootfs generation date ]]
 "kernel_name": "linux-arm-soc-for-next" [[ Kernel name ]]
 "build.id": "117" [[ job id in jenkins ]]
 "rootfs.type": "nano" [[ rootfs type ]]
 "git_commitid": "a63d2e6e8373512f368fe8252dd8b1b162142e00" [[ git commit id ]]
 "git_url": "git://git.linaro.org/people/arnd/arm-soc.git" [[ tree built ]]


and submits them to LAVA for processing the build data to be displayed on the dashboard.
It also instructs the LAVA to test the newly built hwpack.
 

The above step shows instruction for when a kernel build is successful in creating a new hwpack which includes a newly created kernel debian package.
A similar job can be created above for unsuccessful case where a new hwpack is not created or the kernel build failed due to some reason.
The only change that needs to be done to for the build failure notification to be submitted  on LAVA is to change the export BUILD_STATUS='pass' to export BUILD_STATUS='fail'

The script constructs the build status bundle information, creates a kernel testing json containing the tests following metadata information, but to LAVA will not execute any tests on it as there is no hwpack created and exisiting for using.

setting up gcc native build job

More details on building and testing of linaro gcc can be found @ WorkingGroups/ToolChain/Using/GCCNative

Once you have got sufficient permission to create a job, the following information should take few minutes to get a gcc build and testing started

Here are the steps to create a new job:

1) Login to Jenkins and click on the “New Job” icon listed on the top left of the window.

2) Give an meaningful name which describes the purpose of the Job

3) Select the “Build a free-style software project” option.

4) You can either select the “Copy existing job” if there is a job which you want to copy and modify later or you can press ok to go to next step of the setup.[ You may want to copy the gcc-test-build_on_slave which explains how to build gcc Native for x86 on a ec2 slave provisioned dynamically]

5) Click on the “Restrict where this project can be run” option

6) Under the Source Code Management select the Bazaar option and provide lp:gcc-linaro for the Repository URL [ This is to build the upstream gcc linaro, if you have your own private branch which you need to build, then specify the branch name here. There is a local bzr repository that is created on the jenkins master which gets updated every 2 hours with the upstream gcc-linaro, gcc-4.4 and gcc-4.5 changes. To use it you can specify the bzr to use for example http://ci.linaro.org/gcc-linaro to build gcc-linaro. PS: This has not been well tested and I would like to help resolving any problems found with the private branch build]

7) If you want to build the gcc-linaro automatically when a new change is merged in the upstream gcc-linaro then click on the “Poll SCM” under the Build Triggers . Specify the “15 * * * *” , this for example checks the gcc changes every 15 mins everyday.

8) Under the “Build” click on the drop down menu and click on the “Execute Shell” option. Add the commands that you would want to execute for the build and testing. [PS: If you selected “Copy existing job” while creating a new job then this field might already contain some information, you might want to add/modify or leave this steps as is.]

9) If you want the build failures to be notified then you can specify the email information in the whitespace separated formation by checking the “E-mail Notification” option.

10) click save, you can try to verify the instructions work correctly by clicking on the “Build Now” option available on the left side of the window. Sit back and relax or go for a cup of tea while your build is done. [NOTE: If you are interested in building the upstream gcc-linaro for x86 , then you can straight away copy the gcc-test-build_on_slave which includes the steps to build and test the upstream gcc-linaro.]

Existing Kernel jobs on ci.linaro.org

The following kernel jobs exist already on ci.linaro.org which are built on ci.linaro.org and are tested on validation.linaro,org using LAVA.
The following kernel are cross-compiled jobs are built for beagle, panda, origen boards.

LAVA specific jobs

Platform/Systems/LinaroCIBuildTestService/SettingUpNewCIJob (last modified 2014-06-24 17:48:00)