Will redirect to the latest link in readthedocs.org in 2 seconds

Overview

LAVA is an automated validation architecture, and it now has a test framework for running android test tools and parsing the test output. Please NOTE that lava-android-test is just used for running the test tools, parsing the test output, and formatting the test result.

How to

The following covers how to write and add a test wrapper script for lava-android-test. and how to integrate the test into LAVA.

1. checkout the lava-android-test to your local machine

    bzr branch lp:lava-android-test

2. if the test tools are just command that can be run on android system, and the output is well formatted, then congratulations, you can go directly to step 6. You don't need to wrapper script again.

3. If the test tools has already been build into the android image or in the host image(normal Ubuntu image), then you won't need to define some scripts for organizing the test tools, you can skip this step.
Otherwise, put the actual test tools in some place, normally they are in a sub directory of test_definitions, like the busybox test, i.e. the actual test tool is busybox_test.sh, and it is put in the lava_android_test/test_definitions/busybox directory.

  • Note:
    In this case, we should modify the MANIFEST.in file in the root source directory. Like:

    • include lava_android_test/test_definitions/busybox/
    Otherwise the contents in that directory won’t be installed into the system python library.

4. add a test wrapper script for your test into the test_definitions directory. like busybox.py:

    The content of the wrapper script should be something like below:
          Normally, you just need to redefine the red and bold part in the above.
          ###############################################################################################
import os
import lava_android_test.testdef

test_name = 'test_sample'

#linux commands that will be run on the host before INSTALL_STEPS_ADB_PRE"
INSTALL_STEPS_HOST_PRE = []
#adb commands that will be run before install apk file into android
INSTALL_STEPS_ADB_PRE = []
#APK file path list that will be intalled into android
APKS= []
#adb commands that will be run before install apk file into android
INSTALL_STEPS_ADB_POST = []
#linux commands that will be run on the host after INSTALL_STEPS_ADB_POST
INSTALL_STEPS_HOST_POST = []

#linux commands that will be run on the host before RUN_STEPS_ADB_PRE
RUN_STEPS_HOST_PRE = []
#adb commands that will be run before install apk file into android
RUN_STEPS_ADB_PRE = []
#commands that will be run on android
ADB_SHELL_STEPS = []
#adb commands that will be run before install apk file into android
RUN_STEPS_ADB_POST = []
#linux commands that will be run on the host after RUN_STEPS_ADB_POST
RUN_STEPS_HOST_POST = []

#pattern to parse the command output to generate the test result.
PATTERN = "^\s*(?P<test_case_id>\w+)=(?P<result>\w+)\s*$"

inst = lava_android_test.testdef.AndroidTestInstaller(steps_host_pre=INSTALL_STEPS_HOST_PRE,
                                                                           steps_adb_pre=INSTALL_STEPS_ADB_PRE,
                                                                           apks=APKS,
                                                                           steps_adb_post=INSTALL_STEPS_ADB_POST,
                                                                           steps_host_post=INSTALL_STEPS_HOST_POST)

run = lava_android_test.testdef.AndroidTestRunner(steps_host_pre=RUN_STEPS_HOST_PRE,
                                                                          steps_adb_pre=RUN_STEPS_ADB_PRE,
                                                                          adbshell_steps=ADB_SHELL_STEPS,
                                                                          steps_adb_post=RUN_STEPS_ADB_POST,
                                                                          steps_host_post=RUN_STEPS_HOST_POST)

parser = lava_android_test.testdef.AndroidTestParser(PATTERN)
testobj = lava_android_test.testdef.AndroidTest(testname=test_name,
                                    installer=inst,
                                    runner=run,
                                    parser=parser)
          ###############################################################################################

And in the command part, you can use "$(SERIAL)" to represent the device serial number, like:

RUN_STEPS_HOST_POST = [ 'python %s/android-0xbenchmark/android_0xbenchmark_wait.py $(SERIAL)' % curdir]

and "$(OPTIONS)" to represent the option string passed from command line, like:

INSTALL_STEPS_HOST_PRE = [ 'echo $(OPTION)'],
RUN_STEPS_HOST_PRE = [ 'echo $(OPTION)'],

then you can run lava-android-test install -o "install options string" or lava-android-test run -O "run options string"

Note:

Because lava-android-test will be run on lava-lab, and there will be multiple devices connected simultaneously, So we should consider to pass the device serial number for each test tools. If the test tools is defined for steps_adb_pre/adbshell_steps/steps_adb_post, then there is no need to pass the device serial number, lava-android-test will do this for you.

5. you can:

  • use "lava-android-test list-tests" to check if the test wrapper created can be recognized,
  • use "lava-android-test install ${test_name}" to install the test,
  • use "lava-android-test run ${test_name}" to execute the test,
  • use "lava-android-test show ${result_id}" to show the output the test executed,
  • use "lava-android-test parse ${result_id}" to to generate the result bundle for the test executed.

Here is a blog about install/test lava-android-test that you can reference:

6. When you have done the above steps and verified your test that works well, then you can integrate it in LAVA with the android-build. Here is a description about that:

https://wiki.linaro.org/Platform/Android/AndroidBuild-LavaIntegration

7. At last don’t forget to add an entry and some document in the doc/tests.rst file. Like:

busybox
+++++++
.. automodule:: lava_android_test.test_definitions.busybox

Then the information will be listed in the below url:
http://lava-android-test.readthedocs.org/en/latest/tests.html

8. Commit Modification In lava-android-test directory, run the following commands:

bzr launchpad-login ${your-lauchpad-id}
bzr commit -m '${commit msg}
bzr push lp:~${your-launchpad-id}/lava-android-test/${branch-name}

Then you can see your branch in the following page:
https://code.launchpad.net/lava-android-test

Click your branch, and click the “Propose for merging” link in your branch page to submit a merge proposal. In the proposal page, please set Reviewer: to linaro-validation.


If you have any questions, please contact me. Also I have copied it to google docs,
https://docs.google.com/a/linaro.org/document/d/1kVyuFZtnMsganZaszQZhgX3QPOF5KXhKvefCUq4Xh3A/edit

you can comment directly there.

Thanks, Yongqin Liu


Mail: yongqin.liu@linaro.org. IRC Nickname: liuyq

Platform/Android/IntegrateATestIntoLava (last modified 2012-05-09 05:59:26)