Android Performance and Power Testing

This documentation explains how to set up and use workload automation suite to run various workloads for measuring performance and power in Android. The following workloads have been used:

  • CFBench (cfbench)
  • linpack
  • nenamark
  • smartbench
  • recentfling
  • browserfling
  • emailfling
  • galleryfling
  • idle home screen (idle)
  • audio playback (audio)
  • video playback (video)

The names in parenthesis are the workload automation workload names suitable for use in agenda files, described further below.

Download the Android SDK

The easiest way to do this is to download Android Studio. When you run Android Studio for the first time (by running bin/studio.sh) it will prompt you to download and install the SDK, giving you a choice about where to put it. Workload automation needs to know the location of the SDK so once this is done, set the ANDROID_HOME variable to point to the SDK location. For example, export ANDROID_HOME=/home/joeuser/Android/Sdk

Configure the Target

Ideally the target should be configured with wired ethernet. The Cable Matters 202023 USB 2.0 to 10/100 Fast Ethernet Network Adapter works well, it is r8152 based.

This means you cannot use the micro USB connection to your host on the hikey or 410c 96boards since they are not functional when the host USB connectors are in use. But you can connect adb over ethernet by executing the following on the target once (will persist after target reboot):

  • setprop persist.adb.tcp.port 5555

Then on your host, do

  • adb connect <device ip or hostname>:5555

You can then adb connect.

You will want a 1080p monitor connected to your board so you can monitor test progress, and a mouse connected as well for configuring the email account (more on this below).

Install Workload Automation

Pull it from my git repo which has some fixes for usecases on hikey as well as some new workloads.

Install with these steps (from WA docs in repo):

  • python setup.py sdist
  • sudo pip install dist/wlauto-*.tar.gz

It's worth noting that if you plan on tweaking workload automation (including the workloads it comes with) you may want to install it in development mode. This means you can edit the tree and try out your changes without having to reinstall it. Instead of the two commands above:

  • sudo python setup.py develop

Full WA documentation:

Set up Workload Automation

Workload automation by default looks to ~/.workload_automation for target configuration, dependencies and other things. Run WA once and it will create a skeleton config and directory structure:

  • wa run idle

Edit ~/.workload_automation/config.py to change working directory from '/sdcard/wa-working' to '/data/wa-working'. This allows scripts to directly execute files in the working directory.

In ~/.workload_automation/dependencies, create and populate the following directories.

  • browserfling
  • cfbench
  • emailfling
  • galleryfling
  • hwuitest
  • recentfling

Set up Browserfling

Download a copy of a web page to use with the browserfling test. I've been using www.theverge.com. In a web browser of your choice go to the page and save the complete web page (with images etc) into ~/.workload_automation/dependencies/browserfling/testpage/ . Ensure that the web page is named "testpage.html" (without the quotes) and is in testpage/ .

Set up Mail

Set up the AOSP mail client on your target. This will be used with the emailfling jank test. There is a test account here which is populated with 1000+ LKML messages:

Configure the AOSP mail client with this account and manually scroll through the messages, letting it download them.

Push a bunch of pictures to /sdcard/Pictures. You'll want a fair amount so you (or rather, the script) can scroll through them. I've been testing with 400+ pictures. One possibility is to just take a few photos you have lying around and copy them many times.

APK Installation

Install the following free APKs. These are used for recentfling and video playback but haven't yet been integrated into the dependencies properly. You can install APKs with "adb install <apk filename>". Alternatively you can install F-Droid (a free app market) first by downloading it from https://f-droid.org and then install the other apps using the F-Droid app.

  • fdroid
  • duckduckgo
  • amaze
  • twidere
  • icecat
  • docviewer
  • vlc

Other APKs need to be installed in WA's dependency directory. This is located at ~/.workload_automation/dependencies/<workload_name>/. So for example the CF-Bench APK belongs at

  • /home/<username>/.workload_automation/dependencies/cfbench/CF-Bench_1.3.apk

Sometimes WA does not find the apk file despite it being there in the dependencies folder. If this is the case, find your wlauto installation and place the apk files in the corresponding workload directory. For example if wlauto installation is at

  • /usr/local/lib/python2.7/dist-packages/wlauto

Suppose you are copying cfbench apk file , copy it to

  • /usr/local/lib/python2.7/dist-packages/wlauto/workload/cfbench folder. Similarly for all other workloads.

Result Collection

By default the results are collected in the mentioned output directory (wa_output) if no output directory is mentioned in csv format. There are options to change the format. Refer to the worklaod-automation.pdf for instructions.

Target/Hardware Specific Instructions

Juno

When using Juno board, WA has option to collect energy measurements. Add the following line to the config section of the agenda file to enable this.

  • instrumentation: [juno_energy]

When using the above make sure to modify the config.py file in .workload_automation folder to reflect the device as juno instead of generic_android and specify the juno ip address in the adb_name field.

ARM Energy Probe

WA supports the ARM energy probe with the arm-probe instrument, created by Vincent Guittot. The patch for this is available in the repository mentioned above. To use it add this to your WA agenda file:

  • config:
    • instrumentation: [energy_probe_ext]

When running a test this will create an energy_probe directory for the test in the results directory. Inside you will find a summary.txt file that shows the energy consumption during the test.

  • Total nrj: 23.259 J for "first -- duration 29.932 sec -- min 0.000 W -- max 2.886 W Total nrj: 23.259 J for Platform -- duration 29.932 sec

Sample Agenda Files

Juno w/Energy Measurement

  config:
    instrumentation: [juno_energy]
  global: 
    iterations: 5
    runtime_params: 
      sysfile_values: 
        /dev/cpuset/background/cpus: 0-5
        /dev/cpuset/foreground/boost/cpus: 0-5
        /dev/cpuset/foreground/cpus: 0-5
        /dev/cpuset/system-background/cpus: 0-5
        /sys/devices/system/cpu/cpu0/cpufreq/interactive/go_hispeed_load: 85
        /sys/devices/system/cpu/cpu0/cpufreq/interactive/hispeed_freq: 700000
        /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor: interactive
        /sys/devices/system/cpu/cpu1/cpufreq/interactive/go_hispeed_load: 85
        /sys/devices/system/cpu/cpu1/cpufreq/interactive/hispeed_freq: 800000
        /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor: interactive
  workloads: 
    - nenamark
    - linpack
    - smartbench
    - cfbench

Basic Android Power Usecases w/AEP

config:
        instrumentation: [energy_probe_ext]
global:
        iterations: 3

workloads:
        - name: idle
        - name: audio
        - name: video

WorkingGroups/PowerManagement/Resources/android-power-perf-testing (last modified 2016-10-28 18:08:23)