Introduction

The QA Services team main is delivery of reports and early alerts system for release sign off and milestone checkpointing as well as continuous monitoring of automated CI results.

In order to ensure that the quality of such reports and early alert notifications is sane, the QA services team maintains a set of templates for emails and bug reports for the recurring services provided for Linaro engineering.

Build Validation Services

Release Critical Summary

  • Cadence: Monthly

For Linaro monthly release cycle, once release critical test is done, a summary report will be sent to "linaro-release@lists.linaro.org" & "linaro-android@lists.linaro.org" or "linaro-dev@lists.linaro.org". This report will be sent before the end of the first testing day of monthly release cycle, which means usually the end of last Monday in every month. If a release candidate image is delayed, then a separated report will be sent for that specific image. Below is a template as reference:

Hi everyone,

Here is the release critical test summary for following boards. For more details, please click the test result spreadsheet links.

  1. Linaro Android:

    1.1 Samsung Origen: <Paste Test Result Spreadsheet Link>, good or not as Linaro YY.MM release candidate. If not, paste synopsis here.

    1.2 TI Panda 4430: <Paste Test Result Spreadsheet Link>, good or not as Linaro YY.MM release candidate. If not, paste synopsis here.

    1.3 TI Panda 4460: <Paste Test Result Spreadsheet Link>, good or not as Linaro YY.MM release candidate. If not, paste synopsis here.

    1.4 ST Ericsson Snowball: <Paste Test Result Spreadsheet Link>, good or not as Linaro YY.MM release candidate. If not, paste synopsis here.

    1.5 ARM Versatile Express A9: <Paste Test Result Spreadsheet Link>, good or not as Linaro YY.MM release candidate. If not, paste synopsis here.

  2. Linaro ubuntu:

    2.1 Samsung Origen: <Paste Test Result Spreadsheet Link>, good or not as Linaro YY.MM release candidate. If not, paste synopsis here.

    2.2 TI Panda 4430: <Paste Test Result Spreadsheet Link>, good or not as Linaro YY.MM release candidate. If not, paste synopsis here.

    2.3 TI Panda 4460: <Paste Test Result Spreadsheet Link>, good or not as Linaro YY.MM release candidate. If not, paste synopsis here.

    2.4 ARM Versatile Express A9: <Paste Test Result Spreadsheet Link>, good or not as Linaro YY.MM release candidate. If not, paste synopsis here.

Thank you.

Best Regards

<Your Full Name>

Linaro weekly / YY.MM Release Test Summary

  • Cadence: Weekly, Monthly
  • Title: Test Result Summary of Calendar Week <NN> / Linaro YY.MM Release for Linaro <ubuntu> / <Android>.

  • Body:

Calendar Week <NN> / Linaro YY.MM Release: Here is test result summary for Linaro <ubuntu> / <Android> image on following boards:

  • 1) <Company Name + Board Name + Specific Model> (For example, TI Panda Board 4430);

    2) <List boards following item 1>; ......

Synopsis: No more than three sentences to describe general status for all test work.

<More detailed description per board>

1. <Company Name + Board Name + Specific Model> + <ubuntu> / <Android> (Column <X>):

<Test Result Spreadsheet Link for This Board + Platform>

No more than three sentences to describe the main issue and test status for this board + platform.

2. Following above with the same format......

For the previous week (Calendar week <NN-1>) summary, please refer to attachment <Save previous week test result summary as a PDF file then attach to this email. This work can be done in Gmail with "print" option, then select "Print to File">.

Thank you.

Best Regards

<Your Full Name>

Release Critical Bug Reminder

  • Cadence: weekly, monthly

For builds that have one or more open release critical bug, the QA services team sends an alert reminder about all release critical bugs to the release team as well as the build PoC and managing TL/PM for the build

In order to keep signal/noise ratio high, one RC alert reminder mail is sent to each PoC with all the relevant bugs for the PoC included.

Hi,

as part of our Build Validation Services, we deliver weekly and monthly reminders about the release critical bugs in builds we have recorded you as the primary engineering contact for. Please see below for all the important issue. If you need help debugging or reproducing the issue, don't hesitate to get in contact with the  QA services team by replying to this main.

Following your personalized release critical bug summary:

== android-build-name-.... ==
 - bug lp:<XXXXX> - <X weeks/N releases> - <Title of bug 1>
 - first seen: link to build/release this bug was first seen in
 - first release: link to build/release this bug was first seen in
 - confirmed: link to most recent build this bug has been validated against
 - confirmed release: link to build/release this bug was first seen in

== ubuntu-build-name ==

 - bug lp:<XXXXX> - <X weeks/N releases> - <Title of bug 1>
 - first seen: link to build/release this bug was first seen in
 - first release: link to build/release this bug was first seen in
 - confirmed: link to most recent build this bug has been validated against
 - confirmed release: link to build/release this bug was first seen in

== android-big-little-... ==

 - bug lp:<XXXXX> - <X weeks/N releases> - <Title of bug 1>
 - first seen: link to build/release this bug was first seen in
 - first release: link to build/release this bug was first seen in
 - confirmed: link to most recent build this bug has been validated against
 - confirmed release: link to build/release this bug was first seen in

Thanks!

-- 
<Your Name> <<your.name@linaro.org>>
QA Services - Linaro Platform
http://wiki.linaro.org/Platform/QA

Weekly big.LITTLE MP scheduler test

Here is the last week big.LITTLE MP scheduler test execution summary is:

High Lights:

  • New test cases have been added.

    <test case name>

  • Kernel update with latest <XXXX> patches.

  • bugs closed/opened in big.LITTLE System project / linaro Android Project

Release test report (week-9) 13.XX: Here is test result summary for big.LITTLE MP Scheduler test on TC2 platform with Android image

sched_tests.git No of Test Cases        Tests Run       Tests Pass      Tests Fail      Absolute pass rate (%)  Failure Analysis/Comments
  mpregression                20        19      19      0       100     -
  mpbasicsuite          10      9       9       1       90      -
  mpcoresuite            4      4       4       0       75      -
  mpextendedsuite        6      6       4       2       66.7    -
  mploadbalance          2      2       2       0       100     -

Android Build:

  • https://android-build.linaro.org/builds/~linaro-android/vexpress-linaro-mp-13.XX-release/

Linux kernel:

  • Linux version 3.8.0-00928-g7d866cd (jenkins-build@ip-10-10-66-148) (gcc version 4.7.3 20130205 (prerelease) (Linaro GCC 4.7-2013.02) ) #1 SMP Sat Feb 23 11:31:29 UTC 2013

Results spread sheet:

Summary of the all weeks can be found in attachments.

Thank you.

Best Regards

<Your Full Name>

Weekly big.LITTLE IKS core System test

Hi everyone,

Here is the last week big.LITTLE test execution summary is:

High Lights:

  • New test cases have been added.

    <test case name>

  • Kernel update with latest <XXXX> patches.

  • Test executions are performed on LAVA.
  • On Android:
    • Total Tests = XX
    • Tests Passed = XX
    • Tests Failed = XX

    http://validation.linaro.org/lava-server/scheduler/job/XXXX/log_file

Summary of the all weeks can be found in attachments.

Thank you.

Best Regards

<Your Full Name>

Roadmap Project Sign Off

Platform/QA/Communication (last modified 2013-08-12 15:08:29)