Validation Team Meeting

Agenda

  • Actions from last meeting - plars chasing
  • Engineer progress reports - All team members
  • Work Item Progress - plars and JamieBennett questions to the team

  • Any other business (AOB)

Past Action items

  • plars/zyga to discuss libraries shared between linaro-validation projects
  • mirsad to email JamieBennett and plars about scheduler status and blockers

People Present

Actions Recorded

  • plars/lool to discuss uboot mmc possible problems


Engineers Reports

Spring Zhang (springz)

Highlights

  • Start to work on test-abrek implementation, lp:~qzhang/lava/test-abrek: modification on previous comments, start to implement cmd_deploy_abrek.
  • Merged lp:~qzhang/lava/lava-remove-hardcode, which moves hard code to a python module, get comments and in enhancement.
  • Continue with generate_tarball implementation, Paul has almost done a better one, drop it.
  • Update some status in dispatcher blueprints. https://blueprints.launchpad.net/lava/+spec/linaro-n-validation-job-dispatcher

Plans

  • Working on ongoing branches/work items: test-abrek, cmd_deploy_abrek
  • Enhance dispatcher blueprint and spec.

Issues

  • Throat sore when got a cold from last weekend.

Zygmunt Krynicki <zyga>

Highlights

  • Deployed launch-control 0.3 release candidate 3 on validation.linaro.org (http://validation.linaro.org/launch-control/)

  • Fixed all compatibility issues Launch Control 0.3c3 with PostgreSQL
  • Added a workaround for Django 1.2.5 behavior change that affected l-c. In 1.2.5 FileFields are no longer deleted when the corresponding object is deleted. This caused data directory to pile up with unused files. This could prevent normal operation of the application.

  • Did some work on lp:django-debian so that Django applications get proper default database and can generate proper SECRET_KEY. Database settings integrate with Debian's dbconfig-common system and is administrator friendly.
  • While I don't report this often I really push a lot of stuff to my PPA to make deployment possible. I had 59 updates in the last month.

Plans

  • Mirror Django ticket 15622 in Launchpad. Propose for SRU.
  • Release launch-control 0.3c4 with all the PostgreSQL fixes and upgrade validation.linaro.org
  • Kick the RT ticket to upgrade dashboard.linaro.org to 0.3
  • Work on Abrek + power management tests
  • Work on scatter plot charts in launch control (for power management tests)

Issues

  • Got stuck on Django bug (http://code.djangoproject.com/ticket/15622) that affected l-c tests on PostgreSQL. I filed a bug on the upstream bug tracker, proposed a patch, backported the patch to my PPA and tested everything locally.

  • Slow progress on abrek + power management tests, got stuck with releasing launch-control 0.3, mostly analysis of existing tests and proof-of-concept native JSON output from shell scripts. Need to show this to the team to get feedback

Paul Larson <plars>

Highlights

  • Got QEMU continuous integration testing booting some known-good test images everytime there is a commit and new build of qemu in jenkins
  • Merged dependency updates from Torez for power management tests in abrek and had more discussions about next steps for those tests
  • Spent some time looking at how the android images will work, and figuring out how to modify our master images to accommodate them
  • Tested a new partitioning scheme, ran into hard limit of 7 partitions on SD card, will submit a bug to get this limit increased in kernel config
  • Code reviews on launch-control, dispatcher, and scheduler
  • Worked on generate_tarballs branch, got it all working and tested the changes, merge proposed. This should mostly finish up the bits needed to deploy linaro images in LAVA

Plans

  • Work around random mac addresses and dhcp failures to use static ip addresses on test machines, add support for this in LAVA
  • Look more at the new partitioning needed to accommodate android images
  • Work on getting further progress made with the dispatcher and scheduler

Issues

  • Network problems in the Cambridge office tonight, down for several hours now.

Mirsad Vojnikovic <mirsad>

Highlights

Plans

  • Address all issues in database WI and make it done ASAP
  • Start working on scheduler daemon and other WIs

Issues

  • Slow progress on scheduler BP


IRC logs

Started logging meeting in #linaro-meeting
[14:03:49] <plars> [LINK] https://wiki.linaro.org/Platform/Validation/Meetings/2011-03-17
[14:04:00] <plars> [TOPIC] Actions from last meeting
[14:04:11] <plars> * plars/zyga to discuss libraries shared between linaro-validation projects
[14:04:44] <plars> I think we discussed that pretty well... no immediate actions fell out of it I think, but possibly some future work
[14:04:56] <plars> zyga done?
[14:06:05] <plars> we waited too long and zyga fell asleep :)
[14:06:09] <lool> (oy)
[14:06:17] <plars> hi lool
[14:06:34] <zyga> re
[14:06:36] <zyga> sorry
[14:06:38] <plars> np
[14:06:57] <zyga> plars, yes, we can plan something concrete around the O UDS
[14:07:04] <plars> ok
[14:07:11] <plars> * mirsad to email JamieBennett and plars about scheduler status and blockers
[14:07:34] <plars> mirsad: got your email about this, though I don't recall seeing a lot on blockers that we could help with
[14:07:55] <mirsad> ok, I can understand that
[14:08:41] <JamieBennett> mirsad: are there still issues outstanding for you on this?
[14:08:55] <plars> mirsad: as for getting additional people on it, working on that and should have some help there soon
[14:09:17] <mirsad> main blocker is that I don't get so much feedback on my proposals, I would say, since there are many suggestions and questions not answered yet
[14:09:59] <plars> mirsad: if you feel like there is a question not getting answered, please chase people and specifically ask until you get an answer
[14:10:30] <mirsad> I'm trying as good as I can, will try to improve on it
[14:10:32] <springz> sorry for that, i'm not very familiar with web service currently
[14:10:57] <plars> mirsad: there was some feedback on the last merge request, which should be a good place to start
[14:11:19] <mirsad> yes, that was really good
[14:11:32] <plars> ok, moving on
[14:11:40] <plars> [TOPIC] Engineer progress reports
[14:11:52] <plars> let's see
[14:12:06] <plars> mirsad: let's have you go first, since we're already on the subject
[14:12:17] <mirsad> yes
[14:12:31] <mirsad> well, merge proposal for database WI is out there, reivew ongoing
[14:12:42] <mirsad> fixing comments received so far
[14:13:02] <mirsad> also, there is this proposal for daemon
[14:13:06] <plars> mirsad: I hadn't seen any new commits since the original proposal
[14:13:32] <plars> mirsad: when you have something that you think addresses the review comments, please mark it as resubmit... I think it should notify us when you do that
[14:14:04] <mirsad> yes, OK, will do that
[14:14:45] <plars> mirsad: do you have a sense of how close you are to having an update on that branch?
[14:15:06] <mirsad> I will send an update tonight
[14:15:11] <mirsad> when I come home
[14:15:11] <plars> ok, thanks
[14:15:19] <plars> anything else?
[14:15:32] <mirsad> nope
[14:15:40] <plars> ok
[14:15:44] <plars> zyga: your up
[14:16:06] <zyga> re
[14:16:10] <zyga> sorry I'm partially in #tech
[14:16:11] <zyga> okay
[14:16:17] <zyga> so this week was about releasing L-C for me
[14:16:29] <zyga> despite more ambitious plans to do other things
[14:16:46] <zyga> I managed to install LC on validation.linaro.org
[14:17:02] <zyga> I resolved almost all postgresql issues that would prevent it from becoming production ready
[14:17:16] * mirsad likes latest L-C, excellent work zyga!
[14:17:26] <zyga> and I'm struggling with final timezone issue that makes slight data differences between postgresql and sqlite
[14:17:36] <plars> [LINK] http://validation.linaro.org/launch-control
[14:17:40] <zyga> (we're trying to resolve that in #tech with Cody)
[14:17:45] <zyga> righh
[14:17:46] <plars> ah
[14:17:57] <zyga> I encourage everyone to install the RC4 which has just built in the ppa
[14:18:00] <zyga> do this on lucid though
[14:18:05] <plars> so I'd like to note, that the version installed on validation.linaro.org is kind of a tech preview
[14:18:09] <plars> it's not the production box
[14:18:10] <zyga> I did not copy binary packages to the "maverick" part of the PPA
[14:18:16] <zyga> right
[14:18:19] <plars> dashboard.linaro.org is still the official location
[14:18:28] <zyga> I thin RC5 will be final RC and I'll release 0.3 in 1-2 days
[14:18:43] <plars> zyga: have you pushed the updated release process to them for feedback yet?
[14:18:45] <zyga> RC5 has graph fixes so charts _do_ work on both databases now
[14:18:51] <plars> or are you waiting for .3
[14:18:53] <zyga> plars, no I did not
[14:19:18] <zyga> last time they took the impression that they should deploy it _now_ and I don't want to ask until I'm finally sure it's ready for consumption
[14:19:23] <zyga> it's really close now
[14:19:33] <zyga> if all else fails I'll ship 0.3 with timezone bug
[14:19:43] <zyga> we can recover from that later
[14:19:55] <plars> ok
[14:20:06] <plars> anything else?
[14:20:15] <zyga> no, not that I think
[14:20:16] <zyga> oh
[14:20:22] <zyga> I started UDS-O stuff
[14:20:26] <zyga> from Barcelona :)
[14:20:38] <springz> zyga: you mean planning?
[14:20:39] <zyga> I'll update all the wiki pages with my flight details
[14:20:42] <plars> you have all your travel arranged now?
[14:20:50] <plars> ok
[14:20:58] <zyga> plars, sort of, I'm just waiting for final confirmation from the travel agent
[14:21:10] <plars> zyga: is pulling up roots
[14:21:15] * zyga happily moves to Spain in a few days
[14:21:38] <zyga> oh, everyone, I'll be absent starting with 24th of March, my Linaro calendar has all the details
[14:21:57] <zyga> I should be out for around a week
[14:22:07] <zyga> that's all
[14:22:12] <plars> thanks zyga
[14:22:15] <plars> I'll go next
[14:22:36] <plars> qemu continuous integration blueprint I picked up from Peter is complete
[14:22:46] <plars> he said he might like to add a few additional images in the future
[14:23:04] <plars> but this gives him a build in jenkins every time something changes
[14:23:13] <plars> and boots some images to test it
[14:23:29] <plars> spent some time looking into upcoming things
[14:23:37] <plars> like the scheduler daemon and how we might approach that
[14:23:42] <plars> as well as the android stuff
[14:24:11] <plars> they have a very specific partition layout, and if we want to support both android and linaro images, the master image layout needs to change
[14:24:34] <plars> also we will need a kernel config option enabled to allow us to see additional partitions on mmc
[14:24:37] <springz> android image needs lots of partitions
[14:24:48] <plars> I'm not sure how uboot will handle it, or if there will be issues there
[14:24:53] <plars> need to talk to jcrigby about it more
[14:24:58] <plars> yeah, it does
[14:25:18] <plars> updated the code for generate tarballs, pushed merge proposal after doing some testing
[14:25:36] <springz> so the board will be reused between android and normal linux test?
[14:25:40] <plars> there are a few pesky issues that still need to be fixed, mainly dealing with network issues and expect madness
[14:25:51] <plars> but this gets us to where the code checked in to lava should work for deploy
[14:26:07] <plars> the big thing I need to look at now is the networking problems
[14:26:25] <plars> the dhcp server in the lab was just ignoring all requests yesterday
[14:26:35] <plars> so we need to move to a static address on these boxes
[14:26:38] <lool> I'm happy to discuss specific things we could do for u-boot with you if you like
[14:26:48] <plars> but the mac address on the boards cannot be relied on
[14:26:53] <springz> oh, can it recover?
[14:26:58] <lool> we can set the MAC address on boot too if that helps
[14:27:13] <lool> With an udev rule or in /etc/network/interfaces
[14:27:18] <plars> [action] plars/lool to discuss uboot mmc possible problems
[14:28:04] <plars> lool: asac and I discussed quite a bit yesterday, I don't think it's too much trouble at all to just bring up the interface with a static address
[14:28:13] <plars> and that's very predictable
[14:28:26] <springz> what kind of static address? by DHCP server or hardcode
[14:28:43] <lool> Yup
[14:29:07] <plars> springz: originally I was thinking of giving it to it via dhcp, but because of the mac address issues, I'm leaning more toward just having the dispatcher look it up, and push it
[14:29:15] <lool> plars: Eventually, I think we should put machinery in the prebuilt images which makes it easy for validation to run
[14:29:25] <springz> plars: and if it affect the /etc/resolv.conf?
[14:29:27] <lool> If we need an initramfs snippet which sets the MAC address from kernel cmdline, we can add one
[14:29:47] <plars> we have several ways to do this, obviously just pushing an ifconfig command works, as well as through kernel parameter
[14:30:03] <plars> (well, kernel parmeter needs an extra config item turned on)
[14:30:24] <plars> lool: to do it like that though, we'd have to do that in each new test image also
[14:30:51] <lool> plars: To be clear I'm proposing to do this on snapshots and released images
[14:30:54] <lool> so that they are ready to be tested
[14:31:18] <plars> lool: huh? so they would all get the same mac address?
[14:31:25] <plars> oh
[14:31:29] * plars reads that line again
[14:31:31] <lool> plars: Sorry, I'm saying that we'd have support for ethaddr=foo in al images
[14:31:35] <lool> all images
[14:31:41] <lool> and validation infrastructure would just ensure it's set
[14:31:48] <plars> we have several options
[14:31:53] <plars> that's a new one though
[14:32:09] <plars> we will take a look more at all the options this week and decide on the best path forward
[14:32:23] <plars> springz: your turn
[14:32:31] <springz> ok
[14:32:40] <springz> merged lp:~qzhang/lava/lava-remove-hardcode with loic's help
[14:32:55] <springz> Continue with generate_tarball implementation, but Paul has almost done a better one, so I drop it
[14:33:06] <springz> implement a new action cmd_deploy_abrek in test-abrek
[14:33:17] <springz> finish basic command sequence to install abrek to test image by chroot, have a test and pass, update the mp, please comment on it
[14:33:42] <springz> I think test_abrek can split to three parts, first is cmd_deploy_abrek, the second is cmd_install_abrek for abrek installing test suites like ltp, the last is cmd_test_abrek to execute real test suite command
[14:33:52] <springz> what do you think
[14:33:56] <plars> that's not what we discussed
[14:34:11] <plars> why should there be two separate commands for the abrek deploy and install test?
[14:34:34] <plars> I can't imagine where you would want to deploy abrek in preparation for a test, but not the test
[14:34:36] <plars> but if you did
[14:34:51] <plars> you could just leave an empty parameter for which tests to install
[14:34:55] <springz> ok, we can merge the two
[14:35:10] <plars> I thought that's how we had it before, as just a single command
[14:35:56] <springz> so only two actions: cmd_test_abrek and cmd_abrek
[14:36:32] <plars> I would have one of them be cmd_install_abrek or something... cmd_abrek is a little too ambiguous I think
[14:36:43] <plars> I put this in the spec if you want to look at that
[14:36:44] <springz> oh, yes, mistake
[14:37:18] <springz> ok, I'll follow them
[14:37:37] <plars> anything else?
[14:37:46] <springz> and I met some instable issue when sending via serial
[14:37:54] <springz> you can see in the mp comment
[14:37:58] <plars> ah, I saw that
[14:38:07] <plars> something bad with your serial connection perhaps?
[14:38:39] <springz> I think USB-COM converter may have some problem
[14:39:16] <plars> do you have another one you can test with?
[14:39:16] <springz> no others now
[14:39:24] <plars> it shouldn't be related to cu
[14:39:32] <springz> I'm using cu
[14:39:38] <plars> I know
[14:39:43] <plars> I'm saying it should not be that though
[14:39:47] <springz> I'll move to a native serial port to have atry
[14:39:53] <plars> more likely to be hardware I think
[14:39:54] <plars> ok
[14:40:26] <plars> lool, JamieBennett, anyone else have anything more to add before we move on?
[14:40:37] <JamieBennett> plars: not from me in this section
[14:40:51] <plars> that's fine, just wanted to give the opportunity
[14:41:00] <lool> I'm good; I have tons of things to add, but they are distractions :-)
[14:41:06] <plars> heh
[14:41:07] <plars> ok
[14:41:12] <plars> [TOPIC] Work Item Progress
[14:41:20] <plars> [LINK] http://status.linaro.org/linaro-validation.html
[14:41:33] <JamieBennett> So it seems we are behind on the validation master boot image, continuous integration, validation job dispatcher, and benchmark reports in launch control blueprints. The scheduler blueprint still has a lot of work items and it doesn't seem to be progressing. Anything to worry about plars?
[14:42:05] <plars> that seems to cover it
[14:42:22] <plars> the master boot image one I'm not too worried about... a lot of that was dependent on getting hardware in the lab
[14:42:40] <plars> we have a few new pieces of hardware, but we don't have anyone local yet to fiddle with them
[14:42:44] <plars> that will be resolved next week
[14:42:53] <plars> we have someone new starting in cambridge next week
[14:43:00] <plars> so I expect to make additional progress on that
[14:43:10] <springz> plars: is there a master image deployment script for every type of board?
[14:43:10] <plars> but android has thrown a curve ball at it too
[14:43:16] <plars> it's not a script
[14:43:35] <plars> mainly just documentation for how to set up the master image so that it will work for each board type
[14:43:52] <plars> I'm trying to help get us caught up on the dispatcher piece
[14:43:59] <springz> so it is manually done
[14:44:04] <plars> the scheduler I'm really worried about, no progress there
[14:44:26] <plars> zyga: how do you feel about the remainder of the items for launch-control?
[14:46:11] <plars> hmm
[14:46:15] <springz> seems busy on another channel
[14:46:18] <plars> we can come back to him
[14:46:57] <plars> JamieBennett: also, we discussed that I'm trying to bring in some additional help to accelerate development on some of those pieces
[14:47:04] <JamieBennett> plars: right
[14:47:29] <JamieBennett> OK, lets keep these on the radar then
[14:47:35] <JamieBennett> and reassess each week
[14:47:44] <plars> we have a lot of work to get done, but I think we can get a lot of it knocked out
[14:47:52] <JamieBennett> awsome
[14:48:24] <mirsad> I have a question about WIs
[14:48:30] <plars> mirsad: go ahead
[14:48:52] <mirsad> how to know when a WI is done, is there a done criteria?
[14:49:03] <JamieBennett> mirsad: depends on the work item
[14:49:14] <JamieBennett> mirsad: you should know when the item is done or not
[14:49:15] <plars> if you don't know, then it's probably a poorly written work item
[14:49:21] * JamieBennett nods
[14:49:31] <plars> mirsad: is there a specific one you had in mind?
[14:49:36] <mirsad> ok, so it's me who decides?
[14:49:39] <mirsad> plars, no
[14:49:57] <plars> if you are doing the work, yes, if you feel something is done, you should mark it as done
[14:50:06] <plars> anyone subscribed to the blueprint will get a status update on it
[14:50:17] <plars> and can feel free to comment if they disagree
[14:50:43] <mirsad> ok, I understand now
[14:50:53] <plars> I will tend to not mark things done until they are merged
[14:51:03] <plars> unless it's something that doesn't make sense for that
[14:51:32] <plars> anything else on this topic? we're almost out of time
[14:52:24] <plars> ok
[14:52:33] <plars> [TOPIC] AOB
[14:52:46] <plars> as mentioned earlier, we have a new team member starting next week
[14:53:12] <plars> should see him show up, say "hi"
[14:53:29] <springz> ok
[14:53:38] <mirsad> plars, who is it
[14:53:44] <plars> mirsad: Dave Pigott
[14:53:52] <plars> he'll be working out of the Cambridge office
[14:54:15] <plars> so he'll be helping with lava, as well as hardware issues that need a local person to coordinate
[14:54:37] <plars> anything else here?
[14:54:56] <plars> 3...
[14:55:05] <plars> 2...
[14:55:14] <plars> 1...
[14:55:17] <plars> #endmeeting
Meeting ended.

internal/archive/Platform/LAVA/Meetings/2011-03-17 (last modified 2013-08-23 13:19:44)