Platform Validation Team Meeting
- Actions from previous meeting
Work item progress (http://status.linaro.org/linaro-validation.html)
- Brief Progress
- Any other business
Action Items from last Meeting
- zyga to talk to lool about ppa layout recommendations
Action Items this Meeting
- plars/zyga to discuss libraries shared between linaro-validation projects
mirsad to email JamieBennett and plars about scheduler status and blockers
Zygmunt Krynicki (zyga)
- More packaging for ppa:zkrynicki/lava. In particular I spent some time to package a hello-world Django application with the proper Debian integration around database and web server. Mostly research and checking existing packages but I did make five point releases of django-hello (currently at 0.5). It supports apache WSGI integration, dbconfig-common database integration and some debconf settings. Packages and source on launchpad.
Started Django packaging guideline for Debian at http://wiki.debian.org/DjangoPackagingDraft and engaged with debian-python mailing list for feedback.
- Started django-debian package for connecting django database settings to dbconfig-common. Released versions 0.1 and 0.2. Current version seems to be enough to get database integration working correctly. Source code and packages available in my lava PPA, on pypi and launchpad.
- Moving to package launch-control 0.3c2 based on django-hello.
- Had a call with Paul and Torez about power management tests and abrek support. Planned a few weeks of work around integrating power management tests better with abrek and launch control
- I hope to install launch-control 0.3c2 on validation.linaro.org
- I'll start working with Torez on abrek power management tests. The goal is to get current suite of tests extended to use more features offered by launch-control and not directly accessible in current abrek. I will also need to understand how those tests work and what kind of reporting UI is expected in the next step.
- Add native mode to abrek (where tests directly speak dashboard bundle format).
- I did some some tests of Natty Alpha 3 on my Beagle C4. I never managed to get anything to work. My board seems to be stuck on the boot loader. Tried with various cards and l-m-c versions. I'm waiting for Torez to upload a working image to see if it's something specific to my system.
- I was away this Monday (7th of March)
Spring Zhang (springz)
- Working on branch lp:~qzhang/lava/test-abrek, implement a simple version, splits the action to test suite installation and test running.
- Working on lp:~qzhang/lava/lava-remove-hardcode, which moves hard code to a python module, get comments and in enhancement.
Update dispatcher blueprints, replace message queue items and make a detail description for some items. https://blueprints.launchpad.net/lava/+spec/linaro-n-validation-job-dispatcher
Add flow of deploy_linaro_image in spec: https://wiki.linaro.org/Platform/Validation/Specs/JobDispatcher
- Working on ongoing three branches/work items: gen-test-tarballs, remove-hardcode, test-abrek
- Start a new branch on deploy test tools like abrek: deploy-test-tool
- Enhance dispatcher blueprint and spec.
Mirsad Vojnikovic (mirsad)
- Looking at new design for the scheduler, since last week we will not use message queue. New functions are needed in the scheduler, this also affects the spec.
- Discussion about different scheduler things are still ongoing, for instance database part or how to execute and monitor a test job in the dispatcher
- Continue with scheduler design and implementation
- Try to gather more information for the scheduler functionality
Paul Larson (plars)
- worked out procedure for having a local development/test environment for lava with conmux and locally attached boards, documented, and added example configs to lava
- cherry-picked a patch from zygmunt's branch to add support in abrek for saving bundles at the end of a test run
- lot of code reviews, and discussions about specs/work items
- interviews, working on bringing new people in
- tested Alpha3 on panda
- discussions with power management group about how we can help them get their test updates into abrek, and produce useful graphs for them with launch-control
- Finish up getting daily qemu builds to boot a known-working linaro and debian image every day for qemu continuous integration testing
- Work on support for additional boards in lava, that we now have in the lab
- context switching a lot between trying to help with technical aspects of various pieces here, and project management type things
Minutes IRC (Transcript)
<plars> [TOPIC] Actions from previous meeting
Actions from previous meeting
<plars> zyga to talk to lool about ppa layout recommendations
<plars> zyga, lool?
<zyga> plars, so I asked lool about this yesterday
<zyga> plars, but neither of us remembers what it's about
<zyga> plars, no progress I would say
<zyga> lool, ^ ?
<plars> so, if I recall correctly...
<lool> I did remeber yesterday
<lool> but because it was a week old, I had to fix through IRC logs
<plars> you have a plethora of packages you've been working on for all your dependencies for launch control
<lool> Essentially it was about which PPA to use for launch-controls deps
<plars> and it was about just making sure we put them in a sane place
<lool> I didn't know enough to decide whether we basically wanted one PPA for launch-control and one for lp:lava or whether we'd share one for both
<zyga> sorry about mis-interpreting our yesterday conversation then
<lool> So perhaps a simple question is: can either be used standalone, or are they interdependent?
<plars> in this case, I think it would be better to have them separate
<zyga> plars, why?
<plars> they can certainly be used separately
<lool> zyga: We seemed to talk at half an hour offsets yesterday, too high a latency to discuss anything
<plars> and like in our case, they would not even be on the same server
<plars> we may have a small development instance of launch control on v.l.o
<zyga> plars, I don't see why we need more than one ppa for this?
<plars> but it's not the official one
<lool> Can one run launch-control by itself without lp:lava, and lp:lava without launch-control? is that an expected scenario?
<zyga> lool, on one machine, yes, in general, not so much
<zyga> one does not make much sense without the other
<plars> in fact, launch-control existed before lava was even thought about
<zyga> lava is all of that together under one brand
<lool> Ok; so launch-control and lp:lava are really separate products; maybe it would be cleaner to keep them in separate PPAs
<lool> we could have ~linaro-infrastructure/lava PPA for lp:lava and /lauch-control PPA for lp:launch-control
<plars> zyga: right now they are separate projects, you were thinking of merging them somehow?
<plars> zyga: the other thing I was thinking is that l-c has all these deps
<lool> There seems to me confusion between LAVA the project group and lp:lava
<plars> zyga: having them in a single ppa could make it a bit blurry whether all those deps are needed for lava as well
<zyga> ok there are two cases here: should we have many ppas for each thing that exists as a separate project (where I'd say NO because there is no positive effect as far as I can see) and what are the expected usage scenarios (we can argue here), does it really matter?
<lool> Do we have a more precise name for lp:lava?
<zyga> plars, and that is important how exactly?
<zyga> plars, it's a PPA
<zyga> it's meant to have packages
<lool> Or we could stop naming launch control "LAVA" and just say it's useful in conjunction with lp:lava
<plars> zyga: where is launch control called lava?
<zyga> lool, IMHO that's a bad idea
<plars> perhaps I'm missing that somewhere?
<plars> lp:lava should not, in my understanding, include any launch control bits
<zyga> plars, lava is the whole stack, the whole software stack with abrek, l-c, and lp:lava (perhaps that's what causing the confusion)
<plars> with the exception of a single action that allows uploading a bundle to launch control
<plars> but it doesn't use launch control for that
<zyga> plars, as we go forward IMHO we should merge the dashboard with lp:lava web UI as it only makes sense to do so
<plars> doesn't require launch-control deps
<zyga> but that is not what we're discussing here
<zyga> we're discussing PPA layout
<lool> dashboard == launch-control?
<zyga> lool, yes
<plars> zyga: how many packages is it?
<zyga> plars, what exactly?
<plars> zyga: l-c + deps
<zyga> plars, I don't know, l-c itself will most likely be two packages
<plars> that you would need to drop in a ppa
<zyga> plars, but I don't see why the number of packages is relevant
<zyga> plars, we could just make one big package if we really wanted but that's not good way to develop this IMHO
<plars> so, it just seems really unclear to me why someone would go to lava, looking for a ppa to get l-c installed
<plars> when they are separate projects
<lool> If we're going to document launch-control as a separate deployment from lp:lava, I think it would be nicer to split out the deps so that people don't see more unofficial packages than they need to
<zyga> plars, because we're all working on one team for one goal?
<plars> but, you are right that they are from one team
<plars> right, was just getting to that
<zyga> plars, and we should thus have one ppa where all the fruits of our work are?
<plars> so we could just put it all under linaro-validation team ppa
<lool> If they are technically separate but we document that you need to deploy both, then it doesn't really matter and we can merge the deps in the same place
<plars> I would have thought a separate ppa for each though
<zyga> plars, this makes it more messy because things like django backport then belong to yet another ppa
<lool> zyga: all PPAs would be under the team's control, ~linaro-infrastructure
<plars> my original thinking was based on previous conversations with lool about this though
<zyga> we can play this ppa game but I don't see any point
<zyga> lool, I'd like that
<plars> I thought we were trying to go to a model with the ppas where things were separated out more
<zyga> lool, note that's the way it was when we were infrastructure
<lool> zyga: I don't think this is about splitting the team in two or something
<plars> certainly not
<lool> Sorry, I meant ~linaro-validation
<springz> I think we can do it in a PPA, but they are still different software, we need abrek, but it can also be replaced by others
<lool> but you get the idea
<zyga> after we separated I just started my own ppa because I could not do it in ~l-v myself at the time
<zyga> lool, so to get back to the original question
<zyga> lool, what is the ppa model you think we should pursue?
<lool> zyga: Well, it mostly depends on how we hand out these things to the outside world
<lool> if it's a single "Install LAVA suite" solution, then I would likely go for one PPA for everything; if it's more "You can run Launch Control, here's how..." + "You can run lp:lava build farm, here's how..." then I'd go for separate ones
<zyga> lool, why? even if some things are shared?
<lool> e.g. ~linaro-validation/lava and ~linaro-validation/launch-control and ~linaro-validation/abrek
<lool> zyga: What's shared?
<plars> in our case, if IS does decide to pick up launch control from a ppa, then they most certainly do not want to pick up any other pieces there
<zyga> lool, django would be shared at the very least
<zyga> lool, a few libraries I wrote are applicable to all of that, lava, abrek and l-c (currently the only user)
<zyga> plars, they would not have to install them
<zyga> plars, the only things that would get "picked up" would be backports
<lool> Ok; well, we could have a common PPA; I know of teams doing that, but it's more painful
<plars> zyga: I think I need to better understand from you, which libraries you see being needed in lava and abrek that you did for l-c
<zyga> plars, and as soon as we go to the next LTS (if it's the IS deployment we're talking about) we should have none
<plars> but we should probably continue that discussion offline
<lool> Given the number of instances we're speaking of, I don't think any choice is particularly bad
<plars> [action] plars/zyga to discuss libraries shared between linaro-validation projects
plars/zyga to discuss libraries shared between linaro-validation projects
<plars> yeah, I don't want to overthink it
<zyga> we can talk about it later but IMHO if we keep ppas separate it will only discourage common code
<lool> it's just harder to split things out once they are merged :_)
<zyga> and IMHO that's not good idea
<lool> zyga: But we want them to be independent
<zyga> lool, what and how?
<lool> Let's bring it offline
<plars> we've strayed a bit far with that one
<lool> we tried resolving it, needs further discussion
<plars> we have an action to discuss it further
<plars> let's move on for now
<lool> I don't think it's a big deal, no matter what the outcome is
<plars> [TOPIC] Work item progress (http://status.linaro.org/linaro-validation.html)
Work item progress (http://status.linaro.org/linaro-validation.html)
<plars> This is the standing, weekly reminder topic
<plars> to look at our work items
<plars> see if there's anything you can cross off
<plars> I have one I should be able to cross off today
<plars> as soon as it's done
<JamieBennett> Mirsad Vojnikovic still has 37 todo's, is that an issue?
<plars> mirsad: any progress you can cross off on the scheduler part? We haven't seen much code here yet
<mirsad> no, since we changed to new approach and adding stuff to the database, I'm still working on that one
<plars> I don't think we really changed the approach so much
<mirsad> well, I think we did
<plars> and we added a single field to a database that doesn't exist yet (priority, which had been previously discussed)
<JamieBennett> mirsad: beta freeze is 2011-03-24 and it would be nice to get all projects to near completion then even if they are not on the release images. End of cycle is only 2 months or so away.
<mirsad> not just that, we will save json in database, then actions, and most likely some more stuff into db
<JamieBennett> mirsad: is that Blueprint in jeopardy to not be finished this cycle?
<mirsad> JamieBennett, I understand that, and I'm working after my best effort
<mirsad> JamieBennett, probably if things will be added, since I feel that design is still not settled
<springz> I think the change may be that we drop the message queue
<springz> we can still use queue in scheduler to help dispatch the job
<plars> mirsad: we did talk about dropping the message queue, in an attempt to simplify it, not complicate it
<plars> mirsad: but as I mentioned in that thread, if you have code already to support the queue, and believe this to be a simpler approach, that's fine!
<mirsad> yes, but in my original proposal, a deamon process would be on dispatcher part, not scheduler
<springz> that also confuse me sometimes for I think there is some change and some previous work will drop.
<plars> it's really all part of the same code base, so if we have a daemon that picks up the jobs from the queue, and starts them, I really don't see how it's relevant whether we call it part of the scheduler or the dispatcher
<springz> now I think we need to keep on items, if some dicision made, we can also change the code
<mirsad> it is, since it comes under scheduler WIs
<plars> mirsad: but there are a lot of other WIs before you get to that point
<plars> If you have some time in a bit, lets talk about how we can get things rolling on those
<mirsad> plars, that is questionable, please see my latest proposal, since the solution is still not settler
<plars> The purpose of the recent discussion on all of this was not to make things harder, but to try to help you iron out implementation details that I thought might be blocking you
<JamieBennett> mirsad: plars: I think we need to carfully monitor this Blueprint to ensure we move forward. mirsad can you email plars and I a status on the Blueprint and your work and any issues you face that are blocking you?
<JamieBennett> we can then work through them
<mirsad> yes, I can
<plars> [ACTION] mirsad to email JamieBennett and plars about scheduler status and blockers
mirsad to email JamieBennett and plars about scheduler status and blockers
<plars> [TOPIC] Brief Progress
<plars> springz: ?
<springz> I keep on lp:~qzhang/lava/generate-tarball remove-hardcode, and open a new work item test-abrek implementation
<springz> do some modification on dispatcher spec, clear some questions about test-abrek with Paul yesterday
<plars> springz: there has been some progress on those, but would be good to get some of them finished up, review comments handled, and merged
<springz> plars: yes, I'm working on the comments, I'd like to close them
<springz> plars: I do want to make all parts to work as a whole thing in dispatcher
<springz> plars: based on basic implementation, we can enhance them then
<plars> anything else?
<springz> that's all
<plars> mirsad: ?
<mirsad> working on database WI, trying to figure out how to handle actions in best way + will add priority
<mirsad> have done some cleaning of django parts, will try to push some code tonight
<mirsad> still working on the new scheduler design now when MQ is out
<zyga> mirsad, if you want I can review your code
<mirsad> have sent an initial proposal for a deamon, will be good if I could get some feedback
<mirsad> zyga, it would be perfect
<plars> I saw it, and it looked like a high level graphic
<plars> it didn't seem to address the implementation details of it
<plars> I'm more interested in seeing that part
<plars> but as I mentioned earlier
<plars> there are a lot of work items we can look at before that point
<mirsad> plars, yes, but still I don't know how to resolve the mail parts
<plars> mail parts?
<mirsad> main, sorry
<plars> ok, this sounds like one of those blockers you need to talk to me and JamieBennett about
<plars> anything else?
<lool> I'm happy to participate in a call around the scheduler/db stuff
<plars> thanks lool
<zyga> mirsad, if you want we could mumble together, I'm a little out of touch with what's going on in lp:lava now but I'd be glad to offer assistance
<lool> maybe to define a small scope of the very first prototype version, and then expand it
<lool> zero code is not good with a release RSN
<plars> zyga: ?
<mirsad> zyga, it would be nice, but mumble is not allowed inside STE (at least proxy is not allowing it)
<zyga> uh :/
<zyga> plars, ok I will go quickly
<mirsad> lool, agree!
<zyga> so last week has been mostly about packaging and django and debian
<zyga> fortunately this topic is going away now
<lool> is it?
<zyga> I did some extra development on l-c as well
<zyga> I'll push that for review but it's not ready yet
<zyga> (I added the generic chart code)
<zyga> it still has some issues with selecting data and de-duplication of code from the gcc part of the code
<zyga> I've started looking at abrek and power management code
<zyga> today I'll push the final l-c packages and from tomorrow I hope to work on abrek full time
<lool> power management code?
<zyga> for torez
<zyga> and amitk
<zyga> the goal is to get their tests integrated with abrek properly
<lool> Oh ok, I thought you couldn't find it
<plars> torez has some power management tests that we are trying to ensure work in abrek, and also produce useful results in launch-control for them
<zyga> currently there is a disconnect between l-c features and what can be exposed via abrek
<zyga> lool, yes it's gone
<zyga> lool, but I resolved that in another way
<zyga> that's all from me
<plars> thanks zyga
<zyga> oh, and I managed to get my beagle working
<plars> that's good
<zyga> (and natty A3 does not work really)
<plars> but you still need to resolve why it doesn't behave normally for you
<plars> having to download an image from another machine is not a good solution
<plars> lmc should work
<zyga> plars, I made another image this morning, it booted fine
<zyga> plars, with my hacking branch though!
<lool> I built images in EC2 yesterday
<plars> ok, good then
<JamieBennett> zyga: what is wrong with the A3 image?
<lool> it's not trivial as vfat support is broken in the virtual maverick kernels
<zyga> plars, I rebased it on top of the trunk
<lool> but it should be working with lucid + maverick PPA or maverick + -server kernel (pain to install though)
<zyga> JamieBennett, ubuntu desktop stops being visible after a few seconds, HDMI FIFO overflow in syslog
<lool> JamieBennett: I think zyga has l-m-c issues
<plars> ongoing issues
<zyga> lool, no I had issues with the natty A3 image for my beagle as well
<JamieBennett> lool: that doesn't sound like a l-m-c issue, maybe a board issue?
<lool> zyga: As a workaround for the lmc issues, creating an image file and dd-ing should be more reliable
<lool> JamieBennett: I think he has both issues
<zyga> lool, I'm doing that as well, it's much much much faster
<zyga> lool, I can DD with around 9MB, previously I had 1MB
<lool> mucho fastero!
<zyga> mucho fastero
<lool> zyga: Make sure you pass bs= to dd BTW
<lool> bs=30M or something
<zyga> lool, that's what I'm doing
<zyga> lool, I passed 16M
<lool> zyga: </status>?
<plars> ok, anyone else?
<JamieBennett> plars status
<zyga> lool, ?
<zyga> lool, yes
<lool> I can also give a one line update
<plars> mostly code reviews, spec reviews with you all, etc
<plars> tried lool's conmux package
<plars> and documented how to get a local development environment working for lava
<lool> lool: didn't do any code, just some code reviews and chats; I had an idea to simplify creation of testable partitions and tests in general which I'll experiment with today and then submit
<zyga> lool, I have some ideas there too, we can talk offline
<plars> I also merged part of an abrek branch zyga was working on, to add support for saving a bundle at the end of a test run in abrek
<lool> zyga: I was first!
<zyga> plars, I saw, thanks!
<plars> other than that, mostly pm stuff... interviews, working on bringing in additional help for us, getting information about some new stuff michael hope wants us to look at, pm team stuff, etc
<JamieBennett> plars: can you add your activity to the meeting page so I can include it when I do the team report?
<plars> JamieBennett: will do that right after this meeting
<JamieBennett> plars: also any PM stuff you want to offload on to me feel free
<plars> JamieBennett: thanks, good to know
<plars> [TOPIC] Any other business
Any other business
<plars> JamieBennett: do we have new qatracker tests getting kicked off today?
<plars> I tested panda last week
<JamieBennett> plars: I'll set them up after my next call
<plars> we have some new boards in the validation farm
<plars> mx51evk01 is booted to master image now
<plars> thanks to dmart
<springz> plars: with which release?
<plars> and there are two u8500 boards, still in progress getting an image on them
<plars> springz: typically we have been just using the 10.11 release, or latest/most stable
<JamieBennett> plars: infact I've just finished setting up the weekly testing in qatracker (http://qatracker.linaro.org/) so please all test
<plars> thanks JamieBennett
<plars> ok, anything else?
<plars> thanks everyone
internal/archive/Platform/LAVA/Meetings/2011-03-10 (last modified 2013-08-23 13:19:29)