Ubuntu Wiki

Log

UTC

[18:01] <heno_> #startmeeting
[18:01] <heno_> nope :(
[18:02] <heno_> the agenda is here as usual: https://wiki.ubuntu.com/QATeam/Meetings
[18:02] <ara> hello everyone
[18:03] <sbeattie> hey
[18:03] <heno_> [TOPIC] Regression tracking guidelines and web page prototype.
[18:03] <heno_> sbeattie: can you give a summary/update?
[18:03] <sbeattie> Um, sure.
[18:04] <sbeattie> This is an overview https://wiki.ubuntu.com/QATeam/RegressionTracking
[18:05] <LaserJock> so is there any plan on how we're going to deal with and/or triage regression bugs?
[18:05] <sbeattie> We'd like to track 3 things, potential regressions in the devel version (e.g. intrepid), regressions between released versions (e.g. gutsy -> hardy), and regressions in updated versions (e.g. hardy -> hardy-updates)
[18:05] <LaserJock> I can just imagine a flood of bugs
[18:07] <sbeattie> Sort of, I was envisioning web pages to track all of the different tags
[18:07] <heno_> The plan is to first get a better overview of where we are regressing so we can a) minimise that and b) release note better where we cannot fix
[18:07] <heno_> It's not a campaign to collect more bugs as such :)
[18:07] <LaserJock> heno_: so, sorry to be predictable, what about Universe in this?
[18:08] <sbeattie> Plus perhaps another script that looks for untagged bugs that mention regression.
[18:08] <LaserJock> heno_: no, but once you say "we're trying to find out about regressions" you're going to get a lot of people filing regression bugs for every feature that changes
[18:08] <sbeattie> Universe included, I don't see why it would be different
[18:08]  * persia likes the idea of a campaign to collect more bugs: more feedback means better understanding of how to improve
[18:08] <ara> sbeattie: adding a task in the release notes project, as soon as we discover a potential regression... could it be possible that we add a release note of a regression not happening
[18:08] <heno_> LaserJock: does that need a different process? I would think a regression between releases would be the same in main and universe
[18:09] <LaserJock> sbeattie: because it'd perhaps be hard to write release notes if Universe bugs outnumber Main by a lot
[18:09] <ScottK> persia: I think there would be more incentive if there was less closing of bugs not yet fixed.
[18:09] <persia> LaserJock: We might have a lot of wontfix bugs, but that's still useful for later bug filers.
[18:09] <persia> ScottK: That too.
[18:09] <persia> (but unrelated)
[18:09] <ScottK> persia: I think it's related to the odds of success of such a campaign.
[18:10] <LaserJock> heno_: no, not different processes.
[18:10] <heno_> LaserJock: good point - have universe issues been release noted in any way in the past?
[18:10] <LaserJock> heno_: not that I know of
[18:11] <persia> There've been a few cases where release notes mentioned something of especial interest in universe, but it's not common, except for parts of universe that constitute the universe flavours.
[18:11] <LaserJock> I'm also slightly concerned about lots of stats without any developer effort to back them up, if that makes sense
[18:11] <heno_> LaserJock: hm, ok. should we encourage that to be started?
[18:11] <LaserJock> heno_: I'm not exactly sure. I have no idea how it'd be even written
[18:12] <sbeattie> ara: I'm not sure exactly what you're asking, but I could see release notes worthy stuff that aren't regressions being added to the release-notes project; that's more a question for slangasek
[18:12] <heno_> LaserJock: the idea is to use the metrics to inform and focus devel effort
[18:12] <LaserJock> heno_: right, but I think it's important to make sure that actually happens
[18:12] <heno_> does anyone have ideas for other fields in the regression data template?
[18:13] <LaserJock> I love stats, don't get me wrong. I'm a scientist and I live on data
[18:13] <LaserJock> :-)
[18:13] <heno_> I'm a theoretical scientist so I'm not as worried about data ;)
[18:13] <sbeattie> LaserJock: Heh. With actual evidence rather than hand-waving it can become easier to convince people of the need to take action.
[18:13] <LaserJock> but I think the QA team can get a bunch of stats without any connection to actual development efforts
[18:14] <LaserJock> so then users see all these stats and developers not working on them and get upset
[18:14] <LaserJock> i.e. OMG, look at all those regression bugs!!!!
[18:14] <heno_> status.qa.ubuntu.com is an example of how we can use bug data to target the fixing work
[18:14] <LaserJock> so while I think it's a great thing to have, we might want to think about recommendations for ubuntu-devel
[18:15] <heno_> in what sense?
[18:15] <heno_> do you envision objections to using these tags?
[18:16] <LaserJock> things like "heah, we've noticed X number of regression bugs in package Y, you guys might want to get on that" :-)
[18:16] <LaserJock> no, I doubt there will be objections
[18:16] <heno_> and they would be right
[18:17] <sbeattie> LaserJock: I think you're correct to point out that we, as the QA team, need to be advocates for our users to the developers based on the stats we collect.
[18:17] <heno_> ok, please add further comments and ideas to the wiki page
[18:17] <heno_> moving on
[18:17] <LaserJock> sbeattie: I'd go further and say that we are a *part* of the development team and should drive QA within Ubuntu development
[18:18] <heno_> [TOPIC]: QA server setup - who is setting up what services
[18:18] <heno_> Brian has agreed to coordinate placement of services on the server, URLS, etc
[18:18] <heno_> several people here are already moving stuff there
[18:19] <heno_> can we get a brief overview from each?
[18:19] <heno_> ara, bdmurray, sbeattie, ogasawara? others?
[18:19] <bdmurray> I've setup a new bug stats database using postgres and storm on the server
[18:19] <LaserJock> heno_: this is on the new server in the DC?
[18:19] <ara> I just got instructions from bdmurray on the access permissions. I will be adding the automated tests starts there
[18:20] <heno_> LaserJock: yes
[18:20] <bdmurray> I need to import data from p.u.c and then move graph generation to the new server
[18:20] <ogasawara> I haven't officially moved anything over - but could migrate all the pkg-stat, weather report, kernel bug lists, etc over
[18:20] <heno_> dhobach plans to move harvest there as well
[18:21] <heno_> so anyone wishing to migrate stuff there, please coordinate with bdmurray
[18:22] <LaserJock> does that server use the *.qa.ubuntu.com ?
[18:22] <LaserJock> s/the//
[18:22] <heno_> yes, except some drupal things will stay on the old server for some time
[18:23] <bdmurray> like brainstorm and the isotracker right?
[18:23] <heno_> this one is PHP-free
[18:23] <heno_> bdmurray: right
[18:23] <LaserJock> right, but in terms of URL they're all going to use qa.ubuntu.com
[18:23] <heno_> brainstorm will always remain PHP afaik, and is really not related to QA much anymore
[18:24] <bdmurray> LaserJock: Correct, just not *
[18:24] <heno_> LaserJock: right
[18:24] <LaserJock> k
[18:24] <heno_> next
[18:24] <heno_> [TOPIC]: Alpha 5 ISO testing
[18:25] <heno_> images are appearing now and we need testing help!
[18:26] <heno_> Who has working spare HW or VMs ATM?
[18:26] <heno_> I have troble with vbox here ATM
[18:27] <ara> I am downloading now alternate i386 and will give it a try tomorrow. I will run also the automated tests
[18:27] <ogasawara> heno_:  I've got a spare system I can run some tests on
[18:27] <heno_> unfortunately davmor2 who normally does a lot of testing is away this weel
[18:27] <sbeattie> vbox hosted on hardy?
[18:28] <heno_> sbeattie: right, and intrepid as well
[18:28] <heno_> I'll try kvm later though
[18:28] <schwuk> heno_: I've been playing with mvo's sandbox kvm scripts, so my vm's are working
[18:29] <LaserJock>  * I just created https://wiki.ubuntu.com/QATeam/Services to list our hosts/services. Please fill in when you get a chance
[18:29] <heno_> schwuk: excellent. do you have admin access to the iso tracker btw? to help coordinate
[18:29] <schwuk> heno_: let me check
[18:29] <persia> LaserJock: On that page: what about services hosted by Canonical but not on qa.ubuntu.com?
[18:29] <heno_> if not I or stgraber can help set that up
[18:30]  * persia is thinking of things like the conflictschecker
[18:30] <LaserJock> persia: I was gonna put that under "hosted elsewhere"
[18:30] <LaserJock> but I guess that's a good point
[18:30] <heno_> Ideally we should move those over
[18:30] <LaserJock> heno_: that's what I was thinking
[18:31] <heno_> persia: are you interested in running the conflict checker?
[18:31] <heno_> now being open sourced btw
[18:31] <persia> heno_: All of them?  There's quite a few, and for some things (like most of the contents of p.u.c/~ubuntu-archive), there are lots of external scripts depending on the location.
[18:31] <schwuk> heno_: How can I tell if I have admin access?
[18:31] <persia> heno_: Yes, but my plate is far to full to accept anything new right now.
[18:32] <heno_> ok
[18:32] <ara> what is conflict checker used for?
[18:32] <heno_> schwuk: you should be able to add builds
[18:32] <heno_> ara: check for file conflicts in packages
[18:33] <persia> ara: It checks for files that conflict between packages that don't declare the conflict properly, for all possible upgrade paths.  That way, it identifies issues users will encounter before the users encounter them.
[18:33] <schwuk> heno_: In that case I suspect not - nothing like that is jumping out.
[18:33] <sbeattie> schwuk: there should be an Adminstration box in the left column.
[18:33] <ara> heno_, persia: thanks
[18:33] <heno_> bdmurray, pedro_, sbeattie do you have hw/VMs and time to test a bit?
[18:33] <sbeattie> heno_: yes, I can help out.
[18:34] <heno_> we won't aim for full coverage for the alpha but absolutely need that for Beta
[18:34] <bdmurray> I've no vm's but could test with my laptop I believe
[18:34] <pedro_> heno_: yep I'll do some, it will be slow since I'm using an old laptop atm but i can help anyways ;-)
[18:34] <heno_> ok, cool we should have decent coverage then
[18:34] <heno_> thanks folks!
[18:35] <heno_> [TOPIC]: http://status.qa.ubuntu.com - just an fyi
[18:35]  * heno_ cheers ogasawara, stgraber and bdmurray for this work!
[18:35] <ogasawara> yup, just wanted to make an announcement here that the pkg stat pages have been pushed to production
[18:35] <heno_> new we just need to start pimping it :)
[18:35] <ogasawara> I can make a more formal announcement to the ml
[18:36] <schwuk> sbeattie: there isn't. heno_ could you fix my permissions please?
[18:36] <heno_> ogasawara: excellent, please do. -devel and -qa
[18:36]  * ara hugs ogasawara, stgraber, bdmurray
[18:36] <heno_> schwuk: sure, after the meeting
[18:37] <schwuk> well done ogasawara, stgraber and bdmurray. They look good.
[18:37] <heno_> I'm sure we'll tweak the look and content as we go
[18:38] <heno_> the LP pages may also adopt some of this info with time
[18:38] <heno_> in which case we may need to revisit our offering
[18:38] <heno_> [TOPIC]: Posting meeting summaries on the QA blog (as the server team does).
[18:39] <heno_> Someone mentioned this would be a good idea
[18:39] <heno_> other teams may adopt this as well
[18:39] <LaserJock> I would personally rather have individuals blog about things
[18:40] <LaserJock> but meeting summaries should be going to mailing lists for sure
[18:40] <heno_> mailings lists> agree
[18:40] <persia> Actually, I'd like to argue against such double posting.
[18:41] <LaserJock> persia: in what way?
[18:41] <persia> I personally find it annoying to have to skip the server team minutes in one forum or another depending on which I encountered first.
[18:41] <heno_> I think meeting summaries and announcements are appropriate for a team blog though
[18:41] <persia> Then they oughtn't be on the mailing list.  I just object to the duplication.
[18:41] <schwuk> heno_: +1
[18:41] <schwuk> persia: people read in different places - just because you monitor both doesn't mean everyone will.
[18:41] <heno_> persia: but may people don't read both
[18:42] <persia> OK.  Let me put this differently then.
[18:42] <persia> If we want to have an effective means of communication with people, we should direct them to some resource from which they can expect to get e.g. meeting summaries and announcements.
[18:42] <persia> Having multiple places where this information is presented is inherently confusing.
[18:43] <LaserJock> persia: agreed
[18:43] <persia> Some people may prefer one or the other, but there's no reason one can't provide RSS of a mailing list, or receive a blog in one's inbox.
[18:43] <LaserJock> I personally dislike that the Server Team puts the log on Planet, it just seems inappropriate
[18:44] <schwuk> LaserJock: Why? It's Ubuntu related.
[18:44] <LaserJock> schwuk: because it's against Planet Ubuntu's stated purpose
[18:44] <LaserJock> and makes it hard to find the stuff that it *is* supposed to have
[18:44] <heno_> perhaps we should have a team planet where such things go
[18:45] <bdmurray> LaserJock: where is this stated purpose?
[18:45] <LaserJock> bdmurray: on http://planet.ubuntu.com
[18:45] <heno_> then you could get all announcements and summaries there
[18:45] <bdmurray> Is the QA Team not a contributor?
[18:45] <schwuk> "Planet Ubuntu is a window into the world, work and lives of Ubuntu developers and contributors."
[18:45] <persia> heno_: Do you mean like the stated purpose of the fridge?
[18:45] <LaserJock> why not put announcements and summaries where they go?
[18:46] <persia> bdmurray: No, the QA Team is a Team.
[18:46] <schwuk> Meeting minutes seem to fit the bill there LaserJock.
[18:46] <pedro_> do you mean persons and not projects ?
[18:46] <LaserJock> and let Planet Ubuntu do what it's supposed to, aggregate individual blogs
[18:46] <pedro_> so you should move all the projects blogs to somewhere else then
[18:46] <bdmurray> So if were to blog about the pkg landing page who should do it? me? ogasawara? stgraber?
[18:46] <heno_> it provides an insight into our work
[18:46] <LaserJock> pedro_: +1 to that
[18:46] <bdmurray> It was a team effort
[18:46] <pedro_> well the GNOME project does that planet.gnome.org/news <-
[18:46] <heno_> persia: I agree that would be more appropriate actually
[18:46] <pedro_> and i think nobody visit that page
[18:47] <heno_> can we teach the fridge to aggregate team blogs as well as atricles?
[18:47] <LaserJock> bdmurray: anybody/everybody
[18:47] <heno_> *articles
[18:47] <LaserJock> bdmurray: you can individually talk about the stuff you want even
[18:47] <persia> heno_: I'll admit that much of the info on the fridge is duplicate to info in my mailing lists, but I'm less bothered by that, if only because I expect the fridge to have such summaries.
[18:48] <LaserJock> heno_: teams can just submit a story for inclusion in Planet
[18:48] <persia> heno_: Very likely.  Contact ubuntu-news-team@ I've yet to have a request not posted relatively quickly.
[18:48] <heno_> ok, I'll check in with other team leads and fridge editors
[18:49] <LaserJock> Fridge editors want more stuff to put up
[18:49] <heno_> it would be easier if we had a simple technical gateway so we could post post directly
[18:49] <heno_> but I dream :)
[18:49] <LaserJock> heno_: agreed
[18:49] <heno_> any other business?
[18:49] <stgraber> sorry, I wasn't around for the last few minutes :(
[18:49] <stgraber> bdmurray: having something on the fridge seems to be a good idea, we can still individually blog about it
[18:50] <heno_> (so we'll not blog meeting summaries for now)
[18:50] <LaserJock> what about a Testing Day?
[18:50] <LaserJock> I totally dropped the ball on sending an email to ubuntu-qa last week, sorry :(
[18:50] <persia> Thanks.  Apologies for the noise about it, it's just one of my pet peeves with the server team, and I actually care more about reading the QA summaries as I'm often not awake for the meeting.
[18:50] <heno_> today would be a great testing day! :)
[18:51] <LaserJock> heno_: are available to support a testing day?
[18:51] <heno_> LaserJock: do you want to try to organise something for next week?
[18:51] <LaserJock> bah, *are people available
[18:52] <ara> I think that someone should be the person in charge of this and try to coordinate efforts
[18:52] <ara> ping people to add the information to their blogs, create wiki pages, etc.
[18:52] <LaserJock> heno_: I'd personally rather somebody closer to ISO testing were organizing it
[18:52] <heno_> agreed - I have limited bandwidth atm though
[18:53] <heno_> stgraber, sbeattie, schwuk, ara: ^ volunteers?
[18:53] <LaserJock> the thing I'm kind of most concerned about is making sure that the release team is coordinated with
[18:54] <ara> I could do it, maybe for the beta?
[18:54] <LaserJock> when I've done ISO testing before a lot of the time the candidates were invalidated before I even got them downloaded
[18:54] <schwuk> LaserJock: when were you thinking for?
[18:54] <ara> or is it too late? alpha 6 maybe?
[18:55] <LaserJock> it's critical for gaining testing contributors that we don't waste their time
[18:55] <heno_> LaserJock: that's an inherent problem with the process unfortunately
[18:55] <LaserJock> heno_: perhaps, but maybe we can do some stuff about that
[18:55] <heno_> ara: I think we should start with alpha 6
[18:55] <heno_> so we can work out the kinks for beta
[18:55] <heno_> ara: thanks for stepping up!
[18:55] <LaserJock> I think we should plan a Testing Day for *every* alpha, Beta, RC until Intrepid is released
[18:56] <heno_> LaserJock: agreed
[18:56] <LaserJock> we don't have to make a huge deal over it, just blog it, announce it to ubuntu-devel-announce perhaps, and make sure people are around to help
[18:56] <LaserJock> it sets us up well for Intrepid+1
[18:56] <heno_> ara: you may want to take advice from dholbach who coordinated the bug jam
[18:57] <ara> heno_: ok, i will
[18:57] <LaserJock> one thing I've been thinking about is not throwing away results when an ISO is invalidated
[18:57] <ara> monday sep 22nd? it is a couple of days after the alpha, so at least we should know that the minimum test passes
[18:57] <schwuk> ara: let me know if you need any help with organising it
[18:58] <LaserJock> can we let people continue testing whatever .iso they manage to get and add that information somehow to the total testing?
[18:58] <heno_> LaserJock: we had that feature on the tracker before but it's gone missing :(
[18:58] <heno_> 22nd sounds good
[18:59] <LaserJock> so we'd have like the last few .isos on the tracker, but somehow distinguishing from the current candidate
[18:59] <LaserJock> and then people can just not file bugs on known .iso issues
[19:00] <persia> Or can be referred to existing bugs if there is good guidance on how to title the bugs.
[19:00] <LaserJock> yeah
[19:00] <heno_> I don't think we should collect results from obsolete ISOs but we should keep historic data in view when considering release-readiness
[19:01] <LaserJock> I just think it's awful disheartening to download the thing, burn it, and do a test, only to have your results tossed out a minute later, if not earlier
[19:01] <heno_> rsync is pretty quick though if the changes are minimal
[19:02] <heno_> ok, we are out of time
[19:02] <LaserJock> I've got several hours before without getting a single valid candidate tested
[19:02] <LaserJock> *gone
[19:02] <sbeattie> LaserJock: note that for the testing day, we're talking about testing alpha 6 as is, not candidates for alpha 6.
[19:02] <heno_> let's continue in #u-q
[19:02] <LaserJock> it's not just download
[19:02] <heno_> #endmeeting