Ubuntu Wiki

Dev Week -- Getting started with Launchpad development -- gmb -- Wed Sep 2nd, 2009

UTC

(12:01:46 PM) gmb: My name's Graham Binns. I'm a member of the Launchpad Bugs development team.
(12:02:21 PM) gmb: I'm going to talk today about getting started with Launchpad development, in the hope that it might make it easier for you guys to contribute patches to scratch your least favourite itches.
(12:02:32 PM) gmb: Hopefully you'll have all completed the instructions at http://dev.launchpad.net/Getting so that you can follow along with this session. If not, you might struggle a bit, but you can always go back once the session is over and follow it through on your own time.
(12:03:01 PM) gmb: If you've any questions, please shout them out in #ubuntu-classroom-chat and prefix them with QUESTION so that I can see them easier :)
(12:03:25 PM) gmb: Okay, so, first things first, we need to find us a bug to fix. For the purposes of this session I've filed a made-up bug on staging for us to fix https://staging.launchpad.net/bugs/422299. I've gone with this because:
(12:03:44 PM) gmb: 1) It's fairly simple to fix. 2) It's easy to demonstrate our test-driven development process whilst we fix it, which is why I didn't pick a bug in the UI. 3) There were no really trivial bugs available for us to try this out on :).
(12:04:18 PM) gmb: When you're working on fixing a bug in Launchpad, you nearly always want to be doing it in a new branch.
(12:04:45 PM) gmb: We try to keep to one bug per branch, because that means that it's much easier to review the patches when they're done (because they're smaller, natch :))
(12:04:55 PM) mode (+o jcastro ) by dholbach
(12:04:55 PM) gmb: So, let's create a branch in which to fix the bug.
(12:05:09 PM) gmb: If you've set up the Launchpad development environment properly according to http://dev.launchpad.net/Getting, you should be able to run the following command:
(12:05:19 PM) gmb: $ rocketfuel-branch getting-started-with-lp-bug-422299
(12:05:27 PM) gmb: Note that I've appended the bug number to the branch
(12:05:33 PM) gmb: so that I can always refer to it if I need to
(12:05:45 PM) gmb: but I've also given the branch a useful name to help me remember what it's for if I have to leave it for a while.
(12:06:11 PM) gmb: rocketfuel-branch takes a few seconds, so I'll just wait a minute for everyone to catch up.
(12:07:11 PM) gmb: (By the way, if anyone has any problems with rocketfuel-get or any other part of this lesson, please come find me afterwards in #launchpad and I'll try to help you out)
(12:07:33 PM) gmb: s/-get/-branch/ there, sorry.
(12:07:40 PM) gmb: Okay.
(12:07:48 PM) gmb: Now, at this point, once you'd decided how to fix the bug
(12:07:54 PM) gmb: but - importantly - before you start coding
(12:08:03 PM) gmb: you'd ideally have a chat with a member of the Launchpad development team about your intended fix.
(12:08:19 PM) gmb: We normally do this either on IRC or on Skype, depending on your preference.
(12:08:40 PM) gmb: You can usually find a Launchpad developer in #launchpad-dev on Freenode who'll be available for one of these calls.
(12:12:42 PM) gmb: The call gives you a chance to ensure that what you're doing is actually sane.
(12:12:42 PM) gmb: For some bugs there's only one possible fix, complex or otherwise. For others there may be many ways to do it, and it's important to pick the right one.
(12:12:45 PM) gmb: If your solution is particularly complex or you need to demonstrate *why* you want to do things the way you do, it may help to write some tests to reproduce the bug before you have the call.
(12:12:50 PM) gmb: Note that the tests should always fail at this point;
(12:12:52 PM) gmb: you shouldn't make any changes to the actual code until you've had the pre-implementation call or chat with an LP developer.
(12:12:57 PM) gmb: Okay, so that's the info-dumpy bit of this session over for now :)
(12:12:58 PM) gmb left the room (quit: Remote closed the connection).
(12:13:28 PM) gmb [n=gmb@seagoon.grahambinns.com] entered the room.
(12:13:58 PM) jcastro: (gmb is having lag issues, please stand by)
(12:14:24 PM) mode (+o gmb ) by jcastro
(12:14:34 PM) gmb: Sorry about that, all.
(12:14:44 PM) gmb: I have a rather flaky connection today :)
(12:15:12 PM) gmb: As I was saying...
(12:15:46 PM) gmb: Under lib/lp you'll find most of the Launchpad code, split up into its applications.
(12:15:56 PM) gmb: So, `ls lib/lp` in your new getting-started-with-lp-bug-422299 branch should give you something like this:
(12:16:08 PM) gmb: $ ls lib/lp
(12:16:08 PM) gmb: answers           archiveuploader  buildmaster  coop          registry  soyuz
(12:16:11 PM) gmb: app               blueprints       code         __init__.py   scripts   testing
(12:16:14 PM) gmb: archivepublisher  bugs             codehosting  __init__.pyc  services  translations
(12:16:29 PM) gmb: Now, we know that we're working in the bugs application, so lets take a look in there to see where to put our tests:
(12:16:46 PM) gmb: $ ls lib/lp/bugs
(12:16:46 PM) gmb: adapters        emailtemplates      help          model          stories      windmill
(12:16:49 PM) gmb: browser         event               __init__.py   notifications  subscribers  xmlrpc
(12:16:52 PM) gmb: configure.zcml  externalbugtracker  __init__.pyc  pagetests      templates
(12:16:55 PM) gmb: doc             feed                interfaces    scripts        tests
(12:17:38 PM) gmb: There are three types of test in Launchpad: doctests, which live in lib/lp/$app/doc; stories, which live in lib/lp/$app/stories and unittests, which live in lib/lp/$app/tests.
(12:18:06 PM) gmb: In this case we want to add to an existing doctest, so I'll stick with that for now and we can come back to what the others are for later.
(12:18:18 PM) gmb: So, in lib/lp/bugs/doc/ you'll find a file called externalbugtracker-trac.txt.
(12:18:33 PM) gmb: This is the test we want to modify, so feel free to open it in your text editor and take a look at line 110, which is where we're going to add our test.
(12:18:55 PM) gmb: For the sake of making this quicker, I've already created a diff of the change that I'd make here: http://pastebin.ubuntu.com/263869/plain/
(12:19:16 PM) gmb: You can save that to disk somewhere (e.g. /tmp/diff) and then apply it as a patch using `bzr patch /tmp/diff` in the root of your new Launchpad branch.
(12:19:35 PM) gmb: The test we've just added is really simple.
(12:20:09 PM) gmb: It passes 'frobnob' to the convertRemoteStatus() method of a Trac instance (which is just an abstraction that lets us talk to an actual Trac server)
(12:20:21 PM) gmb: and expects to get "Fix Released" back.
(12:20:43 PM) gmb: Of course, it doesn't since we haven't implemented that yet :).
(12:20:51 PM) gmb: Once we've written the test, we run it to make sure it fails.
(12:21:07 PM) gmb: This part is very important: your tests should always fail first and only after they fail do you write the code to make them pass.
(12:21:13 PM) gmb:  That means that you can use the tests to build a good spec of how your module / class / function / whatever should behave.
(12:21:50 PM) gmb: It also means that, like I said before, you can use the failing tests to demonstrate what your fix will actually change to whoever you have a call with.
(12:22:01 PM) gmb: To run this specific test only, we use the `bin/test` command:
(12:22:13 PM) gmb: $ bin/test -vvt externalbugtracker-trac.txt
(12:22:42 PM) gmb: That might take a short while to run (Launchpad's test suite can be frustratingly slow sometimes, but don't let that put you off; the payoff is worth it)
(12:22:57 PM) gmb: The output from which should look something like this: http://pastebin.ubuntu.com/263874/
(12:23:16 PM) gmb: Note the important bit:
(12:23:16 PM) gmb:     File "lib/lp/bugs/tests/../doc/externalbugtracker-trac.txt", line 111, in externalbugtracker-trac.txt
(12:23:20 PM) gmb:     Failed example:
(12:23:22 PM) gmb:         trac.convertRemoteStatus('frobnob').title
(12:23:25 PM) gmb:     Exception raised:
(12:23:27 PM) gmb:         Traceback (most recent call last):
(12:23:30 PM) gmb:           File "/home/graham/canonical/lp-sourcedeps/eggs/zope.testing-3.8.1-py2.4.egg/zope/testing/doctest.py", line 1361, in __run
(12:23:33 PM) gmb:             compileflags, 1) in test.globs
(12:23:35 PM) gmb:           File "<doctest externalbugtracker-trac.txt[line 111, example 35]>", line 1, in ?
(12:23:38 PM) gmb:           File "/home/graham/canonical/lp-branches/lesson/lib/lp/bugs/externalbugtracker/trac.py", line 265, in convertRemoteStatus
(12:23:41 PM) gmb:             raise UnknownRemoteStatusError(remote_status)
(12:23:44 PM) gmb:         UnknownRemoteStatusError: frobnob
(12:23:44 PM) gmb: This tells us that the test failed, which is exactly what we wanted.
(12:23:46 PM) gmb: (Yes, copying and pasting in IRC makes me a bad man.)
(12:23:51 PM) gmb: nvertRemoteStatus() raised an UnknownRemoteStatusError instead of giving us back the status we wanted.
(12:24:03 PM) gmb: Which was, of course, the 'Fix Released' status.
(12:24:10 PM) gmb: At this point, you might want to commit the changes:
(12:24:17 PM) gmb: $ bzr commit -m "Added tests for bug 422299."
(12:24:53 PM) gmb: Again - I can't emphasise this enough - the fact that your test fails is a Good Thing. If it didn't fail, it wouldn't be a good test, since we know that the bug actually exists in the code.
(12:25:25 PM) gmb: Now that we have a test that fails, we want to add some code to make it pass
(12:26:02 PM) gmb: We want to add this to lib/lp/bugs/externalbugtracker/trac.py.
(12:26:22 PM) gmb: Now, as it happens, I knew that before I started, but you can work it out by looking at the top of the doctest file that we just edited.
(12:26:41 PM) gmb: So, open lib/lp/bugs/externalbugtracker/trac.py now and take a look at line 258. We'll add our fix here.
(12:27:15 PM) gmb: The fix is really simple, and we can pretty much copy line 255 and alter it to suit our needs.
(12:27:29 PM) gmb: We want 'frobnob' to map to 'Fix Released', so we add the following line:
(12:27:38 PM) gmb:     ('frobnob', BugTaskStatus.FIXRELEASED),
(12:27:59 PM) gmb: I'll not go into the nitty-gritty of how status lookups work here, because it's unimportant.
(12:28:26 PM) gmb: Suffice it to say that in Trac's case it's a simple pair of values, (remote_status, launchpad_status).
(12:28:50 PM) gmb: Here's a diff of that change: http://pastebin.ubuntu.com/263882/
(12:29:00 PM) gmb: Now that we've added a fix for the bug, we run the test again:
(12:29:07 PM) gmb: $ bin/test -vvt externalbugtracker-trac.txt
(12:29:31 PM) gmb: This time, it should pass without any problems...
(12:29:39 PM) gmb: and it does
(12:29:57 PM) gmb: http://pastebin.ubuntu.com/263885/
(12:30:06 PM) gmb: So, now we commit our changes:
(12:30:07 PM) gmb: $ bzr ci -m "Fixed bug 422299"
(12:30:18 PM) gmb: (Note that this is a lame description of the fix; you should use something more descriptive).
(12:30:44 PM) gmb: So, we now have a branch that fixes a bug. Hurrah and all that.
(12:31:02 PM) gmb: Now we need to get it into the Launchpad tree.
(12:31:23 PM) gmb: Launchpad developers use the Launchpad code review system to review Launchpad branches.
(12:31:32 PM) gmb: You can't land a branch without having it reviewed first
(12:31:43 PM) gmb: This allows us to ensure that code quality stays high
(12:32:08 PM) gmb: And it also acts as a  sanity check to make sure that the developer hasn't done something unnecessarily odd in their fix.
(12:32:36 PM) gmb: So at this point, you need to push your branch to Launchpad using the `bzr push` command:
(12:32:41 PM) gmb: $ bzr push
(12:33:17 PM) gmb: Once the branch has been pushed up to Launchpad it gets its own page in the Launchpad web interface, which you can look at by running:
(12:33:22 PM) gmb: $ bzr lp-open
(12:33:28 PM) gmb: This should open the page in your default browser.
(12:33:45 PM) gmb: Now that you've fixed the bug and pushed the branch to Launchpad you need to request a review for it.
(12:34:16 PM) gmb: To do this, go to the branch page in your browser and click the "Propose for merging into another branch" link.
(12:35:17 PM) gmb: This will take you to a page that looks like this:
(12:35:31 PM) gmb: http://people.ubuntu.com/~gbinns/propose-merge.png
(12:35:57 PM) gmb: In the "Initial comment" box, you need to type a description of the branch.
(12:36:07 PM) gmb: For example, for this branch I'd write something like:
(12:36:33 PM) gmb: "This branch fixes bug 422299 by making Trac.convertRemoteStatus() map the "frobnob" status to Launchpad's Fix Released status."
(12:37:38 PM) gmb: After you've typed in your description, hit the "Propose merge" button and you should see a page that looks something like this: https://code.edge.launchpad.net/~gmb/launchpad/lesson/+merge/11068
(12:38:02 PM) gmb: You then need to head on over to #launchpad-reviews on Freenode and ask if anyone's available to review your branch.
(12:38:19 PM) gmb: If there's no-one available at the time, don't worry.
(12:38:46 PM) gmb: We have a reviewer schedule: http://dev.launchpad.net/ReviewerSchedule, so someone should take a look at it withing 24 hours.
(12:39:21 PM) gmb: The reviewer may ask you to make changes to your branch
(12:39:30 PM) gmb: To bring your fix into line with our coding standards
(12:39:38 PM) gmb: Or maybe to fix a bug that they've spotted in your fix.
(12:39:57 PM) gmb: Once the reviewer has signed off on the changes, they'll submit the branch for merging for you.
(12:40:39 PM) gmb: When a branch gets merged, the entire test suite is run against it
(12:41:11 PM) gmb: If any of the tests fail
(12:41:18 PM) gmb: The reviewer may ask you to help fix them
(12:41:30 PM) gmb: But it's likely that someone else will take care of it if you're not around at the time
(12:42:07 PM) gmb: And that's about all there is to simple Launchpad development :)
(12:42:21 PM) gmb: Are there any questions? Please shout them out in #ubuntu-classroom-chat
(12:47:23 PM) gmb: < ahe> QUESTION: When will launchpad be available as a package in the standard distribution?
(12:48:07 PM) gmb: ahe: At this point, there aren't any plans for that. We released the code for Launchpad because we wanted to let people help to improve the service, but we've no plans as far as I'm aware to distribute it as a package.
(12:52:08 PM) gmb: < Andphe> question: have you planned guys, offer launchpad in another languages than english, example spanish ?
(12:52:37 PM) gmb: Andphe: It's something that we've considered and that we would like to do at some point, at least for certain parts of the interface.
(12:53:12 PM) gmb: The problem is that launchpad is meant to be a global collaboration tool, and if we translate it wholesale into other languages that automatically means that a certain amount of collaboration will be lost
(12:53:45 PM) gmb: For exampel, if a user reads the interface in Spanish and files a bug in Spanish, how am I, an non-Spanish speaker, going to be able to deal with that bug report?
(12:54:13 PM) gmb: However, internationalisation would work quite well for the Answers application, and it's already built with that in mind.
(12:54:25 PM) gmb: < ahe> QUESTION: Do you deploy launchpad manually or are there some helper scripts or stuff like that to ease the deployment in a production environment?
(12:54:36 PM) gmb: It's a combination of the two.
(12:54:51 PM) gmb: edge.launchpad.net is deployed by a script every night, as is staging.launchpad.net.
(12:55:21 PM) gmb: The production servers are updated manually by our sysadmins at least once per cycle (though it's usually more than that since we discover urgent bugs that need to be fixed).
(12:56:43 PM) gmb: < Andphe> question: if answers already support another languages, how can we help to translate it ?
(12:57:44 PM) gmb: Andphe: It's built with translation in mind, but I don't know what work needs doing to make it translatable.
(12:58:09 PM) gmb: Andphe: Your best bet would be to join the Launchpad Developers mailing list (http://launchpad.net/~launchpad-dev) and post a question about it there.
(12:59:24 PM) gmb: I think that's about all we've got time for.
(12:59:49 PM) gmb: If you've any further questions, please feel free to join the Launchpad Dev list (above)
(12:59:54 PM) gmb: And ask there.
(12:59:59 PM) gmb: Everyone's welcome to contribute.
(01:00:06 PM) gmb: Thanks very much for your time.