Ubuntu Wiki

Dev Week -- How to get started hacking Launchpad -- kfogel -- Thu, Jan 28

UTC

(03:04:15 PM) jelmer: The next speaker up is Karl Fogel; he works on Launchpad and will introduce you to the wonderful world of Launchpad development!
(03:04:19 PM) kfogel: Indeed!
(03:04:23 PM) kfogel: Thank you, Jelmer.
(03:04:38 PM) kfogel: Hi, everyone.  To start with, I am by no means the world's greatest expert in lp development.
(03:04:45 PM) kfogel: I have only recently started doing it.
(03:04:55 PM) kfogel: That said, I'm happy to help you get started too -- at least it's fresh in my mind!
(03:04:56 PM) kfogel: :-)
(03:05:08 PM) kfogel:   The intended audience is people who use Launchpad -- especially Ubuntu maintainers, but not limited to them -- and see things they might want to improve in Launchpad.  Some users who have already contributed to Launchpad are listed here:
(03:05:17 PM) kfogel: https://dev.launchpad.net/Contributions
(03:05:21 PM) kfogel: You too can be listed on that page :-).
(03:05:30 PM) kfogel: (That list is automatically updated, so if you hack Launchpad, your name will eventually show up there too.)
(03:05:45 PM) kfogel: For this class, you should have a web browser open.  You should have a little familiarity with Bazaar or some other decentralized version control system, plus basic comfort with Python and HTML -- but you do not need to be an expert in any of these things.
(03:05:58 PM) kfogel: I hope that by the end of the session you will feel like you can contribute an improvement to Launchpad whenever you see something that needs fixing.
(03:06:08 PM) kfogel: This isn't going to be a write-a-change-in-realtime kind of session, because there's too much to tell you about and I want to keep things moving.  Instead, the idea is that you would keep the transcript of this session around, and come back to it later for a closer look (especially at the examples) if/when you're ready to start writing your own changes.
(03:06:14 PM) kfogel: Note that much of whath I say here will be stuff you can also find in the dev.launchpad.net wiki.  But the wiki can be a little daunting.  I hope this narrative presentation will give a useful overview of how things go.
(03:06:25 PM) kfogel: GETTING LAUNCHPAD
(03:06:48 PM) kfogel: You may want to start downloading the Launchpad source code if you don't have it already (although it's not really necessary for what we're going to talk about today, it's just good to have so you can explore around).
(03:06:56 PM) kfogel: Now, normally, the "right" way to obtain the Launchpad source code is to follow the instructions at https://dev.launchpad.net/Getting .
(03:07:01 PM) kfogel: But that's actually a somewhat non-trivial process, and involves running a script (I'll explain more later).  So for now, if you just want to get the code fast, so you can explore it during this session, you can download some pre-packaged, compressed Bazaar repositories I've made for you :-).  (But note that when you actually start hacking on Launchpad, you'll want to go back and follow the instructions at https://dev.launchpad.ne
(03:07:01 PM) kfogel: t/Getting .)
(03:07:10 PM) kfogel: So...
(03:07:12 PM) kfogel: To avoid overload an any one server, let's do it this way: if your family name starts with the letters "A-M", you can start the download from here:
(03:07:18 PM) kfogel: http://www.red-bean.com/kfogel/lp-hacking/lp-repos.tar.gz
(03:07:23 PM) kfogel: And if your family name starts with the letters "N-Z", get it from here:
(03:07:28 PM) kfogel: http://people.canonical.com/~kfogel/lp-hacking/lp-repos.tar.gz
(03:07:33 PM) kfogel: (How do you like my load-balancing algorithm?  I'm thinking of patenting it!)
(03:07:41 PM) kfogel: Just kidding.  I sure hope you're all on slow connections, though :-).
(03:07:53 PM) kfogel: Anyway, when you unpack it with
(03:07:56 PM) kfogel:     tar zxvf lp-repos.tar.gz
(03:08:04 PM) kfogel: you'll get a new subdirectory named lp-repos, and inside that will be a .bzr subdirectory and a branch named "devel".  "devel" is where the Launchpad sources are.  To actually build and run Launchpad, you need a bunch of other dependencies -- all of which is described at (you guessed it) https://dev.launchpad.net/Getting .
(03:08:06 PM) kfogel:  
(03:08:18 PM) kfogel: At this point, I will pause, both for questions, and so people who want to can start download
(03:08:21 PM) kfogel: downloading.
(03:08:39 PM) kfogel: Note it is not really necessary to have the sources for this talk.  I will give you pointers to all examples.
(03:08:48 PM) kfogel: Anyone there? :-)
(03:08:53 PM) ***kfogel waits
(03:09:09 PM) kfogel: (or say something in #ubuntu-classroom-chat if you can't speak here)
(03:09:53 PM) kfogel: Heh.  I'll keep going, since the transcript will be useful.  If anyone has a question at any point, please just speak up here or in #ubuntu-classroom-chat.
(03:10:02 PM) kfogel: RUNNING A LOCAL DEVELOPMENT LAUNCHPAD INSTANCE
(03:10:30 PM) kfogel: Basically, do what https://dev.launchpad.net/Running says .  I won't actually try to get your instance running during this class, since there are so many people and we have other stuff to cover.  But do note a couple of things:
(03:10:44 PM) kfogel: At one point in the setup, you run a script as root.  Yes, that's terrible and awful.  But you can look at the script first :-).  The reason to do that is that a launchpad development instance runs on your local box but using the hostname "launchpad.dev" -- to test stuff, you'll point your browser at URLs that start like this:
(03:10:56 PM) kfogel:      https://launchpad.dev/
(03:10:56 PM) kfogel:      https://bugs.launchpad.dev/
(03:10:56 PM) kfogel:      https://code.launchpad.dev/
(03:10:56 PM) kfogel:      https://blueprints.launchpad.dev/
(03:10:56 PM) kfogel:      https://translations.launchpad.dev/
(03:10:57 PM) kfogel:      https://answers.launchpad.dev/
(03:11:01 PM) kfogel:      https://lists.launchpad.dev/
(03:11:07 PM) kfogel: Those would all be running on your local box during development.
(03:11:14 PM) kfogel: So the setup scripts modify your /etc/hosts (and a few other files, I think) to take care of this all automatically.
(03:11:26 PM) kfogel: The other thing is that if you use PostgreSQL, the scripts will blow any data you have in there.  If that's a problem, just run it in a virtual machine.
(03:11:56 PM) kfogel: So, that's how you get it and set it up to run -- just do what that page says.
(03:12:12 PM) kfogel: And understand why it's so complicated!
(03:12:14 PM) kfogel: GETTING HELP
(03:12:44 PM) kfogel: The most important thing to realize is, doing Launchpad development is primarily about communications, not coding.  You do not need mad coding skills, you just need to be comfortable talking to people about your proposed bugfix / enhancement / new feature.
(03:12:48 PM) kfogel: QUESTION: is there a choice as to which DB you can run it on
(03:13:00 PM) kfogel: ANSWER: not right now, AFAIK.  It's tied to PostgreSQL.
(03:13:21 PM) kfogel: Except for really trivial changes (typo fixes and such), most of your time will be spent communicating with other developers, and writing the automated tests for the change.  The actual time writing the change is often the smallest part of it.
(03:13:40 PM) kfogel: So even before I tell you how to get the source code, I'm going to tell you how to get help -- how to find the right people to talk to!
(03:13:49 PM) kfogel: Start here:
(03:13:51 PM) kfogel: dev.launchpad.net/Help
(03:14:04 PM) kfogel: join the mailing list launchpad-dev {AT} lists.launchpad.net
(03:14:22 PM) kfogel: And use these two IRC channels here on freenode;
(03:14:33 PM) kfogel: #launchpad-dev and #launchpad-reviews (for reviewing changes you've submitted)
(03:14:43 PM) kfogel: the first channel is the main one for conversation about Launchpad development.
(03:14:56 PM) kfogel: It is not to be confused with #launchpad, which is for *usage* of Launchpad
(03:15:06 PM) kfogel: Next thing is:
(03:15:10 PM) kfogel: WHAT TO WORK ON?
(03:15:24 PM) kfogel: Most likely, your own usage of Launchpad will drive what you're interested in improving.  But if you want to get an idea of what we think needs fixing, take a look at https://dev.launchpad.net/FixBugs
(03:15:48 PM) kfogel: However, the usual way things work is that you, as a user of Launchpad, run into something that you think should be fixed, and you fix it.
(03:16:11 PM) kfogel: So our list of existing bugs is interesting, but not necessarily the driving force for community developers (do check there to see if your bug has already been filed, though).
(03:16:21 PM) kfogel: HOW TO START TO WORK ON SOMETHING
(03:16:34 PM) kfogel: You'll need three key things: a bug, a branch, and a merge proposal.
(03:16:55 PM) kfogel: If there isn't already a bug filed for what you want to do, then discuss the proposed change first, in #launchpad-dev or on the launchpad-dev {AT} lists.launchpad.net mailing list.  Once there is general agreement, file a bug describing what you plan to do.  (If it's a new feature, there may be a Blueprint involved too.)
(03:17:14 PM) kfogel: So, let's assume that by one route or another, you have a bug filed now.
(03:17:38 PM) kfogel: The next step is to roughly describe your proposed solution to an experienced Launchpad developer  -- this is known as the "pre-implementation call", although
(03:17:45 PM) kfogel: it can also be an IRC chat session instead of a phone call.
(03:18:05 PM) kfogel: Once you've got the basic design agreed on (which is usually a pretty quick process), you make a branch.
(03:18:27 PM) kfogel: A "branch" is equivalent to a patch -- it's how you submit a change to the project.  In order to get the dependencies set up right, I recommend starting one with 'rocketfuel-branch'
(03:18:36 PM) kfogel:   $ cd MY_LP_BRANCHES
(03:18:36 PM) kfogel:   $ ls
(03:18:36 PM) kfogel:   devel db-devel stable db-stable
(03:18:36 PM) kfogel:   $ utilities/rocketfuel-branch 123456-improve-fish
(03:18:36 PM) kfogel:   <<... watch output fly by ...>
(03:18:37 PM) kfogel:   $ ls
(03:18:39 PM) kfogel:   123456-improve-fish devel db-devel stable db-stable
(03:18:41 PM) kfogel:   $
(03:18:47 PM) kfogel:  
(03:19:01 PM) kfogel: By the way, if you're wondering what that "devel", "db-devel", "stable", "db-stable" thing is,
(03:19:04 PM) kfogel: the answer is here:
(03:19:14 PM) kfogel: https://dev.launchpad.net/Trunk
(03:19:33 PM) kfogel: Basically, Launchpad has four trunks, instead of one trunk.  There's stable vs devel, and there's db-changes vs no-db-changes.
(03:19:39 PM) kfogel: It's all explained on that page.
(03:19:53 PM) kfogel: 95% of the time, you'll be working against 'devel', so when I say "trunk", that's what I mean.
(03:20:29 PM) kfogel: In your branch, edit the code and use Bazaar as you normally would.  Make as many commits as you want!
(03:20:40 PM) kfogel: It's okay to make lots of small commits.  This is all just in yuor local branch.
(03:20:52 PM) kfogel: You should also push the branch up to launchpad, so others can see it.
(03:21:01 PM) kfogel: just type 'bzr push' inside the top level of your new branch
(03:21:05 PM) kfogel: bzr will do the right thing
(03:21:17 PM) kfogel: and you'll get a branch visible at some place like this:
(03:21:20 PM) kfogel: https://code.launchpad.net/~your_name_here/launchpad/123456-improve-fish
(03:21:47 PM) kfogel: Note the bug number in the branch name -- this is a convention.  It can go at the beginning or the end, but it's very handy to have it in there.
(03:21:53 PM) kfogel: You should also link the branch *to* the bug.
(03:22:08 PM) kfogel: If you go to the branch page, there will be a link for that.  Do it immediately, so anyone who
(03:22:14 PM) kfogel: visits the bug can see it is being worked on.
(03:22:22 PM) kfogel: You might also want to mark the bug's status as "in progress".
(03:22:48 PM) kfogel: In a moment, I'll digress into talking about what you actually do to develop.
(03:23:00 PM) kfogel: But first, let's assume you're done with your branch, and you want to submit it for review.
(03:23:27 PM) kfogel: First make a merge proposal, in the usual way (there's a link on the branch page for that).
(03:23:28 PM) kfogel: I recommend going into #launchpad-reviews and seeing who the on-call reviewer is
(03:23:55 PM) kfogel: and entering them as a reviewer, if they agree.  Or if you're already talking to another lanuchpad developer about the change, they may be available to review it.
(03:24:18 PM) kfogel: If your change involves a DB schema change, then click "extra options" on the merge proposa page and put "db" for review type.
(03:24:29 PM) kfogel: If it involves visible UI changes, then put "ui" for review type.
(03:24:52 PM) kfogel: (You can submit to different reviewers for different types of reviews -- ask the developers you're working with who the appropriate people might be.)
(03:25:10 PM) kfogel: brief pause for questions...
(03:25:43 PM) kfogel: okay, continuing
(03:25:53 PM) kfogel: Launchpad development is very test-driven.
(03:26:09 PM) kfogel: Most changes of any size (except maybe for trivial typo fixes and stuff) need automated tests to accompany them.
(03:26:21 PM) kfogel: The testing system looks complicated, but don't worry, it's actually complicated.
(03:26:26 PM) kfogel: There.  That was comforting, right?
(03:26:49 PM) kfogel: It's not so bad.  There are three basic kinds of tests: doc tests, story tests, and windmill (javascript/ui) tests.
...