HackLP

Differences between revisions 4 and 5
Revision 4 as of 2010-01-28 22:38:17
Size: 27961
Editor: pool-71-123-25-78
Comment:
Revision 5 as of 2010-01-29 10:12:03
Size: 26993
Editor: i59F765F3
Comment:
Deletions are marked like this. Additions are marked like this.
Line 4: Line 4:
{{{
(03:04:15 PM) jelmer: The next speaker up is Karl Fogel; he works on Launchpad and will introduce you to the wonderful world of Launchpad development!
(03:04:19 PM) kfogel: Indeed!
(03:04:23 PM) kfogel: Thank you, Jelmer.
(03:04:38 PM) kfogel: Hi, everyone. To start with, I am by no means the world's greatest expert in lp development.
(03:04:45 PM) kfogel: I have only recently started doing it.
(03:04:55 PM) kfogel: That said
, I'm happy to help you get started too -- at least it's fresh in my mind!
(03:04:56 PM) kfogel:
:-)
(03:05:08 PM) kfogel: The intended audience is people who use Launchpad -- especially Ubuntu maintainers, but not limited to them -- and see things they might want to improve in Launchpad. Some users who have already contributed to Launchpad are listed here:
(03:05:17 PM) kfogel: https://dev.launchpad.net/Contributions
(03:05:21 PM) kfogel: You too can be listed on that page :-).
(03:05:30 PM) kfogel: (That list is automatically updated, so if you hack Launchpad, your name will eventually show up there too.)
(03:05:45 PM) kfogel: For this class, you should have a web browser open. You should have a little familiarity with Bazaar or some other decentralized version control system, plus basic comfort with Python and HTML -- but you do not need to be an expert in any of these things.
(03:05:58 PM) kfogel: I hope that by the end of the session you will feel like you can contribute an improvement to Launchpad whenever you see something that needs fixing.
(03:06:08 PM) kfogel: This isn't going to be a write-a-change-in-realtime kind of session, because there's too much to tell you about and I want to keep things moving. Instead, the idea is that you would keep the transcript of this session around, and come back to it later for a closer look (especially at the examples) if/when you're ready to start writing your own changes.
(03:06:14 PM) kfogel: Note that much of whath I say here will be stuff you can also find in the dev.launchpad.net wiki. But the wiki can be a little daunting. I hope this narrative presentation will give a useful overview of how things go.
(03:06:25 PM) kfogel:
GETTING LAUNCHPAD
(03:06:48 PM) kfogel: You may want to start downloading the Launchpad source code if you don't have it already (although it's not really necessary for what we're going to talk about today, it's just good to have so you can explore around).
(03:06:56 PM) kfogel:
Now, normally, the "right" way to obtain the Launchpad source code is to follow the instructions at https://dev.launchpad.net/Getting .
(03:07:01 PM) kfogel: But that's actually a somewhat non-trivial process, and involves running a script (I'll explain more later). So for now, if you just want to get the code fast, so you can explore it during this session, you can download some pre-packaged, compressed Bazaar repositories I've made for you :-). (But note that when you actually start hacking on Launchpad, you'll want to go back and follow the instructions at https://dev.launchpad.ne
(03:07:01 PM) kfogel: t/Getting .)
(03:07:10 PM) kfogel: So...
(03:07:12 PM) kfogel: To avoid overload an any one server, let's do it this way: if your family name starts with the letters "A-M", you can start the download from here:
(03:07:18 PM) kfogel: http://www.red-bean.com/kfogel/lp-hacking/lp-repos.tar.gz
(03:07:23 PM) kfogel: And if your family name starts with the letters "N-Z", get it from here:
(03:07:28 PM) kfogel: http://people.canonical.com/~kfogel/lp-hacking/lp-repos.tar.gz
(03:07:33 PM) kfogel: (How do you like my load-balancing algorithm? I'm thinking of patenting it!)
(03:07:41 PM) kfogel: Just kidding. I sure hope you're all on slow connections, though :-).
(03:07:53 PM) kfogel: Anyway, when you unpack it with
(03:07:56 PM) kfogel: tar zxvf lp-repos.tar.gz
(03:08:04 PM) kfogel: you'll get a new subdirectory named lp-repos, and inside that will be a .bzr subdirectory and a branch named "devel". "devel" is where the Launchpad sources are. To actually build and run Launchpad, you need a bunch of other dependencies -- all of which is described at (you guessed it) https://dev.launchpad.net/Getting .
(03:08:06 PM) kfogel:
(03:08:18 PM) kfogel: At this point, I will pause, both for questions, and so people who want to can start download
(03:08:21 PM) kfogel: downloading.
(03:08:39 PM) kfogel: Note it is not really necessary to have the sources for this talk. I will give you pointers to all examples.
(03:08:48 PM) kfogel:
Anyone there? :-)
(03:08:53 PM) ***kfogel waits
(03:09:09 PM) kfogel: (or say something in #ubuntu-classroom-chat if you can't speak here)
(03:09:53 PM) kfogel: Heh. I'll keep going, since the transcript will be useful. If anyone has a question at any point, please just speak up here or in #ubuntu-classroom-chat.
(03:10:02 PM) kfogel: RUNNING A LOCAL DEVELOPMENT LAUNCHPAD INSTANCE
(03:10:30 PM) kfogel: Basically, do what https://dev.launchpad.net/Running says . I won't actually try to get your instance running during this class, since there are so many people and we have other stuff to cover. But do note a couple of things:
(03:10:44 PM) kfogel: At one point in the setup, you run a script as root. Yes, that's terrible and awful. But you can look at the script first :-). The reason to do that is that a launchpad development instance runs on your local box but using the hostname "launchpad.dev" -- to test stuff, you'll point your browser at URLs that start like this:
(03:10:56 PM) kfogel: https://launchpad.dev/
(03:10:56 PM) kfogel: https://bugs.launchpad.dev/
(03:10:56 PM) kfogel: https://code.launchpad.dev/
(03:10:56 PM) kfogel: https://blueprints.launchpad.dev/
(03:10:56 PM) kfogel: https://translations.launchpad.dev/
(03:10:57 PM) kfogel: https://answers.launchpad.dev/
(03:11:01 PM) kfogel: https://lists.launchpad.dev/
(03:11:07 PM) kfogel: Those would all be running on your local box during development.
(03:11:14 PM) kfogel: So the setup scripts modify your /etc/hosts (and a few other files, I think) to take care of this all automatically.
(03:11:26 PM) kfogel: The other thing is that if you use PostgreSQL, the scripts will blow any data you have in there. If that's a problem, just run it in a virtual machine.
(03:11:56 PM) kfogel: So, that's how you get it and set it up to run -- just do what that page says.
(03:12:12 PM) kfogel: And understand why it's so complicated!
(03:12:14 PM) kfogel: GETTING HELP
(03:12:44 PM) kfogel: The most important thing to realize is, doing Launchpad development is primarily about communications, not coding. You do not need mad coding skills, you just need to be comfortable talking to people about your proposed bugfix / enhancement / new feature.
(03:12:48 PM) kfogel: QUESTION: is there a choice as to which DB you can run it on
(03:13:00 PM) kfogel: ANSWER: not right now, AFAIK. It's tied to PostgreSQL.
(03:13:21 PM) kfogel: Except for really trivial changes (typo fixes and such), most of your time will be spent communicating with other developers, and writing the automated tests for the change. The actual time writing the change is often the smallest part of it.
(03:13:40 PM) kfogel: So even before I tell you how to get the source code, I'm going to tell you how to get help -- how to find the right people to talk to!
(03:13:49 PM) kfogel: Start here:
(03:13:51 PM) kfogel: dev.launchpad.net/Help
(03:14:04 PM) kfogel: join the mailing list launchpad-dev {AT} lists.launchpad.net
(03:14:22 PM) kfogel: And use these two IRC channels here on freenode;
(03:14:33 PM) kfogel: #launchpad-dev and #launchpad-reviews (for reviewing changes you've submitted)
(03:14:43 PM) kfogel: the first channel is the main one for conversation about Launchpad development.
(03:14:56 PM) kfogel: It is not to be confused with #launchpad, which is for *usage* of Launchpad
(03:15:06 PM) kfogel: Next thing is:
(03:15:10 PM) kfogel: WHAT TO WORK ON?
(03:15:24 PM) kfogel: Most likely, your own usage of Launchpad will drive what you're interested in improving. But if you want to get an idea of what we think needs fixing, take a look at https://dev.launchpad.net/FixBugs
(03:15:48 PM) kfogel: However, the usual way things work is that you, as a user of Launchpad, run into something that you think should be fixed, and you fix it.
(03:16:11 PM) kfogel: So our list of existing bugs is interesting, but not necessarily the driving force for community developers (do check there to see if your bug has already been filed, though).
(03:16:21 PM) kfogel: HOW TO START TO WORK ON SOMETHING
(03:16:34 PM) kfogel: You'll need three key things: a bug, a branch, and a merge proposal.
(03:16:55 PM) kfogel: If there isn't already a bug filed for what you want to do, then discuss the proposed change first, in #launchpad-dev or on the launchpad-dev {AT} lists.launchpad.net mailing list. Once there is general agreement, file a bug describing what you plan to do. (If it's a new feature, there may be a Blueprint involved too.)
(03:17:14 PM) kfogel: So, let's assume that by one route or another, you have a bug filed now.
(03:17:38 PM) kfogel: The next step is to roughly describe your proposed solution to an experienced Launchpad developer -- this is known as the "pre-implementation call", although
(03:17:45 PM) kfogel: it can also be an IRC chat session instead of a phone call.
(03:18:05 PM) kfogel: Once you've got the basic design agreed on (which is usually a pretty quick process), you make a branch.
(03:18:27 PM) kfogel: A "branch" is equivalent to a patch -- it's how you submit a change to the project. In order to get the dependencies set up right, I recommend starting one with 'rocketfuel-branch'
(03:18:36 PM) kfogel: $ cd MY_LP_BRANCHES
(03:18:36 PM) kfogel: $ ls
(03:18:36 PM) kfogel: devel db-devel stable db-stable
(03:18:36 PM) kfogel: $ utilities/rocketfuel-branch 123456-improve-fish
(03:18:36 PM) kfogel: <<... watch output fly by ...>
(03:18:37 PM) kfogel: $ ls
(03:18:39 PM) kfogel: 123456-improve-fish devel db-devel stable db-stable
(03:18:41 PM) kfogel: $
(03:18:47 PM) kfogel:
(03:19:01 PM) kfogel: By the way, if you're wondering what that "devel", "db-devel", "stable", "db-stable" thing is,
(03:19:04 PM) kfogel: the answer is here:
(03:19:14 PM) kfogel: https://dev.launchpad.net/Trunk
(03:19:33 PM) kfogel: Basically, Launchpad has four trunks, instead of one trunk. There's stable vs devel, and there's db-changes vs no-db-changes.
(03:19:39 PM) kfogel: It's all explained on that page.
(03:19:53 PM) kfogel: 95% of the time, you'll be working against 'devel', so when I say "trunk", that's what I mean.
(03:20:29 PM) kfogel: In your branch, edit the code and use Bazaar as you normally would. Make as many commits as you want!
(03:20:40 PM) kfogel: It's okay to make lots of small commits. This is all just in yuor local branch.
(03:20:52 PM) kfogel: You should also push the branch up to launchpad, so others can see it.
(03:21:01 PM) kfogel: just type 'bzr push' inside the top level of your new branch
(03:21:05 PM) kfogel: bzr will do the right thing
(03:21:17 PM) kfogel: and you'll get a branch visible at some place like this:
(03:21:20 PM) kfogel:
https://code.launchpad.net/~your_name_here/launchpad/123456-improve-fish
(03:21:47 PM) kfogel: Note the bug number in the branch name -- this is a convention. It can go at the beginning or the end, but it's very handy to have it in there.
(03:21:53 PM) kfogel: You should also link the branch *to* the bug.
(03:22:08 PM) kfogel: If you go to the branch page, there will be a link for that. Do it immediately, so anyone who
(03:22:14 PM) kfogel: visits the bug can see it is being worked on.
(03:22:22 PM) kfogel: You might also want to mark the bug's status as "in progress".
(03:22:48 PM) kfogel: In a moment, I'll digress into talking about what you actually do to develop.
(03:23:00 PM) kfogel: But first, let's assume you're done with your branch, and you want to submit it for review.
(03:23:27 PM) kfogel: First make a merge proposal, in the usual way (there's a link on the branch page for that).
(03:23:28 PM) kfogel: I recommend going into #launchpad-reviews and seeing who the on-call reviewer is
(03:23:55 PM) kfogel: and entering them as a reviewer, if they agree. Or if you're already talking to another lanuchpad developer about the change, they may be available to review it.
(03
:24:18 PM) kfogel: If your change involves a DB schema change, then click "extra options" on the merge proposa page and put "db" for review type.
(03:24:29 PM) kfogel: If it involves visible UI changes, then put "ui" for review type.
(03:24:52 PM) kfogel: (You can submit to different reviewers for different types of reviews -- ask the developers you're working with who the appropriate people might be.)
(03:25:10 PM) kfogel: brief pause for questions...
(03:25:43 PM) kfogel: okay, continuing
(03:25:53 PM) kfogel: Launchpad development is very test-driven.
(03
:26:09 PM) kfogel: Most changes of any size (except maybe for trivial typo fixes and stuff) need automated tests to accompany them.
(03:26:21 PM) kfogel: The testing system looks complicated, but don't worry, it's actually complicated.
(03:26:26 PM) kfogel:
There. That was comforting, right?
(03:26:49 PM) kfogel: It's not so bad. There are three basic kinds of tests: doc tests, story tests, and windmill (javascript/ui) tests.
(03:27:08 PM) kfogel: Running "bin/test -v -t bug.txt" runs the doc test lib/lp/bugs/doc/bug.txt
(03:27:49 PM) kfogel: bin/test -v -v -t test_bugtask runs
(03:27:59 PM) kfogel: lib/lp/bugs/browser/tests/test_bugtask.py
(03:28:09 PM) kfogel: (for example again, and this is all from inside the top of "devel", by the way)
(03:28:50 PM) kfogel: and bin/test -v -v -t patches-view.txt
(03:28:56 PM) kfogel: runs tests lib/lp/bugs/stories/patches-view/patches-view.txt .
(03:29:38 PM) kfogel: I won't go into depth about how those tests work, since the best way to learn them is just to look at the examples there already. Basically, they are a way to test both the Python-level interfaces you may have added, and a way to test browser behavior (the story tests actually simulate a browser)
(03:30:12 PM) kfogel: for more on these, see lib/canonical/launchpad/pagetests/README.txt
(03:30:18 PM) kfogel: and lib/canonical/launchpad/pagetests/REFERENCE.txt
(03:30:56 PM) kfogel: By the way, since you run tests at the command line, the MOST USEFUL TRICK IN THE WORLD is this:
(03:31:02 PM) kfogel: in the code, you can do the standard python debugger thing:
(03:31:04 PM) kfogel: import pdb
(03:31:04 PM) kfogel: pdb.set_trace()
(03:31:10 PM) kfogel: (that's in a .py file somewhere)
(03:31:27 PM) kfogel: and then when you run your tests, when you get to that spot, you'll be right at an interactive debugger.
(03:31:34 PM) kfogel: It is an incredible timesaver. Highly recommended.
(03:32:38 PM) kfogel:
another trick is, in the story tests, you'll see how when you show browser page contents, there is a certain output expected -- it's just the raw text (with HTML tags stripped out) from the page, with whitespace all folded
(03:33:07 PM) kfogel: So if you want to see the output you're actually getting, just put a bunch of blank lines where the output would be expected, and you'll get a nice diff showing what the browser put out instead
(03:33:28 PM) kfogel: Okay, now really on to some examples, which I hope will be good references.
(03:33:31 PM) kfogel: EXAMPLES
(03:34:02 PM) kfogel: I'm going to use two examples from code changes I happened to make recently.
(03:34:20 PM) kfogel: I think these are actually fairly clean examples of two types of changes you might find yourself having to make.
(03:34:30 PM) kfogel: The first is "What does making a change to the DB schema look like?"
(03:34:48 PM) kfogel: Here's a branch that makes a very small change to the schema (and it landed recently, I believe):
(03:34:50 PM) kfogel: https://code.edge.launchpad.net/~kfogel/launchpad/509194-506514-bug-heat-db-fixes
(03:34:58 PM) kfogel: Note the two bug numbers both in the branch name.
(03:35:13 PM) kfogel: (of course, if there had been many bugs, I would have not done that, or would have had an overarching master bug)
(03:35:28 PM) kfogel: I'll pause so people can take a look at that code.
(03:36:19 PM) kfogel: Note how the SQL file we changed was database/schema/patch-2207-27-0.sql. When the change first started, the file we added and edited was:
(03:36:33 PM) kfogel: database/schema/patch-2207-99-0.sql
(03:37:01 PM) kfogel: For complicated, historical reasons, DB changes come in distinct SQL files, and the "99" convention tells the DB merge person "fit this one in wherever you have the next slot".
(03:37:17 PM) kfogel: So by the end, when we merged, we were assigned "27". But when we were working, it was "99".
(03:37:29 PM) kfogel: ("we" is Abel Deuring and myself, btw -- he's adeuring in IRC)
(03:37:51 PM) kfogel: That change went into the db-devel branch, of course, and so our merge proposal was targeted to that.
(03:38:27 PM) kfogel: Regarding merge proposals, there are two good wiki links about them (thanks to ajmitch for these):
(03:38:29 PM) kfogel: https://dev.launchpad.net/PatchSubmission
(03:38:42 PM) kfogel: https://dev.launchpad.net/CoverLetters
(03:39:01 PM) kfogel: A "merge proposal cover letter" is the sort of summary description of the change, that helps the reviewer know what they're looking at and how to test it.
(03:39:15 PM) kfogel: It is important to give the reviewer the context they need to properly review the change.
(03:39:37 PM) kfogel: There will be some more about that in the next example,
(03:39:43 PM) kfogel: which is about making a UI change.
(03:40:05 PM) kfogel: This next example comes from https://bugs.edge.launchpad.net/malone/+bug/506018
(03:40:36 PM) kfogel: which is about making a new "+patches" view available (you put "+patches" on the end of certain Launchpad URLs, and you get a listing of patches available for, say, that source package, or project, or whatever).
(03:40:46 PM) kfogel: The code branch is here:
(03:40:48 PM) kfogel:
https://code.edge.launchpad.net/~kfogel/launchpad/506018-patch-report
(03:40:59 PM) kfogel: And the merge proposal is here:
(03:41:01 PM) kfogel: https://code.edge.launchpad.net/~kfogel/launchpad/506018-patch-report/+merge/18181
(03:41:29 PM) kfogel: That's a terrific example of what you can expect from launchpad code review, by the way (above).
(03:41:42 PM) kfogel: Note how the merge proposal describes the change, makes it clear to the reviewer how to test it.
(03:41:55 PM) kfogel: And the bug itself contains a screenshot (http://launchpadlibrarian.net/38192596/screenshot-2010-01-21_rich_popup.png) so the reviewer knows what to look for.
(03:42:28 PM) kfogel: <PAUSE FOR QUESTIONS>
(03:42:48 PM) kfogel: Okay, so there are two more things I want to talk about.
(03:42:56 PM) kfogel:
The first is, how does a change actually land?
(03:43:19 PM) kfogel: The second is just some standard Launchpad development "gotchas" that tend to bite newbies. Maybe I can save you some time by telling you about them now!
(03:43:22 PM) kfogel: HOW TO LAND A CHANGE
(03:44:01 PM) kfogel: Well, right now, once your branch has passed review (and note that review is something *all* Launchpad developers get, including full-time experts)
(03:44:17 PM) kfogel: an existing Launchpad developer has to land it. What they'll do is use the Bazaar PQM plugin:
(03:44:21 PM) kfogel: https://edge.launchpad.net/bzr-pqm
(03:44:27 PM) kfogel: and run a command like this:
(03:44:56 PM) kfogel: $ bzr pqm-submit -m "[r=REVIEWER_NICKNAME][ui=none] Description of change here..."
(03:45:04 PM) kfogel: inside a branch with your changes.
(03:45:41 PM) kfogel: Your commits will be landed, even though they were gated through by someone else (and by the PQM -- "Patch Queue Manager" system -- which does some build sanity checks before putting the change on trunk.
(03:45:42 PM) kfogel: )
(03:46:02 PM) kfogel: After that, your name will show up at https://dev.launchpad.net/Contributions , within a day or so :-).
(03:46:35 PM) kfogel: To see what other people's submit messages and commit messages look like, I highly recommend running 'bzr log -n0 > log-n0.out' in one of the four trunk branches, say, "devel".
(03:46:43 PM) kfogel: It gives you a good sense for how development goes.
(03:47:01 PM) kfogel: Okay, I'll pause for a moment before launching into the final section of this class...
(03:47:09 PM) kfogel: any questions?
(03:47:25 PM) kfogel: Okay, it's time for...
(03:47:28 PM) kfogel: STANDARD LAUNCHPAD DEVELOPMENT GOTCHAS
(03:47:42 PM) kfogel: Watch out for 'make schema' -- it will blow away any test data you generated while using your instance.
(03:47:56 PM) kfogel: So how do you save previously-generated test data? Easy: 'make newsampledata'
(03:48:02 PM) kfogel: like this:
(03:48:05 PM) kfogel: $ pwd
(03:48:05 PM) kfogel: ...../lp-branches/devel/database/sampledata
(03:48:05 PM) kfogel: $ ls -l current* newsampledata*
(03:48:05 PM) kfogel: -rw-r--r-- 1 kfogel kfogel 1635602 2009-12-07 14:11 current-dev.sql
(03:48:05 PM) kfogel: -rw-r--r-- 1 kfogel kfogel 1583524 2010-01-18 09:01 current.sql
(03:48:06 PM) kfogel: -rw-r--r-- 1 kfogel kfogel 1712790 2010-01-18 09:54 newsampledata-dev.sql
(03:48:08 PM) kfogel: -rw-r--r-- 1 kfogel kfogel 1583524 2010-01-18 09:54 newsampledata.sql
(03:48:18 PM) kfogel:
(03:48:24 PM) kfogel: that file, newsampledata-dev.sql, contains the sample data.
(03:48:42 PM) kfogel: This is very useful for your reviewers, too! Save it, attach it where they can use it, so they don't have to generate this data themselves.
(03:48:50 PM) kfogel:
(03:49:00 PM) kfogel: Who are the test users to use in your "launchpad.dev" instance?
(03:49:18 PM) kfogel: log in as "test@canonical.com" for a standard test user (I think the password is "test")
(03:49:32 PM) kfogel: and log in as "foo.bar@canonical.com" for an admin-privs user (same pw I think)
(03:49:33 PM) kfogel:
(03:49:50 PM) kfogel: Hmm, I'll start numbering the tips now, so you know where the boundaries are.
(03:50:12 PM) kfogel: Tip 3: if things aren't working, you should check the "launchpad.log" file in the top level of your branch
(03:50:29 PM) kfogel:
that is, when you do a "make run" and then visit the launchpad.dev instance in your browser,
(03:50:52 PM) kfogel: your code changes may or may not have the effect you expected. If they don't have the effect you expected, the answers are often to be found in launchpad.log.
(03:51:05 PM) kfogel: Also, /var/log/postgresql/postgresql-8.3-main.log has all your DB queries
(03:51:15 PM) kfogel:
(03:51:19 PM) kfogel: Tip 4: Visiting foo.launchpad.dev for the first time may bring up a cert failure thing. Just accept the cert.
(03:51:20 PM) kfogel:
(03:51:32 PM) kfogel: Tip 5: You don't need to restart on a template change (a .pt file). You DO need to restart on a .py file change.
(03:51:39 PM) kfogel: These templates are Zope templates, by the way.
(03:51:51 PM) kfogel: If you're new to Zope (I am!) then http://www.owlfish.com/software/simpleTAL/tal-guide.html is your friend.
(03:52:07 PM) kfogel: And we use the Yahoo User Interface (YUI) library for javascript.
(03:52:11 PM) kfogel: If you're new to that, see
(03:52:12 PM) kfogel: http://developer.yahoo.com/yui/3 and http://developer.yahoo.com/yui/3/api/module_event.html
(03:52:20 PM) kfogel:
(03:52:43 PM) kfogel: Tip 5: terminology. You may be used to seeing "project" to mean a software program.
(03:53:07 PM) kfogel: But inside the code, "project" means "project group", and "product" means what you normally think of as a project. E.g, firefox is a "product" object in the code.
(03:53:25 PM) kfogel: Watch out for confusion. Yes, we're sorry. No, it's too late to change it now - that would be a massive string-replace :-).
(03:53:31 PM) kfogel:
(03:53:52 PM) kfogel: Tip 6: use staging.launchpad.net to test behaviors. the database gets reset every night or so, so you can't hurt anything.
(03:53:58 PM) kfogel:
(03:54:25 PM) kfogel: Tip 7: most of your page UI will be defined by .pt (template) files, for example see lib/lp/bugs/templates/bugtarget-patches.pt
(03:54:38 PM) kfogel: (the new file added by that 506018 branch I referred to earlier).
(03:55:01 PM) kfogel: In a template file, "context/foo" accesses the model, whereas "view/foo" accesses the view. If you already know Zope, you probably know that. For the rest of us, it's a pretty basic thing.
(03:55:19 PM) kfogel: Also, in the code, such as in a View class (like "BugsPatchesView(LaunchpadView):"), self.context also accesses the model.
(03:55:30 PM) kfogel:
(03:56:10 PM) kfogel: Tip 8: we observe an "XXX" convention. If you leave an XXX comment in the code, meaning "this isn't quite right, but I'm going to leave it for now so I can just finish this feature", then the deal is, file a bug about the XXX, and go back and mark the XXX with your name, the date, and the bug number.
(03:56:57 PM) kfogel: For example: "# XXX Jill Random 2010-01-28 bug=123456 We really should use a database query here instead of looping over the objects in Python."
(03:57:00 PM) kfogel:
(03:57:08 PM) kfogel: QUESTION: what is, or what was the deal with open sourcing Launchpad, exactly? To the average user, and many experienced ones, Launchpad is not software, it's just a web page.
(03:57:12 PM) kfogel:
(03:57:21 PM) kfogel: Interesting question.
(03:57:25 PM) kfogel: Couple of reasons:
(03:58:04 PM) kfogel: One, many people become more willing to use Launchpad, and invest time & data into it, if they feel that they are not locked in -- that is, no matter what happens, the source code is open so it can't be lost.
(03:58:42 PM) kfogel: We heard from people that they didn't want to host their free software on a proprietary system. We did want them to use it, so we solved it by making the system not proprietary :-).
(03:59:19 PM) kfogel: The second reason is that, although it is a web service, it's one that has a very technically skilled user base, and there's no reason they shouldn't be able to fix problems when they see them.
(03:59:49 PM) kfogel: So we're doing something a bit unusual: forming a development community around a piece of software that runs as a web service, but forming the community around that actual service -- around Launchpad.net itself.
(04:00:10 PM) kfogel: The code is open, but we try to shepherd all changes onto the running service as quickly as we can.
(04:00:27 PM) kfogel: We have a monthly release cycle, and changes become visible on edge.launchpad.net even sooner for beta testers.
(04:00:40 PM) kfogel: QUESTION: why are there only 14 people outside Canonical with registered contributions?
(04:00:56 PM) kfogel: ANSWER: because one of them is William Grant, who seems to actually about 10 people himself? :-)
(04:01:19 PM) kfogel: No, seriously, I think the answer is that Launchpad is complex, and it takes people a while to get up to speed.
(04:01:42 PM) kfogel: It's not clear to me whether 14 is too high or too low. I hope it goes up, of course; but we didn't know what to expect when we open sourced.
(04:01:53 PM) kfogel: Remember also that we only open sourced six or seven months ago!
(04:01:59 PM) kfogel: I think that's it.
(04:02:20 PM) kfogel: I hope this helped those who might consider hacking on Launchpad. It's been a pleasure; find me in #launchpad-dev if ever have questions of course.
{{{#!IRC
[20:03] <kfogel> AFAIK, I'm teaching the next class. Should I wait for some official signal to start? :-)
[20:03] <jelmer> Thanks Karl :-)
[20:03] <jelmer> I don't think there is a chair at the moment
[20:04] <kfogel> jelmer: did dholbach or someone lead you in?
(I don't see any record of it...)
[20:04] <jelmer>
The next speaker up is Karl Fogel; he works on Launchpad and will introduce you to the wonderful world of Launchpad development!
[20:04] <kfogel> Indeed!
[20:04] <kfogel> Thank you, Jelmer.
[20:04] <kfogel> Hi, everyone. To start with, I am by no means the world's greatest expert in lp development.
[20:04] <kfogel> I have only recently started doing it.
[20:04] <kfogel> That said, I'm happy to help you get started too
-- at least it's fresh in my mind!
[20:04] <kfogel> :-)
[20:05] <kfogel> The intended audience is people who use Launchpad
-- especially Ubuntu maintainers, but not limited to them -- and see things they might want to improve in Launchpad. Some users who have already contributed to Launchpad are listed here:
[20:05] <kfogel> https://dev.launchpad.net/Contributions
[20:05] <kfogel> You too can be listed on that page :-).
[20:05] <kfogel> (That list is automatically updated, so if you hack Launchpad, your name will eventually show up there too.)
[20:05] <kfogel> For this class, you should have a web browser open. You should have a little familiarity with Bazaar or some other decentralized version control system, plus basic comfort with Python and HTML -- but you do not need to be an expert in any of these things.
[20:05] <kfogel> I hope that by the end of the session you will feel like you can contribute an improvement to Launchpad whenever you see something that needs fixing.
[20:06] <kfogel> This isn't going to be a write-a-change-in-realtime kind of session, because there's too much to tell you about and I want to keep things moving. Instead, the idea is that you would keep the transcript of this session around, and come back to it later for a closer look
(especially at the examples) if/when you're ready to start writing your own changes.
[20:06] <kfogel> Note that much of whath
I say here will be stuff you can also find in the dev.launchpad.net wiki. But the wiki can be a little daunting. I hope this narrative presentation will give a useful overview of how things go.
[20:06] <kfogel> GETTING LAUNCHPAD
[20:06] <kfogel> You may want to start downloading the Launchpad source code if you don't have it already (although it's not really necessary for what we're going to talk about today, it's just good to have so you can explore around).
[20
:06] <kfogel> Now, normally, the "right" way to obtain the Launchpad source code is to follow the instructions at https://dev.launchpad.net/Getting .
[20:07] <kfogel> But that's actually a somewhat non-trivial process, and involves running a script (I'll explain more later). So for now, if you just want to get the code fast, so you can explore it during this session, you can download some pre-packaged, compressed Bazaar repositories I've made for you :-). (But note that when you actually start hacking on Launchpad, you'll want to go back and follow the instructions at https://dev.launchpad.ne
[20:07] <kfogel> t/Getting .)
[20:07] <kfogel> So...
[20:07] <kfogel> To avoid overload an any one server, let's do it this way: if your family name starts with the letters "A-M", you can start the download from here:
[20:07] <kfogel> http://www.red-bean.com/kfogel/lp-hacking/lp-repos.tar.gz
[20:07] <kfogel> And if your family name starts with the letters "N-Z", get it from here:
[20:07] <kfogel> http://people.canonical.com/~kfogel/lp-hacking/lp-repos.tar.gz
[20:07] <kfogel> (How do you like my load-balancing algorithm? I'm thinking of patenting it!)
[20:07] <kfogel> Just kidding. I sure hope you're all on slow connections, though :-).
[20:07] <kfogel> Anyway, when you unpack it with
[20:07] <kfogel> tar zxvf lp-repos.tar.gz
[20:08] <kfogel> you'll get a new subdirectory named lp-repos, and inside that will be a .bzr subdirectory and a branch named "devel". "devel" is where the Launchpad sources are. To actually build and run Launchpad, you need a bunch of other dependencies -- all of which is described at (you guessed it) https://dev.launchpad.net/Getting .
[20:08] <kfogel>
[20:08] <kfogel> At this point, I will pause, both for questions, and so people who want to can start download
[20:0
8] <kfogel> downloading.
[20:08] <kfogel> Note it is not really necessary to have the sources for this talk.
I will give you pointers to all examples.
[20:08] <kfogel> Anyone there? :-)
[20:08] * kfogel waits
[20:09] <kfogel> (or say something in #ubuntu-classroom-chat if you can't speak here)
[20:09] <kfogel> Heh. I'll keep going, since the transcript will be useful. If anyone has a question at any point, please just speak up here or in #ubuntu-classroom-chat.
[20:10] <kfogel> RUNNING A LOCAL DEVELOPMENT LAUNCHPAD INSTANCE
[20:10] <kfogel> Basically, do what https://dev.launchpad.net/Running says . I won't actually try to get your instance running during this class, since there are so many people and we have other stuff to cover. But do note a couple of things:
[20:10] <kfogel> At one point in the setup, you run a script as root. Yes, that's terrible and awful. But you can look at the script first :-). The reason to do that is that a launchpad development instance runs on your local box but using the hostname "launchpad.dev" -- to test stuff, you'll point your browser at URLs that start like this:
[20:10] <kfogel> https://launchpad.dev/
[20:10] <kfogel> https://bugs.launchpad.dev/
[20:10] <kfogel> https://code.launchpad.dev/
[20:10] <kfogel> https://blueprints.launchpad.dev/
[20:10] <kfogel> https://translations.launchpad.dev/
[20:10] <kfogel> https://answers.launchpad.dev/
[20:11] <kfogel> https://lists.launchpad.dev/
[20:11] <kfogel> Those would all be running on your local box during development.
[20:11] <kfogel> So the setup scripts modify your /etc/hosts (and a few other files, I think) to take care of this all automatically.
[20:11] <kfogel> The other thing is that if you use PostgreSQL, the scripts will blow any data you have in there. If that's a problem, just run it in a virtual machine.
[20:11] <kfogel> So, that's how you get it and set it up to run -- just do what that page says.
[20:12] <kfogel> And understand why it's so complicated!
[20:12] <kfogel> GETTING HELP
[20:12] <kfogel> The most important thing to realize is, doing Launchpad development is primarily about communications, not coding. You do not need mad coding skills, you just need to be comfortable talking to people about your proposed bugfix / enhancement / new feature.
[20:12] <kfogel> QUESTION: is there a choice as to which DB you can run it on
[20:13] <kfogel> ANSWER: not right now, AFAIK. It's tied to PostgreSQL.
[20:13] <kfogel> Except for really trivial changes (typo fixes and such), most of your time will be spent communicating with other developers, and writing the automated tests for the change. The actual time writing the change is often the smallest part of it.
[20:13] <kfogel> So even before I tell you how to get the source code, I'm going to tell you how to get help -- how to find the right people to talk to!
[20:13] <kfogel> Start here:
[20:13] <kfogel> dev.launchpad.net/Help
[20:14] <kfogel> join the mailing list launchpad-dev {AT} lists.launchpad.net
[20:14] <kfogel> And use these two IRC channels here on freenode;
[20:14] <kfogel> #launchpad-dev and #launchpad-reviews (for reviewing changes you've submitted)
[20:14] <kfogel> the first channel is the main one for conversation about Launchpad development.
[20:14] <kfogel> It is not to be confused with #launchpad, which is for *usage* of Launchpad
[20:15] <kfogel> Next thing is:
[20:15] <kfogel> WHAT TO WORK ON?
[20:15] <kfogel> Most likely, your own usage of Launchpad will drive what you're interested in improving. But if you want to get an idea of what we think needs fixing, take a look at https://dev.launchpad.net/FixBugs
[20:15] <kfogel> However, the usual way things work is that you, as a user of Launchpad, run into something that you think should be fixed, and you fix it.
[20:16] <kfogel> So our list of existing bugs is interesting, but not necessarily the driving force for community developers (do check there to see if your bug has already been filed, though).
[20:16] <kfogel> HOW TO START TO WORK ON SOMETHING
[20:16] <kfogel> You'll need three key things: a bug, a branch, and a merge proposal.
[20:16] <kfogel> If there isn't already a bug filed for what you want to do, then discuss the proposed change first, in #launchpad-dev or on the launchpad-dev {AT} lists.launchpad.net mailing list. Once there is general agreement, file a bug describing what you plan to do. (If it's a new feature, there may be a Blueprint involved too.)
[20:17] <kfogel> So, let's assume that by one route or another, you have a bug filed now.
[20:17] <kfogel> The next step is to roughly describe your proposed solution to an experienced Launchpad developer -- this is known as the "pre-implementation call", although
[20:17] <kfogel> it can also be an IRC chat session instead of a phone call.
[20:18] <kfogel> Once you've got the basic design agreed on (which is usually a pretty quick process), you make a branch.
[20:18] <kfogel> A "branch" is equivalent to a patch -- it's how you submit a change to the project. In order to get the dependencies set up right, I recommend starting one with 'rocketfuel-branch'
[20:18] <kfogel> $ cd MY_LP_BRANCHES
[20:18] <kfogel> $ ls
[20:18] <kfogel> devel db-devel stable db-stable
[20:18] <kfogel> $ utilities/rocketfuel-branch 123456-improve-fish
[20:18] <kfogel> <<... watch output fly by ...>
[20:18] <kfogel> $ ls
[20:18] <kfogel> 123456-improve-fish devel db-devel stable db-stable
[20:18] <kfogel> $
[20:18] <kfogel>
[20:19] <kfogel> By the way, if you're wondering what that "devel", "db-devel", "stable", "db-stable" thing is,
[20:19] <kfogel> the answer is here:
[20:19] <kfogel> https://dev.launchpad.net/Trunk
[20:19] <kfogel> Basically, Launchpad has four trunks, instead of one trunk. There's stable vs devel, and there's db-changes vs no-db-changes.
[20:19] <kfogel> It's all explained on that page.
[20:19] <kfogel> 95% of the time, you'll be working against 'devel', so when I say "trunk", that's what I mean.
[20:20] <kfogel> In your branch, edit the code and use Bazaar as you normally would. Make as many commits as you want!
[20:20] <kfogel> It's okay to make lots of small commits. This is all just in yuor local branch.
[20:20] <kfogel> You should also push the branch up to launchpad, so others can see it.
[20:21] <kfogel> just type 'bzr push' inside the top level of your new branch
[20:2
1] <kfogel> bzr will do the right thing
[20:21] <kfogel> and you
'll get a branch visible at some place like this:
[20:21] <kfogel> https://code.launchpad.net/~your_name_here/launchpad/123456-improve-fish
[20:21] <kfogel> Note the bug number in the branch name -- this is a convention. It can go at the beginning or the end, but it's very handy to have it in there.
[20:21] <kfogel> You should also link the branch *to* the bug.
[20:22] <kfogel> If you go to the branch page, there will be a link for that. Do it immediately, so anyone who
[20:22] <kfogel> visits the bug can see it is being worked on.
[20:22] <kfogel> You might also want to mark the bug's status as "in progress".
[20:22] <kfogel> In a moment, I'll digress into talking about what you actually do to develop.
[20:23] <kfogel> But first, let's assume you're done with your branch, and you want to submit it for review.
[20:23] <kfogel> First make a merge proposal, in the usual way
(there's a link on the branch page for that).
[20:2
3] <kfogel> I recommend going into #launchpad-reviews and seeing who the on-call reviewer is
[20
:23] <kfogel> and entering them as a reviewer, if they agree. Or if you're already talking to another lanuchpad developer about the change, they may be available to review it.
[20:24] <kfogel> If your change involves a DB schema change, then click "extra options" on the merge proposa page and put "db" for review type.
[20:24] <kfogel> If it involves visible UI changes, then put "ui" for review type.
[20:24] <kfogel> (You can submit to different reviewers for different types of reviews -- ask the developers you're working with who the appropriate people might be.)
[20:25] <kfogel> brief pause for questions...
[20:25] <kfogel> okay, continuing
[20:25] <kfogel> Launchpad development is very test-driven.
[20:26] <kfogel> Most changes of any size
(except maybe for trivial typo fixes and stuff) need automated tests to accompany them.
[20:26] <kfogel> The testing system looks complicated, but don't worry, it's actually complicated.
[20:26] <kfogel> There. That was comforting, right?
[20:26] <kfogel> It's not so bad. There are three basic kinds of tests: doc tests, story tests, and windmill (javascript/ui) tests.
[20:27] <kfogel> Running "bin/test -v -t bug.txt" runs the doc test lib/lp/bugs/doc/bug.txt
[20:27] <kfogel> bin/test -v -v -t test_bugtask runs
[20:27] <kfogel> lib/lp/bugs/browser/tests/test_bugtask.py
[20:28] <kfogel> (for example again, and this is all from inside the top of "devel", by the way)
[20:28] <kfogel> and bin/test -v -v -t patches-view.txt
[20:28] <kfogel> runs tests lib/lp/bugs/stories/patches-view/patches-view.txt .
[20:29] <kfogel> I won't go into depth about how those tests work, since the best way to learn them is just to look at the examples there already. Basically, they are a way to test both the Python-level interfaces you may have added, and a way to test browser behavior (the story tests actually simulate a browser)
[20:30] <kfogel> for more on these, see lib/canonical/launchpad/pagetests/README.txt
[20:30] <kfogel> and lib/canonical/launchpad/pagetests/REFERENCE.txt
[20:30] <kfogel> By the way, since you run tests at the command line, the MOST USEFUL TRICK IN THE WORLD is this:
[20:31] <kfogel> in the code, you can do the standard python debugger thing:
[20:31] <kfogel> import pdb
[20:31] <kfogel> pdb.set_trace()
[20:31] <kfogel> (that's in a .py file somewhere)
[20:31] <kfogel> and then when you run your tests, when you get to that spot, you'll be right at an interactive debugger.
[20:31] <kfogel> It is an incredible timesaver. Highly recommended.
[20
:32] <kfogel> another trick is, in the story tests, you'll see how when you show browser page contents, there is a certain output expected -- it's just the raw text (with HTML tags stripped out) from the page, with whitespace all folded
[20:33] <kfogel> So if you want to see the output you're actually getting, just put a bunch of blank lines where the output would be expected, and you'll get a nice diff showing what the browser put out instead
[20:33] <kfogel> Okay, now really on to some examples, which I hope will be good references.
[20:33] <kfogel> EXAMPLES
[20:34] <kfogel> I'm going to use two examples from code changes I happened to make recently.
[20:34] <kfogel> I think these are actually fairly clean examples of two types of changes you might find yourself having to make.
[20:34] <kfogel> The first is "What does making a change to the DB schema look like?"
[20:34] <kfogel> Here's a branch that makes a very small change to the schema (and it landed recently, I believe):
[20:34] <kfogel> https://code.edge.launchpad.net/~kfogel/launchpad/509194-506514-bug-heat-db-fixes
[20:34] <kfogel> Note the two bug numbers both in the branch name.
[20:35] <kfogel> (of course, if there had been many bugs, I would have not done that, or would have had an overarching master bug)
[20:35] <kfogel> I'll pause so people can take a look at that code.
[20:36] <kfogel> Note how the SQL file we changed was database/schema/patch-2207-27-0.sql. When the change first started, the file we added and edited was:
[20:36] <kfogel> database/schema/patch-2207-99-0.sql
[20:37] <kfogel> For complicated, historical reasons, DB changes come in distinct SQL files, and the "99" convention tells the DB merge person "fit this one in wherever you have the next slot".
[20:37] <kfogel> So by the end, when we merged, we were assigned "27". But when we were working, it was "99".
[20:37] <kfogel> ("we" is Abel Deuring and myself, btw -- he's adeuring in IRC)
[20:37] <kfogel> That change went into the db-devel branch, of course, and so our merge proposal was targeted to that.
[20:38] <kfogel> Regarding merge proposals, there are two good wiki links about them (thanks to ajmitch for these):
[20:38] <kfogel> https://dev.launchpad.net/PatchSubmission
[20:38] <kfogel> https://dev.launchpad.net/CoverLetters
[20:39] <kfogel> A "merge proposal cover letter" is the sort of summary description of the change, that helps the reviewer know what they're looking at and how to test it.
[20:39] <kfogel> It is important to give the reviewer the context they need to properly review the change.
[20:39] <kfogel> There will be some more about that in the next example,
[20:39] <kfogel> which is about making a UI change.
[20:40] <kfogel> This next example comes from https://bugs.edge.launchpad.net/malone/+bug/506018
[20:40] <kfogel> which is about making a new "+patches" view available (you put "+patches" on the end of certain Launchpad URLs, and you get a listing of patches available for, say, that source package, or project, or whatever).
[20:40] <kfogel> The code branch is here:
[20
:40] <kfogel> https://code.edge.launchpad.net/~kfogel/launchpad/506018-patch-report
[20:40] <kfogel> And the merge proposal is here:
[20:41] <kfogel> https://code.edge.launchpad.net/~kfogel/launchpad/506018-patch-report/+merge/18181
[20:41] <kfogel> That's a terrific example of what you can expect from launchpad code review, by the way (above).
[20:41] <kfogel> Note how the merge proposal describes the change, makes it clear to the reviewer how to test it.
[20:41] <kfogel> And the bug itself contains a screenshot (http://launchpadlibrarian.net/38192596/screenshot-2010-01-21_rich_popup.png) so the reviewer knows what to look for.
[20:42] <kfogel> <PAUSE FOR QUESTIONS>
[20:42] <kfogel> Okay, so there are two more things I want to talk about.
[20
:42] <kfogel> The first is, how does a change actually land?
[20:43] <kfogel> The second is just some standard Launchpad development "gotchas" that tend to bite newbies. Maybe I can save you some time by telling you about them now!
[20:43] <kfogel> HOW TO LAND A CHANGE
[20:44] <kfogel> Well, right now, once your branch has passed review (and note that review is something *all* Launchpad developers get, including full-time experts)
[20:44] <kfogel> an existing Launchpad developer has to land it. What they'll do is use the Bazaar PQM plugin:
[20:44] <kfogel> https://edge.launchpad.net/bzr-pqm
[20:44] <kfogel> and run a command like this:
[20:44] <kfogel> $ bzr pqm-submit -m "[r=REVIEWER_NICKNAME][ui=none] Description of change here..."
[20:45] <kfogel> inside a branch with your changes.
[20:45] <kfogel> Your commits will be landed, even though they were gated through by someone else (and by the PQM -- "Patch Queue Manager" system -- which does some build sanity checks before putting the change on trunk.
[20:45] <kfogel> )
[20:46] <kfogel> After that, your name will show up at https://dev.launchpad.net/Contributions , within a day or so :-).
[20:46] <kfogel> To see what other people's submit messages and commit messages look like, I highly recommend running 'bzr log -n0 > log-n0.out' in one of the four trunk branches, say, "devel".
[20:46] <kfogel> It gives you a good sense for how development goes.
[20:47] <kfogel> Okay, I'll pause for a moment before launching into the final section of this class...
[20:47] <kfogel> any questions?
[20:47] <kfogel> Okay, it's time for...
[20:47] <kfogel> STANDARD LAUNCHPAD DEVELOPMENT GOTCHAS
[20:47] <kfogel> Watch out for 'make schema' -- it will blow away any test data you generated while using your instance.
[20:47] <kfogel> So how do you save previously-generated test data? Easy: 'make newsampledata'
[20:48] <kfogel> like this:
[20:48] <kfogel> $ pwd
[20:48] <kfogel> ...../lp-branches/devel/database/sampledata
[20:48] <kfogel> $ ls -l current* newsampledata*
[20:48] <kfogel> -rw-r--r-- 1 kfogel kfogel 1635602 2009-12-07 14:11 current-dev.sql
[20:48] <kfogel> -rw-r--r-- 1 kfogel kfogel 1583524 2010-01-18 09:01 current.sql
[20:48] <kfogel> -rw-r--r-- 1 kfogel kfogel 1712790 2010-01-18 09:54 newsampledata-dev.sql
[20:48] <kfogel> -rw-r--r-- 1 kfogel kfogel 1583524 2010-01-18 09:54 newsampledata.sql
[20:48] <kfogel>
[20:48] <kfogel> that file, newsampledata-dev.sql, contains the sample data.
[20:48] <kfogel> This is very useful for your reviewers, too! Save it, attach it where they can use it, so they don't have to generate this data themselves.
[20:48] <kfogel>
[20:49] <kfogel> Who are the test users to use in your "launchpad.dev" instance?
[20:49] <kfogel> log in as "test@canonical.com" for a standard test user (I think the password is "test")
[20:49] <kfogel> and log in as "foo.bar@canonical.com" for an admin-privs user (same pw I think)
[20:49] <kfogel>
[20:49] <kfogel> Hmm, I'll start numbering the tips now, so you know where the boundaries are.
[20:50] <kfogel> Tip 3: if things aren't working, you should check the "launchpad.log" file in the top level of your branch
[20:
50] <kfogel> that is, when you do a "make run" and then visit the launchpad.dev instance in your browser,
[20:50] <kfogel> your code changes may or may not have the effect you expected. If they don't have the effect you expected, the answers are often to be found in launchpad.log.
[20:51] <kfogel> Also, /var/log/postgresql/postgresql-8.3-main.log has all your DB queries
[20:51] <kfogel>
[20:51] <kfogel> Tip 4: Visiting foo.launchpad.dev for the first time may bring up a cert failure thing. Just accept the cert.
[20:51] <kfogel>
[20:51] <kfogel> Tip 5: You don't need to restart on a template change (a .pt file). You DO need to restart on a .py file change.
[20:51] <kfogel> These templates are Zope templates, by the way.
[20:51] <kfogel> If you're new to Zope (I am!) then http://www.owlfish.com/software/simpleTAL/tal-guide.html is your friend.
[20:52] <kfogel> And we use the Yahoo User Interface (YUI) library for javascript.
[20:52] <kfogel> If you're new to that, see
[20:52] <kfogel> http://developer.yahoo.com/yui/3 and http://developer.yahoo.com/yui/3/api/module_event.html
[20:52] <kfogel>
[20:52] <kfogel> Tip 5: terminology. You may be used to seeing "project" to mean a software program.
[20:53] <kfogel> But inside the code, "project" means "project group", and "product" means what you normally think of as a project. E.g, firefox is a "product" object in the code.
[20:53] <kfogel> Watch out for confusion. Yes, we're sorry. No, it's too late to change it now - that would be a massive string-replace :-).
[20:53] <kfogel>
[20:53] <kfogel> Tip 6: use staging.launchpad.net to test behaviors. the database gets reset every night or so, so you can't hurt anything.
[20:53] <kfogel>
[20:54] <kfogel> Tip 7: most of your page UI will be defined by .pt (template) files, for example see lib/lp/bugs/templates/bugtarget-patches.pt
[20:54] <kfogel> (the new file added by that 506018 branch I referred to earlier).
[20:55] <kfogel> In a template file, "context/foo" accesses the model, whereas "view/foo" accesses the view. If you already know Zope, you probably know that. For the rest of us, it's a pretty basic thing.
[20:55] <kfogel> Also, in the code, such as in a View class (like "BugsPatchesView(LaunchpadView):"), self.context also accesses the model.
[20:55] <kfogel>
[20:56] <kfogel> Tip 8: we observe an "XXX" convention. If you leave an XXX comment in the code, meaning "this isn't quite right, but I'm going to leave it for now so I can just finish this feature", then the deal is, file a bug about the XXX, and go back and mark the XXX with your name, the date, and the bug number.
[20:56] <kfogel> For example: "# XXX Jill Random 2010-01-28 bug=123456 We really should use a database query here instead of looping over the objects in Python."
[20:57] <kfogel>
[20:57] <kfogel> QUESTION: what is, or what was the deal with open sourcing Launchpad, exactly? To the average user, and many experienced ones, Launchpad is not software, it's just a web page.
[20:57] <kfogel>
[20:57] <kfogel> Interesting question.
[20:57] <kfogel> Couple of reasons:
[20:58] <kfogel> One, many people become more willing to use Launchpad, and invest time & data into it, if they feel that they are not locked in -- that is, no matter what happens, the source code is open so it can't be lost.
[20:58] <kfogel> We heard from people that they didn't want to host their free software on a proprietary system. We did want them to use it, so we solved it by making the system not proprietary :-).
[20:59] <kfogel> The second reason is that, although it is a web service, it's one that has a very technically skilled user base, and there's no reason they shouldn't be able to fix problems when they see them.
[20:59] <kfogel> So we're doing something a bit unusual: forming a development community around a piece of software that runs as a web service, but forming the community around that actual service -- around Launchpad.net itself.
[21:00] <kfogel> The code is open, but we try to shepherd all changes onto the running service as quickly as we can.
[21:00] <kfogel> We have a monthly release cycle, and changes become visible on edge.launchpad.net even sooner for beta testers.
[21:00] <kfogel> QUESTION: why are there only 14 people outside Canonical with registered contributions?
[21:00] <kfogel> ANSWER: because one of them is William Grant, who seems to actually about 10 people himself? :-)
[21:01] <kfogel> No, seriously, I think the answer is that Launchpad is complex, and it takes people a while to get up to speed.
[21:01] <kfogel> It's not clear to me whether 14 is too high or too low. I hope it goes up, of course; but we didn't know what to expect when we open sourced.
[21:01] <kfogel> Remember also that we only open sourced six or seven months ago!
[21:01] <kfogel> I think that's it.
[21:02] <kfogel> I hope this helped those who might consider hacking on Launchpad. It's been a pleasure; find me in #launchpad-dev if ever have questions of course.

Dev Week -- How to get started hacking Launchpad -- kfogel -- Thu, Jan 28

UTC

   1 [20:03] <kfogel> AFAIK, I'm teaching the next class.  Should I wait for some official signal to start? :-)
   2 [20:03] <jelmer> Thanks Karl :-)
   3 [20:03] <jelmer> I don't think there is a chair at the moment
   4 [20:04] <kfogel> jelmer: did dholbach or someone lead you in?  (I don't see any record of it...)
   5 [20:04] <jelmer> The next speaker up is Karl Fogel; he works on Launchpad and will introduce you to the wonderful world of Launchpad development!
   6 [20:04] <kfogel> Indeed!
   7 [20:04] <kfogel> Thank you, Jelmer.
   8 [20:04] <kfogel> Hi, everyone.  To start with, I am by no means the world's greatest expert in lp development.
   9 [20:04] <kfogel> I have only recently started doing it.
  10 [20:04] <kfogel> That said, I'm happy to help you get started too -- at least it's fresh in my mind!
  11 [20:04] <kfogel> :-)
  12 [20:05] <kfogel>   The intended audience is people who use Launchpad -- especially Ubuntu maintainers, but not limited to them -- and see things they might want to improve in Launchpad.  Some users who have already contributed to Launchpad are listed here:
  13 [20:05] <kfogel> https://dev.launchpad.net/Contributions
  14 [20:05] <kfogel> You too can be listed on that page :-).
  15 [20:05] <kfogel> (That list is automatically updated, so if you hack Launchpad, your name will eventually show up there too.)
  16 [20:05] <kfogel> For this class, you should have a web browser open.  You should have a little familiarity with Bazaar or some other decentralized version control system, plus basic comfort with Python and HTML -- but you do not need to be an expert in any of these things.
  17 [20:05] <kfogel> I hope that by the end of the session you will feel like you can contribute an improvement to Launchpad whenever you see something that needs fixing.
  18 [20:06] <kfogel> This isn't going to be a write-a-change-in-realtime kind of session, because there's too much to tell you about and I want to keep things moving.  Instead, the idea is that you would keep the transcript of this session around, and come back to it later for a closer look (especially at the examples) if/when you're ready to start writing your own changes.
  19 [20:06] <kfogel> Note that much of whath I say here will be stuff you can also find in the dev.launchpad.net wiki.  But the wiki can be a little daunting.  I hope this narrative presentation will give a useful overview of how things go.
  20 [20:06] <kfogel> GETTING LAUNCHPAD
  21 [20:06] <kfogel> You may want to start downloading the Launchpad source code if you don't have it already (although it's not really necessary for what we're going to talk about today, it's just good to have so you can explore around).
  22 [20:06] <kfogel> Now, normally, the "right" way to obtain the Launchpad source code is to follow the instructions at https://dev.launchpad.net/Getting .
  23 [20:07] <kfogel> But that's actually a somewhat non-trivial process, and involves running a script (I'll explain more later).  So for now, if you just want to get the code fast, so you can explore it during this session, you can download some pre-packaged, compressed Bazaar repositories I've made for you :-).  (But note that when you actually start hacking on Launchpad, you'll want to go back and follow the instructions at https://dev.launchpad.ne
  24 [20:07] <kfogel> t/Getting .)
  25 [20:07] <kfogel> So...
  26 [20:07] <kfogel> To avoid overload an any one server, let's do it this way: if your family name starts with the letters "A-M", you can start the download from here:
  27 [20:07] <kfogel> http://www.red-bean.com/kfogel/lp-hacking/lp-repos.tar.gz
  28 [20:07] <kfogel> And if your family name starts with the letters "N-Z", get it from here:
  29 [20:07] <kfogel> http://people.canonical.com/~kfogel/lp-hacking/lp-repos.tar.gz
  30 [20:07] <kfogel> (How do you like my load-balancing algorithm?  I'm thinking of patenting it!)
  31 [20:07] <kfogel> Just kidding.  I sure hope you're all on slow connections, though :-).
  32 [20:07] <kfogel> Anyway, when you unpack it with
  33 [20:07] <kfogel>     tar zxvf lp-repos.tar.gz
  34 [20:08] <kfogel> you'll get a new subdirectory named lp-repos, and inside that will be a .bzr subdirectory and a branch named "devel".  "devel" is where the Launchpad sources are.  To actually build and run Launchpad, you need a bunch of other dependencies -- all of which is described at (you guessed it) https://dev.launchpad.net/Getting .
  35 [20:08] <kfogel>  
  36 [20:08] <kfogel> At this point, I will pause, both for questions, and so people who want to can start download
  37 [20:08] <kfogel> downloading.
  38 [20:08] <kfogel> Note it is not really necessary to have the sources for this talk.  I will give you pointers to all examples.
  39 [20:08] <kfogel> Anyone there? :-)
  40 [20:08]  * kfogel waits
  41 [20:09] <kfogel> (or say something in #ubuntu-classroom-chat if you can't speak here)
  42 [20:09] <kfogel> Heh.  I'll keep going, since the transcript will be useful.  If anyone has a question at any point, please just speak up here or in #ubuntu-classroom-chat.
  43 [20:10] <kfogel> RUNNING A LOCAL DEVELOPMENT LAUNCHPAD INSTANCE
  44 [20:10] <kfogel> Basically, do what https://dev.launchpad.net/Running says .  I won't actually try to get your instance running during this class, since there are so many people and we have other stuff to cover.  But do note a couple of things:
  45 [20:10] <kfogel> At one point in the setup, you run a script as root.  Yes, that's terrible and awful.  But you can look at the script first :-).  The reason to do that is that a launchpad development instance runs on your local box but using the hostname "launchpad.dev" -- to test stuff, you'll point your browser at URLs that start like this:
  46 [20:10] <kfogel>      https://launchpad.dev/
  47 [20:10] <kfogel>      https://bugs.launchpad.dev/
  48 [20:10] <kfogel>      https://code.launchpad.dev/
  49 [20:10] <kfogel>      https://blueprints.launchpad.dev/
  50 [20:10] <kfogel>      https://translations.launchpad.dev/
  51 [20:10] <kfogel>      https://answers.launchpad.dev/
  52 [20:11] <kfogel>      https://lists.launchpad.dev/
  53 [20:11] <kfogel> Those would all be running on your local box during development.
  54 [20:11] <kfogel> So the setup scripts modify your /etc/hosts (and a few other files, I think) to take care of this all automatically.
  55 [20:11] <kfogel> The other thing is that if you use PostgreSQL, the scripts will blow any data you have in there.  If that's a problem, just run it in a virtual machine.
  56 [20:11] <kfogel> So, that's how you get it and set it up to run -- just do what that page says.
  57 [20:12] <kfogel> And understand why it's so complicated!
  58 [20:12] <kfogel> GETTING HELP
  59 [20:12] <kfogel> The most important thing to realize is, doing Launchpad development is primarily about communications, not coding.  You do not need mad coding skills, you just need to be comfortable talking to people about your proposed bugfix / enhancement / new feature.
  60 [20:12] <kfogel> QUESTION: is there a choice as to which DB you can run it on
  61 [20:13] <kfogel> ANSWER: not right now, AFAIK.  It's tied to PostgreSQL.
  62 [20:13] <kfogel> Except for really trivial changes (typo fixes and such), most of your time will be spent communicating with other developers, and writing the automated tests for the change.  The actual time writing the change is often the smallest part of it.
  63 [20:13] <kfogel> So even before I tell you how to get the source code, I'm going to tell you how to get help -- how to find the right people to talk to!
  64 [20:13] <kfogel> Start here:
  65 [20:13] <kfogel> dev.launchpad.net/Help
  66 [20:14] <kfogel> join the mailing list launchpad-dev {AT} lists.launchpad.net
  67 [20:14] <kfogel> And use these two IRC channels here on freenode;
  68 [20:14] <kfogel> #launchpad-dev and #launchpad-reviews (for reviewing changes you've submitted)
  69 [20:14] <kfogel> the first channel is the main one for conversation about Launchpad development.
  70 [20:14] <kfogel> It is not to be confused with #launchpad, which is for *usage* of Launchpad
  71 [20:15] <kfogel> Next thing is:
  72 [20:15] <kfogel> WHAT TO WORK ON?
  73 [20:15] <kfogel> Most likely, your own usage of Launchpad will drive what you're interested in improving.  But if you want to get an idea of what we think needs fixing, take a look at https://dev.launchpad.net/FixBugs
  74 [20:15] <kfogel> However, the usual way things work is that you, as a user of Launchpad, run into something that you think should be fixed, and you fix it.
  75 [20:16] <kfogel> So our list of existing bugs is interesting, but not necessarily the driving force for community developers (do check there to see if your bug has already been filed, though).
  76 [20:16] <kfogel> HOW TO START TO WORK ON SOMETHING
  77 [20:16] <kfogel> You'll need three key things: a bug, a branch, and a merge proposal.
  78 [20:16] <kfogel> If there isn't already a bug filed for what you want to do, then discuss the proposed change first, in #launchpad-dev or on the launchpad-dev {AT} lists.launchpad.net mailing list.  Once there is general agreement, file a bug describing what you plan to do.  (If it's a new feature, there may be a Blueprint involved too.)
  79 [20:17] <kfogel> So, let's assume that by one route or another, you have a bug filed now.
  80 [20:17] <kfogel> The next step is to roughly describe your proposed solution to an experienced Launchpad developer  -- this is known as the "pre-implementation call", although
  81 [20:17] <kfogel> it can also be an IRC chat session instead of a phone call.
  82 [20:18] <kfogel> Once you've got the basic design agreed on (which is usually a pretty quick process), you make a branch.
  83 [20:18] <kfogel> A "branch" is equivalent to a patch -- it's how you submit a change to the project.  In order to get the dependencies set up right, I recommend starting one with 'rocketfuel-branch'
  84 [20:18] <kfogel>   $ cd MY_LP_BRANCHES
  85 [20:18] <kfogel>   $ ls
  86 [20:18] <kfogel>   devel db-devel stable db-stable
  87 [20:18] <kfogel>   $ utilities/rocketfuel-branch 123456-improve-fish
  88 [20:18] <kfogel>   <<... watch output fly by ...>
  89 [20:18] <kfogel>   $ ls
  90 [20:18] <kfogel>   123456-improve-fish devel db-devel stable db-stable
  91 [20:18] <kfogel>   $
  92 [20:18] <kfogel>  
  93 [20:19] <kfogel> By the way, if you're wondering what that "devel", "db-devel", "stable", "db-stable" thing is,
  94 [20:19] <kfogel> the answer is here:
  95 [20:19] <kfogel> https://dev.launchpad.net/Trunk
  96 [20:19] <kfogel> Basically, Launchpad has four trunks, instead of one trunk.  There's stable vs devel, and there's db-changes vs no-db-changes.
  97 [20:19] <kfogel> It's all explained on that page.
  98 [20:19] <kfogel> 95% of the time, you'll be working against 'devel', so when I say "trunk", that's what I mean.
  99 [20:20] <kfogel> In your branch, edit the code and use Bazaar as you normally would.  Make as many commits as you want!
 100 [20:20] <kfogel> It's okay to make lots of small commits.  This is all just in yuor local branch.
 101 [20:20] <kfogel> You should also push the branch up to launchpad, so others can see it.
 102 [20:21] <kfogel> just type 'bzr push' inside the top level of your new branch
 103 [20:21] <kfogel> bzr will do the right thing
 104 [20:21] <kfogel> and you'll get a branch visible at some place like this:
 105 [20:21] <kfogel> https://code.launchpad.net/~your_name_here/launchpad/123456-improve-fish
 106 [20:21] <kfogel> Note the bug number in the branch name -- this is a convention.  It can go at the beginning or the end, but it's very handy to have it in there.
 107 [20:21] <kfogel> You should also link the branch *to* the bug.
 108 [20:22] <kfogel> If you go to the branch page, there will be a link for that.  Do it immediately, so anyone who
 109 [20:22] <kfogel> visits the bug can see it is being worked on.
 110 [20:22] <kfogel> You might also want to mark the bug's status as "in progress".
 111 [20:22] <kfogel> In a moment, I'll digress into talking about what you actually do to develop.
 112 [20:23] <kfogel> But first, let's assume you're done with your branch, and you want to submit it for review.
 113 [20:23] <kfogel> First make a merge proposal, in the usual way (there's a link on the branch page for that).
 114 [20:23] <kfogel> I recommend going into #launchpad-reviews and seeing who the on-call reviewer is
 115 [20:23] <kfogel> and entering them as a reviewer, if they agree.  Or if you're already talking to another lanuchpad developer about the change, they may be available to review it.
 116 [20:24] <kfogel> If your change involves a DB schema change, then click "extra options" on the merge proposa page and put "db" for review type.
 117 [20:24] <kfogel> If it involves visible UI changes, then put "ui" for review type.
 118 [20:24] <kfogel> (You can submit to different reviewers for different types of reviews -- ask the developers you're working with who the appropriate people might be.)
 119 [20:25] <kfogel> brief pause for questions...
 120 [20:25] <kfogel> okay, continuing
 121 [20:25] <kfogel> Launchpad development is very test-driven.
 122 [20:26] <kfogel> Most changes of any size (except maybe for trivial typo fixes and stuff) need automated tests to accompany them.
 123 [20:26] <kfogel> The testing system looks complicated, but don't worry, it's actually complicated.
 124 [20:26] <kfogel> There.  That was comforting, right?
 125 [20:26] <kfogel> It's not so bad.  There are three basic kinds of tests: doc tests, story tests, and windmill (javascript/ui) tests.
 126 [20:27] <kfogel> Running "bin/test -v -t bug.txt"  runs the doc test lib/lp/bugs/doc/bug.txt
 127 [20:27] <kfogel> bin/test -v -v -t test_bugtask runs
 128 [20:27] <kfogel> lib/lp/bugs/browser/tests/test_bugtask.py
 129 [20:28] <kfogel> (for example again, and this is all from inside the top of "devel", by the way)
 130 [20:28] <kfogel> and bin/test -v -v -t patches-view.txt
 131 [20:28] <kfogel> runs tests lib/lp/bugs/stories/patches-view/patches-view.txt .
 132 [20:29] <kfogel> I won't go into depth about how those tests work, since the best way to learn them is just to look at the examples there already.  Basically, they are a way to test both the Python-level interfaces you may have added, and a way to test browser behavior (the story tests actually simulate a browser)
 133 [20:30] <kfogel> for more on these, see lib/canonical/launchpad/pagetests/README.txt
 134 [20:30] <kfogel> and lib/canonical/launchpad/pagetests/REFERENCE.txt
 135 [20:30] <kfogel> By the way, since you run tests at the command line, the MOST USEFUL TRICK IN THE WORLD is this:
 136 [20:31] <kfogel> in the code, you can do the standard python debugger thing:
 137 [20:31] <kfogel>         import pdb
 138 [20:31] <kfogel>         pdb.set_trace()
 139 [20:31] <kfogel> (that's in a .py file somewhere)
 140 [20:31] <kfogel> and then when you run your tests, when you get to that spot, you'll be right at an interactive debugger.
 141 [20:31] <kfogel> It is an incredible timesaver.  Highly recommended.
 142 [20:32] <kfogel> another trick is, in the story tests, you'll see how when you show browser page contents, there is a certain output expected -- it's just the raw text (with HTML tags stripped out) from the page, with whitespace all folded
 143 [20:33] <kfogel> So if you want to see the output you're actually getting, just put a bunch of blank lines where the output would be expected, and you'll get a nice diff showing what the browser put out instead
 144 [20:33] <kfogel> Okay, now really on to some examples, which I hope will be good references.
 145 [20:33] <kfogel> EXAMPLES
 146 [20:34] <kfogel> I'm going to use two examples from code changes I happened to make recently.
 147 [20:34] <kfogel> I think these are actually fairly clean examples of two types of changes you might find yourself having to make.
 148 [20:34] <kfogel> The first is "What does making a change to the DB schema look like?"
 149 [20:34] <kfogel> Here's a branch that makes a very small change to the schema (and it landed recently, I believe):
 150 [20:34] <kfogel> https://code.edge.launchpad.net/~kfogel/launchpad/509194-506514-bug-heat-db-fixes
 151 [20:34] <kfogel> Note the two bug numbers both in the branch name.
 152 [20:35] <kfogel> (of course, if there had been many bugs, I would have not done that, or would have had an overarching master bug)
 153 [20:35] <kfogel> I'll pause so people can take a look at that code.
 154 [20:36] <kfogel> Note how the SQL file we changed was database/schema/patch-2207-27-0.sql.  When the change first started, the file we added and edited was:
 155 [20:36] <kfogel> database/schema/patch-2207-99-0.sql
 156 [20:37] <kfogel> For complicated, historical reasons, DB changes come in distinct SQL files, and the "99" convention tells the DB merge person "fit this one in wherever you have the next slot".
 157 [20:37] <kfogel> So by the end, when we merged, we were assigned "27".  But when we were working, it was "99".
 158 [20:37] <kfogel> ("we" is Abel Deuring and myself, btw -- he's adeuring in IRC)
 159 [20:37] <kfogel> That change went into the db-devel branch, of course, and so our merge proposal was targeted to that.
 160 [20:38] <kfogel> Regarding merge proposals, there are two good wiki links about them (thanks to ajmitch for these):
 161 [20:38] <kfogel> https://dev.launchpad.net/PatchSubmission
 162 [20:38] <kfogel> https://dev.launchpad.net/CoverLetters
 163 [20:39] <kfogel> A "merge proposal cover letter" is the sort of summary description of the change, that helps the reviewer know what they're looking at and how to test it.
 164 [20:39] <kfogel> It is important to give the reviewer the context they need to properly review the change.
 165 [20:39] <kfogel> There will be some more about that in the next example,
 166 [20:39] <kfogel> which is about making a UI change.
 167 [20:40] <kfogel> This next example comes from https://bugs.edge.launchpad.net/malone/+bug/506018
 168 [20:40] <kfogel> which is about making a new "+patches" view available (you put "+patches" on the end of certain Launchpad URLs, and you get a listing of patches available for, say, that source package, or project, or whatever).
 169 [20:40] <kfogel> The code branch is here:
 170 [20:40] <kfogel> https://code.edge.launchpad.net/~kfogel/launchpad/506018-patch-report
 171 [20:40] <kfogel> And the merge proposal is here:
 172 [20:41] <kfogel> https://code.edge.launchpad.net/~kfogel/launchpad/506018-patch-report/+merge/18181
 173 [20:41] <kfogel> That's a terrific example of what you can expect from launchpad code review, by the way (above).
 174 [20:41] <kfogel> Note how the merge proposal describes the change, makes it clear to the reviewer how to test it.
 175 [20:41] <kfogel> And the bug itself contains a screenshot (http://launchpadlibrarian.net/38192596/screenshot-2010-01-21_rich_popup.png) so the reviewer knows what to look for.
 176 [20:42] <kfogel>   <PAUSE FOR QUESTIONS>
 177 [20:42] <kfogel> Okay, so there are two more things I want to talk about.
 178 [20:42] <kfogel> The first is, how does a change actually land?
 179 [20:43] <kfogel> The second is just some standard Launchpad development "gotchas" that tend to bite newbies.  Maybe I can save you some time by telling you about them now!
 180 [20:43] <kfogel> HOW TO LAND A CHANGE
 181 [20:44] <kfogel> Well, right now, once your branch has passed review (and note that review is something *all* Launchpad developers get, including full-time experts)
 182 [20:44] <kfogel> an existing Launchpad developer has to land it.  What they'll do is use the Bazaar PQM plugin:
 183 [20:44] <kfogel> https://edge.launchpad.net/bzr-pqm
 184 [20:44] <kfogel> and run a command like this:
 185 [20:44] <kfogel>    $ bzr pqm-submit -m "[r=REVIEWER_NICKNAME][ui=none] Description of change here..."
 186 [20:45] <kfogel> inside a branch with your changes.
 187 [20:45] <kfogel> Your commits will be landed, even though they were gated through by someone else (and by the PQM -- "Patch Queue Manager" system -- which does some build sanity checks before putting the change on trunk.
 188 [20:45] <kfogel> )
 189 [20:46] <kfogel> After that, your name will show up at https://dev.launchpad.net/Contributions , within a day or so :-).
 190 [20:46] <kfogel> To see what other people's submit messages and commit messages look like, I highly recommend running 'bzr log -n0 > log-n0.out' in one of the four trunk branches, say, "devel".
 191 [20:46] <kfogel> It gives you a good sense for how development goes.
 192 [20:47] <kfogel> Okay, I'll pause for a moment before launching into the final section of this class...
 193 [20:47] <kfogel>   any questions?
 194 [20:47] <kfogel> Okay, it's time for...
 195 [20:47] <kfogel> STANDARD LAUNCHPAD DEVELOPMENT GOTCHAS
 196 [20:47] <kfogel> Watch out for 'make schema' -- it will blow away any test data you generated while using your instance.
 197 [20:47] <kfogel> So how do you save previously-generated test data?  Easy:  'make newsampledata'
 198 [20:48] <kfogel> like this:
 199 [20:48] <kfogel>     $ pwd
 200 [20:48] <kfogel>     ...../lp-branches/devel/database/sampledata
 201 [20:48] <kfogel>     $ ls -l current* newsampledata*
 202 [20:48] <kfogel>     -rw-r--r-- 1 kfogel kfogel 1635602 2009-12-07 14:11 current-dev.sql
 203 [20:48] <kfogel>     -rw-r--r-- 1 kfogel kfogel 1583524 2010-01-18 09:01 current.sql
 204 [20:48] <kfogel>     -rw-r--r-- 1 kfogel kfogel 1712790 2010-01-18 09:54 newsampledata-dev.sql
 205 [20:48] <kfogel>     -rw-r--r-- 1 kfogel kfogel 1583524 2010-01-18 09:54 newsampledata.sql
 206 [20:48] <kfogel>  
 207 [20:48] <kfogel> that file, newsampledata-dev.sql, contains the sample data.
 208 [20:48] <kfogel> This is very useful for your reviewers, too!  Save it, attach it where they can use it, so they don't have to generate this data themselves.
 209 [20:48] <kfogel>  
 210 [20:49] <kfogel> Who are the test users to use in your "launchpad.dev" instance?
 211 [20:49] <kfogel> log in as "test@canonical.com" for a standard test user (I think the password is "test")
 212 [20:49] <kfogel> and log in as "foo.bar@canonical.com" for an admin-privs user (same pw I think)
 213 [20:49] <kfogel>  
 214 [20:49] <kfogel> Hmm, I'll start numbering the tips now, so you know where the boundaries are.
 215 [20:50] <kfogel> Tip 3: if things aren't working, you should check the "launchpad.log" file in the top level of your branch
 216 [20:50] <kfogel> that is, when you do a "make run" and then visit the launchpad.dev instance in your browser,
 217 [20:50] <kfogel> your code changes may or may not have the effect you expected.  If they don't have the effect you expected, the answers are often to be found in launchpad.log.
 218 [20:51] <kfogel> Also, /var/log/postgresql/postgresql-8.3-main.log has all your DB queries
 219 [20:51] <kfogel>  
 220 [20:51] <kfogel> Tip 4: Visiting foo.launchpad.dev for the first time may bring up a cert failure thing.  Just accept the cert.
 221 [20:51] <kfogel>  
 222 [20:51] <kfogel> Tip 5: You don't need to restart on a template change (a .pt file).  You DO need to restart on a .py file change.
 223 [20:51] <kfogel> These templates are Zope templates, by the way.
 224 [20:51] <kfogel> If you're new to Zope (I am!) then http://www.owlfish.com/software/simpleTAL/tal-guide.html is your friend.
 225 [20:52] <kfogel> And we use the Yahoo User Interface (YUI) library for javascript.
 226 [20:52] <kfogel> If you're new to that, see
 227 [20:52] <kfogel> http://developer.yahoo.com/yui/3 and http://developer.yahoo.com/yui/3/api/module_event.html
 228 [20:52] <kfogel>  
 229 [20:52] <kfogel> Tip 5: terminology.  You may be used to seeing "project" to mean a software program.
 230 [20:53] <kfogel> But inside the code, "project" means "project group", and "product" means what you normally think of as a project.  E.g, firefox is a "product" object in the code.
 231 [20:53] <kfogel> Watch out for confusion.  Yes, we're sorry.  No, it's too late to change it now - that would be a massive string-replace :-).
 232 [20:53] <kfogel>  
 233 [20:53] <kfogel> Tip 6: use staging.launchpad.net to test behaviors.  the database gets reset every night or so, so you can't hurt anything.
 234 [20:53] <kfogel>  
 235 [20:54] <kfogel> Tip 7: most of your page UI will be defined by .pt (template) files, for example see lib/lp/bugs/templates/bugtarget-patches.pt
 236 [20:54] <kfogel> (the new file added by that 506018 branch I referred to earlier).
 237 [20:55] <kfogel> In a template file, "context/foo" accesses the model, whereas "view/foo" accesses the view.  If you already know Zope, you probably know that.  For the rest of us, it's a pretty basic thing.
 238 [20:55] <kfogel>     Also, in the code, such as in a View class (like "BugsPatchesView(LaunchpadView):"), self.context also accesses the model.
 239 [20:55] <kfogel>  
 240 [20:56] <kfogel> Tip 8: we observe an "XXX" convention.  If you leave an XXX comment in the code, meaning "this isn't quite right, but I'm going to leave it for now so I can just finish this feature", then the deal is, file a bug about the XXX, and go back and mark the XXX with your name, the date, and the bug number.
 241 [20:56] <kfogel> For example: "# XXX Jill Random 2010-01-28 bug=123456  We really should use a database query here instead of looping over the objects in Python."
 242 [20:57] <kfogel>  
 243 [20:57] <kfogel> QUESTION: what is, or what was the deal with open sourcing Launchpad, exactly? To the average user, and many experienced ones, Launchpad is not software, it's just a web page.
 244 [20:57] <kfogel>  
 245 [20:57] <kfogel> Interesting question.
 246 [20:57] <kfogel> Couple of reasons:
 247 [20:58] <kfogel> One, many people become more willing to use Launchpad, and invest time & data into it, if they feel that they are not locked in -- that is, no matter what happens, the source code is open so it can't be lost.
 248 [20:58] <kfogel> We heard from people that they didn't want to host their free software on a proprietary system.  We did want them to use it, so we solved it by making the system not proprietary :-).
 249 [20:59] <kfogel> The second reason is that, although it is a web service, it's one that has a very technically skilled user base, and there's no reason they shouldn't be able to fix problems when they see them.
 250 [20:59] <kfogel> So we're doing something a bit unusual: forming a development community around a piece of software that runs as a web service, but forming the community around that actual service -- around Launchpad.net itself.
 251 [21:00] <kfogel> The code is open, but we try to shepherd all changes onto the running service as quickly as we can.
 252 [21:00] <kfogel> We have a monthly release cycle, and changes become visible on edge.launchpad.net even sooner for beta testers.
 253 [21:00] <kfogel> QUESTION: why are there only 14 people outside Canonical with registered contributions?
 254 [21:00] <kfogel> ANSWER: because one of them is William Grant, who seems to actually about 10 people himself? :-)
 255 [21:01] <kfogel> No, seriously, I think the answer is that Launchpad is complex, and it takes people a while to get up to speed.
 256 [21:01] <kfogel> It's not clear to me whether 14 is too high or too low.  I hope it goes up, of course; but we didn't know what to expect when we open sourced.
 257 [21:01] <kfogel> Remember also that we only open sourced six or seven months ago!
 258 [21:01] <kfogel> I think that's it.
 259 [21:02] <kfogel> I hope this helped those who might consider hacking on Launchpad.  It's been a pleasure; find me in #launchpad-dev if ever have questions of course.

MeetingLogs/devweek1001/HackLP (last edited 2010-01-29 10:12:03 by i59F765F3)