Ubuntu Wiki

Dev Week -- Using The Launchpad Web Service API -- Tue, Sep 2

hi, everybody

My name is Leonard Richardson. I'm on the Launchpad Foundations team and I'm the co-author of the O'Reilly book "RESTful web Services". I'm here to talk about the Launchpad web service API that we released a few weeks ago. I'll do a slow infodump and then take some questions. if you have questions in the meantime just put them in #u-c-c

1. Intro

First, we've got docs talking about the API here: https://help.launchpad.net/API Put simply, we've created an easy way for you to integrate Launchpad into your own applications. If you perform the same tasks on Launchpad over and over again, you can now write a script that automates the tasks for you. You don't have to rely on fragile screen-scraping. If you're a developer of an IDE, testing framework, or some other program that has something to do with software development, you can integrate Launchpad into the program to streamline the development processes. If you run a website for a project hosted on Launchpad, you can get project data from Launchpad and publish it on your website. And so on. You'll eventually be able to use the API to do most of the things you can do through the Launchpad web site.

2. Tools

The simplest way to integrate is to use launchpadlib, a Python library we've written. (see https://help.launchpad.net/API/launchpadlib) This gives you a Pythonic interface to the Launchpad API, so you don't have to know anything about HTTP client programming: >>> launchpad.me.name u'leonardr' >>> launchpad.bugs[1].title u'Microsoft has a majority market share' But it's also easy to learn the API's HTTP-based protocol and write your own client in some other language. (see https://help.launchpad.net/API/Hacking for that)

3. Roadmap

Right now the web service publishes information about people, bugs, and the Launchpad registry (the projects, milestones, etc.). We've divided up the remaining work by team. The bugs team is in charge of publishing bugs through the web service, the code team is in charge of publishing branches, and so on. I got information from the team leads about where this, publication of their data through the web service, fits in their priorities for the next 3-4 months. I think it's important to tell you this to set expectations--you might have a cool idea for a program but you won't be able to write it until we publish the objects you need

• Bugs: A top priority. Bugs are almost completely published right now. The big thing missing is bug search, which should come online within a few days.
• Registry (projects, milestones, etc.): A top priority. We're working on this in the Foundations team now.
• Code: Branches will be published soon.
• Soyuz: There will be some basic objects published to help with the Ubuntu workflow.
• The translations, answers, and blueprints teams don't plan to work on this in the next 3-4 months.

That's my infodump, so i'll take questions now.

<thekorn_> QUESTION: are there plans for 'official' bindings for other languages?

no, we're only writing an official client in Python. oh, actually flacoste reminds me that we'll probably also write a javascript client for use in Ajax applications and because we publish the capabilities of our web service in WADL documents, it'll be easier to write a client in some other language than if you were writing the whole client from scratch

<qense> I couldn't find a good WADL library for PHP...

that's a good point. WADL support itself isn't very good in a lot of languages, because it's a new standard

<charliecb> leonardr: QUESTION: What is WADL ?

good question. first, here's the url: https://wadl.dev.java.net/ it's an XML vocabulary, the name stands for Web Application Description Language. I think of it as a kind of super-enhanced version of HTML forms. When you get an HTML form in your browser, you know that you can send data to a certain URL, formatted a certain way, and something will happen. WADL is a way of talking about which HTTP requests you can send to URLs. it lets you specify what the data should look like, in more detail than HTML forms allow and it also lets you specify what's likely to come back in response to the HTTP request. If you look at the reference documentation https://edge.launchpad.net/+apidoc (which is out of date, but that'll be fixed soon) that document is generated from the WADL document using an XSLT transform. You can see that it talks about what kinds of objects there are on the web service, what data they contain, and what you can do to them. You can use that information to generate docs, or you can use it to go from your mental picture of what you want to do to launchpad, to an HTTP request that will have that effect.

<charliecb> leonardr: QUESTION: How can i delete a branch over the web-api ? and can i delete a project over the web-api ?

i don't think the web service publishes branches yet. give me a minute to look into the project question. well, there's no way to delete projects right now, but it's likely that there will be something of that kind later, even if it's only to help admins delete unapproved spam projects. assuming we did publish branches now, and you could delete them. in launchpadlib, you would probably call a method on the branch object .delete(). this would translate into an HTTP request that invoked a POST operation on the branch object. i hope that answers your question--i can come back to it if now.

<qense> QUESTION: Does an API request user more or less bandwidth?

It depends on what you're doing. If you want the details of your user account, or a bug, it'll be a lot less bandwidth. ***leonardr runs a quick test*** Yeah, about 2-3k for a bug in the API versus 25k on the website. it's about the same size for a bug in the API as for the textual representation of a bug at /+bug/[number]/+text but if you get a huge list of bugs it's going to be 75 * the size of a bug and you probably wouldn't make that kind of request on the website. So depending on usage, you could use a lot more, even though individual objects take less data to represent.

<qense> QUESTION: Is there a page (or trunk) where you can watch the new API changes in detail?

API changes land on Launchpad trunk, and sometimes on launchpadlib trunk. There's no page listing the changes in detail, but every week i write a summary of the week's progress on the launchpad news blog http://news.launchpad.net/category/api flacoste points out that you can also diff the wadl file to see what's new, because the wadl file gives a complete description of the web service

<tacone> QUESTIONS: when will attachment sending be implemented ?

(this is attachments to a bug) I'm pretty sure the bugs team is working on that now. I may change the way it works in the future because I think it's a little hacky. But if I'm right it should be in sometime this week.

<tacone> QUESTION(2): is it possible to delete a comment from your project, using the api ?

In general, if it's possible in Launchpad, it's likely that we will eventually add it to the web service. it's not possible in launchpad, or hidden Nothing will show up in the web service that's not already present in Launchpad first. It sounds like flacoste in #chat is saying that that's a feature the bugs team is working on. They'll get it into Launchpad and then they'll export it through the web service.

<qense> QUESTION: What are some nifty things the API can do and everyone should know about? Right now the coolest thing is probably the scriptable access to bugs. I think that's the most interesting part of Launchpad that's been exposed to far. For me most of the excitement is behind the scenes. We have a design that makes it very easy for the bugs, code, etc. programmers to publish their objects through the web service without having to learn a lot about how the web service works. And our design also makes it easy to write a loosely-coupled client like launchpadlib.

<thekorn_> QUESTION: to understand this right, the functionallity of the API will always be a sub-set of the functionallity provided by the web interface?

I never like to say "always", but it's pretty likely that we won't publish features through the web service that aren't present in the web site. Certainly not generally useful things like hiding spam comments. But we do have _architectural_ features on the web service, like caching, that aren't on the website. And in the future the line between the two might blur: we might have some special way of performing batch operations through the API that wouldn't make sense to expose through the website. (that's just an example, we're not actually planning that)

<qense> QUESTION: What are things you should watch out for when using the API?

If you're using launchpadlib, make sure you have a cache directory that you reuse between runs of your program. This will save you a huge amount of time. But also note that launchpadlib's cache is not thread-safe. If you're running multiple incarnations at once, you'll need to give each one a separate cache directory. Be careful when iterating over lists, because the list might be enormous. It's usually safer to slice a list and iterate over the slice. If you're writing your own client you'll need to follow some good HTTP practices so you don't waste time and bandwidth. I talk about most of those in the hacking document.

barry: leonardr: isn't launchpadlib's cache based on httplib2?

hi, barry yes, launchpadlib is based on httplib2, and httplib2's FileCache is also not thread-safe

barry: thx

<thekorn_> QUESTION: launchpadlib: why is the no default directory set for caching but a temp dir is used

If I understand your question correctly, it's because I didn't want to dictate a directory structure to launchpadlib users

<mok0> QUESTION: how does one develop and test code without doing weird stuff to Launchpads bug database?

I believe you can make your code run against the staging server, which gets wiped every night. barry, is that right?

barry: leonardr: right

flacoste: and it's the default

You can specify which server to run against when you create a Launchpad object

mok0: thx

barry: you also don't need to worry about spamming the world when testing against staging, since it doesn't send emails

The example in https://help.launchpad.net/API/launchpadlib shows how to run against staging

<jpds> QUESTION: Why does running: ./setup.py install on lp:launchpadlib try to download and install a ton of stuff I have installed: http://paste.ubuntu.com/42817/

barry: jpds: that's the way setuptools works by default

<mok0> QUESTION: How does launchpadlib relate to the older launchpadbugs module?

i think that's thekorn's module, and he's working on making it use launchpadlib ok, any other last-minute questions?

mok0: leonardr: thanks, great lesson

mok0, my pleasure

<thekorn_> Question: one quick one: is there any update on updating ubuntu's python-httplib2

thekorn_: thanks leonardr great session, great API and launchpadlib

No, i don't know any more than you do on that. thekorn_, thanks for your support and your help

charliecb: leonardr: thx for the sesion!!