Dev Week -- Syncing your app data everywhere with U1DB -- aquarius -- Thu, 3rd Feb, 2012
1 [19:44] <aquarius> Hi all, and welcome back. I'm still Stuart Langridge from Ubuntu One :)
2 [19:44] <aquarius> (thanks, akgraner :))
3 [19:44] <aquarius> I'm now going to talk about U1DB.
4 [19:45] <aquarius> In addition to syncing files and music and photos around, it's good to be able to sync data, too
5 [19:45] <aquarius> U1DB is our solution to that.
6 [19:45] <aquarius> It's still being worked on, and we made a preview release in late December.
7 [19:45] <aquarius> Basically, U1DB is for syncing data -- that is, something structured -- to every device you want.
8 === Guest43512 is now known as ejat
9 [19:45] <aquarius> So, preferences or lists or data of any kind where you don't want to have to represent the data as separate files
10 [19:45] <aquarius> U1DB is an API.
11 [19:45] <aquarius> The reason for this is so it can be implemented in any language.
12 [19:45] <aquarius> So someone could build the U1DB API in Python, and then you could use that U1DB library in a Python Quickly app on Ubuntu.
13 [19:46] <aquarius> Someone else could build the U1DB API in JavaScript, and then you could use that U1DB library in a web app.
14 [19:46] <aquarius> At that point, you can build the Ubuntu app and the web app, and they can share data.
15 [19:46] <aquarius> Then you could build an Android app which uses U1DB -- again, this would be a standard Android app, in Java -- and it can also share and sync that data with the other apps in Python and JavaScript and whatever.
16 [19:46] <aquarius> So you've got the ability to build apps everywhere, for every platform, Ubuntu and the web and smartphones and desktops and netbooks, and have them all work with the same data: your data, and your users' data.
17 [19:46] <aquarius> At the moment, what we've built is called the "reference implementation": it's using Python and SQLite, and includes both a Python client (for use in Python apps), and a server.
18 [19:47] <aquarius> (by the way, you can ask questions in #ubuntu-classroom-chat; just type QUESTION: <your question> )
19 [19:47] <aquarius> Let's try a little bit of code.
20 [19:47] <aquarius> "bzr branch lp:u1db" to get the latest trunk code of u1db.
21 [19:47] <aquarius> We have early documentation online at http://people.canonical.com/~aquarius/u1db-docs
22 [19:47] <aquarius> If you take a look at http://people.canonical.com/~aquarius/u1db-docs/quickstart.html#starting-u1db that gives you an example of how to use u1db itself
23 [19:48] <aquarius> You'll see that U1DB stores "documents" -- JSON documents, that is
24 [19:48] <aquarius> A document can contain anything you want; there's no particular structure imposed on you
25 [19:48] <aquarius> So if you choose to store all your app's data in many separate documents, or group those documents together somehow, or require certain keys in them, that's fine
26 [19:48] <aquarius> So, creating a document is done with
27 [19:48] <aquarius> >>> content = json.dumps({"name": "Alan Hansen"}) # create a document
28 [19:48] <aquarius> >>> doc = db.create_doc(content)
29 [19:49] <aquarius> and that's saved that document into the database
30 [19:49] <aquarius> You use put_doc to overwrite an existing document:
31 [19:49] <aquarius> >>> doc.content = json.dumps({"name": "Alan Hansen", "position": "defence"}) # update the document's content
32 [19:49] <aquarius> >>> rev = db.put_doc(doc)
33 [19:49] <aquarius> (and that returns a revision number)
34 [19:49] <aquarius> U1DB is a revisioned database, meaning that it keeps track of the revision number of a document
35 [19:49] <aquarius> So it knows when things have changed
36 [19:50] <aquarius> Syncing two U1DBs together is manually commanded by your app, whenever it wants
37 [19:50] <aquarius> Because U1DB comes with a server, you can test this out for yourself
38 [19:50] <aquarius> U1DB can also be controlled from the command line, which makes testing this stuff easy
39 [19:50] <ClassBot> There are 10 minutes remaining in the current session.
40 [19:50] <aquarius> In one terminal, do:
41 [19:50] <aquarius> $ u1db-client init-db first.u1db # this creates a database
42 [19:50] <aquarius> $ echo '{"name": "Stuart Langridge"}' | u1db-client create first.u1db # create a document in it
43 [19:50] <aquarius> (This will print the ID of the new document, and its revision: something like this)
44 [19:50] <aquarius> id: D-cf8a96bea58b4b5ab2ce1ab9c1bfa053
45 [19:50] <aquarius> rev: f6657904254d474d9a333585928726df:1
46 [19:50] <aquarius> You can retrieve that document back again:
47 [19:50] <aquarius> $ u1db-client get first.u1db D-cf8a96bea58b4b5ab2ce1ab9c1bfa053 # fetch it
48 [19:50] <aquarius> {"key": "value"}
49 [19:51] <aquarius> rev: f6657904254d474d9a333585928726df:1
50 [19:51] <aquarius> Now, let's run the server in this folder:
51 [19:51] <aquarius> $ u1db-serve --verbose
52 [19:51] <aquarius> listening on: 127.0.0.1:43632
53 [19:51] <aquarius> Now, you have a U1DB server running on port 43632
54 [19:51] <aquarius> So, in another terminal:
55 [19:51] <aquarius> $ u1db-client init-db second.u1db # create a second database
56 [19:51] <aquarius> $ u1db-client get second.u1db D-cf8a96bea58b4b5ab2ce1ab9c1bfa053 # try and fetch a doc
57 [19:51] <aquarius> And you'll see that that says: Document not found (id: D-cf8a96bea58b4b5ab2ce1ab9c1bfa053)
58 [19:51] <aquarius> because that document doesn't exist in second.u1db
59 [19:51] <aquarius> Now, let's sync second with first:
60 [19:52] <aquarius> $ u1db-client sync second.u1db http://127.0.0.1:43632/first.u1db
61 [19:52] <aquarius> And now, the document exists in second:
62 [19:52] <aquarius> $ u1db-client get second.u1db D-cf8a96bea58b4b5ab2ce1ab9c1bfa053
63 [19:52] <aquarius> {"key": "value"}
64 [19:52] <aquarius> rev: f6657904254d474d9a333585928726df:1
65 [19:52] <aquarius> So syncing has worked!
66 [19:52] <aquarius> Syncing is over http -- the server is http and provides a nice RESTful API
67 [19:52] <aquarius> We already have implementations of U1DB under way on other platforms and languages
68 [19:53] <aquarius> The U1DB team are building a C + SQLite implementation
69 [19:53] <aquarius> dobey is working on a Vala implementation for Ubuntu (lp:shardbridge)
70 [19:53] <aquarius> and I'm working on a JavaScript implementation so that I can write web apps and mobile web apps which sync data with U1DB
71 [19:53] <aquarius> The documentation at http://people.canonical.com/~aquarius/u1db-docs should tell you all you need to know to get started
72 [19:53] <aquarius> We hang out in #u1db on freenode and on the mailing list at https://launchpad.net/~u1db-discuss
73 [19:53] <aquarius> So we'd be very interested in helping you use u1db in your apps.
74 [19:54] <aquarius> That's a very quick tour around U1DB, what it's going to be like, and how you can get started
75 [19:54] <aquarius> sorry for the rush -- I had to fit this talk into 15 minutes :)
76 [19:55] <aquarius> so, if anyone has any questions about U1DB, now's the time to ask them; I've got five minutes before I hand over to kelemengabor
77 [19:55] <ClassBot> There are 5 minutes remaining in the current session.
78 [19:56] <aquarius> (er, james_w :))
79 [19:57] <aquarius> <jderose> QUESTION: what's the u1db equivalent of couchdb views?
80 [19:57] <aquarius> U1DB indexes. :)
81 [19:57] <aquarius> Create an index with create_index
82 [19:57] <aquarius> and then you can query that index
83 [19:58] <aquarius> http://people.canonical.com/~aquarius/u1db-docs/quickstart.html#starting-u1db has an example
84 [19:58] <aquarius> http://people.canonical.com/~aquarius/u1db-docs/high-level-api.html#document-storage-and-retrieval has more examples :)
85 [19:59] <aquarius> I've got one minute, so thank you all for listening!
86 [20:00] <aquarius> Chase us down on #u1db if you have further questions