== Dev Week -- Syncing your app data everywhere with U1DB -- aquarius -- Thu, 3rd Feb, 2012 == {{{#!irc [19:44] Hi all, and welcome back. I'm still Stuart Langridge from Ubuntu One :) [19:44] (thanks, akgraner :)) [19:44] I'm now going to talk about U1DB. [19:45] In addition to syncing files and music and photos around, it's good to be able to sync data, too [19:45] U1DB is our solution to that. [19:45] It's still being worked on, and we made a preview release in late December. [19:45] Basically, U1DB is for syncing data -- that is, something structured -- to every device you want. === Guest43512 is now known as ejat [19:45] So, preferences or lists or data of any kind where you don't want to have to represent the data as separate files [19:45] U1DB is an API. [19:45] The reason for this is so it can be implemented in any language. [19:45] So someone could build the U1DB API in Python, and then you could use that U1DB library in a Python Quickly app on Ubuntu. [19:46] Someone else could build the U1DB API in JavaScript, and then you could use that U1DB library in a web app. [19:46] At that point, you can build the Ubuntu app and the web app, and they can share data. [19:46] Then you could build an Android app which uses U1DB -- again, this would be a standard Android app, in Java -- and it can also share and sync that data with the other apps in Python and JavaScript and whatever. [19:46] So you've got the ability to build apps everywhere, for every platform, Ubuntu and the web and smartphones and desktops and netbooks, and have them all work with the same data: your data, and your users' data. [19:46] At the moment, what we've built is called the "reference implementation": it's using Python and SQLite, and includes both a Python client (for use in Python apps), and a server. [19:47] (by the way, you can ask questions in #ubuntu-classroom-chat; just type QUESTION: ) [19:47] Let's try a little bit of code. [19:47] "bzr branch lp:u1db" to get the latest trunk code of u1db. [19:47] We have early documentation online at http://people.canonical.com/~aquarius/u1db-docs [19:47] If you take a look at http://people.canonical.com/~aquarius/u1db-docs/quickstart.html#starting-u1db that gives you an example of how to use u1db itself [19:48] You'll see that U1DB stores "documents" -- JSON documents, that is [19:48] A document can contain anything you want; there's no particular structure imposed on you [19:48] So if you choose to store all your app's data in many separate documents, or group those documents together somehow, or require certain keys in them, that's fine [19:48] So, creating a document is done with [19:48] >>> content = json.dumps({"name": "Alan Hansen"}) # create a document [19:48] >>> doc = db.create_doc(content) [19:49] and that's saved that document into the database [19:49] You use put_doc to overwrite an existing document: [19:49] >>> doc.content = json.dumps({"name": "Alan Hansen", "position": "defence"}) # update the document's content [19:49] >>> rev = db.put_doc(doc) [19:49] (and that returns a revision number) [19:49] U1DB is a revisioned database, meaning that it keeps track of the revision number of a document [19:49] So it knows when things have changed [19:50] Syncing two U1DBs together is manually commanded by your app, whenever it wants [19:50] Because U1DB comes with a server, you can test this out for yourself [19:50] U1DB can also be controlled from the command line, which makes testing this stuff easy [19:50] There are 10 minutes remaining in the current session. [19:50] In one terminal, do: [19:50] $ u1db-client init-db first.u1db # this creates a database [19:50] $ echo '{"name": "Stuart Langridge"}' | u1db-client create first.u1db # create a document in it [19:50] (This will print the ID of the new document, and its revision: something like this) [19:50] id: D-cf8a96bea58b4b5ab2ce1ab9c1bfa053 [19:50] rev: f6657904254d474d9a333585928726df:1 [19:50] You can retrieve that document back again: [19:50] $ u1db-client get first.u1db D-cf8a96bea58b4b5ab2ce1ab9c1bfa053 # fetch it [19:50] {"key": "value"} [19:51] rev: f6657904254d474d9a333585928726df:1 [19:51] Now, let's run the server in this folder: [19:51] $ u1db-serve --verbose [19:51] listening on: 127.0.0.1:43632 [19:51] Now, you have a U1DB server running on port 43632 [19:51] So, in another terminal: [19:51] $ u1db-client init-db second.u1db # create a second database [19:51] $ u1db-client get second.u1db D-cf8a96bea58b4b5ab2ce1ab9c1bfa053 # try and fetch a doc [19:51] And you'll see that that says: Document not found (id: D-cf8a96bea58b4b5ab2ce1ab9c1bfa053) [19:51] because that document doesn't exist in second.u1db [19:51] Now, let's sync second with first: [19:52] $ u1db-client sync second.u1db http://127.0.0.1:43632/first.u1db [19:52] And now, the document exists in second: [19:52] $ u1db-client get second.u1db D-cf8a96bea58b4b5ab2ce1ab9c1bfa053 [19:52] {"key": "value"} [19:52] rev: f6657904254d474d9a333585928726df:1 [19:52] So syncing has worked! [19:52] Syncing is over http -- the server is http and provides a nice RESTful API [19:52] We already have implementations of U1DB under way on other platforms and languages [19:53] The U1DB team are building a C + SQLite implementation [19:53] dobey is working on a Vala implementation for Ubuntu (lp:shardbridge) [19:53] and I'm working on a JavaScript implementation so that I can write web apps and mobile web apps which sync data with U1DB [19:53] The documentation at http://people.canonical.com/~aquarius/u1db-docs should tell you all you need to know to get started [19:53] We hang out in #u1db on freenode and on the mailing list at https://launchpad.net/~u1db-discuss [19:53] So we'd be very interested in helping you use u1db in your apps. [19:54] That's a very quick tour around U1DB, what it's going to be like, and how you can get started [19:54] sorry for the rush -- I had to fit this talk into 15 minutes :) [19:55] so, if anyone has any questions about U1DB, now's the time to ask them; I've got five minutes before I hand over to kelemengabor [19:55] There are 5 minutes remaining in the current session. [19:56] (er, james_w :)) [19:57] QUESTION: what's the u1db equivalent of couchdb views? [19:57] U1DB indexes. :) [19:57] Create an index with create_index [19:57] and then you can query that index [19:58] http://people.canonical.com/~aquarius/u1db-docs/quickstart.html#starting-u1db has an example [19:58] http://people.canonical.com/~aquarius/u1db-docs/high-level-api.html#document-storage-and-retrieval has more examples :) [19:59] I've got one minute, so thank you all for listening! [20:00] Chase us down on #u1db if you have further questions }}}