DesktopCouchWishList

Differences between revisions 8 and 9
Revision 8 as of 2011-04-15 22:56:07
Size: 5811
Editor: 173-14-15-225-Colorado
Comment:
Revision 9 as of 2011-04-15 23:37:15
Size: 7173
Editor: 173-14-15-225-Colorado
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
This is where I'm keeping my notes on changes I would like to see in [[https://launchpad.net/desktopcouch|desktopcouch]] based on his experience working on [[https://launchpad.net/dmedia|dmedia]]. Note that this is written out of love of desktopcouch, despite being a list of pain points I would like to see changed. This is where I'm keeping my notes on changes I would like to see in [[https://launchpad.net/desktopcouch|desktopcouch]] based on my experience working on [[https://launchpad.net/dmedia|dmedia]]. Note that this is written out of love of desktopcouch, despite being a list of pain points I would like to see changed.
Line 12: Line 12:

Ideally, the standard way of using desktopcouch should abstract out the desktop part, so that applications can easily run as hosted web apps also, without each application rolling their own abstraction.
Line 61: Line 63:
Although the '''record_type''' convention addresses an important need (standardizing document schema for use across apps), it's rather awkward and verbose. Because the type (or record_type) needs to referenced so often, and in many contexts (view functions, UI code, etc), I've settled on something more concise for dmedia/Novacut.

For example, the desktopcouch spec dictates something like this:

 {{{
{
    "_id": "ZZZATIZG6IA3DJOEANQCFT3FHR4IU2FC",
    "record_type": "http://www.freedesktop.org/wiki/Specifications/dmedia/file"
}
}}}

But dmedia uses something much less verbose:

 {{{
{
    "_id": "ZZZATIZG6IA3DJOEANQCFT3FHR4IU2FC",
    "type": "dmedia/file"
}
}}}

This is a very typical dmedia view function:

 {{{
function(doc) {
    if (doc.type == 'dmedia/file') {
        emit(doc.mtime, null);
    }
}
}}}

Which becomes awkward and far less readable if the desktopcouch convention:

 {{{
function(doc) {
    if (doc.record_type == 'http://www.freedesktop.org/wiki/Specifications/dmedia/file') {
        emit(doc.mtime, null);
    }
}
}}}

Line 69: Line 112:
 * The '''included_names''' list should be stored in a non-replicated '''_local/foo''' document, or local file  * Preferably, the '''included_names''' list should be stored in a non-replicated '''_local/foo''' document, or local file
Line 75: Line 118:

== Make web apps first class citizens ==

This is where I'm keeping my notes on changes I would like to see in desktopcouch based on my experience working on dmedia. Note that this is written out of love of desktopcouch, despite being a list of pain points I would like to see changed.

May Stuart and Jason's bromance continue.

Abstract away the Desktop part

In addition to using desktopcouch, Both Novacut and dmedia must be able to run on headless servers and talk to a system-wide CouchDB. Although as of dmedia 0.6 this will finally be possible, it took a while to get there, plus there is still one fundamental issue: the desktopcouch reconnection hack for working around CouchDB crashing after resuming from suspend.

Ideally, the standard way of using desktopcouch should abstract out the desktop part, so that applications can easily run as hosted web apps also, without each application rolling their own abstraction.

reconnection hack must go

To have the reconnection hack work you must use a desktopcouch.records.database.Database, which makes it basically impossible to abstract away the desktop.

Jason's #1 request for Oneiric is for this to change. Instead, the port must remain the same throughout the desktop session. It's fine if the port is randomly chosen at the start of the session, but it must remain the same once desktopcouch starts.

When it comes down to it, the API needs to be plain HTTP. Requiring a specialized wrapper library like desktopcouch closes off too many cool use cases (like having the same application run both on the web and on desktopcouch more or less transparently).

For what it's worth, desktopcouch.Database isn't close enough to the CouchDB REST API to be usable by dmedia (DC makes too many assumptions, certain functionality isn't available). In fact, even python-couchdb has been a constant headache for dmedia... again, certain functionality isn't exposed, and there is way too much magic/ambiguity behind the scenes.

Not that this sort wrapper isn't probably a perfect fit for some applications. But dmedia and Novacut are pretty demanding, and something close to the metal like microfiber would make Jason's life much easier.

abstractcouch.get_env()

As discussed in lp:722035, there needs to be a single entry point from which an app can get information about the CouchDB environment it is running in. If you were running against desktopcouch, it would return something like this:

  • {
      "oauth": {
        "consumer_key": "oRTyTHKiKu", 
        "consumer_secret": "bdXSzITryM", 
        "token": "lyrygLlsbk", 
        "token_secret": "QbqvZaiBGV"
      }, 
      "port": 45484, 
      "url": "http://localhost:45484/"
    }

Or if you were running against the system-wide CouchDB, it would return something like this:

  • {
      "port": 5984, 
      "url": "http://localhost:5984/"
    }

dmedia.core.get_env() implements the above behavior (but the point is apps shouldn't have to abstract away the desktop part on their own).

When talking to CouchDB from Python, dmedia.abstractcouch.get_server() is used to create an appropriately configured couchdb.Server based on the env it is passed.

When talking to CouchDB from JavaScript (UI running in embedded WebKit), the dmedia CouchView is used to transparently sign OAuth requests. This would be a great piece to standardize and upstream into desktopcouch.

enforcing record_type is too restrictive

Although the record_type convention addresses an important need (standardizing document schema for use across apps), it's rather awkward and verbose. Because the type (or record_type) needs to referenced so often, and in many contexts (view functions, UI code, etc), I've settled on something more concise for dmedia/Novacut.

For example, the desktopcouch spec dictates something like this:

  • {
        "_id": "ZZZATIZG6IA3DJOEANQCFT3FHR4IU2FC",
        "record_type": "http://www.freedesktop.org/wiki/Specifications/dmedia/file"
    }

But dmedia uses something much less verbose:

  • {
        "_id": "ZZZATIZG6IA3DJOEANQCFT3FHR4IU2FC",
        "type": "dmedia/file"
    }

This is a very typical dmedia view function:

  • function(doc) {
        if (doc.type == 'dmedia/file') {
            emit(doc.mtime, null);
        }
    }

Which becomes awkward and far less readable if the desktopcouch convention:

  • function(doc) {
        if (doc.record_type == 'http://www.freedesktop.org/wiki/Specifications/dmedia/file') {
            emit(doc.mtime, null);
        }
    }

Per DB sync should be opt-in, not opt-out

Developing with desktopcouch I've found it annoying that all the databases are synced to UbuntuOne by default... I frequently create test databases for testing dmedia (not just unit tests, but also using dmedia for an extended period, while not wanting to hose up my production dmedia DB). UbuntuOne sync is, of course, 100% awesome, but having it op-in rather than opt-out would make life easier for developers. It also avoids potential unexpected privacy oopses... when I was first learning desktopcouch I was surprised everything was synced by default. In my case, nothing in the database was sensitive, but for some, that wont be the case.

So, in reverse order of preference, the solution I would like is:

  • At the very least, desktopcouch should not sync databases whose names start with test_

  • Change from opt-out to opt-in, eg have included_names rather than excluded_names

  • Preferably, the included_names list should be stored in a non-replicated _local/foo document, or local file

The last point needs some explanation. Currently the opt-out list is stored in a replicated document in the management database, which syncs through UbuntuOne to all devices. This means that all devices get the same set of databases, which doesn't always make sense as not all devices will have the apps that use those databases. This is particularly an issue with mobile devices, when syncing unnecessary databases will needlessly burn through precious storage, bandwidth, and CPU time. Note that tablets in particular are a very high priority for Novacut.

There is also a security and privacy issue here. Whether opt-in or opt-out, if this critical decision is stored in a replicated document, an attacker need only gain control of a single device or service to cause other devices to replicate databases intended to be local-only.

Make web apps first class citizens

Standard location for static webUI files

DesktopCouchWishList (last edited 2011-05-09 16:06:31 by 137)