Ubuntu One is a blanket name for what is destined to be a suite of online services from Canonical.

So far there is only one service, a Dropbox-like service for file exchange and mirroring. It is currently in beta.

Invitations

Invitations are no longer required to join the Ubuntu One beta test and subscribe to a file sharing plan.

Introduction

• The log file is at ~/.cache/ubuntuone/log/syncdaemon.log

• There is as yet no indicator of the remaining time/bytes.
• The sync algorithm handles partial downloads of files and resumes from interruptions.
• If a file changes the entire file is re-copied; deltas are planned but not yet implemented.
• At the moment it will only sync files in the ~/Ubuntu One folder, but it could do more in future, perhaps synchronising /var/www from a server for example.
• The plans are 2GB of free storage and a paid for 10GB option. More than 10GB is not currently available.
• The data is stored on the Amazon S3 servers in both the UK and the USA.

Areas needing extensive testing

• Putting files in and out whilst synchronising
• Interrupted synchronisations
• Deletions
• Copy the ~/Ubuntu One folder or some part of it from one computer to another via sftp or otherwise and let U1 pick up synchronisation
• Conflicts ("simultaneous" modifications to the same file from different locations)

Reporting bugs

You may be requested to submit part of your log file. Please bear in mind this contains filenames when posting a public bug report or posting on IRC.

The command

• tail -n 100 logfile | pastebinit

will put the last 100 lines of logfile on pastebin and give you a URL suitable for discussing on IRC.

• grep -n -A 30 -B 10 ERROR syncdaemon.log |pastebinit

will hunt the syncdaemon.log file for an ERROR and will print the preceeding 10 lines and following 30 lines to the pastebin.

Bugs can be reported on Launchpad

Support & Tutorials

Support options

There are various support options available to Ubuntu One users:

• File a bug or submit a feature suggestion: https://bugs.launchpad.net/ubuntuone-client/+filebug

• Ask a question: https://answers.launchpad.net/ubuntuone-client/

• Post to the Ubuntu One channel on Ubuntu Forums: http://ubuntuforums.org/forumdisplay.php?f=367

• Join the discussion on IRC: #ubuntuone on freenode - Project team members are available in this channel.

• Receive account-related assistance (eg, billing): https://ubuntuone.com/support/account-assistance/

Tutorials

The following are some quick tutorials to get new Ubuntu One users up and running:

• Files

• Notes

Technical details

• Details as viewed by an outsider. Blame sladen for E&OE and for the insistence on using cute names.

As of 2009-06-08, the available services are:

• Storage: remote storage

Note: Because the "Ubuntu One" branding is non-specific and because most of the components do not have succinct official names, we'll use shorter (unique) names to succinctly identify each of the components and layers. You can find a table mapping these simple names to download packages and URLs at the bottom. (See the "#storagefs" section for just how long-winded and suboptimal these actual names can get).

"Ubunet" was the original codename for the online services initiative (the Canonical department itself is called "Online Services"), and is the name of the Launchpad project encompassing the backend components (there is also a "control tower" group which is used when landing branches).

Release notes

The release notes for the latest Ubuntu One releases can be found here.

Code and licensing

For contributions to be accepted back and integrated into the tree, copyright on the changes must be assigned to Canonical.

Packaging

Packages should be versioned in the following way. Here, $VERSION refers to the version specified in configure.ac or setup.py. rN refers to the bzr revision number of the source being packaged, if not packaged from a release, and $BUILDID is the build string (aka. "package version", typically of the form 0ubuntu1 for packages in Ubuntu, for build strings for PPA builds, see https://help.launchpad.net/Packaging/PPA ), which may include the PPA index, and SERIES index as applicable for building in a PPA, or repository which is not the official distribution package repository.

• Released tarball versions: $VERSION-$BUILDID
• Snapshot from bzr (nightlies): $VERSION+rN-$BUILDID

"Storage"

This provides 2GB-10GB of remotely accessible disk space stored on Amazon S3. Rather than using an existing protocol (eg. FTP, SFTP, NFS, Webdav) access to this remote disk space is via an invented-here communication protocol called "u1storage".

The system is intended to be for mirroring of content between computers, opportunistically synchronizing those mirrors when the user is online. There is a delay because a file must be completely uploaded before the changes can start to be shared. The heavy duty backend storage is ultimately out-sourced to Amazon S3 (Amazon Simple Storage Service).

At a high-level, Storage has been compared to Dropbox. In June 2009, Mandriva have produced something similar called mdvbackup, (branded as "Mandriva Click'n'Backup").

During early development, the codename for this facility was "Hammertime".

General concepts

• There is a backing database (currently on S3) which stores file objects, directory objects, and the BLOB objects (files) themselves.

• On the user's computer, a Share is a directory that is actively being mirrored.

• When a file is modifid, the complete new version is uploaded and saved in a BLOB. The file object is updated to point at the new BLOB (directories are not themselves changed for a simple overwrite operation).

• A Subtree is a particular directory object, representing itself and all of its descendants.

• A Subtree can be shared with other users. This is called a View.

• The View given to another user can be either read-only, or read-write.

• On connection, or reconnection, the u1sync-agent is responsible for spidering the full directory tree and identifying any changes on its own.

• While connected, the u1sync-agent will receive unsolicited notifications of changed objects. Notifications are an optimisation. They allow for lower latency in synchronisation and they avoid unnecessary polling.

• u1sync is an example of a simple one-shot client that ignores unsolicited notifications.

Authentication

A long-term token is used for communication between the desktop and server side infrastructures. This token is stored in the gnome-keyring. Actual login is done entirely within a web-browser, so that any password/user ID is kept within Firefox and not exported. There are no username/passwords stored by u1sync-agent.

u1sync-agent calls out via DBus to com.ubuntuone.Authentication. This is currently answered by gnome-u1sync-applet. The applet queries the GNOME keyring, and if the token does not exist, it starts the OAuth/OpenID web-login process and has the user enter their credentials that way. The token is saved and also returned to the u1sync-agent. At the same time, there is some ACL fiddling to allow any (future) other UbuntuOne service programs to also access the token(s).

Fixme: needs more and it still isn't clear in my head and so is unlikely to be accurate.

updown

This is a backend librarian service. It listens for connections via HTTP and accepts the upload, or returns the download requested. It does not have a user-visible interface.

It does not use the "u1storage" protocol for access, talking instead to the backend database and is therefore effectively a raw HTTP<->S3 gateway.

There is one implementation, which is proprietary. Little is known, but an instance of it runs on an EC2 server at updown.ubuntu.com. You can get some HTTP headers from it:

$ telnet-ssl -z ssl updown.ubuntuone.com 443
...
Server: PasteWSGIServer/0.5 Python/2.5.2
X-Bzr-Revision-Number: 1486
Via: 1.0 domU-12-31-38-00-4D-75.compute-1.internal

webstorage

Also known as "web ui". This is a pretty HTTP/web-browser interface to view directory listings and files held in the user's storage service (and other Shares which they have access to). It is universal and cross-platform and allows viewing of files, plus uploading and downloading.

The user's own files are displayed under "My Files" and incoming Shares from other users show up under "Shared With Me". An advantage is that it allows direct "cherry picking" of individual files, without waiting for a (potentially lengthy) synchronisation process to complete.

Going via the browser appears to be "closest to bare metal" interface.

There is one implementation, which is proprietary. Little is known, but an instance of it runs at https://www.ubuntuone.com/files/ (subscribers only).

u1storage

This protocol is built on top of Google Protocol Buffers (a sort of fast binary XMLish schema language). This abstraction is to allow extension of the u1storage protocol with breaking compatibility.

The intention of the protocol is to enabling synchronisation by mirroring of content. The protocol itself uses SHA-1 hashes as identifiers for directory objects and their contents (files). Requests are always initiated from the client end. The first versions does not support any kind of delta-optimisation and each object (file) must be sent in its entirety.

Currently there is only one implementation, it written in Python and called python-u1storage (AGPLv3+copyright assignment required). The .proto descriptions (AGPLv3+assignment) can be used as starting points for other language implementations.

storagefs

This is the fuzzy term for a monolithic(?) daemon running on the server fs-1.ubuntuone.com (174.129.241.144, currently an Amazon EC2 virtual machine instance). It accepts incoming SSL/TLS wrapped u1storage connections on port 443. Note that although port 443 (HTTPS) is used to allow passage through firewalls, the protocol has nothing to do with HTTP.

Storagefs sends a client connected via the u1storage protocol unsolicited updates of any hash reference that have been updated, so that the other end can re-download the changed objects.

Currently there is one implementation, which is proprietary. This links to python-u1storage. That implemetnation is started by /etc/init.d/ubuntuoneapiserver which probably executes something like /usr/bin/env python .../canonical/ubuntuone/storage/server/server.py. It seems to save the partially uploaded files to a temporary directory; then presumably indexes/moves them off to Amazon S3 where the real storage is. You can connect to this instance with:

$ telnet-ssl -z ssl fs-1.ubuntuone.com 443
...
3 ubuntu one storage server revision 1486.

u1sync-agent

This is a per-user daemon run on each desktop computer and has no user interface. It runs in the background and exposes a DBus API interface. The agent does the actual work of deciding what to sychronise in which direction and handles doing so. Automatic resumption and partial uploading/downloading is supported, these files will be visible as filename.partial until the transfer has completed, when they are required to the proper filename.

By default, one fold called ~/Ubuntu One/ is automatically created and configured for mirroring. Changes to this fold (and any others added) are watched using inotifiy. Synchronisation meta-data about which directories are being mirrored and what their is stored in ~/.cache/ubuntuone. When remote content has changed, the agent acts on the incoming unsolicited notification sent by storagefs and starts downloading.

Authenication is via OAuth.

There is one implementation, written in Python (GPLv3), this may also be referred to as chicharra. It is started by /usr/share/dbus-1/services/com.ubuntuone.SyncDaemon.service and exposes its DBus interface under com.ubuntuone.SyncDaemon. Logs files are kept in ~/.cache/ubuntuone/log/syncdaemon.log.

gnome-u1sync-applet

Tray icon to control the behaviour of the a running u1sync-agent instance, by communicating over the DBus interface. Provides a constant status update on "what's happening".

There is one implementation, which is written in Python (GPL+copyright assignment hassle status unknown).

u1sync-tool

Command line utility to control the behaviour of a running u1sync-agent instance. The tool does not perform any actions directly, but sends messages to the running agent.

There is one implementation, which is written in Python (GPLv3).

libnautilus-u1sync

An extension (plugin) for the default GNOME file-browser called Nautilus to adds a status icon/button. This can send connection and disconnection requests to a running u1sync-agent instance. Additional directories (Shares) can be nominated for sharing using a new right-click context menu.

There are two implementations. One currently being shipped, written in Python (GPLv3), and one written in C (GPLv3) under development as of 2009-06-08.

u1sync

This is a command line and test utility tool more akin to bzr branch/push or git clone/push (version control systems). Synchronisation metadata is stored in a hidden directory at the root folder of each share named $dir/.ubuntuone-sync.

Note: It is incompatible with the rest of the u1sync-agent infrastructure.

Apparently, this utility has concepts borrowed from Amazon's s3sync tool.

There is one implementation, which is written in Python (GPLv3).

fuse-u1storage

Apparently there was a FUSE (Filesystem in User Space) implementation, but it was "a disaster", so probably got canned. No details on whether this worked via mirroring, or simply exposing/mounting the remote filesystem locally.

There are zero current implementations.

Name mappings

• Components denoted '(N)' and having the same number are each part of the same suite.

Security

More details on Ubuntu One security and privacy can be found here.

QA

Bug Triage

A bug travels through this general flow (not including Wishlist & Invalid bugs):

1. New/Incomplete
2. Triage
3. In-Progress
4. Fix Committed
5. Fix Released

General Process

1. Classify incoming bugs (ie bugs that are new, unassigned, and not tagged (desktop+, foundations+, ops+)) to determine if they are:

   • Emergency

     • Set status to Triaged

     • Set Importance to Critical

     • Let the team know on #ubuntuone ASAP

   • Invalid

     • Set Status to Invalid

     • Comment should note the reason it is invalid

   • Needs More Info

     • Set Status to Incomplete

     • Ask the bug reporter nicely for more information, possibly using apport: $ apport-collect -p ubuntuone-client replace-with-bug-#

   • Ready for Investigation/Fix

     • Set Status to Triaged

     • If you think you can gauge the Importance, go ahead & set it (see guidance below on setting importance)

   • New Features (Wishlist)

     • Set Status to Wishlist

     • Comment should let bug reporter know that we are either planning on adding the feature (only if you know it's already on the roadmap) or that we'll consider adding it to our future plans
2. On first touch, some nice comment in acknowledgement should be made to our kind bug reporter.
3. Bugs should be assigned to one of the + teams using a LP tag. Team Leads should have searches keyed on these tags for further triage.

   • desktop+ - Desktop applications, applets, etc.

   • foundations+ - Web, sync daemon, etc.

   • ops+ - Infrastructure, testing, etc.

4. If the optimal assignee is known, the bug can be placed in that team member's name. This may be advisable in the case of:
   • Critical bugs
   • Requested placeholders
   • Team member has already engaged the reporter without triaging

Other tags that you may see in use are:

• story-NNNN - link notes to the high-level storyboarding system (see #Side-projects).

• ubunet-only - bug reports misfiled against the open-source desktop components which can only be fixed in the backend/closed parts.

Setting Importance

If you think you can set the importance of a bug, then you should. Here is some guidance on the various levels of Importance (thank you Launchpad team - https://dev.launchpad.net/BugTriage:

• Critical - The bug dramatically impairs users. Users may lose their data. Users cannot complete crucial tasks. The feature is needed to encourage adoption or prevent abandonment of the project.

• High - The bug prevents users from completing their tasks. The feature provides new kinds of tasks or new ways of completing tasks.

• Medium - The bug is an inconvenience for many users. The feature provides new ways of completing tasks.

• Low - The bug is an inconvenience to users, but it does not prevent them from completing their tasks. The feature is a convenience to users.

Vapourware

Anything below here is scraped together from the desktop-karmic-integrating-with-ubuntu-one spec, the irclogs, from bug reports and blogs and identi.ca/statik+group/ubuntuone+tags/ubuntuone. Take it all with a pinch of salt:

• u1-ss Screen-sharing ("VNC/vino based"?)

• Tomboy note sharing (upstream's Snowy API + OAuth)

• "graphite", no clue

• "desktopcouch"

  • Evolution data server (e-d-s) talking to local CouchDB instance. (Now not really vapourware anymore).

• Phone syncing (Funambol-based(?))

Side-projects

High-level storyboarding (internal project management across many components) for the Online Services team is being performed by a sophisticated Python mashup that pulls in launchpadlib and gdata libraries. Work nuggets are broken down into one bug/branch per day or ~500 lines of code; the story-NNNN tags allow information from Launchpad, dashboard/general progress overviews and information from a shared Google Spreadsheet document to be tied together in one place.

Some of the ideas provided by the extra glue are being sent back to the Launchpad team to evaluate as future wish-list items.

Other things seen:

• "openSUSE packaging"