Summary

This spec discusses integrating support for synchronization of PDA's (Palm/PocketPC), Phones (Nokia/Blackberry), Multimedia Players (iPods), Desktop applications (Evolution/Rythembox) and Online Services (GoogleCalendar/Flicker)

The proposed solution is complex and involves a number of different intergration projects to culminate in a robust and effective syncing platform.

Rationale

Portable data storage for multiple types of data and databases of information are becoming more important; keeping this data in sync with desktops or internet services is more and more demanded by users.

Use cases

• John has an iPaq h1940 which he uses as a GPS device and PIM, he also uses a desktop pc at work with Evolution. He would like to be able to sync his PDA with Evolution.
• Clarise uses a Nokia 6021, this is a mobile phone with basic PIM functionality and support for syncml-obex.

• Michael is a web 2.0 man and uses several devices (internet tablet, mobile phone, laptop and desktop PC). He would like to keep his favorites synced so he uses del.icio.us. He would like to see his photos synced, so he uses flickr and picasa. He would like to keep his documents synced, so he uses google docs. But, ... he would like to be able accessing these info offline in any device when he wishes, update it, and re-sync when he go online again

• Michael has a new sci-fi ebook for his 770's FBReader. When he visits Ivan's house, he plugs his 770 with Ivan's PC, and he share his Documents folder, so Ivan's PC detects a new ebook available. After reading its metadata, Ivan's adds it to his Documents/eBooks folder. When Ivan connects his N810 to his PC, the ebook becomes available on it, under sci-fi category created from ebook metadata. Now think about sharing/syncing music, videos, etc.. using its embedded metada

• Fleur is an artist and uses her Dell Axim X3 for keeping track on her calendar, to do lists and excel-sheets. She also uses her Sony Ericcson phone as a "snapshot" camera. She would like to be able to sync her PDA with Evolution and sync her excel-sheets and photos with her home-folder.
• Winko is using his HTC Kaiser for calendar and MP3 player. He would like to be able to sync his phone with Evolution and share/update his MP3's.
• Stefan has a Palm m515 which he uses to keep track of expenses on his business travel. When in the office he wants to sync these expenses with gnome-expense, gnucash and/or a gnumeric spreadsheet. His appointments and contacts he manages in evolution and wants to sync them to his palm, the mobile phone and his corporate online calendar using caldav. He would like to be able to keep personal contacts from being synced to the company groupware server along with the other groups in his address books (e.g. clients). The same goes for private tasks and personal birthday reminders.
• Needs expanding (remember to keep to use cases, not solutions)

Scope

Integratation of existing tools which allow people to sync data into the same stack.

Design

• HAL

• OpenSync

• Conduit
• SynCE
• Bluetooth

Integration Low Level

HAL via udev to call scripts mentioned in FDI files for devices, these scripts are to populate HAL with the following information:

• Unique Identifier
• Refined model data

• OpenSync plugin to use

• Available data types supported (pim/photo/music/video/etc)

For each pda/phone that's detected we should run a script to identify who on that computer owns the device enabeling a degree of security and prevent the user from syncing to the wrong user logon; This script will just add to HAL the username, in the future it could be made more intergrated and secure.

Conduit will be changed to use the hardware endpoints automatically; it will tie into OpenSync who will provide a way for tempory sync configurations to be set up and executed.

Integration Bluetooth

Devices which allow syncing over bluetooth should be detected and handled once paired as if the device was plugged in via usb. This requires that the bluetooth stack report a new device to udev/hal so they can benefit from the same model as will be used for USB. There is no reason to create any special use cases between the bluetooth software, Conduit or Opensync at the detection and hardware configuration layer. This is a feature useful for other bluetooth hardware.

Integration GUI

On top of Conduit we wish to add a very simple python app which sits in the user's panel when active; this app will talk to Conduit via dbus and display all enabled and active syncing actions that can be taken. It will have a quick link to pairing bluetooth devices as well as a link into opening up Conduit GUI fully. It will also pop up information bubbles when a syncing service has been enabled by the plugging in or nearing of a device and when a new and unrecognised device is plugged in it will ask the user if this device stores their personal information and configure a new sync profile in Conduit automatically depending on the kind of device.

Notes

SynCE

• going to have fun packaging it... (Are the wbxml patches still needed? There's some talk of doing this inline)

• Needs kernel patch. Establishing a connection excites NetworkManager and wifi connection is lost

• Need to start SyncEngine, odccm and DTPT.py.

• Odccm probably on boot, ideally on connect. DTPT on connect.

• SyncEngine on dbus activation

• If you see the forum posts on the IRC chatter, SynCE is and absolute nightmare for users...

USB Serial

• Automatically establish PPP connection - don't interfere with NetworkManager

• Packaging and streamlining plugins

Conduit

• Uses OpenSyncModule to communicate to open sync

• Supports iPods and Nokia N800 hardware directly intergrated into HAL
• Supports many internet services
• Integrated into GNOME - e.g. plugins for Nautilus, EOG that allow them to initiate sync
• Dataproviders for Evolution, Tomboy and F-spot, Rhythmbox, Banshee, and any GnomeVFS folders and files

• Closely following GNOME Online desktop and trying to work with them to make sure they don't go and implement yet more sync engines... (There are at least 2 around right now)

• Core does not depend on any toolkit - You can run Conduit without any gtk - with conduit -c (?)
• Needs to move existing hardware support back to opensync plugins. This means converting all of the conversion stuff too. N800 Music sync isnt a priority for opensync, more functionality in open sync, more media formats to centralise the hardware support
• But lets think about how to do this in steps. No developer takes well to orders to dissolve and merge. We are working to get Opensync plugins in conduit. Thats step 1. Then Conduit plugins in Opensync

• Porting for portings sake? It should be clear that Conduit is making steps to play nice with opensync, GNOME online desktop, and others. E.g. they have patches to synce, tomboy, f-spot and even helped a little with opensync python bindings. So let us make this rock, rather than spending the next 6 months wasting time.. Convergence will happen as it makes sense. - Agree

• Also don't overlook fact that Online Desktop will likely bring more disconnected ways of syncing... gconf-sync-daemon+Tomboy springs to mind

• Will be using Opensync functionality when we finish the work on hosting their plugins. - **phase 1**. Phase 2 is extending the HAL stuff from phase 1 and connecting sync engine from opensync in to conduits model.. calling Sync on our dbus will take care of setting everything up on their engine and running it. Phase 3 will see our dataproviders run on their engine. More work from Jc2k with Opensync guys as those unfold. dgollub and Jc2k talked briefly about specialised engines like Opensync for PIM and csync (http://www.csync.org/) for files. Conduit could host multiple engines if that is the decided best direction.

• Is following GNOME so can expect it to follow GNOME release cycle this cycle

OpenSync

• OpenSync vision is to be a library. Small. Embedded. ***

• Aims just to be a sync engine
• Integration is up to whoever is embedding the sync logic
• HAL integration needs to happen outside of the Opensync project - this means that it won't be upstreamable and will be duplicating Conduit.

gnome-phone-manager

• it's a different thing -- doesn't sync (yet); should talk to Conduit for doing this
• Developer is in discussion with Conduit devs atm (I have spoken to Bastien before - this is bluesky type interaction and UI issues, not some endorsement of any engine)
• At the least would this benefit from bluetooth capabilities?

Semantic Desktop

• There are many technologies from Semantic Web that can apply nicely here: RDF managers (librdf, raptor, ...), FOAF as PIM data model with extra-info, geoRDF and geoglue, Dublin Core for files metadata models, ...
• There are some apps that already use these technologies like Tracker, Semperwiki, ...

History

• Opensync does pim because that's its heritage
• Conduit does web-centric because that's its heritage
• Lets not construe these historical things for some active intent to duplicate all the work
• So converging is an option? - opensync just want to provide an engine.
• Converging is always and option, the tone of the email discussion seems to be 'everyone converge NOW! GO!' while the pragmatic reality of FOSS is that both projects will wander together based on technical choices. Conduit is currently making serious work to walk over to use opensync plugins. That is common work needing to be done when we go to their sync engine in future.

  • Further to that, OpenSyncModule is conduits initial attempt to converge. Or at least, step 1 of it.

Old Notes

Sync integration with nokia n770
e-d-s running on both sides,
770 is debian-based
opensync could be ported to the device
primary mode would be sync via bluetooth
it could be taken to the extent where syncs are done in a proactive way - either side could initiate a sync on various changes
syncml could be used - server would run on desktop side, more options on desktop
syncml spec allows for either side to initiate
actually both sides would have to be a server (could be resource-intensive)
connection must be done from client->server
the protocol should act like a peer-to-peer, but doesn't really
conflict resolution server engine only has to run on the desktop, however
server would send message to opensync via dbus to initiate sync
don't *need* a full syncml server on desktop, just something to prod opensync into doing the sync
opensync is a pull-based system, not a push-based system,
it expects a process to make requests, and pushes results
opensync would poll e-d-s, or e-d-s would have to notify opensync to start pulling
syncml from the 770 to trigger opensync
e-d-s is backend, opensync is transport
syncml needs to listen on dbus, bluetooth, and initiate events

eds pokes opensync
opensync uses syncml client to talk to syncml server, which puts data into eds on the remote side
the same thing is done in the other direction
conflict resolution is done inside opensync
both sides would need eds, opensync, syncml client & syncml server
syncml server would have to be listening all the time, in some fashion
syncml library that could be used is http://libsyncml.opensync.org (not released, only in CVS)
advantage of this model is that any system using syncml could use this

Implementation:
 writing a binding for eds/syncml
 writing a trigger such that eds can trigger opensync cleanly

can we use parts of opensync to have syncml talk to eds?
dependencies of opensync: glib2, libxml2, maybe needs libsqlite3
each plugin has different build requirements, each is packaged independently
libsyncml needs to be packaged, need to check & test if it's in a good shape
   svn co http://svn.opensync.org/libsyncml/trunk libsyncml

syncml server should be a bluetooth-independent project, but start off with bluetooth support