== Summary == This spec discusses integrating support for synchronization of PDA's (Palm/PocketPC), Phones (Nokia/Blackberry), Multimedia Players (iPods), Desktop applications (Evolution/Rythembox) and Online Services (GoogleCalendar/Flicker) The proposed solution is complex and involves a number of different intergration projects to culminate in a robust and effective syncing platform. == Rationale == Portable data storage for multiple types of data and databases of information are becoming more important; keeping this data in sync with desktops or internet services is more and more demanded by users. == Use cases == * John has an iPaq h1940 which he uses as a GPS device and PIM, he also uses a desktop pc at work with Evolution. He would like to be able to sync his PDA with Evolution. * Clarise uses a Nokia 6021, this is a mobile phone with basic PIM functionality and support for syncml-obex. * Michael is a web 2.0 man and uses several devices (internet tablet, mobile phone, laptop and desktop PC). He would like to keep his favorites synced so he uses del.icio.us. He would like to see his photos synced, so he uses flickr and picasa. He would like to keep his documents synced, so he uses google docs. But, ... he would like to be able accessing these info offline in any device when he wishes, update it, and re-sync when he ''go'' online again * Michael has a new sci-fi ebook for his 770's FBReader. When he visits Ivan's house, he plugs his 770 with Ivan's PC, and he ''share'' his ''Documents'' folder, so Ivan's PC detects a new ebook available. After reading its metadata, Ivan's adds it to his ''Documents/eBooks'' folder. When Ivan connects his N810 to his PC, the ebook becomes available on it, under ''sci-fi'' category created from ebook metadata. Now think about ''sharing/syncing'' music, videos, etc.. using its embedded metada * Fleur is an artist and uses her Dell Axim X3 for keeping track on her calendar, to do lists and excel-sheets. She also uses her Sony Ericcson phone as a "snapshot" camera. She would like to be able to sync her PDA with Evolution and sync her excel-sheets and photos with her home-folder. * Winko is using his HTC Kaiser for calendar and MP3 player. He would like to be able to sync his phone with Evolution and share/update his MP3's. * Stefan has a Palm m515 which he uses to keep track of expenses on his business travel. When in the office he wants to sync these expenses with gnome-expense, gnucash and/or a gnumeric spreadsheet. His appointments and contacts he manages in evolution and wants to sync them to his palm, the mobile phone and his corporate online calendar using caldav. He would like to be able to keep personal contacts from being synced to the company groupware server along with the other groups in his address books (e.g. clients). The same goes for private tasks and personal birthday reminders. * Needs expanding (remember to keep to use cases, not solutions) == Scope == Integratation of existing tools which allow people to sync data into the same stack. == Design == * HAL * OpenSync * Conduit * SynCE * Bluetooth == Integration Low Level == HAL via udev to call scripts mentioned in FDI files for devices, these scripts are to populate HAL with the following information: * Unique Identifier * Refined model data * OpenSync plugin to use * Available data types supported (pim/photo/music/video/etc) For each pda/phone that's detected we should run a script to identify who on that computer owns the device enabeling a degree of security and prevent the user from syncing to the wrong user logon; This script will just add to HAL the username, in the future it could be made more intergrated and secure. Conduit will be changed to use the hardware endpoints automatically; it will tie into OpenSync who will provide a way for tempory sync configurations to be set up and executed. == Integration Bluetooth == Devices which allow syncing over bluetooth should be detected and handled once paired as if the device was plugged in via usb. This requires that the bluetooth stack report a new device to udev/hal so they can benefit from the same model as will be used for USB. There is no reason to create any special use cases between the bluetooth software, Conduit or Opensync at the detection and hardware configuration layer. This is a feature useful for other bluetooth hardware. == Integration GUI == On top of Conduit we wish to add a very simple python app which sits in the user's panel when active; this app will talk to Conduit via dbus and display all enabled and active syncing actions that can be taken. It will have a quick link to pairing bluetooth devices as well as a link into opening up Conduit GUI fully. It will also pop up information bubbles when a syncing service has been enabled by the plugging in or nearing of a device and when a new and unrecognised device is plugged in it will ask the user if this device stores their personal information and configure a new sync profile in Conduit automatically depending on the kind of device. = Notes = == SynCE == * going to have fun packaging it... (Are the wbxml patches still needed? There's some talk of doing this inline) * Needs kernel patch. Establishing a connection excites NetworkManager and wifi connection is lost * Need to start SyncEngine, odccm and DTPT.py. * Odccm probably on boot, ideally on connect. DTPT on connect. * SyncEngine on dbus activation * If you see the forum posts on the IRC chatter, SynCE is and absolute nightmare for users... == USB Serial == * Automatically establish PPP connection - don't interfere with NetworkManager * Packaging and streamlining plugins == Conduit == * Uses OpenSyncModule to communicate to open sync * Supports iPods and Nokia N800 hardware directly intergrated into HAL * Supports many internet services * Integrated into GNOME - e.g. plugins for Nautilus, EOG that allow them to initiate sync * Dataproviders for Evolution, Tomboy and F-spot, Rhythmbox, Banshee, and any GnomeVFS folders and files * Closely following [[http://live.gnome.org/OnlineDesktop/|GNOME Online desktop]] and trying to work with them to make sure they don't go and implement yet more sync engines... (There are at least 2 around right now) * Core does not depend on any toolkit - You can run Conduit without any gtk - with conduit -c (?) * Needs to move existing hardware support back to opensync plugins. This means converting all of the conversion stuff too. N800 Music sync isnt a priority for opensync, more functionality in open sync, more media formats to centralise the hardware support * But lets think about how to do this in steps. No developer takes well to orders to dissolve and merge. We are working to get Opensync plugins in conduit. Thats step 1. Then Conduit plugins in Opensync * Porting for portings sake? It should be clear that Conduit is making steps to play nice with opensync, [[http://live.gnome.org/OnlineDesktop/|GNOME online desktop]], and others. E.g. they have patches to synce, tomboy, f-spot and even helped a little with opensync python bindings. So let us make this rock, rather than spending the next 6 months wasting time.. Convergence will happen as it makes sense. - Agree * Also don't overlook fact that Online Desktop will likely bring more disconnected ways of syncing... gconf-sync-daemon+Tomboy springs to mind * Will be using Opensync functionality when we finish the work on hosting their plugins. - **phase 1**. Phase 2 is extending the HAL stuff from phase 1 and connecting sync engine from opensync in to conduits model.. calling Sync on our dbus will take care of setting everything up on their engine and running it. Phase 3 will see our dataproviders run on their engine. More work from Jc2k with Opensync guys as those unfold. dgollub and Jc2k talked briefly about specialised engines like Opensync for PIM and csync (http://www.csync.org/) for files. Conduit could host multiple engines if that is the decided best direction. * Is following GNOME so can expect it to follow GNOME release cycle this cycle == OpenSync == * OpenSync vision is to be a library. Small. Embedded. *** * Aims just to be a sync engine * Integration is up to whoever is embedding the sync logic * HAL integration needs to happen outside of the Opensync project - this means that it won't be upstreamable and will be duplicating Conduit. == gnome-phone-manager == * it's a different thing -- doesn't sync (yet); should talk to Conduit for doing this * Developer is in discussion with Conduit devs atm (I have spoken to Bastien before - this is bluesky type interaction and UI issues, not some endorsement of any engine) * At the least would this benefit from bluetooth capabilities? == Semantic Desktop == * There are many technologies from Semantic Web that can apply nicely here: RDF managers (librdf, raptor, ...), FOAF as PIM data model with extra-info, geoRDF and geoglue, Dublin Core for files metadata models, ... * There are some apps that already use these technologies like Tracker, Semperwiki, ... = History = * Opensync does pim because that's its heritage * Conduit does web-centric because that's its heritage * Lets not construe these historical things for some active intent to duplicate all the work * So converging is an option? - opensync just want to provide an engine. * Converging is always and option, the tone of the email discussion seems to be 'everyone converge NOW! GO!' while the pragmatic reality of FOSS is that both projects will wander together based on technical choices. Conduit is currently making serious work to walk over to use opensync plugins. That is common work needing to be done when we go to their sync engine in future. * Further to that, OpenSyncModule is conduits initial attempt to converge. Or at least, step 1 of it. = Old Notes = {{{ Sync integration with nokia n770 e-d-s running on both sides, 770 is debian-based opensync could be ported to the device primary mode would be sync via bluetooth it could be taken to the extent where syncs are done in a proactive way - either side could initiate a sync on various changes syncml could be used - server would run on desktop side, more options on desktop syncml spec allows for either side to initiate actually both sides would have to be a server (could be resource-intensive) connection must be done from client->server the protocol should act like a peer-to-peer, but doesn't really conflict resolution server engine only has to run on the desktop, however server would send message to opensync via dbus to initiate sync don't *need* a full syncml server on desktop, just something to prod opensync into doing the sync opensync is a pull-based system, not a push-based system, it expects a process to make requests, and pushes results opensync would poll e-d-s, or e-d-s would have to notify opensync to start pulling syncml from the 770 to trigger opensync e-d-s is backend, opensync is transport syncml needs to listen on dbus, bluetooth, and initiate events eds pokes opensync opensync uses syncml client to talk to syncml server, which puts data into eds on the remote side the same thing is done in the other direction conflict resolution is done inside opensync both sides would need eds, opensync, syncml client & syncml server syncml server would have to be listening all the time, in some fashion syncml library that could be used is http://libsyncml.opensync.org (not released, only in CVS) advantage of this model is that any system using syncml could use this Implementation: writing a binding for eds/syncml writing a trigger such that eds can trigger opensync cleanly can we use parts of opensync to have syncml talk to eds? dependencies of opensync: glib2, libxml2, maybe needs libsqlite3 each plugin has different build requirements, each is packaged independently libsyncml needs to be packaged, need to check & test if it's in a good shape svn co http://svn.opensync.org/libsyncml/trunk libsyncml syncml server should be a bluetooth-independent project, but start off with bluetooth support }}}