MigrationAssistance

Differences between revisions 3 and 38 (spanning 35 versions)
⇤ ← Revision 3 as of 2006-10-20 14:29:36 →
Size: 14600
Editor: ip37-9-173-82
Comment:
← Revision 38 as of 2008-08-06 16:19:54 → ⇥
Size: 8865
Editor: localhost
Comment: converted to 1.6 markup
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
 * '''Launchpad Entry''': https://launchpad.net/distros/ubuntu/+spec/migration-assistance
 * '''Created''': 10/20/06 by EvanDandrea		  * '''Launchpad Entry''': UbuntuSpec:migration-assistant-gutsy
 * '''Created''': 17/5/07 by EvanDandrea
Line 5: Line 5:
 * See also: KubuntuUbiquityMigrationAssistant
Line 8: Line 9:
Further the development of migration-assistant to the point that it can be included in Fiesty Fawn. ''Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.''

Further the development of migration-assistant to the point that it can be included in Feisty Fawn. Work prior to Feisty is covered in the [[/Edgy]] specification. Work for Feisty is covered in the [[/Feisty]] specification.
##(see the SpecSpec for an explanation)

 * Support for Windows 2000 and Vista, as well as Mac OS X support will be added.
 * Support for migrating data to an external device or source, such as a USB key, for importing from using a desktop application after install, will be added.
 * migration-assistant will be enabled on the alternate CD installer.

== Release Note ==

The migration functionality in the installer now allows users to export their migrated data to a USB thumb drive or other external source to later be merged into their Ubuntu system using the accompanied desktop application.
Line 12: Line 24:
The reality for most potential Ubuntu users is that they've bought their computer with Windows pre-installed or have already installed another Linux distribution. These users have probably already used their existing operating system for a decent period of time and have accumulated documents, email, settings, and other files that can be imported and translated for use in Ubuntu, but the existing Ubuntu graphical installer mercilessly destroys them. If users could install Ubuntu knowing that all of their existing Office documents would be placed in their Documents folder, their email would be imported into Evolution, and their bookmarks imported into Firefox, they would have much more incentive to make the switch, be it from Windows or another Linux distribution. The desktop application and ability to migrate data to a source other than the root filesystem are required for users who wish to migrate their data without keeping their old partitions around, and for those users who either did not have the ability to use migration-assistant in a previous release or chose not to, but wish to use it now.
Line 14: Line 26:
== Use cases == == Use Cases ==
 * Mark wants to migrate his data from Windows XP without having to keep his Windows XP partition around.
 * Evan wants to import his Apple OS X users when installing Ubuntu on his iMac.
 * Joe would like Ubuntu to be able to automatically authenticate to the various wireless access points he's associated with previously without requiring him to re-enter WEP and WPA keys.
Line 16: Line 31:
 * Andrew is a long time Gentoo user, tired of having to manually configure his system, and wants a distribution that "just works". He'd like to copy over his home directory to the new system when he installs Ubuntu.
 * Dave is a busy parent who has been using Windows XP for the past year, and he and his children have accumulated a large number of pictures, music, documents, and bookmarks between their four user accounts. He'd like to be able to copy over all of the user accounts while selecting what files to keep and what to throw away when installing Ubuntu.
 * Joe is a CAD engineer who would like to switch his home computer to Ubuntu. While Ubuntu does not have software to read Autodesk files, he would nevertheless like to save his set. He does not have a CD burner and the files are too big for email attachments.		 == Assumptions ==
Line 20: Line 33:
== Scope ==

A package containing all of the importing code and the debian-installer hooks was uploaded as a source package in Universe. The changes to Ubiquity will need to be merged in to the main branch.		 None
Line 25: Line 36:
{{{
+----------------------------------------+
| |
| Select the operating systems that you |
| would like to import documents and |
| settings for. Press next without |
| selecting anything to skip importing. |
| |
| [X] Windows XP (/dev/hda1) |
| [X] Linux (/dev/hda5) |
| |
| |
| |
| [ Back ] [ Next ] |
| |
+----------------------------------------+
}}}
If Ubuntu is being installed alongside another operating system, the above dialog is presented to the user, otherwise migration-assistant is skipped.
{{{
+----------------------------------------+
| |
| Select the users that you would like |
| to import for Windows XP (/dev/hda1). |
| |
| [X] Evan |
| [X] Joe User |
| |
| |
| |
| [ Back ] [ Next ] |
| |
+----------------------------------------+
}}}
The above dialog is presented for each operating system the user chooses to import from.
{{{
+----------------------------------------+
| |
| +---+ |
| | | Evan |
| +---+ |
| |
| Select the items you would like to |
| import for this user. |
| |
| +------------------------------------+ |
| | |/\| |
| | [X] My Documents | | |
| | [X] My Music | | |
| | [X] My Pictures | | |
| | [X] User Picture | | |
| | [X] AOL Instant Messenger | | |
| | [X] Yahoo Instant Messenger | | |
| | [X] IE Bookmarks | | |
| | [X] Firefox Bookmarks | | |
| | [X] Microsoft Outlook | | |
| | |\/| |
| +------------------------------------+ |
| |
| Select an account to import this user |
| into: |
| +-------------+--+ |
| | New Account |\/| |
| +-------------+--+ |
| | New Account | |
| | evan | |
| +----------------+ |
| |
| [ Back ] [ Next ] |
| |
+----------------------------------------+
}}}
The user is presented with the above page for each user they selected to import. The user picture is used along with the user name to clarify exactly which account the user is dealing with. All items are selected by default and "New Account" is selected by default.
If the user does not select any items, then the account will be created, but nothing will be imported into it.
{{{
+----------------------------------------+
| |
| Type in the full name, username, and |
| password for this account below. |
| |
| Full Name: [ Evan Dandrea ] |
| User Name: [ evand ] |
| Password: [ ] |
| Confirm: [ ] |
| |
| [ ] Administrator Account |
| |
| |
| [ Back ] [ Next ] |
| |
+----------------------------------------+
}}}
If the user selected "New Account" on the previous page, they are presented with the above dialog. The account they create will be visible in the accounts drop-down list after they press next, so if they wanted to, they could import multiple users into a single account.
Full Name is already filled in if it can be obtained from the existing operating system. User Name is filled in with an all lowercase, spaces stripped version of the previous username. If the user is importing the account from a Linux operating system, the password and confirm fields are filled in with a set number of asterisks. Administrator Account is not selected by default, but if it is checked the user will be added to sudoers.
If the user types in a username that already exists, or Password and Confirm do not match, an error message is presented alongside the offending field.
Line 120: Line 37:
A stand-alone desktop application will be added that will enable the user to import from external sources used during the install for exporting migration data to, such as a USB key; FTP, sftp, or WebDAV site; or, possibly in the event of two CD recorder devices, a CD. Exporting to and importing from a USB key will be developed first, allowing other, less commonly used sources to be added as time permits.

Support for importing from Windows Vista, Mac OS X, and Windows 2000 will be added.

Additional import sources will be added. These may include:
 * Wireless access points with WPA and WEP keys.
 * Files on the desktop.
 * OS X applications, including Photo Booth pictures, Mail.app settings, Dashboard widgets (into Jackfield), iTunes music, and web browser bookmarks from Safari, Firefox, and Camino.
 * All instant messenger settings, including logs and passwords (if possible).
 * All PIM data from Outlook and Evolution including actual email, calendar data, and notes.
 * Mozilla Thunderbird.
 * Fonts.
 * Web browser data such as cookies, proxy settings, and cross-platform Firefox plugins.
Line 123: Line 53:
migration-assistant is broken into five programs:  * migration-assistant will be added to the alternate CD installer via a change of priority.
 * migration-assistant will have to be moved before partman-apply in debian-installer to accommodate exporting to a source other than the root filesystem, as the goal of that feature is to enable users who are electing to format their disk completely to still make sure of the migration functionality. This will require changes to {{{ma-ask}}}'s partition state detection code.
 * migration-assistant will be modified to use choices-c to allow translating source names, such as "Internet Explorer", into the user's native language.
 * A progress dialog will be added to give the user more feedback on the detection process and allow them to cancel and skip the migration step, if they so desire.
 * Support for importing a user's entire /home directory will be added and a validation check will be created to ensure that the user does not select both the entire home directory and an item contained within it.
 * The structure of the migration data when exporting to an external device or source will be a {{{migration-assistant}}} directory in the root of the device or source, with /home below that, containing all the migrated files.
 * migration-assistant will need to be modified to detect and mount loopback filesystems created by Wubi to provide a migration path from the test installs created by Wubi as part of the UbuntuSpec:installer-for-windows specification.
Line 125: Line 61:
 * '''ma-ask:'''[[BR]]
 This is shell code that asks the user all of the questions via debconf that ma-apply needs to import the documents and settings.
 * '''ma-search-users:'''[[BR]]
 This finds users suitable for importing from given the operating system and location specified in its arguments.
 * '''ma-search-items:'''[[BR]]
 This looks for all available items to import given the operating system and location specified in its arguments.
 * '''ma-apply:'''[[BR]]
 This reads all of the debconf questions creates any needed users, and feeds the rest of the data to ma-search-items.
 * '''ma-import:'''[[BR]]
 This imports a type of item given arguments of operating system type, location, existing user, location to import to, and user to import into.		 === UI Changes ===
Line 136: Line 63:
ma-ask is called within debian-installer and a modified version of Ubiquity seeds the questions that ma-ask asks.  * Minor changes to the wording of the text used to instruct the user may be made.
 * Additional text that lets the user know that they can wait until after install to migrate their documents and settings (using the desktop application) will be added.
 * The full name of Windows users and the computer name will be preseeded by first grabbing the data from SAM and the registry, respectively.
 * The following section will be added with an accompanying modal dialog similar to the "Connect to Server..." dialog used in GNOME, allowing the user to specify a source to export the migrated data to.
Line 138: Line 68:
migration-assistant as a whole searches the hard drive on which it is being installed for user accounts as well as the documents and settings that go along with each account, and presents its findings as options in the installer that the user can select to be imported into their new Ubuntu installation. Configuration data is translated into its appropriate Ubuntu equivelent. AOL Instant Messenger accounts, for example, are translated into XML and added to the .gaim/accounts.xml file in that user's home directory. {{{
                           +-----------+
Import into: This computer | Change... |
                           +-----------+
}}}
Line 140: Line 74:
migration-assistant will merge data from multiple operating systems, or multiple applications that map to a single application in Ubuntu, into single entities in Ubuntu. For example, if the user has AOL IM, MSN IM, and Yahoo! IM installed, migration-assistant will import all of them into the .gaim/accounts.xml file so they can all be used in Gaim. If the user has both Firefox and Internet Explorer bookmarks then migration-assistant will import both of them into Firefox. == Test/Demo Plan ==
Line 142: Line 76:
The transferring of data is done by means of running ma-import, which copies the files and settings the user has selected to the new Ubuntu filesystem, after the LiveCD filesystem is copied. Testing migration-assistant will be done by running make check before upload, and by ensuring that all items imported successfully upon reboot when installing with either the LiveCD or alternate CD.
Line 144: Line 78:
'''Data:'''

If Ubuntu is being installed over a Linux system, the import function will select all of the home directories as well as the matching sections of /etc/passwd and /etc/shadow. However, if Ubuntu is being installed over Windows, the import tool will select each user from Documents and Settings, their My Documents folder, and what other files and settings it's preprogrammed to look for. As the development of the import function progresses, this should include:

'''Files:'''
 * Documents are imported into the default Ubuntu location (~/Documents).
 * Pictures are imported into the default Ubuntu location (~/Pictures, I think).
 * Music is imported into the default Ubuntu location and the appropriate gconf keys are set so Rhythmbox automatically finds and displays it when first run.
 * Email messages
 * Fonts

'''Settings:'''
 * User accounts
 * Email clients (Outlook, Thunderbird, Opera Mail, etc)
 * Instant messaging accounts (AOL, MSN, Yahoo!, ICQ, etc)
 * Bookmarks (Firefox, Opera, Internet Explorer, etc)
 * Desktop background
 * User picture (This will compliment the new face browser)
 * Internet connection settings (proxy, wireless access points, etc)
 * Address book		 The logging verbosity of migration-assistant will be increased for the course of the development cycle.
Line 167: Line 82:
What is the best way to enable migration-assistant to work when not setting up a dual boot environment?
  * This cannot be done by creating a temporary partition, as that is highly unreliable.
  * MatthewPaulThomas suggested:
   Future work: From an implementation point of view it's easiest to migrate documents and settings for someone who's dual-booting. But from a user's point of view, this is a lot less useful than migrating documents and settings for someone who won't have their Windows partition to read from. For these people, the installer should offer to copy their files to CDs, DVDs, or an external drive before blanking the Windows partition. -- MatthewPaulThomas
  * Ralf12, simple suggestion:
   When replacing an operating system, ''only transfer settings''. This can be kept in memory. For safety reasons, it might be wise, to store it (temporarily) on a internet-server. Or, a little bit simpler: ''email them''. Opening the email-attachment (My-Settings.Ubiquity) would launch an, 'would you like to import these settings?' dialog.
  * Ralf12, more complicated suggestion:
  Offer the removal of an other operating systems ''after'' installation. Someone can try Ubuntu out for a while, and when they are sure, they can select 'remove this operating system and used freed space for Ubuntu'. This can be part of a graphical grub configuration utility. Where you can select the operating systems offered at boot time and/or remove them. This way the dual-boot setup would be the recommended way of transferring data and settings.
  * Ralf12, a-lot-of-work future suggestion:
   Create a cross-platform utility that you can install on windows and linux. This program will create a ubuntu-live-dvd with all of your data and current settings. This you can take with you, and you can install it. In other words: unify the creation of a live-cd/dvd with backing up of (selected files) and settings. Off course this only works with relatively moderate amounts of data. It would also be a great personal ubuntu rescue/backup system. 		 * Should the desktop application allow the creation of user accounts, like the Ubiquity component does, by including /etc/passwd on the device or source that contains the migrated data?

== BoF agenda and discussion ==
{{{
Review the possibility of importing to other targets (FTP, WebDAV, USB key, sftp, CD-R, etc).
- Run before or after partman-apply.
Discuss the possibility of a desktop application.
- Do not encourage using for sync.
- Add text to say "you do not have to do this now, you can do this later"
Plan KDE UI and KDE application support. update https://wiki.kubuntu.org/KubuntuUbiquityMigrationAssistant
Dealing with /home.
- multual exclusivity by validation.
multi-process
try to get in debian
get into alternate cd
Checking if the data will fit in RAM, using a temporary directory.
choices-c
}}}
Line 180: Line 103:
  * Ralf12: Why would a temporary partition be unreliable? When transfering files and settings on a dual-boot, we are doing ''the very same thing''. We're resizing one partition, and creating another one. Why couldn't we after moving the files, remove the windows/linux partition(s) entirely? Resizing an ext3 partition is not more unreliable than resizing an ntfs partition. Yet ''this is offered by default'' and ''used by many''. The only problem could be the lack of enough free space. Which is why it is crucial we have some control over the directories and types of files we choose to transfer.
  * Ralf12: I honestly think the settings are way more important than the files: people know how to backup files. They can't import settings easily. Even with the risk of loosing the settings, at least try to import the settings when replacing an operating system, just by storing them in memory. It shouldn't be more than say 1-2 mb maximum, right? Just the account-settings, firefox/ie passwords, homepages and bookmarks.
  * Ralf12: Offer to move files based on file-type as well as directory. It would be best to make a distinction between an image and a photo, movie and a video, as well as, between a song and an album/radio-show based on length/size of the media.
  * Ralf12: Don't forget the shared folder. A lot of multi-user setups keep their music collection there.
  * Ralf12: Perhaps it would be nice to automatically convert the files to 'free formats' when possible. Violating patents to convert a file is protected under law in most, if not all, countries. Since it has to do with interoperability, being compatible with, instead of depending on, someone's patented ''invention''. It would also solve the easy-codec discussion. But even without media conversion, we can at least convert the MS Office documents and such.		  * What about migrating from one Ubuntu to another? Apple has an application helping migrating to new hardware. --AndersWallenquist
  * The intention of Ubuntu's general hardware support is that you should be able to pull out the hard drive, put it in another machine, and have it Just Work. Cases where this does not happen are bugs and should be filed as such. An application to take care of this case shouldn't be necessary. --ColinWatson
   * It's true Ubuntu has good hardware management, but when you buy a new machine you rarely keep your old hard-disk and waste the (faster and bigger) disk that comes with the new box. Use Case: Carls new Dell Laptop with Ubuntu preinstalled has just arrived and Carl wants to start using this in favor for his old HP stationary office computer migrateing his personal configaton using Wifi or USB in a mouseclick.
     * Wouldn't this fall under importing to other targets, as described above in the specification? --EvanDandrea
      * The problem is not the target, but the source. I agree, that other sources, namely Linux distributions, including Ubuntu, should be supported as well. Nowadays, where laptops become more common, simply switching the hard drive is not always an option. Of course, there are other tools (rsync etc), but an expanded migration assistant would really be a boon to users. At the same time I think that the spec should not be expanded, before importing from Windows is in a stable state. --Sokraates
        * By target I meant target to import from. So "source" by your definition. If you run migration-assistant in Feisty it will import from whatever other Linux distributions are installed, unless that distro uses a separate home directory, but fails to use a UUID to label it. It could not import the entire contents of /home/$USER in Feisty, but the solution to that is discussed in the above specification. Importing to a USB disk is also discussed. -- EvanDandrea
   * Working with Xen and virtual machines you often want to copy your personal configuations in a mouseclick without copying your whole home-catalog -- AndersWallenquist

----
CategorySpec

Summary

Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.

Further the development of migration-assistant to the point that it can be included in Feisty Fawn. Work prior to Feisty is covered in the /Edgy specification. Work for Feisty is covered in the /Feisty specification.

  • Support for Windows 2000 and Vista, as well as Mac OS X support will be added.
  • Support for migrating data to an external device or source, such as a USB key, for importing from using a desktop application after install, will be added.
  • migration-assistant will be enabled on the alternate CD installer.

Release Note

The migration functionality in the installer now allows users to export their migrated data to a USB thumb drive or other external source to later be merged into their Ubuntu system using the accompanied desktop application.

Rationale

The desktop application and ability to migrate data to a source other than the root filesystem are required for users who wish to migrate their data without keeping their old partitions around, and for those users who either did not have the ability to use migration-assistant in a previous release or chose not to, but wish to use it now.

Use Cases

  • Mark wants to migrate his data from Windows XP without having to keep his Windows XP partition around.
  • Evan wants to import his Apple OS X users when installing Ubuntu on his iMac.
  • Joe would like Ubuntu to be able to automatically authenticate to the various wireless access points he's associated with previously without requiring him to re-enter WEP and WPA keys.

Assumptions

None

Design

A stand-alone desktop application will be added that will enable the user to import from external sources used during the install for exporting migration data to, such as a USB key; FTP, sftp, or WebDAV site; or, possibly in the event of two CD recorder devices, a CD. Exporting to and importing from a USB key will be developed first, allowing other, less commonly used sources to be added as time permits.

Support for importing from Windows Vista, Mac OS X, and Windows 2000 will be added.

Additional import sources will be added. These may include:

  • Wireless access points with WPA and WEP keys.
  • Files on the desktop.
  • OS X applications, including Photo Booth pictures, Mail.app settings, Dashboard widgets (into Jackfield), iTunes music, and web browser bookmarks from Safari, Firefox, and Camino.
  • All instant messenger settings, including logs and passwords (if possible).
  • All PIM data from Outlook and Evolution including actual email, calendar data, and notes.
  • Mozilla Thunderbird.
  • Fonts.
  • Web browser data such as cookies, proxy settings, and cross-platform Firefox plugins.

Implementation

  • migration-assistant will be added to the alternate CD installer via a change of priority.

  • migration-assistant will have to be moved before partman-apply in debian-installer to accommodate exporting to a source other than the root filesystem, as the goal of that feature is to enable users who are electing to format their disk completely to still make sure of the migration functionality. This will require changes to ma-ask's partition state detection code.

  • migration-assistant will be modified to use choices-c to allow translating source names, such as "Internet Explorer", into the user's native language.
  • A progress dialog will be added to give the user more feedback on the detection process and allow them to cancel and skip the migration step, if they so desire.
  • Support for importing a user's entire /home directory will be added and a validation check will be created to ensure that the user does not select both the entire home directory and an item contained within it.

  • The structure of the migration data when exporting to an external device or source will be a migration-assistant directory in the root of the device or source, with /home below that, containing all the migrated files.

  • migration-assistant will need to be modified to detect and mount loopback filesystems created by Wubi to provide a migration path from the test installs created by Wubi as part of the installer-for-windows specification.

UI Changes

  • Minor changes to the wording of the text used to instruct the user may be made.
  • Additional text that lets the user know that they can wait until after install to migrate their documents and settings (using the desktop application) will be added.
  • The full name of Windows users and the computer name will be preseeded by first grabbing the data from SAM and the registry, respectively.
  • The following section will be added with an accompanying modal dialog similar to the "Connect to Server..." dialog used in GNOME, allowing the user to specify a source to export the migrated data to. 

                           +-----------+
Import into: This computer | Change... |
                           +-----------+

Test/Demo Plan

Testing migration-assistant will be done by running make check before upload, and by ensuring that all items imported successfully upon reboot when installing with either the LiveCD or alternate CD.

The logging verbosity of migration-assistant will be increased for the course of the development cycle.

Outstanding Issues

  • Should the desktop application allow the creation of user accounts, like the Ubiquity component does, by including /etc/passwd on the device or source that contains the migrated data?

BoF agenda and discussion

Review the possibility of importing to other targets (FTP, WebDAV, USB key, sftp, CD-R, etc).
- Run before or after partman-apply.
Discuss the possibility of a desktop application.
- Do not encourage using for sync.
-  Add text to say "you do not have to do this now, you can do this later"
Plan KDE UI and KDE application support. update https://wiki.kubuntu.org/KubuntuUbiquityMigrationAssistant
Dealing with /home.
- multual exclusivity by validation.
multi-process
try to get in debian
get into alternate cd
Checking if the data will fit in RAM, using a temporary directory.
choices-c

Comments

  • What about migrating from one Ubuntu to another? Apple has an application helping migrating to new hardware. --AndersWallenquist

    • The intention of Ubuntu's general hardware support is that you should be able to pull out the hard drive, put it in another machine, and have it Just Work. Cases where this does not happen are bugs and should be filed as such. An application to take care of this case shouldn't be necessary. --ColinWatson

      • It's true Ubuntu has good hardware management, but when you buy a new machine you rarely keep your old hard-disk and waste the (faster and bigger) disk that comes with the new box. Use Case: Carls new Dell Laptop with Ubuntu preinstalled has just arrived and Carl wants to start using this in favor for his old HP stationary office computer migrateing his personal configaton using Wifi or USB in a mouseclick.

        • Wouldn't this fall under importing to other targets, as described above in the specification? --EvanDandrea

          • The problem is not the target, but the source. I agree, that other sources, namely Linux distributions, including Ubuntu, should be supported as well. Nowadays, where laptops become more common, simply switching the hard drive is not always an option. Of course, there are other tools (rsync etc), but an expanded migration assistant would really be a boon to users. At the same time I think that the spec should not be expanded, before importing from Windows is in a stable state. --Sokraates

            • By target I meant target to import from. So "source" by your definition. If you run migration-assistant in Feisty it will import from whatever other Linux distributions are installed, unless that distro uses a separate home directory, but fails to use a UUID to label it. It could not import the entire contents of /home/$USER in Feisty, but the solution to that is discussed in the above specification. Importing to a USB disk is also discussed. -- EvanDandrea

      • Working with Xen and virtual machines you often want to copy your personal configuations in a mouseclick without copying your whole home-catalog -- AndersWallenquist

CategorySpec

MigrationAssistance (last edited 2008-08-06 16:19:54 by localhost)