FoundationsQPythonVersions

Differences between revisions 1 and 19 (spanning 18 versions)
⇤ ← Revision 1 as of 2011-11-09 19:49:06 →
Size: 2135
Editor: mail
Comment:
← Revision 19 as of 2012-04-25 20:30:18 → ⇥
Size: 5523
Editor: mail
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= Shipping only Python 3 on the 12.04 LTS CD = ## page was copied from Python/FoundationsPPythonVersions
= Shipping only Python 3 on the 12.10 CD =
Line 3: Line 4:
Okay, that's pretty ambitious, and it's not going to happen for 12.04 LTS. This is basically work we've tried to do for several cycles leading up to 12.04, and our goal is now the more modest one of porting at least one desktop application, including its full dependency stack, to Python 3 for inclusion on the CD images. We'll claim some space for Python 3 that way (mitigated by the removal of Python 2.6 from 12.04), and we'll learn a lot about the porting process, i.e. what don't we know? For many cycles Ubuntu developers have been working on a migration to Python 3. This has taken modest steps leading up to 12.04, with many packages (primarily libraries) getting Python 3 versions upstream, into Debian, and into Ubuntu. Python 3 is not distributed on the Ubuntu 12.04 CD images though, but major milestones have been reached, such as the porting of dbus-python to Python 3.2.

'''It is a release goal for Ubuntu 12.10 (Q cycle) that we ship only Python 3 on the desktop CD images.''' Don't worry, Python 2.7 will still be available in the archive, and more specifically 'main' for now. But we want all libraries and applications on the desktop CD (or more specifically, those packages with '''Task: ubuntu-desktop''' (also ''minimal'' and ''standard'') to be running on Python 3.
Line 7: Line 10:
The [[https://blueprints.launchpad.net/ubuntu/+spec/foundations-p-python-versions|blueprint]] contains the work items and some additional details. This wiki page will be used to further specific the task for 12.04. The [[https://blueprints.launchpad.net/ubuntu/+spec/foundations-q-python-versions|Q-series blueprint]] exists for planning the UDS-Q session, holding work items, etc.

This [[https://docs.google.com/a/canonical.com/spreadsheet/ccc?key=0AiT4gOXSkmapdFA1anRkWERsaXgtWnllUG9QWXhDVWc#gid=0|Google document]] captures the set of packages and leaf applications that still need packaging or porting as of Ubuntu 12.04. This list was generated with the {{{oncd.py}}} script from [[https://code.launchpad.net/~barry/+junk/pydeps|this branch]]. '''WE NEED [[https://wiki.ubuntu.com/Python/3|YOUR HELP]]!'''. Some packages have Python 3 support upstream but are not yet packages for Ubuntu. Others do not yet have Python 3 support. In those cases we must either help the upstream to land Python 3 support or rewrite our applications to not require the unported dependency.

Spreadsheet legend:
 * white background - status unknown or Python 3 version currently unavailable
 * yellow background - Python available (or nearly so) in upstream and/or Debian. May need Debian packaging/update, or sync to Ubuntu
 * blue - Canonical/Ubuntu is the upstream. This doesn't mean you can't or shouldn't help port it, just that we know who to lean on at crunch time. :)
 * green background - Python 3 version now available in Ubuntu!
 * red background - Python 3 port likely impossible due to upstream orphan, resistance, or infeasibility

(See the Status column of each package for additional details.)

== Plan ==

The plan is to identify a handful of desktop applications included on the default install, follow their dependency stack, and investigate whether their upstreams already support Python 3 (either in a released version, a development branch, or fork). For those which have no upstream support yet, we have several choices:

 * Find an alternative that does already support Python 3
 * Encourage and help upstream to do the port
 * Port it ourselves and contribute to upstream
 * Fork or rewrite from scratch

Candidates for Python 3 porting:
 * Software Center (LP: #823254)
 * Update Manager
 * Jockey
 * gwibber backend
 * Computer Janitor

There is '''no''' plan to make Python 3 the default {{{/usr/bin/python}}}. See [[http://www.python.org/dev/peps/pep-0394/|for upstream policy on this]].

== Known blockers and issues ==

 * If we're to carry multiple versions of Python 3 (e.g. 3.2 and 3.3), how does dh_python3 handle situations where modules don't support all supported Python 3 versions? This is unsolved, although often discussed in debian-python.
 * Namespaces (not PEP 382 or 402 though; Eric Smith's as yet unreleased new PEP)
 * xapian
 * gstreamer

'' '''Warning:''' Ubuntu Software Center currently plans to stay on Python 2.7. dbus and xapian are the main blockers to moving to Python 3. Talk to Michael Vogt for more details. —mpt''

== Resources ==

 * [[https://docs.google.com/a/canonical.com/spreadsheet/ccc?key=0AiT4gOXSkmapdFA1anRkWERsaXgtWnllUG9QWXhDVWc#gid=0|Ubuntu porting status page]] '''-- START HERE'''
 * http://python3wos.appspot.com/
 * There were discussions with Fedora at !PyCon 2011
 * [[AllisonRandal/Python3Status|Allison Randal's Python 3 status page]]
 * [[http://getpython3.net|Get Python 3 website]]
 * [[http://python3porting.com/|Python 3 Porting Guide]]

== Python 3 packaging ==

Python3 packaging is covered in this [[http://wiki.debian.org/Python/LibraryStyleGuide|packaging best practices guide]]. However {{{dh_auto-*}}} doesn't yet support Python 3. There's was a Debian GSoC project to write a new tool before fixing {{{dh}}} (python-multibuild tool to be later integrated). It needs work (Piotr in debian-python is leading this now). Until this lands, we should just continue using overrides in the meantime.
Distutils packages can override these easily, however other packages may be non-trivial.
Line 13: Line 68:
* https://blueprints.launchpad.net/ubuntu/+spec/foundations-m-python-2.7
* https://blueprints.launchpad.net/ubuntu/+spec/foundations-m-python-3		  * https://blueprints.launchpad.net/ubuntu/+spec/foundations-q-python-versions
 * https://blueprints.launchpad.net/ubuntu/+spec/foundations-p-python-versions
 * https://blueprints.launchpad.net/ubuntu/+spec/foundations-m-python-2.7
 * https://blueprints.launchpad.net/ubuntu/+spec/foundations-m-python-3
Line 18: Line 75:
https://wiki.ubuntu.com/MaverickMeerkat/TechnicalOverview/Python
Line 22: Line 77:

Work items (oneiric):
[barry] Ensure that 3.1 is gone: DONE
[barry] look at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=622279 for blockers which we could apply resources to to help unblock: DONE
[allison] take transition discussion to DEX: DONE
[barry] research python3 disk usage for CDs. (Size estimate of python3 binary plus duplicates of all current python 2 packages, or at least the ones that have been ported to 3): DONE
[kate.stewart] at the june rally: reassess adding Python3 or python3 only on the CD: DONE

Since 2.6 will not be in 12.04, the Launchpad team wants a 10.04 PPA with Python 2.7 in it so that they can transition to 2.7 before transitioning to 12.04. This is available here: https://launchpad.net/~pythoneers/+archive/lts

Shipping only Python 3 on the 12.10 CD

For many cycles Ubuntu developers have been working on a migration to Python 3. This has taken modest steps leading up to 12.04, with many packages (primarily libraries) getting Python 3 versions upstream, into Debian, and into Ubuntu. Python 3 is not distributed on the Ubuntu 12.04 CD images though, but major milestones have been reached, such as the porting of dbus-python to Python 3.2.

It is a release goal for Ubuntu 12.10 (Q cycle) that we ship only Python 3 on the desktop CD images. Don't worry, Python 2.7 will still be available in the archive, and more specifically 'main' for now. But we want all libraries and applications on the desktop CD (or more specifically, those packages with Task: ubuntu-desktop (also minimal and standard) to be running on Python 3.

The ultimate goal is really to relegate Python 2 to universe for 14.04 LTS.

The Q-series blueprint exists for planning the UDS-Q session, holding work items, etc.

This Google document captures the set of packages and leaf applications that still need packaging or porting as of Ubuntu 12.04. This list was generated with the oncd.py script from this branch. WE NEED YOUR HELP!. Some packages have Python 3 support upstream but are not yet packages for Ubuntu. Others do not yet have Python 3 support. In those cases we must either help the upstream to land Python 3 support or rewrite our applications to not require the unported dependency.

Spreadsheet legend:

  • white background - status unknown or Python 3 version currently unavailable
  • yellow background - Python available (or nearly so) in upstream and/or Debian. May need Debian packaging/update, or sync to Ubuntu

  • blue - Canonical/Ubuntu is the upstream. This doesn't mean you can't or shouldn't help port it, just that we know who to lean on at crunch time. Smile :)

  • green background - Python 3 version now available in Ubuntu!
  • red background - Python 3 port likely impossible due to upstream orphan, resistance, or infeasibility

(See the Status column of each package for additional details.)

Plan

The plan is to identify a handful of desktop applications included on the default install, follow their dependency stack, and investigate whether their upstreams already support Python 3 (either in a released version, a development branch, or fork). For those which have no upstream support yet, we have several choices:

  • Find an alternative that does already support Python 3
  • Encourage and help upstream to do the port
  • Port it ourselves and contribute to upstream
  • Fork or rewrite from scratch

Candidates for Python 3 porting:

  • Software Center (LP: #823254)
  • Update Manager
  • Jockey
  • gwibber backend
  • Computer Janitor

There is no plan to make Python 3 the default /usr/bin/python. See for upstream policy on this.

Known blockers and issues

  • If we're to carry multiple versions of Python 3 (e.g. 3.2 and 3.3), how does dh_python3 handle situations where modules don't support all supported Python 3 versions? This is unsolved, although often discussed in debian-python.
  • Namespaces (not PEP 382 or 402 though; Eric Smith's as yet unreleased new PEP)
  • xapian
  • gstreamer

Warning: Ubuntu Software Center currently plans to stay on Python 2.7. dbus and xapian are the main blockers to moving to Python 3. Talk to Michael Vogt for more details. —mpt

Resources

Python 3 packaging

Python3 packaging is covered in this packaging best practices guide. However dh_auto-* doesn't yet support Python 3. There's was a Debian GSoC project to write a new tool before fixing dh (python-multibuild tool to be later integrated). It needs work (Piotr in debian-python is leading this now). Until this lands, we should just continue using overrides in the meantime. Distutils packages can override these easily, however other packages may be non-trivial.

Historical reference

As mentioned, we've actually been working on this spec since the Maverick days. Here is some historical information from the whiteboard:

See also this wiki page for detailed background:

Full UDS-O session notes: http://pad.ubuntu.com/uds-o-foundations-o-python-versions

Python/FoundationsQPythonVersions (last edited 2012-06-01 15:44:50 by mail)