Shipping only Python 3 on the 14.04 CD
For many cycles Ubuntu developers have been working on a migration to Python 3. This will continue as we strive to remove Python 2 from various images, in decreasing order of priority:
- Ubuntu Touch
- Ubuntu Server
- Ubuntu Desktop
It is a release goal for Ubuntu 14.04 (Trusty Tahr) that we ship only Python 3 on the touch, server, and desktop images. Don't worry, Python 2.7 will still be available in the archive, and more specifically 'main' for now. But we want all libraries and applications to be running on Python (specifically, those packages with Task: ubuntu-desktop (also minimal, standard, and server - there is no task for Touch images).
The T-series blueprint exists for planning the vUDS 1310 session, holding work items, etc.
This Google document captures the set of packages and leaf applications that still need packaging or porting as of Ubuntu 14.04. This list was generated with the oncd.py script from this branch. WE NEED YOUR HELP!. Some packages have Python 3 support upstream but are not yet packages for Ubuntu. Others do not yet have Python 3 support. In those cases we must either help the upstream to land Python 3 support or rewrite our applications to not require the unported dependency.
Spreadsheet legend:
- white background - status unknown or Python 3 version currently unavailable
- yellow background - Python 3 version available (or nearly so) in upstream and/or Debian. May need Debian packaging/update, or sync to Ubuntu
blue - Canonical/Ubuntu is the upstream. This doesn't mean you can't or shouldn't help port it, just that we know who to lean on at crunch time.
- green background - Python 3 version now available in Ubuntu!
- dark gray - Will likely go away in 14.04
- red background - Python 3 port likely impossible due to upstream orphan, resistance, or infeasibility
(See the Status column of each package for additional details.)
Rationale
Python 2 is EOL upstream. Python 2.7 is the last Python 2 that will ever be released. There will never be a Python 2.8.
- While Python 2.7 is officially supported upstream for bug fixes until 2015, and probably longer for security fixes, in reality, the enthusiasm will wane considerably before then for upstream to continue to maintain Python 2.7 when all the good stuff is happening in Python 3.
- We don't want to have to actively support Python 2.7 after the next LTS.
- Porting to Python 3 is a multi-cycle, disruptive, change. Best to do this now so that we have plenty of time to shake out problems before the next LTS.
- If we wait to switch to Python 3, the costs do not magically go away. A few more packages may be ported by then, but by driving this effort now, we can improve the overall momentum to Python 3, speeding up the arrival of the tipping point.
- Doing ports now allows us to develop strategies for success, both from a language POV and a platform POV, which we can communicate to others, multiplying the positive effect and thus quality of all libraries undertaking the port.
Driving the ports ourselves allows us to identify issues in Python that uniquely affect the Debian/Ubuntu platform. We thus have the opportunity to effect changes in upstream Python 3 (since it can't happen in Python 2) that improve the robustness of our Python platform (e.g. PEP 3147, PEP 3149, and PEP 420.
Moving to Python 3 allows us to finally deal with the pain of Unicode errors in non-English locales in a much more principled way (rather than sheer luck and liberal cargo cult sprinkling of try/excepts
) - Gives us an opportunity to clean up tons of technical debt incurred in code we've written as old as Python 2.3.
- We can take advantage of all the new features and modules in Python 3, where all the new development is happening.
Plan
The plan is to identify a handful of desktop applications included on the default install, follow their dependency stack, and investigate whether their upstreams already support Python 3 (either in a released version, a development branch, or fork). For those which have no upstream support yet, we have several choices:
- Find an alternative that does already support Python 3
- Encourage and help upstream to do the port
- Port it ourselves and contribute to upstream
- Fork or rewrite from scratch
There is no plan to make Python 3 the default /usr/bin/python. See for upstream policy on this.
Side Effects
Even though this task is ostensibly to port only those packages on the various official images, it will have some effects on other flavors and installations. For example, once Ubiquity and Update Manager are ported, all flavors will have to ship the Python 3 version, meaning they will probably end up shipping both Python 2 and 3 (consuming some additional disk space).
Also, there is an impact on optional software. A good example are the Suggests for duplicity, which add some optional behavior. These packages (e.g. python-boto, python-cloudfiles, python-paramiko) are not seeded on the desktop CD, so aren't in the critical path for porting, but not having them would reduce the functionality of the parent tool. In these cases, we would still like to have Python 3 ports, and in the spreadsheet above, you'll see them listed under "Good to Have".
It is indeed true that this means /usr/bin/python will not exist on a fresh install from desktop CD until it's installed explicitly, or implicitly by way of an unported dependency. /usr/bin/python3 will exist though.
Known blockers and issues
- If we're to carry multiple versions of Python 3 (e.g. 3.2 and 3.3), how does dh_python3 handle situations where modules don't support all supported Python 3 versions? This is unsolved, although often discussed in debian-python.
- xapian
Warning: Ubuntu Software Center currently plans to stay on Python 2.7. Xapian is the main blockers to moving to Python 3. Upstream Xapian has an experimental port not yet merged to their trunk. Software Center may ultimately go away anyway.
Ubuntu porting status page -- START HERE There were discussions with Fedora at PyCon 2011 #python3 on Freenode The python-porting mailing list on python.org.
Python3 packaging is covered in this packaging best practices guide. Use pybuild for best results.
As mentioned, we've actually been working on this spec since the Maverick days. Here is some historical information from the whiteboard: https://blueprints.launchpad.net/ubuntu/+spec/foundations-q-python-versions https://blueprints.launchpad.net/ubuntu/+spec/foundations-p-python-versions https://blueprints.launchpad.net/ubuntu/+spec/foundations-m-python-2.7 https://blueprints.launchpad.net/ubuntu/+spec/foundations-m-python-3 See also this wiki page for detailed background: Full UDS-O session notes: http://pad.ubuntu.com/uds-o-foundations-o-python-versions Resources
Python 3 packaging
Historical reference