AutomagicPythonBuildSystem

Differences between revisions 1 and 4 (spanning 3 versions)
Revision 1 as of 2009-05-23 18:43:17
Size: 2678
Editor: 86
Comment:
Revision 4 as of 2009-05-23 20:07:37
Size: 6159
Editor: 86
Comment:
Deletions are marked like this. Additions are marked like this.
Line 10: Line 10:
This should provide an overview of the issue/functionality/change proposed here. Focus here on what will actually be DONE, summarising that so that other people don't have to read the whole spec. See also CategorySpec for examples. In this blueprint we propose extending python-distutils-extra by a "do what I
mean" mode where it automatically chooses the right destination path and action
for various known file types such as Python modules, exectuable scripts, icons,
desktop files, etc. The goal is that the typical setup.py only has to
contain meta information like description, version, and author.

We also discuss Debian packaging generation.
Line 20: Line 26:
This should cover the _why_: why is this change being proposed, what justifies it, where we see this justified. One of Python's native and very popular build systems is "distutils" which
comes shipped with Python itself. It is fairly limited, though, and requires
the user to learn a lot about build systems. python-distutils-extra adds some
functionality for i18n, but there are still some standard use cases missing.

With our goal of supporting the opportunistic programmer and our pre-selection
of technologies, a lot of the build system design can be made implicit
("convention over configuration"), which will greatly reduce the amount of
learning necessary for installing a project and building a package.
Line 24: Line 38:
=== Jockey ===
The current Jockey project needs a very large and redundant
[[http://bazaar.launchpad.net/%7Ejockey-hackers/jockey/trunk/annotate/548/setup.py|setup.py]],
[[http://bazaar.launchpad.net/%7Ejockey-hackers/jockey/trunk/annotate/548/setup.cfg|setup.cfg]], and
[[http://bazaar.launchpad.net/%7Ejockey-hackers/jockey/trunk/annotate/548/setup.cfg|POTFILES.in]].

With the improved distutils-extras, `setup.cfg` and `POTFILES.in` can be dropped completely, and `setup.py` can be dropped to the metadata:

{{{
from DistUtilsExtra.auto import setup

setup(
    name='jockey',
    version='0.5.1',

    description='UI for managing third-party and non-free drivers',
    url='https://launchpad.net/jockey',
    license='GPL v2 or later',
    author='Martin Pitt',
    author_email='martin.pitt@ubuntu.com',
)
}}}
Line 25: Line 62:

 * Uses standard PyGTK and Python libraries and technologies.
 * Automatically generated Debian package produces single binary only. No library/public package building.
Line 28: Line 68:
You can have subsections that better describe specific parts of the issue. === Project structure ===

 * All directories with `__init__.py` are Python packages which are used and shipped
 * TBD: Scripts in `./bin/` ? `main.py`? ''projectname''`.py`?

=== setup.py ===

For a standard project which uses well-known file types and a standard
directory layout, `setup.py` should only contain metadata about the project
which cannot be inferred from the source code.

Required fields:

 * name
 * version
 * license
 * description
 * author

Optional fields:

 * author_email
 * url (homepage)

=== Debian packaging ===

 * fully automatically generated from scratch, working, Policy compliant
Line 32: Line 98:
This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like: === distutils-extra extensions ===
Line 34: Line 100:
=== UI Changes ===  * Compiling PyKDE .ui files to .py files with `pykdeuic4`
Line 36: Line 102:
Should cover changes required to the UI, or specific UI that is required to implement this === distutils-extra automatic setup.py ===
 
 * Python packages (directory with `__init__.py`)
Line 38: Line 106:
=== Code Changes ===  * D-Bus configuration (`*.conf` with magic string)
Line 40: Line 108:
Code changes should include an overview of what needs to change, and in some cases even the specific details.  * D-Bus service (`*.service` with magic string)
  * If it has an `User=` line → system service, otherwise session service
Line 42: Line 111:
=== Migration ===  * `.desktop.in`
Line 44: Line 113:
Include:
 * data migration, if any
 * redirects from old URLs to new ones, if any
 * how users will be pointed to the new way of doing things, if necessary.
 * `.policy.in`

 * `.notifyrc.in`

 * `*.po`, build `*.pot`: already provided by distutils-extras

 * Icons in `data/icons/size/category/*.{png,svg}`: already provided by distutils-extras

 * `po/POTFILES.in`

 * `cmdclass` (default to distutils-extra classes)

 * supplementary data files: `data/foo` → `/usr/share/`''project''`/foo`

 * scripts: TBD

=== Debian packaging ===

Based on the information in `setup.py` and the source code, we can also
generate a working Debian packaging.

TBD:
 * integration? `./setup.py debian` ?
 * into distutils-extras or quickly or ???
 * create if not existing/update if existing? (changelog/dependencies)

==== changelog ====

 * project name, version, author, email from setup.py
 * description: "new release"
 * distro target: needs argument from environment; default to `lsb_release -c`?

==== compat ====

constant (6 for hardy support)

==== rules =====

 * static, cdbs+python-distutils.mk

==== copyright ====
TBD

==== control ====

 * no dh_install, upstream build system DTRT
 * Source, Package, Maintainer, Description: from setup.py
 * Section, Priority, Standards-Version, XS-Python-Version: constant
 * Build-Depends: static (no tests), or binary depends (if test cases available)
 * Depends: grep all *.py for import statements, check which package provides them

Summary

In this blueprint we propose extending python-distutils-extra by a "do what I mean" mode where it automatically chooses the right destination path and action for various known file types such as Python modules, exectuable scripts, icons, desktop files, etc. The goal is that the typical setup.py only has to contain meta information like description, version, and author.

We also discuss Debian packaging generation.

Release Note

This section should include a paragraph describing the end-user impact of this change. It is meant to be included in the release notes of the first release in which it is implemented. (Not all of these will actually be included in the release notes, at the release manager's discretion; but writing them is a useful exercise.)

It is mandatory.

Rationale

One of Python's native and very popular build systems is "distutils" which comes shipped with Python itself. It is fairly limited, though, and requires the user to learn a lot about build systems. python-distutils-extra adds some functionality for i18n, but there are still some standard use cases missing.

With our goal of supporting the opportunistic programmer and our pre-selection of technologies, a lot of the build system design can be made implicit ("convention over configuration"), which will greatly reduce the amount of learning necessary for installing a project and building a package.

User stories

Jockey

The current Jockey project needs a very large and redundant setup.py, setup.cfg, and POTFILES.in.

With the improved distutils-extras, setup.cfg and POTFILES.in can be dropped completely, and setup.py can be dropped to the metadata:

from DistUtilsExtra.auto import setup

setup(
    name='jockey',
    version='0.5.1',

    description='UI for managing third-party and non-free drivers',
    url='https://launchpad.net/jockey',
    license='GPL v2 or later',
    author='Martin Pitt',
    author_email='martin.pitt@ubuntu.com',
)

Assumptions

  • Uses standard PyGTK and Python libraries and technologies.
  • Automatically generated Debian package produces single binary only. No library/public package building.

Design

Project structure

  • All directories with __init__.py are Python packages which are used and shipped

  • TBD: Scripts in ./bin/ ? main.py? projectname.py?

setup.py

For a standard project which uses well-known file types and a standard directory layout, setup.py should only contain metadata about the project which cannot be inferred from the source code.

Required fields:

  • name
  • version
  • license
  • description
  • author

Optional fields:

  • author_email
  • url (homepage)

Debian packaging

  • fully automatically generated from scratch, working, Policy compliant

Implementation

distutils-extra extensions

  • Compiling PyKDE .ui files to .py files with pykdeuic4

distutils-extra automatic setup.py

  • Python packages (directory with __init__.py)

  • D-Bus configuration (*.conf with magic string)

  • D-Bus service (*.service with magic string)

    • If it has an User= line → system service, otherwise session service

  • .desktop.in

  • .policy.in

  • .notifyrc.in

  • *.po, build *.pot: already provided by distutils-extras

  • Icons in data/icons/size/category/*.{png,svg}: already provided by distutils-extras

  • po/POTFILES.in

  • cmdclass (default to distutils-extra classes)

  • supplementary data files: data/foo/usr/share/project/foo

  • scripts: TBD

Debian packaging

Based on the information in setup.py and the source code, we can also generate a working Debian packaging.

TBD:

  • integration? ./setup.py debian ?

  • into distutils-extras or quickly or ???
  • create if not existing/update if existing? (changelog/dependencies)

changelog

  • project name, version, author, email from setup.py
  • description: "new release"
  • distro target: needs argument from environment; default to lsb_release -c?

compat

constant (6 for hardy support)

==== rules =====

  • static, cdbs+python-distutils.mk

TBD

control

  • no dh_install, upstream build system DTRT
  • Source, Package, Maintainer, Description: from setup.py
  • Section, Priority, Standards-Version, XS-Python-Version: constant
  • Build-Depends: static (no tests), or binary depends (if test cases available)
  • Depends: grep all *.py for import statements, check which package provides them

Test/Demo Plan

It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to http://testcases.qa.ubuntu.com/Coverage/NewFeatures for tracking test coverage.

This need not be added or completed until the specification is nearing beta.

Unresolved issues

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

BoF agenda and discussion

Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.


CategorySpec

DesktopTeam/Specs/Karmic/AutomagicPythonBuildSystem (last edited 2009-07-29 08:00:04 by pD9EB3788)