ReducingDuplication

Differences between revisions 6 and 7
Revision 6 as of 2005-11-02 19:28:23
Size: 5630
Editor: 209
Comment: de-hat-ify
Revision 7 as of 2005-11-02 19:51:58
Size: 5870
Editor: 209
Comment:
Deletions are marked like this. Additions are marked like this.
Line 146: Line 146:
grep "^Package: " /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_dapper_main_binary-i386_Packages | awk '{print $2}' | sort | sed -e "s/c2//" -e "s/c102//" -e "s/[-0-9.]//g" | sort | uniq -d

http://cthulhu.ubz/~adconrad/dupes.txt

Summary

We need to reduce library (and app) duplication in main as much as possible to make dapper's "supported for 5 years on the server" goal easier on all of us.

Rationale

Supporting multiple versions of similar codebases can be incredibly difficult and time-consuming. We should, rather, be concentrating on keeping ONE of everything (one libdb, one libmysqlclient, one libssl, one libpng, one set of mozilla libs, etc) in main, and punting the rest to universe (or oblivion)

Use cases

A security problem in libpcre requires an urgent update. Pitti does not want, nor have the time, to find and fix all the libpcre copies hidden in the Ubuntu sources, nor does he want to rebuild everything that links statically to libpcre.a.

A support company wants to commit to providing technical support for Dapper, but only wants to support packages that have security support. Jeff, the support person, needs to reduce the set of packages in main to avoid crying himself to sleep every night.

Scope

Design

Implementation

  • Identify duplicate libs, make sure everything in main links to the "best" version, and kick the rest out.
  • Audit packages for local copies of libs (static libz, libdb, libpng, imlib, libpcre, and libneon have all been common in the past), and get us linking dynamically to packaged libraries everywhere possible.

Code

Data preservation and migration

Outstanding issues

Packages which ship their own version of a library, as opposed to statically linking with the Ubuntu version, might have local patches which need to be audited.

BoF agenda and discussion

Static linking

zlib

  • dpkg (justified)
  • aide (justified)
  • ia32-libs (warty, hoary)
  • amd64-libs (warty, hoary)
  • rpm (in lsb-rpm, might be justified)

libpcre3

  • python2.{1,2,3} (hacked upstream, fixed in 2.4)
  • gnumeric (warty, hoary, breezy)
  • apache2 (warty)

modified copies of xpdf in source

  • cupsys (warty, hoary)
  • tetex-bin (not really justified probably)

libnspr4/libnss3

  • build from one source, get mozilla into universe

mozilla-dev

  • enigmail, librsvg2, openoffice.org2 all build-dep on mozilla-dev currently
  • librsvg2 works fine with firefox-dev; enigmail could require a package split (e-thunderbird and e-mozilla); for OO.o we should check whether it builds and works with firefox-dev, and if not, split out the three shared libraries in the mozilla-browser package into a mozilla-libs package, which would allow us to demote at least mozilla-browser to universe

Seed Justification

The seeds should be audited, as they've built up a lot of cruft. We should add a rationale for everything in the seeds. This isn't directly 'code duplication' but it's fairly obviously related.

(Even less orthogonally, 'extra' should be cleared up as we're missing a bunch of -doc and -dev packages for libraries in main.)

As a rule there should be no libraries listed in the seeds; they should all be pulled in as Depends or Build-Depends of seeded packages.

Multiple versions

libmysql*

  • libmysqlclient10 and libmysqlclient12 are going away completely.
  • If MySQL 5.0 proves stable, MySQL 4.1 and libmysqlclient14 will also go away
    • in favour of 5.0 and client15.

libdb*

gnutls/gcrypt

  • gnutls10 should go away (if it isn't already)
  • gnutls11 currently is necessary for openldap, needs an interface rewrite (old openssl compatibility code)
  • everything else should eventually use gnutls12 (which isn't in dapper yet ...)
  • gcrypt{!11} should go away if it hasn't already

Python2.*

Python2.3 is still required for Zope2, which is not certified with Python 2.4.

libgtk.*

1.2 still used by xmms, kicker-applets, and evms-gui

libmpeg1

gimp build-depends on it, but doesn't appear to USE it, so test dropping the build-dep

libmpeg3

The only thing using this is directfb, the only thing using that is libggi. Nothing is using libggi, it's only pulled in because it's directly seeded and shouldn't be.

libgd*

  • python-gdchart should be transitioned to libgd2

gnome1

Users:

  • evms-gui (can't be dropped without splitting source)
  • glife (Ogra says, "DROP IT, DO IT, POP THE TRUNK")
  • gnome-pilot (Pitti says, "Doesn't evolution do that these days?")
  • unixodbc (Adam says he'll fix)

Other easy candidates

Old versions of ncurses and readline, doko says easy.

Semi-automated pkg duplication finding

Simple and stupid checks like this, might be useful? e.g.

zgrep "^Package: " dists/dapper/main/binary-i386/Packages.gz | awk '{print $2}' | sort | sed -e "s/[0-9]//g" | sort | uniq -d

which gives us:

http://people.ubuntu.com/~james/tmp/pkg-duplication.txt

grep "^Package: " /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_dapper_main_binary-i386_Packages | awk '{print $2}' | sort | sed -e "s/c2//" -e "s/c102//" -e "s/[-0-9.]//g" | sort | uniq -d

http://cthulhu.ubz/~adconrad/dupes.txt

Germinatate Improvements

Colin Watson has kindly volunteered to improve germinate thusly:

  • If a package is in extra and matches:
    • ^*-dev$, ^*-doc$, ^*-dbg$
    They get promoted if the source package is going to main.

Adam Conrad would like germinate to support promotion of whole source packages, perhaps by prefixing the package name with '%', a la quinn diff.

ReducingDuplication (last edited 2008-08-06 16:41:21 by localhost)