ChangingTheOrigTarball

Differences between revisions 2 and 8 (spanning 6 versions)
Revision 2 as of 2006-01-20 15:07:18
Size: 3466
Editor: DSL01
Comment: add CategoryMOTU
Revision 8 as of 2007-05-22 11:47:43
Size: 4806
Editor: p1033-ipbf37marunouchi
Comment: Added get-orig-source examples for watch files and cvs imports
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
## page was renamed from CommonPackagingMistakes/ChangingTheOrigTarball
Line 17: Line 18:
If you use svn (svn-buildpackage) (I guess that counts for other VCS as well, although I don't have experience there yet) to handle your debian package, you usually don't store the orig-tarball inside. If s.o. else will do a checkout, he'll need to get the orig-tarball seperately... (see point 1 from here on). If you use svn (svn-buildpackage) (I guess that counts for other VCS as well, although I don't have experience there yet) to handle your debian package, you usually don't store the orig-tarball inside. Another person doing a checkout will need to get the orig-tarball seperately... (see point 1 from here on).
Line 21: Line 22:
Consider a situation, where s.o. *wants* to introduce a backdoor/rootkit or other evil stuff. If the orig-tarball is intact, I can easily scan through the .diff.gz and see if the one who debianized the package tries to do s.th. evil. If the orig-tarball is changed however, I also need to check the differences between the tarball and the really original source if I want to check if the one who debianized the software introduces evil stuff. (note that you still have to trust upstream for not doing evil things here, but this is valid whether the orig is unchanged or not). Consider a situation, where someone *wants* to introduce a backdoor/rootkit or other evil stuff. If the orig-tarball is intact, one can easily scan through the .diff.gz and see if the one who debianized the package tries to do something evil. If the orig-tarball is changed however, one also needs to check the differences between the tarball and the really original source in order to check if the one who debianized the software introduced evil stuff. (note that you still have to trust upstream to not do evil things here, but this is valid whether the orig is unchanged or not).
Line 48: Line 49:
    * Do not repackage. Ask upstream to delete the debian/ dir and provide a diff.gz instead. This makes it easier to review his upstream work, his packaging work and your packaging work.     * Do not repackage. Ask upstream to delete the debian/ dir and provide a diff.gz instead. This makes it easier to review upstream development, upstream packaging work and your packaging work.
Line 57: Line 58:
    * FIXME can a watch file handle this?       Example:
{{{
get-orig-source:
        cd ..; wget http://somesite.org/stuff/somesoftware-4.2.tar.bz2
        bzcat ../somesoftware-4.2.tar.bz2 | gzip --best -c - > somesoftware-4.2.tar.gz
        ln -s ../somesoftware-4.2.tar.gz ../somesoftware_4.2.orig.tar.gz
Line 59: Line 65:
  * directly imported from svn
    * provide get-orig-source in debian/rules
    * FIXME watch file?
}}} (and maybe also provide the rule {{{ ../somesoftware_4.2.orig.tar.gz: get-orig-source }}}, or list get-orig-source within .PHONY).

  * if you use a watch file, this can be:
{{{
VER:=4.2

get-orig-source:
        -uscan --force-download
        bzcat ../somesoftware-$(VER).tar.bz2 | gzip --best -c - \
                > somesoftware-$(VER).tar.gz
        ln -s ../somesoftware-$(VER).tar.gz ../somesoftware_$(VER).orig.tar.gz
}}}

  * directly imported from cvs

{{{
CVSDATE+=22 May 2007
SW_VERSION+=4.2

TARFILE+=somesoftware_$(SW_VERSION)~cvs$(shell date -d "$(CVSDATE)" +%Y%m%d).orig.tar.gz
CVSHOME+=:pserver:anonymous@somesoftware.cvs.sourceforge.net:/cvsroot/somesoftware

get-orig-source::
        cvs -d$(CVSHOME) login
        cvs -d$(CVSHOME) export -D "$(CVSDATE)" somesoftware
        tar czf $(CURDIR)/../$(TARFILE) $(CURDIR)/somesoftware
        rm -rf $(CURDIR)/somesoftware

../$(TARFILE):: get-orig-source
}}}

 * Always remember to create debian/README.Debian-source when you need to repack the orig.tar.gz, explaining why you repacked it, and how others can verify your work.

P.S.:
It's always a good idea to contact upstream and ask that stuff like autoconf-issues or directory layout (or old FSF-adress) or other things you need to "patch" afterwards in .diff.gz be corrected.
Line 64: Line 102:
P.S.:
It's always a good idea to contact upstream and ask him if he can correct
stuff like autoconf-issues or directory layout (or old FSF-adress) or other
things, you need to "patch" afterwards in .diff.gz.
----

Problems, that occur when changing the orig-tarball

1) reproducability

If you take just the .diff.gz and .dsc, you or someone else has no means to reproduce the changes in the orig-tarball.

2) upgradeability

No easy way to upgrade to a new upstream version.

3) debian <-> ubuntu

Differing orig-tarballs make it hard to automatically sync from debian to ubuntu.

4) Usage of VCS for debian package

If you use svn (svn-buildpackage) (I guess that counts for other VCS as well, although I don't have experience there yet) to handle your debian package, you usually don't store the orig-tarball inside. Another person doing a checkout will need to get the orig-tarball seperately... (see point 1 from here on).

5) security tracking

Consider a situation, where someone *wants* to introduce a backdoor/rootkit or other evil stuff. If the orig-tarball is intact, one can easily scan through the .diff.gz and see if the one who debianized the package tries to do something evil. If the orig-tarball is changed however, one also needs to check the differences between the tarball and the really original source in order to check if the one who debianized the software introduced evil stuff. (note that you still have to trust upstream to not do evil things here, but this is valid whether the orig is unchanged or not).

6) You already have the option to use the .diff.gz to reflect changes to the orig-tarball.

Under what circumstances can the orig-tarball be changed and when it shouldn't be

Allowed

  • upstream tarball contains (non-free) stuff, that cannot be redistributed -> remove the portions of non-free stuff

Not allowed

  • directory layout wrong -> dpkg-source is quite flexible with this and manages to produce the correct directory layout even if

    • the directory inside the tarball is not named <upstream>-<version>

    • there is no subdirectory inside the tarball
  • files that need to be removed to keep the .diff.gz small (e.g. autotools-stuff):
    • everything, that needs to be deleted, should be done in the clean rule. Since the .diff.gz is created with diff -u, you'll not see removed files in .diff.gz
  • files to be modified
    • always need to go into .diff.gz. That's the purpose of it Wink ;)

  • wrong permissions on files
    • These won't be represented in .diff.gz. However you have the means to change this in debian/rules. <evilcynicalmode> If you don't know how to do it with cdbs, use debhelper!</evilcynicalmode>

  • What to do with .orig.tar.gz which already include a debian/ dir?
    • Do not repackage. Ask upstream to delete the debian/ dir and provide a diff.gz instead. This makes it easier to review upstream development, upstream packaging work and your packaging work.

Kind of allowed

  • upstream provides only bzip2
    • just do bunzip2 on the .tar.bz2 and gzip -9 on the tar.
    • the md5sums of the .tar you provide and the original .tar must match!

    • eventually provide get-orig-source in debian/rules, that does this converting
      • Example:

get-orig-source:
        cd ..; wget http://somesite.org/stuff/somesoftware-4.2.tar.bz2
        bzcat ../somesoftware-4.2.tar.bz2 | gzip --best -c - > somesoftware-4.2.tar.gz
        ln -s ../somesoftware-4.2.tar.gz ../somesoftware_4.2.orig.tar.gz

(and maybe also provide the rule  ../somesoftware_4.2.orig.tar.gz: get-orig-source , or list get-orig-source within .PHONY).

  • if you use a watch file, this can be:

VER:=4.2

get-orig-source:
        -uscan --force-download
        bzcat ../somesoftware-$(VER).tar.bz2 | gzip --best -c - \
                > somesoftware-$(VER).tar.gz
        ln -s ../somesoftware-$(VER).tar.gz ../somesoftware_$(VER).orig.tar.gz
  • directly imported from cvs

CVSDATE+=22 May 2007
SW_VERSION+=4.2

TARFILE+=somesoftware_$(SW_VERSION)~cvs$(shell date -d "$(CVSDATE)" +%Y%m%d).orig.tar.gz
CVSHOME+=:pserver:anonymous@somesoftware.cvs.sourceforge.net:/cvsroot/somesoftware

get-orig-source::
        cvs -d$(CVSHOME) login
        cvs -d$(CVSHOME) export -D "$(CVSDATE)" somesoftware
        tar czf $(CURDIR)/../$(TARFILE) $(CURDIR)/somesoftware
        rm -rf $(CURDIR)/somesoftware

../$(TARFILE):: get-orig-source
  • Always remember to create debian/README.Debian-source when you need to repack the orig.tar.gz, explaining why you repacked it, and how others can verify your work.

P.S.: It's always a good idea to contact upstream and ask that stuff like autoconf-issues or directory layout (or old FSF-adress) or other things you need to "patch" afterwards in .diff.gz be corrected.


[:CategoryMOTU]

MOTU/Packages/CommonPackagingMistakes/ChangingTheOrigTarball (last edited 2008-08-06 16:30:52 by localhost)