Here are some proof-of-concept scripts for creating and applying binary diffs of debian packages for smaller downloads on upgrades. You need to install bsdiff for this to work. JamesHall

If debian packages were all recompressed using the script below (since dpkg currently uses zlib), its possible to create binary diffs between packages, preventing lengthy downloads when only a small update is needed. Here are some statistics:

These diffs are made between packages currently found in breezy and the latest dapper releases:

2805228 gimp_2.2.8-2ubuntu6_i386.deb
2781226 gimp_2.2.11-1ubuntu1_i386.deb
607138  gimp_2.2.8-2ubuntu6_2.2.11-1ubuntu1_i386.diff
Saved bandwidth: 78.2%

1061534 synaptic_0.57.4ubuntu10_i386.deb
1035092 synaptic_0.57.8ubuntu10_i386.deb
235820  synaptic_0.57.4ubuntu10_0.57.8ubuntu10_i386.diff
Saved bandwidth: 77.2%

476670 gedit_2.12.1-0ubuntu1_i386.deb
590960 gedit_2.14.2-0ubuntu3_i386.deb
379656 gedit_2.12.1-0ubuntu1_2.14.2-0ubuntu3_i386.diff
Saved bandwidth: 35.8%

861612 gedit-common_2.12.1-0ubuntu1_all.deb
958216 gedit-common_2.14.2-0ubuntu3_all.deb
379656 gedit_2.12.1-0ubuntu1_2.14.2-0ubuntu3_i386.diff
Saved bandwidth: 60.3%

2508272 abiword_2.4.1-1ubuntu1_i386.deb
2510884 abiword_2.4.2-0ubuntu5_i386.deb
409418  abiword_2.4.1-1ubuntu1_2.4.2-0ubuntu5_i386.diff
Saved bandwidth: 83.7%

Recompress deb script

This will recompress debian packages using proper versions of gzip and ar. (required to make checksums match)

echo "Usage: recompress-deb input output"
mkdir ./working

cd ./working
ar xo ../$1
gunzip -fN control.tar.gz
gunzip -fN data.tar.gz

gzip control.tar
gzip data.tar

ar r new.deb debian-binary control.tar.gz data.tar.gz 
sed 's/data.tar.gz\//data.tar.gz /' new.deb > new1.deb
sed 's/control.tar.gz\//control.tar.gz /' new1.deb > new2.deb
sed 's/binary\//binary /' new2.deb > new3.deb

cd ../

mv ./working/new3.deb $2
rm -fr working/

diffdeb script

echo "Usage: diffdeb old new diff"

mkdir ./working
mkdir ./working/old
mkdir ./working/new

# Extract old deb (ar)
cd ./working/old
ar xo ../../$1

cd ../../

# Extract new deb (ar)
cd ./working/new
ar xo ../../$2

cd ../../

# Extract old deb contents (gzip)
cd ./working/old
gunzip -fN control.tar.gz
gunzip -fN data.tar.gz

cd ../../

# Extract new deb contents (gzip)
cd ./working/new
gunzip -fN control.tar.gz
gunzip -fN data.tar.gz

cd ../../

# Make each into a tar
cd ./working/old
tar cf old.tar *

cd ../../

cd ./working/new
tar cf new.tar *

cd ../../
# Make binary diff
bsdiff ./working/old/old.tar ./working/new/new.tar $3

# Force recursive removal of folder 'working'
rm -fr working/

patchdeb script

echo "Usage: patchdeb olddeb newdeb patch"

mkdir ./working
mkdir ./working/old
mkdir ./working/new

# Extract old deb (ar)
cd ./working/old
ar xo ../../$1

# Extract old deb contents (gzip)
gunzip -fN control.tar.gz
gunzip -fN data.tar.gz

# Make into a tar
tar cf old.tar *
cd ../../

# Patch it
bspatch ./working/old/old.tar ./working/new/new.tar $3

cd ./working/new
tar xf new.tar
rm new.tar

cd ../../
cd ./working/old
rm old.tar

cd ../../

# Recreate proper debian ar's
cd ./working/new
gzip control.tar
gzip data.tar
ar r new.deb debian-binary control.tar.gz data.tar.gz 
sed 's/data.tar.gz\//data.tar.gz /' new.deb > new1.deb
sed 's/control.tar.gz\//control.tar.gz /' new1.deb > new2.deb
sed 's/binary\//binary /' new2.deb > new3.deb

cd ../../
mv ./working/new/new3.deb $2


# Force recursive removal of folder 'working'
rm -fr working/

Comments