Scripts

Differences between revisions 1 and 15 (spanning 14 versions)
⇤ ← Revision 1 as of 2009-09-09 12:13:47 →
Size: 314
Editor: gw
Comment:
← Revision 15 as of 2010-06-02 21:49:05 → ⇥
Size: 3207
Editor: eth0
Comment: Break things.
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
To be able to create a mirror people can use and trust, you need to have all the files and at the right moment. 'File not found'-errors cause a lot of issues and annoyances for users. This page provides you with scripts to sync from other mirrors and prevent 404's. ||<tablestyle="float:right; width:30%; background:#F1F1ED; margin: 0 0 1em 1em;" style="padding:0.5em;">'''Contents'''||
||<<TableOfContents(2)>>||
To be able to create a mirror people can rely on, you need to have all the files and you need them at the right moment. 'File not found'-errors cause a lot of issues and annoyances for users. This page provides you with scripts to sync from other mirrors and prevent 404's.

'''Please be sure to use a local mirror

= ubumirror =

One can use the scripts from the '''[[https://code.launchpad.net/~ubumirror-devs/ubumirror/trunk|ubumirror]]''' project to keep their mirror in sync, or use the scripts provided below.

This project has been [[https://launchpad.net/ubuntu/+source/ubumirror|packaged since Lucid]] onwards, please file any bug reports you find [[https://bugs.launchpad.net/ubuntu/+source/ubumirror|against it]].
Line 5: Line 15:
For archive mirrors, it is very important '''not''' to delete packages before the Packages.gz-files (which hold information about the packages available) are updated. Therefor, you need a 'Two stage sync'. This means that you download new packages first, and new Packages.gz after that. After you've downloaded the Packages.gz files, it's safe to delete old packages.

== The script ==

{{{
#/bin/dash

fatal() {
  echo "$1"
  exit 1
}

warn() {
  echo "$1"
}

# Find a source mirror near you which supports rsync on
# https://launchpad.net/ubuntu/+archivemirrors
# rsync://<iso-country-code>.rsync.archive.ubuntu.com/ubuntu should always work
RSYNCSOURCE=rsync://archive.ubuntu.mirror.isp.com/ubuntu

# Define where you want the mirror-data to be on your mirror
BASEDIR=/var/www/ubuntuarchive/

if [ ! -d ${BASEDIR} ]; then
  warn "${BASEDIR} does not exist yet, trying to create it..."
  mkdir -p ${BASEDIR} || fatal "Creation of ${BASEDIR} failed."
fi

rsync --recursive --times --links --hard-links \
  --stats \
  --exclude "Packages*" --exclude "Sources*" \
  --exclude "Release*" \
  ${RSYNCSOURCE} ${BASEDIR} || fatal "First stage of sync failed."

rsync --recursive --times --links --hard-links \
  --stats --delete --delete-after \
  ${RSYNCSOURCE} ${BASEDIR} || fatal "Second stage of sync failed."
}}}
Line 6: Line 56:

For Releases mirrors, stuff is a little less complicated. There are no dependencies between files, so you can just rsync away.

== The script ==

{{{
#/bin/dash

fatal() {
  echo "$1"
  exit 1
}

warn() {
  echo "$1"
}

# Find a source mirror near you which supports rsync on
# https://launchpad.net/ubuntu/+cdmirrors
# rsync://<iso-country-code>.rsync.releases.ubuntu.com/releases should always work
RSYNCSOURCE=rsync://releases.ubuntu.mirror.isp.com/releases

# Define where you want the mirror-data to be on your mirror
BASEDIR=/var/www/ubuntureleases/

if [ ! -d ${BASEDIR} ]; then
  warn "${BASEDIR} does not exist yet, trying to create it..."
  mkdir -p ${BASEDIR} || fatal "Creation of ${BASEDIR} failed."
fi

rsync --verbose --recursive --times --links --hard-links \
  --stats --delete-after \
  ${RSYNCSOURCE} ${BASEDIR} || fatal "Failed to rsync from ${RSYNCSOURCE}."

}}}

To be able to create a mirror people can rely on, you need to have all the files and you need them at the right moment. 'File not found'-errors cause a lot of issues and annoyances for users. This page provides you with scripts to sync from other mirrors and prevent 404's.

Please be sure to use a local mirror

ubumirror

One can use the scripts from the ubumirror project to keep their mirror in sync, or use the scripts provided below.

This project has been packaged since Lucid onwards, please file any bug reports you find against it.

Archive mirrors

For archive mirrors, it is very important not to delete packages before the Packages.gz-files (which hold information about the packages available) are updated. Therefor, you need a 'Two stage sync'. This means that you download new packages first, and new Packages.gz after that. After you've downloaded the Packages.gz files, it's safe to delete old packages.

The script

#/bin/dash

fatal() {
  echo "$1"
  exit 1
}

warn() {
  echo "$1"
}

# Find a source mirror near you which supports rsync on
# https://launchpad.net/ubuntu/+archivemirrors
# rsync://<iso-country-code>.rsync.archive.ubuntu.com/ubuntu should always work
RSYNCSOURCE=rsync://archive.ubuntu.mirror.isp.com/ubuntu

# Define where you want the mirror-data to be on your mirror
BASEDIR=/var/www/ubuntuarchive/

if [ ! -d ${BASEDIR} ]; then
  warn "${BASEDIR} does not exist yet, trying to create it..."
  mkdir -p ${BASEDIR} || fatal "Creation of ${BASEDIR} failed."
fi

rsync --recursive --times --links --hard-links \
  --stats \
  --exclude "Packages*" --exclude "Sources*" \
  --exclude "Release*" \
  ${RSYNCSOURCE} ${BASEDIR} || fatal "First stage of sync failed."

rsync --recursive --times --links --hard-links \
  --stats --delete --delete-after \
  ${RSYNCSOURCE} ${BASEDIR} || fatal "Second stage of sync failed."

Releases mirrors

For Releases mirrors, stuff is a little less complicated. There are no dependencies between files, so you can just rsync away.

The script

#/bin/dash

fatal() {
  echo "$1"
  exit 1
}

warn() {
  echo "$1"
}

# Find a source mirror near you which supports rsync on
# https://launchpad.net/ubuntu/+cdmirrors
# rsync://<iso-country-code>.rsync.releases.ubuntu.com/releases should always work
RSYNCSOURCE=rsync://releases.ubuntu.mirror.isp.com/releases

# Define where you want the mirror-data to be on your mirror
BASEDIR=/var/www/ubuntureleases/

if [ ! -d ${BASEDIR} ]; then
  warn "${BASEDIR} does not exist yet, trying to create it..."
  mkdir -p ${BASEDIR} || fatal "Creation of ${BASEDIR} failed."
fi

rsync --verbose --recursive --times --links --hard-links \
  --stats --delete-after \
  ${RSYNCSOURCE} ${BASEDIR} || fatal "Failed to rsync from ${RSYNCSOURCE}."

Mirrors/Scripts (last edited 2017-10-17 23:22:07 by hloeung)