PersonalPackageCacheSpec

Summary

Provide a method to designate a computer on a user's personal LAN as a Personal Package Cache. The cache would proxy all requestes for packages to its local copy rather than the original Internet repository. The cache would contain only packages that have been requested for a local install on a personal LAN.

Release Note

This feature will bring a wrapper to the ability to pull a package once and install manually on many computer on a personal lan making it a suitable service for the average desktop user.

Rationale

The feature would allow users in bandwith capped or metered locales to conserve and better manger their internet usage. It may also allow users to more quickly install their most used packages on computer in their personal lan.

See Also: AptProxyCache on the server side and apt-zeroconf for a mesh network of apt-caches.

This would be a more end-user friendly variation that requires manual updates with no pushing of a packages than what has been proposed before. It would also perform most of the reading and writing configuration for the user.

User stories

Raul is a user with a bandwith cap on his internet usage. He opts to use Personal Package Cache to setup his families three computers. Once he has installed his perfect system, he designates it as the cache via a simple pannel. Now when sets up his other two computer, he first designates the computer with the Personal Package Cache as the location for any repository request via another simple pannel. He now saves two-thirds on his normal usage.

Paul is a sysadmin for Acme Enterprise. He decideds to try Personal Package Cache to speed the retrieval of packages from the repositories. He executes a simple script that sets one of his computers as the cahce. Since he only uses a specific subset of packages, his storage needs are much less then if he chose to mirror all the repositories. Now on his other computers, he can designate the machine with the Personal Package Cache remotely via another simple script. His downloads drop from minutes to seconds for each package retrieval.

Assumptions

  • Bandwith conservation and quicker installations are desirable.
  • Repurposing parts of current projects like approx or apt-cache-ng would be effective.

  • Verification the packages retrievedby its signature should be sufficient for most cases.

Design

A Personal Cache Service

  • A proxy-like service that mediates package retrieval from the various repositories a user may chose.
  • An Avahi advertised service would let other computers on a LAN learn
    • Its existence
    • Available labeled repositories
  • A cron service that would maintain and keep the cache in sync. (Perhaps with pdiffs)
  • A script and graphical interface
    • Translation of sources.list to a set of labeled retrievable locations
    • Modification of the set of retrievable locations and their labels

A Personal Cache Client

  • A setup script and graphical interface
    • Tranformation of a sources.list to for use via a Personal Package Cache
  • Future development might see integration of the feature into programs that modify the sources.list

Original Sources List

...
deb http://us.archive.ubuntu.com/ubuntu/ jaunty main restricted
deb-src http://us.archive.ubuntu.com/ubuntu/ jaunty main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://us.archive.ubuntu.com/ubuntu/ jaunty-updates main restricted
deb-src http://us.archive.ubuntu.com/ubuntu/ jaunty-updates main restricted
...

approx

modified sources list

...
deb http://desktop.local:9999/ubuntu/ jaunty main restricted
deb-src http://desktop.local:9999/ubuntu/ jaunty main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://desktop.local:9999/ubuntu/ jaunty-updates main restricted
deb-src http://desktop.local:9999/ubuntu/ jaunty-updates main restricted
...

proxy conf support change

/etc/approx/approx.conf

...
ubuntu http://us.archive.ubuntu.com/ubuntu/
...

apt cache ng

modified sources list

...
deb http://desktop.local:3142/ubuntu/ jaunty main restricted
deb-src http://desktop.local:3142/ubuntu/ jaunty main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://desktop.local:3142/ubuntu/ jaunty-updates main restricted
deb-src http://desktop.local:3142/ubuntu/ jaunty-updates main restricted
...

proxy conf support change

/etc/apt-caher-ng/acng.conf

...
Remap-ubuntu: /ubuntu ; http://us.archive.ubuntu.com/ubuntu/
...

A special note for apt-cache-ng: While it is true that can we just add a proxy line to apt sources, I feel rewritting the sources to a label on the proxy leads to cleaner and more manageable configuration.

proxy support page changes

In addition, the support pages accessible via the cache proxy should bettter reflect the usage of Personal Package Cache. Perhaps mimic the style of other Ubuntu web pages too.

  • /etc/apt-cacher-ng/maint.html -> /usr/lib/apt-cacher-ng/maint.html

  • /etc/apt-cacher-ng/report.html -> /usr/lib/apt-cacher-ng/report.html

  • /etc/apt-cacher-ng/style.css -> /usr/lib/apt-cacher-ng/style.css

  • /etc/apt-cacher-ng/userinfo.html -> /usr/lib/apt-cacher-ng/userinfo.html. This page should show the current available labels and repository location it knows about.

Packaging Considerations

Personal Package Cache Service will depend on approx or apt-cacher-ng for its caching. Personal Package Cache Client would not. It will then provide a 'core' terminal package and 'gtk/kde' graphical interface package which would use the library of core for its interaction with caching configuration and sources.list.

Implementation

In my opinion approx and apt-cache-ng is close to what I think the Personal Cache Service should provide. Both lack a friendly interface or helpful scripts on the service or client side. A draw back of approx is that it written in OCAML which may or not be the best language for the long haul.

We need to provide easy scripts and interfaces for the user's Personal Package Cache.

Server side: Easy conversion of sources.list to the proxy choice without intervention.

Client side: Easy selection of avaiable repos on the server and auto modification of sources list.

Avahi should be used to advertise and listen for an available Personal Package Cache and their available repositories. Code updating the proxy should auto refresh the proxy conf on changes.

A future goal might be to intergate these features into the existing code that currently modify the sources.list.

Work is occuring on https://code.edge.launchpad.net/~ausimage/+junk/apt-zeroconf to implement the spec. It is starting from apt-zeroconf-0.4 code and is being modified to add in features from approx and apt-cacher-ng. Work will also need to be done in creating tools to easily enable the use of this proxy/redirect service for the average user once the service features are complete.

Migration

Migration would be handled on installation of the package and perhaps notification of the new method of adding repositories be presented to the user upon completion of the install.

Test/Demo Plan

  1. Designate a computer on a LAN as the package cache.
  2. On another machine on the LAN and set the machine to use the package cache.
  3. Notice that all requests for packages go through the cache and only packages not in cache are pulled from the Internet.

Unresolved issues

BoF agenda and discussion


CategorySpec

PersonalPackageCacheSpec (last edited 2009-05-10 18:07:33 by pool-70-16-48-183)