AutomaticUpdates

Differences between revisions 9 and 10
Revision 9 as of 2005-11-02 22:06:09
Size: 3736
Editor: 209
Comment: reviewers comments
Revision 10 as of 2005-11-04 17:36:48
Size: 5836
Editor: 209
Comment: updated
Deletions are marked like this. Additions are marked like this.
Line 7: Line 7:

== Comments from CoreyBurger ==
 * Where are the use cases. The rationale and the use case should be better
 * Where is your scope?
 * Did you decide on dpkg --force-conf-old or not?
 * What are you going to do about those packages with postinst questions? I didn't see an answer
 * Where is the policy for making certain that there is no postinst. Have the various security people been informed?
 * What does the code do?
Line 30: Line 22:
Alice runs a Ubuntu/stable system and has broadband. She is not interseted in the system a lot but wants to get timely security updates automatically.

Bob runs Ubuntu/stable on a server and wants a easy way to automaticaly check/install security updates nightly.

Malone runs a server and is very conservative. He only wants to install security upgrades manually after he read the changelog of all packages.
Line 31: Line 29:

This spec is covers Ubuntu.

== Problems ==

There are various problems associated with unattended upgrades. The debian packages are not designed for unattended upgrades. Various things may require user intervention.

Problems:
 * package may prompt in the postinst script (using the read command)
 * package may ask questions with debconf
 * conffile questions asked by dpkg

Fortunately there are only few packages that ask prompt using "read". But especially for universe it can not guranteed that they do not do this. Importend packages like the kernel, libc, etc do it still.

Debconf questions can be tackled by runing with it with the `noninteractive` frontend.

Conffile questions may come up if a package ships with a insecure configuration by default and the conffile is updated in the package but the user modified that conffile (security upgrades like this happend in the past).
Line 34: Line 49:
The problem with unattended upgrades are packages that ask questions in
postinst. Fortunately there are few of them nowdays, but we still have
kernel, libc, etc. Another problem is that there may be conffile questions during the upgrade. [[BR]]

We may run dpkg with --force-conf-old. A problem with this is that:

 * a security upgrade may be using a new version for a (default) conffile
 * a upgrade may need a new conffile format

We need a policy for security upgrades that the packages must be prepared for it (don't ask questions, don't prompt, no new conffile).
To adress the above problems a policy for security upgrades must be put into place. the packages must be prepared for it and it should include:
 * don't prompt
 * do the right thing with debconf set to non-interactive
 * no new conffile (if it is not *absolutely* required)
Line 46: Line 55:
(origin: ubuntu-security), if anything is installed/upgraded that does not comes from ubuntu-security we will ignore it.
(origin: $disto-security), if anything is installed/upgraded that does not comes from ubuntu-security we will ignore it. Conffile changes must be detected before the package is actually installed and if they ocur the package must not be installed but kept back. A logfile for each upgrade (or keep back) should be written to /var/log/auto-upgrade (to be confirmed). For kept back upgrades the update-notifier icon should show up.
Line 51: Line 59:
Write it in python-apt, check what's upgradable and comes from security;
verify if it does not a) break/remove anything b) installs
stuff outside security; upgrade it. It will be tied into the apt cron-job
we have already.
The actual unattended upgrade tool will be written in python with python-apt. It will check for available upgradable packages and check if they come from $distro-security (e.g. dapper-security). If so, it will simulate a install in it's cache and checks if all dependencies (at may be required to install) come from security as well and if no removals are required (and the cache does not break from this change). It will then download the packages and examine them for possible conffile prompts. Any package that will prompt will be kept back. The rest is installed.

It will be tied into the apt cron-job we have already have. The cron job will be extended to have a APT::Periodic::Post-Run-Hook (to be confirmed) with a list of commands that are run after the regular APT::Periodic actions happend. That will ensure that the packages list is always up-to-date and that (if the user selected that) the packages are already downloaded to the cache.
Line 59: Line 66:
No production code has yet been written. The below code gives a overview about what is planed. It will try to upgrade the various upgradable packages and see how that affects the cache. This is not final, but it gives a idea what is planed.
Line 62: Line 71:
def check_changes_for_sanity(): def check_changes_for_sanity(cache):
 if cache.BrokenCount != 0:
  return False
Line 64: Line 75:
  if pkg.markedRemove:
   return False
Line 65: Line 78:
                pkg.candidateOrigion != "breezy-security":                 pkg.candidateComponent != "breezy-security":
Line 70: Line 83:
pkgs = [] pkgs_to_upgrade = []
Line 73: Line 86:
           pkg.candidateOrigin == "breezy-security":
  pkg.markUpgradable()
  if check_changes_for_sanity():
   #cache = apt.Cache()
   for pkg in cache:
    pkg.markKeep()
   for pkg2 in pkgs:
           pkg.candidateComponent == "dapper-security":
  pkg.markUpgrade()
  if check_changes_for_sanity(cache):
   cache.clean()
   assert cache.BrokenCount == 0 and \
   cache.InstallCount == 0 and cache.RemoveCount == 0
   
for pkg2 in pkgs_to_upgrade:
Line 83: Line 96:

# TODO: download and check for possible conffile prompts

Summary

Automatic Updates - automatic installation of (security) updates, possible even when current user has no sudo privileges

Rationale

When ubuntu is used by a user without sudo privileges, he cannot upgrade it with security updates. This can mean a machine will be vulnerable to security risks for a prolonged period of time. Also some people just don't care about updates and other technical stuff. They just don't want to be bothered and have ubuntu to keep itself updated and secure.

Use cases

Alice runs a Ubuntu/stable system and has broadband. She is not interseted in the system a lot but wants to get timely security updates automatically.

Bob runs Ubuntu/stable on a server and wants a easy way to automaticaly check/install security updates nightly.

Malone runs a server and is very conservative. He only wants to install security upgrades manually after he read the changelog of all packages.

Scope

This spec is covers Ubuntu.

Problems

There are various problems associated with unattended upgrades. The debian packages are not designed for unattended upgrades. Various things may require user intervention.

Problems:

  • package may prompt in the postinst script (using the read command)
  • package may ask questions with debconf
  • conffile questions asked by dpkg

Fortunately there are only few packages that ask prompt using "read". But especially for universe it can not guranteed that they do not do this. Importend packages like the kernel, libc, etc do it still.

Debconf questions can be tackled by runing with it with the noninteractive frontend.

Conffile questions may come up if a package ships with a insecure configuration by default and the conffile is updated in the package but the user modified that conffile (security upgrades like this happend in the past).

Design

To adress the above problems a policy for security upgrades must be put into place. the packages must be prepared for it and it should include:

  • don't prompt
  • do the right thing with debconf set to non-interactive
  • no new conffile (if it is not *absolutely* required)

We will limit ourself to security upgrades for the installed distro (origin: $disto-security), if anything is installed/upgraded that does not comes from ubuntu-security we will ignore it. Conffile changes must be detected before the package is actually installed and if they ocur the package must not be installed but kept back. A logfile for each upgrade (or keep back) should be written to /var/log/auto-upgrade (to be confirmed). For kept back upgrades the update-notifier icon should show up.

Implementation

The actual unattended upgrade tool will be written in python with python-apt. It will check for available upgradable packages and check if they come from $distro-security (e.g. dapper-security). If so, it will simulate a install in it's cache and checks if all dependencies (at may be required to install) come from security as well and if no removals are required (and the cache does not break from this change). It will then download the packages and examine them for possible conffile prompts. Any package that will prompt will be kept back. The rest is installed.

It will be tied into the apt cron-job we have already have. The cron job will be extended to have a APT::Periodic::Post-Run-Hook (to be confirmed) with a list of commands that are run after the regular APT::Periodic actions happend. That will ensure that the packages list is always up-to-date and that (if the user selected that) the packages are already downloaded to the cache.

Code

No production code has yet been written. The below code gives a overview about what is planed. It will try to upgrade the various upgradable packages and see how that affects the cache. This is not final, but it gives a idea what is planed.

   1 import apt
   2 
   3 def check_changes_for_sanity(cache):
   4         if cache.BrokenCount != 0:
   5                 return False
   6         for pkg in cache:
   7                 if pkg.markedRemove:
   8                         return False
   9                 if (pkg.markedInstall or pkg.markedUpgrade) and \
  10                 pkg.candidateComponent != "breezy-security":
  11                         return False
  12         return True
  13 
  14 cache = apt.Cache()
  15 pkgs_to_upgrade = []
  16 for pkg in cache:
  17         if pkg.isUpgradable() and \
  18            pkg.candidateComponent == "dapper-security":
  19                 pkg.markUpgrade()
  20                 if check_changes_for_sanity(cache):
  21                         cache.clean()
  22                         assert cache.BrokenCount == 0 and \
  23                         cache.InstallCount == 0 and cache.RemoveCount == 0
  24                         for pkg2 in pkgs_to_upgrade:
  25                                 pkg2.markUpgradable()
  26                 else:
  27                         pkgs.append(pkg)
  28 
  29 # TODO: download and check for possible conffile prompts

Data preservation and migration

Outstanding issues

  • update-notifier would need a way to figure if the pkg-database is locked or not. Write an informational file next to the dpkg lock when acquiring it? Stale locks may be a problem here. Frontends should catch SIGINT and cancel their locks, the boot process should remove any stale locks.
  • flock has advisory locks, which can be used on directory you cannot write in. (DoS? -- one option would be to use an advisory lock to inform user processes, but not to make that the locking protocol between APT frontends; i.e., it should be entirely advisory for frontends; only the main fcntl lock should be mandatory)

BoF agenda and discussion

AutomaticUpdates (last edited 2008-08-06 16:27:19 by localhost)