AutomaticUpdates

Differences between revisions 5 and 15 (spanning 10 versions)
⇤ ← Revision 5 as of 2005-11-02 16:33:31 →
Size: 3361
Editor: 209
Comment: results after the first bof
← Revision 15 as of 2007-07-22 12:08:39 → ⇥
Size: 6386
Editor: e176205151
Comment: Spelling only
Deletions are marked like this. Additions are marked like this.
Line 9: Line 9:
Automatic Updates - automatic installation of (security) updates, possible even
when current user has no sudo privileges
Automatic Updates - automatic installation of (security) updates, possibly even when the current user has no sudo privileges
Line 15: Line 15:
with security updates. This can mean a machine will be vulnerable to security
risks for a prolonged period of time. Also some people just don't care about		 with security updates. This can mean that a machine will be vulnerable to security
risks for a prolonged period of time. Also, some people just don't care about
Line 18: Line 18:
have ubuntu to keep itself updated and secure. want ubuntu to keep itself updated and secure.
Line 22: Line 22:
Alice runs a Ubuntu/stable system and has broadband. She is not interested in the system a lot but wants to get timely security updates automatically.

Bob runs Ubuntu/stable on a server and wants an easy way to automaticaly check/install security updates nightly.

Malone runs a server and is very conservative. He only wants to install security upgrades manually after he read the changelog of all packages.
Line 23: Line 29:

This spec describes a tool which will allow unattended package upgrades for a certain set of packages on a ubuntu system.

== Problems ==

There are various problems associated with unattended upgrades. The debian packages are not designed for unattended upgrades. Various things may require user intervention.

Problems:
 * package may prompt in the postinst script (using the read command)
 * package may ask questions with debconf
 * conffile questions may be asked by dpkg

Fortunately, there are only few packages that ask prompt using "read". But especially for universe, it can not be guaranteed that they do not do this. Important packages like the kernel, libc, etc do it still.

Debconf questions can be tackled by running with it with the `noninteractive` frontend.

Conffile questions may come up if a package ships with an insecure configuration by default and the conffile is updated in the package, but the user modified that conffile (security upgrades like this happend in the past).
Line 26: Line 49:
One problem with unattended upgrades are packages that ask questions in
postinst. Fortunately there are few of them nowdays, but we still have
kernel, libc, etc. Another problem is that there may be conffile questions
during the upgrade.		 To address the above problems, a policy for security upgrades must be put into place. Packages must be prepared to adhere to it.
Line 31: Line 51:
We may run dpkg with --force-conf-old. A problem with this is that: The policy should include:
Line 33: Line 53:
 * a security upgrade may be using a new version for a (default) conffile
 * a upgrade may need a new conffile format		  * don't prompt
 * do the right thing if debconf is set to non-interactive
 * no changed conffiles (if not *absolutely* required)
Line 36: Line 57:
The packages must be prepared for it (don't ask questions, don't prompt, no
new conffile).
We will limit ourself to security upgrades for the installed distro
(origin: ubuntu-security), if anything is installed/upgraded that does not
comes from ubuntu-security we will ignore it.
 We will limit ourselves to security upgrades for the installed distro (origin: $disto-security).
If anything is installed/upgraded that does not come from ubuntu-security, we will ignore it. Conffile changes must be detected before the package is actually installed; if they occur, the package must not be installed, but be kept back. A logfile for each upgrade (or keep-back) should be written to /var/log/auto-upgrade (to be confirmed). For kept-back upgrades, the update-notifier icon should show up.
Line 45: Line 62:
Write it in python-apt, check what's upgradable and comes from security;
verify if it does not a) break/remove anything b) installs
stuff outside security; upgrade it. It will be tied into the apt cron-job
we have already.		 The actual unattended upgrade tool will be written in python with python-apt. It will check for available upgradeable packages and check if they originate from $distro-security (e.g. dapper-security). If so, it will simulate an install in its cache and check that
 1. all dependencies (that may be required to install) come from security as well
 1. no removals are required (and the cache does not break from this change).
It will then download the packages and examine them for possible conffile prompts. Any package that would prompt will be kept back. The rest is installed if possible.

The tool will be tied into the apt cron-job we have already have. The cron job will be extended to have a APT::Periodic::Post-Run-Hook (to be confirmed) with a list of commands that are executed after the regular APT::Periodic actions. This will ensure that the Packages list is always up-to-date, and that the packages are already downloaded to the cache (if the user selected that).

With the hook in the apt.cron script, we can ship the unattended upgrade tool in a seperate package.

Alternatively a option like APT::Periodic::Upgrade-Interval for the automatic upgrades can be added to the cron-job directly. This means that the python-apt code would have to be shiped with apt and the user must have python-apt installed to make it useful (if python-apt is not installed, it must output a error to the log).
Line 52: Line 75:
{{{#!python
import apt
cache = apt.Cache()
for pkg in cache:
    if pkg.isUpgradable() and pkg.candidateOrigin == "breezy-security":
        pkg.markUpgrade()
        # TODO: Check if something unwanted was marked for upgrade/removal
        if cache.BrokenCount > 0:
            # TODO: Undo the last action and try something else.
            pkg.unmarkUpgrade()
}}}
No production code has yet been written. The code below gives an overview of what is planed. It will try to upgrade the various upgradeable packages and see how that affects the cache. This is not final, but it shows what is planed.
Line 66: Line 80:

def check_changes_for_sanity(cache):
 if cache.BrokenCount != 0:
  return False
        for pkg in cache:
  if pkg.markedRemove:
   return False
                if (pkg.markedInstall or pkg.markedUpgrade) and \
                pkg.candidateComponent != "breezy-security":
                        return False
        return True
Line 67: Line 93:
pkgs = [] pkgs_to_upgrade = []
Line 69: Line 95:
 if pkg.isUpgradable() and pkg.candidateOrigin == "breezy-security":
  pkg.markUpgradable()
  if we_shouldnt_continue():
   #cache = apt.Cache()
   for pkg in cache:
    pkg.markKeep()
   for pkg2 in pkgs:		  if pkg.isUpgradable() and \
           pkg.candidateComponent == "dapper-security":
  pkg.markUpgrade()
  if check_changes_for_sanity(cache):
   cache.clean()
   assert cache.BrokenCount == 0 and \
   cache.InstallCount == 0 and cache.RemoveCount == 0
   for pkg2 in pkgs_to_upgrade:
Line 80: Line 107:
=== Data preservation and migration === # TODO: download and check for possible conffile prompts
}}}
Line 84: Line 112:
 * update-notifier would need a way to figure if the pkg-database is locked or
 not. Write an informational file next to the dpkg lock when acquiring it?
 Stale locks may be a problem here. Frontends should catch SIGINT and cancel their locks, the boot process should remove any stale locks.		  * update-notifier would need a way to determine whether (and why) the pkg database is locked. Write an informational file next to the dpkg lock when acquiring it?
    * Stale locks may be a problem. Frontends should catch SIGINT and cancel their locks; the boot process should remove any stale locks.
Line 88: Line 115:
 * flock has advisory locks, which can be used on directory you cannot write in.  
 (DoS? -- one option would be to use an advisory lock to inform user processes, but not to make that the locking protocol between APT frontends; i.e., it should be entirely advisory for frontends; only the main fcntl lock should be mandatory)

== BoF agenda and discussion ==		  * flock has advisory locks which can be used on a directory you cannot write in.
   * DoS? -- one option would be to use an advisory lock to inform user processes, but not to make that the locking protocol between APT frontends; i.e., it should be entirely advisory for frontends; only the main fcntl lock should be mandatory.
----
CategorySpec

Summary

Automatic Updates - automatic installation of (security) updates, possibly even when the current user has no sudo privileges

Rationale

When ubuntu is used by a user without sudo privileges, he cannot upgrade it with security updates. This can mean that a machine will be vulnerable to security risks for a prolonged period of time. Also, some people just don't care about updates and other technical stuff. They just don't want to be bothered and want ubuntu to keep itself updated and secure.

Use cases

Alice runs a Ubuntu/stable system and has broadband. She is not interested in the system a lot but wants to get timely security updates automatically.

Bob runs Ubuntu/stable on a server and wants an easy way to automaticaly check/install security updates nightly.

Malone runs a server and is very conservative. He only wants to install security upgrades manually after he read the changelog of all packages.

Scope

This spec describes a tool which will allow unattended package upgrades for a certain set of packages on a ubuntu system.

Problems

There are various problems associated with unattended upgrades. The debian packages are not designed for unattended upgrades. Various things may require user intervention.

Problems:

  • package may prompt in the postinst script (using the read command)
  • package may ask questions with debconf
  • conffile questions may be asked by dpkg

Fortunately, there are only few packages that ask prompt using "read". But especially for universe, it can not be guaranteed that they do not do this. Important packages like the kernel, libc, etc do it still.

Debconf questions can be tackled by running with it with the noninteractive frontend.

Conffile questions may come up if a package ships with an insecure configuration by default and the conffile is updated in the package, but the user modified that conffile (security upgrades like this happend in the past).

Design

To address the above problems, a policy for security upgrades must be put into place. Packages must be prepared to adhere to it.

The policy should include:

  • don't prompt
  • do the right thing if debconf is set to non-interactive
  • no changed conffiles (if not *absolutely* required)

We will limit ourselves to security upgrades for the installed distro (origin: $disto-security). If anything is installed/upgraded that does not come from ubuntu-security, we will ignore it. Conffile changes must be detected before the package is actually installed; if they occur, the package must not be installed, but be kept back. A logfile for each upgrade (or keep-back) should be written to /var/log/auto-upgrade (to be confirmed). For kept-back upgrades, the update-notifier icon should show up.

Implementation

The actual unattended upgrade tool will be written in python with python-apt. It will check for available upgradeable packages and check if they originate from $distro-security (e.g. dapper-security). If so, it will simulate an install in its cache and check that

  1. all dependencies (that may be required to install) come from security as well
  2. no removals are required (and the cache does not break from this change).

It will then download the packages and examine them for possible conffile prompts. Any package that would prompt will be kept back. The rest is installed if possible.

The tool will be tied into the apt cron-job we have already have. The cron job will be extended to have a APT::Periodic::Post-Run-Hook (to be confirmed) with a list of commands that are executed after the regular APT::Periodic actions. This will ensure that the Packages list is always up-to-date, and that the packages are already downloaded to the cache (if the user selected that).

With the hook in the apt.cron script, we can ship the unattended upgrade tool in a seperate package.

Alternatively a option like APT::Periodic::Upgrade-Interval for the automatic upgrades can be added to the cron-job directly. This means that the python-apt code would have to be shiped with apt and the user must have python-apt installed to make it useful (if python-apt is not installed, it must output a error to the log).

Code

No production code has yet been written. The code below gives an overview of what is planed. It will try to upgrade the various upgradeable packages and see how that affects the cache. This is not final, but it shows what is planed. 

   1 import apt
   2 
   3 def check_changes_for_sanity(cache):
   4         if cache.BrokenCount != 0:
   5                 return False
   6         for pkg in cache:
   7                 if pkg.markedRemove:
   8                         return False
   9                 if (pkg.markedInstall or pkg.markedUpgrade) and \
  10                 pkg.candidateComponent != "breezy-security":
  11                         return False
  12         return True
  13 
  14 cache = apt.Cache()
  15 pkgs_to_upgrade = []
  16 for pkg in cache:
  17         if pkg.isUpgradable() and \
  18            pkg.candidateComponent == "dapper-security":
  19                 pkg.markUpgrade()
  20                 if check_changes_for_sanity(cache):
  21                         cache.clean()
  22                         assert cache.BrokenCount == 0 and \
  23                         cache.InstallCount == 0 and cache.RemoveCount == 0
  24                         for pkg2 in pkgs_to_upgrade:
  25                                 pkg2.markUpgradable()
  26                 else:
  27                         pkgs.append(pkg)
  28 
  29 # TODO: download and check for possible conffile prompts

Outstanding issues

  • update-notifier would need a way to determine whether (and why) the pkg database is locked. Write an informational file next to the dpkg lock when acquiring it?
    • Stale locks may be a problem. Frontends should catch SIGINT and cancel their locks; the boot process should remove any stale locks.
  • flock has advisory locks which can be used on a directory you cannot write in.
    • DoS? -- one option would be to use an advisory lock to inform user processes, but not to make that the locking protocol between APT frontends; i.e., it should be entirely advisory for frontends; only the main fcntl lock should be mandatory.

CategorySpec

AutomaticUpdates (last edited 2008-08-06 16:27:19 by localhost)