AptAuthenticationReliability

Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.

Summary

Improve apt's authentication checks to be more robust against transient network failures.

Release Note

The package management system is now more robust against transient network failures, and no longer claims that packages are unauthenticated when it really just had problems downloading certain files.

Rationale

apt's authentication checks often fail if a mirror is slightly out of sync, confusing users and generating bug reports. We should be able to do better, although it may involve archive improvements.

At present, .gpg files in /var/lib/apt/lists/ are removed after a failed check in order to signal that authentication failed.

Some of the transient failures are believed to be due to interactions with round-robin DNS for mirrors. So-called "transparent" proxies are also often a source of difficult-to-debug problems; they do not universally honour the usual cache control headers.

Use Cases

  • apt fails to download Release.gpg due to a transient network failure. Instead of telling the user that their packages are now unauthenticated (and thereby confusing the user and teaching them to believe that this message is somehow "normal"), it rolls back the whole update for that hostname.

  • Users whose ISPs impose a transparent HTTP proxy find that apt updates more reliably and with fewer spurious errors.

Design

Network reliability

We will change apt to use pipelined HTTP to download all indices on the same hostname in a single HTTP session. This will avoid problems with round-robin DNS, since the same host will serve all requests for a given hostname.

KeesCook has a test case in which apt consistently fails to update but network problems are not believed to be at fault. He will investigate this further and report back.

Downgrade attack prevention

apt should not allow downgrading from a signed archive to an unsigned archive without manual override, because this exposes straightforward downgrade attacks on the archive's signature system provided that users can be socially-engineered into ignoring authentication warnings. Instead, it should simply restore all the old index files and refuse to update if it cannot verify the new Release signature.

apt-setup should insert valid Release and Release.gpg files at installation time, in order that the archive starts out signed. (The contents of these files does not matter as long as the signature is valid, so files downloaded from the archive at an arbitrary point in time would do fine.) This requires an apt-get option to print the appropriate filenames in /var/lib/apt/lists/.

If /etc/cron.daily/apt fails due to signature checking, then we will notify the user via update-notifier that the update failed and that they may have to check manually.

Archive changes

The above changes will make the situation much better for Ubuntu 8.04. However, fundamentally, much of the difficulty is that the objects being signed are in a different file from the signature, which means that there will always be a race during mirroring (unless the mirror takes steps to atomically update the dists/ subdirectory; however, this is not entirely straightforward and no mirrors are currently believed to do it). Merging Release and Release.gpg into a single file with an inline signature would be relatively straightforward and would help, but the problem remains that Release might be out of sync with some or all of the Packages files.

A valid but backward-incompatible solution would be to include an inline signature in every Packages file (and probably rename it to SignedPackages or similar at the same time in order to avoid breaking old tools). We will discuss this with affected parties in Debian and prepare the way for this in apt. The archive should not be changed until we are confident that this is a viable approach.

Implementation

This feature is now part of apt 0.7.9ubuntu5 in hardy. The implementation will refuse to continue if Release/Release.gpg file authentication fails and use the previously available files instead. It will also print a warning then.

Testcase were added to the test/ directory that simulate a previously authenticated repository were the signature no longer matches. The tests are run with both If-Modified-Since hits and without.

apt-setup 1:0.31ubuntu3 in hardy populates /var/lib/apt/lists with signed Release files.

Test/Demo Plan

It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during CD testing, and to show off after release.

This need not be added or completed until the specification is nearing beta.

Future work

There was discussion of disallowing unsigned repositories unless they are explicitly allowed by the user. However, PPAs are not yet signed, and many small private archives (e.g. for personal use) may not be signed.


CategorySpec

AptAuthenticationReliability (last edited 2008-08-06 16:36:58 by localhost)