Launchpad Entry: server-karmic-bzr-etckeeper-integration
Created: June 8th, 2009
Packages affected: etckeeper, bzr
etckeeper is a great way of keeping /etc under revision control, however it doesn't come without any pitfalls. These are mostly due to lack of integration between the VCS used and the permissions/ownership metadata. This spec tries to address the most obvious issues so that etckeeper can be more widely used in Ubuntu in the future.
etckeeper in 9.10 is easier than ever. Enabling it is just a apt-get command away and etckeeper integration with bzr allows to visualize and revert permissions/ownership changes.
The etckeeper features introduced in Jaunty (track by SUDO_USER, daily autocommit...) allow for a richer out-of-the-box experience in tracking changes in configuration for an Ubuntu system. However, it is still a little confusing (bzr diff) or potentially harmful (bzr revert) to use as far as permissions/ownership changes are concerned. By leveraging the extra control over VCS that the etckeeper-bzr plugin allows, we can make etckeeper/bzr a more integrated experience and target regular users in future releases.
- Anthony runs "apt-get install etckeeper" in Karmic and gets a working setup out of the box.
- Bonnie uses etckeeper as a "captain's log" of all changes made to her servers configuration. Having a "bzr diff" command that really shows the differences, including ownership/permissions differences, at each file level, makes it a lot more readable and exploitable by other tools.
- Charles uses etckeeper since Jaunty but was always reluctant to run "bzr revert" since it would not restore ownership or permissions. With the Karmic release he can run this command with confidence.
The required bzr hooks to implement this functionality need to be accepted in Bazaar.
etckeeper should be working once "sudo apt-get install etckeeper" completes. As of Jaunty you need to run "etckeeper init" and "etckeeper commit" before /etc is really put under revision control. Current upstream version does "etckeeper init" but still requires you to run "etckeeper commit". This should be done in one step.
Support for running bzr under sudo
bzr doesn't really support running under sudo. In particular, it creates (if missing) $HOME/.bazaar directories as owner root rather than as the $SUDO_USER. In order to work properly in the specific etckeeper use case, those issues in bzr need to be addressed.
The .bazaar issue is tracked in https://bugs.launchpad.net/ubuntu/+source/etckeeper/+bug/376388
Integration of ownership/permissions metadata in "bzr diff" output
With the current version, a change in permissions or ownership will be displayed as a change in the .etckeeper file rather than a change in the file itself. This leads to confusion, and lack of integration with other tools (lsdiff...) which show the difference in the wrong place.
So rather than having:
$ sudo bzr diff /etc === modified file '.etckeeper' --- .etckeeper 2009-01-28 08:47:07 +0000 +++ .etckeeper 2009-06-08 11:49:08 +0000 @@ -5,2 +5,3 @@ maybe chgrp adm ./resolv.conf + maybe chmod 666 ./tomcat6/server.xml maybe chgrp tomcat6 ./tomcat6/policy.d
We should have:
$ sudo bzr diff /etc === modified file 'tomcat6/server.xml' (permissions: 644 -> 666)
This requires at least two hooks in bzr:
- One hook to add the files which just have metadata changes to the changes iterator (and remove .etckeeper from changed files)
- One hook to display metadata changes (if any) in all modified files
Design discussion is tracked in https://bugs.launchpad.net/ubuntu/+source/etckeeper/+bug/322327
Integration of ownership/permissions metadata in bzr file restore operations
With the current version, running "bzr revert" to restore a previous version of a file (or any other command that restore another version of a file) doesn't take permissions/ownership metadata into account. The file is restored with default ownership/permissions, and you have to manually look up the metadata in .etckeeper and restore it accordingly. Given the importance of permissions/ownership in /etc contents, this can be misleading and source for errors.
Ideally we need to hook at file creation time to look up corresponding metadata and restore it as well. The difficulty is in trying to cover all file restoration cases. One option is to fix "merge" and then implement etckeeper's "revert" in terms of "merge".
Design discussion is tracked in https://bugs.launchpad.net/ubuntu/+source/etckeeper/+bug/322339
This section should describe a plan of action (the "how") to implement the changes discussed.
It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to http://testcases.qa.ubuntu.com/Coverage/NewFeatures for tracking test coverage. This need not be added or completed until the specification is nearing beta.
This need not be added or completed until the specification is nearing beta.
This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.
BoF agenda and discussion
Introduction: etckeeper in 9.04
- Using bzr by default
- Daily autocommits
- Tracking commits by SUDO_USER
- repo-init and commit at install-time
- Making operations fool-proof:
- diff and revert should take .etckeeper permissions metadata into account
- revert is particularly dangerous, can create an unusable system
- diff/revert/log are most common operations, merge would be nice to have
- These features can be delivered through the "etckeeper" bzr plugin
- Where to hook in bzr to deliver that functionality ?
- Implement etckeeper's revert in terms of merge?
- bzr developers will have a deeper look into it to advise, then etckeeper hackers can do a PoC
- Users run "etckeeper commit" rather than "bzr commit", but users are allowed to use
bzr directly (and the etckeeper plugin should make that work)
- Support for running under sudo in bzr
- Running "sudo bzr ..." creates .bazaar and .bzr.log owned by root in user's home dir (bug 376388).
- Could maybe use $BZR_HOME to control that?
- Admittedly a bzr bug
Discussion of etckeeper usage potential
- Would be very useful installed by default, e.g. helpful for support
Discussion of future improvements
- An ignore.d directory would be convenient for packages (e.g. a package that creates private keys/certs would add those files/dirs to the ignore list).