NOTE: This page is part of the Ubuntu Specification process. Please check the status and details in Launchpad before editing. If the spec is Approved then you should contact the Assignee, or another knowledgeable person, before making changes.
Launchpad entry: https://features.launchpad.net/distros/ubuntu/+spec/compressed-memory
Packages affected: linux-image
Summary
This spec aims to deploy compressed memory in Ubuntu Linux using the new http://linuxcompressed.sf.net patches.
Rationale
It has been shown that compressed memory increases performance greatly when memory is overdrawn and decreases performance negligibly when ample memory is available. We should take advantage of this for both desktop and LiveCD performance.
Use cases
There are several.
- Alice tries to run the LiveCD runs on 192MiB of RAM; with compressed memory, fewer disc reads occur and more programs can run at once.
- Bob is running several applications on 1GiB of RAM; they total 800MiB and the kernel asides 400MiB for page cache due to swappiness, but doesn't write to disk because page cache and some memory pages are compressed before resorting to actual swapping.
- Ben sets up a Web server with 2GiB of RAM servicing thousands of users. At peak usage it avoids expensive disk swaps by compressing some page cache and LRU memory.
Scope
This spec focuses on LiveCD, desktop, and server environments. Embedded environments such as OLPC should be handled case by case.
Design
The http://linuxcompressed.sourceforge.net/ compressed memory patches will be used to enable memory compression. These patches should be stable by Edgy+1.
The compressed memory infrastructure is patched into the kernel; but the actual compression engine is a module. This module shall be separately loaded.
It should be possible to disable auto-loading of the compression module using a kernel command line switch such as nocompress.
Implementation
- Merge patches to the 2.6 kernel with the Ubuntu kernel
- Build the compressed memory module as a separate module
- Write a loader script for the compressed memory module
- Minimalistic and simple
- Check a kernel command line parameter
- Designed to be executed by anything; possibly execute in the initrd
- Test heavily
Run both desktop and LiveCD with restrictive mem= parameters
Data preservation and migration
None.
Unresolved issues
Tuning of the compressed memory parameters is important. The current experimental patches allow direct tuning of the number of pages used for each of swap cache and page cache; it would be interesting to make this a percentage and let the kernel decide how to distribute it between the two.
BoF agenda and discussion
- It would be wise to attempt to merge the compressed memory patches with mainline. Nitin Gupta states in e-mail that he has no interest in this because it's difficult to negotiate with the kernel developers; however, as with the 2.4 patch, if not merged with mainline use will be cumbersome and the project may eventually be abandoned.
The Ubiquity installer does not run on machines with low amounts of memory. I asked Nitin Gupta if it could expand the amount of effective amount of virtual memory available before a swap partition was set up. Apparently not yet, but the next release may be able to reduce the amount of physical memory needed to run Ubiquity. If this could reduce the number of users needing to download and burn the alternate install CD this could save a fair amount of hassle. (Maybe more hassle than maintaining this outside the official kernel? Often the maintainers are willing to include a patch if it proves itself first in a large scale distro, so it may not need to be maintained for long). -- JohnMccabeDansted
Another use: James has a old laptop with 128MB. The CPU is fast enough, but growing memory requirements make even Xubuntu quite slow. Adding memory is a pain, especially since murphies law states that they will be lost or stolen shortly afterwards. -- JohnMccabeDansted
Also Compressed Caching will have the most effect when the system is under heavy memory pressure, when you are likely to have plenty of CPU time that would otherwise be wasted waiting for disk. The benchmarks aren't specifically designed for this purpose, but they do seem to show that Compressed Caching would cause the system performace to degrade far more gracefully as memory load increases. E.g. httperf is ~5 times quicker with 32MB of ram. This should reduce the tendency of systems to appear to freeze when unexpected spikes in memory usage occur. -- JohnMccabeDansted