ZFS

Differences between revisions 10 and 11
Revision 10 as of 2007-05-30 23:40:11
Size: 5191
Editor: adsl-074-232-249-027
Comment:
Revision 11 as of 2007-06-07 16:00:09
Size: 5287
Editor: 199
Comment: Confirmed: OSX 10.5 will feature ZFS
Deletions are marked like this. Additions are marked like this.
Line 44: Line 44:
 * Port ZFS to Linux. N.b. people are working on BSD and OSX ports already.  * Port ZFS to Linux. N.b. people are working on a BSD port already.
 * ZFS will be available in OS X 10.5 ("Leopard") and will be the default filesystem for new systems.

Summary

ZFS is an open source filesystem used in Solaris 10. It has powerful applications for server and workstation use cases, and would be a radical and attractive feature or basis for Ubuntu. It is released under the [http://en.wikipedia.org/wiki/CDDL CDDL] licence, which looks like it will be in universe, not main for now.

Rationale

Ubuntu server, and linux servers in general compete with other unices and windows. ZFS is a killer-app for Solaris, as it allows straightforward administration of a pool of disks, while giving intelligent performance and data integrity.

ZFS does away with partitioning, EVMS, LVM, MD, etc., etc.. The available disks (of any size) are used to the best of their ability. Compression can be used to increase bandwidth. (q.v. Reiser 4, and cloop?)

ZFS is 128 bit, meaning it is excessively scalable. (e.g. 16 exabyte limit)

ZFS achieves its impressive performance through a number of techniques:

  • Dynamic striping across all devices to maximize throughput
  • Copy-on-write design makes most disk writes sequential
  • Multiple block sizes, automatically chosen to match workload
  • Explicit I/O priority with deadline scheduling
  • Globally optimal I/O sorting and aggregation
  • Multiple independent prefetch streams with automatic length and stride detection
  • Unlimited, instantaneous read/write snapshots
  • Parallel, constant-time directory operations

[http://www.sun.com/2004-0914/feature/ Sun ZFS article]

Use cases

  • Dani is a sys admin for a medium-size business. She spends 20% of her time planning/changing/administering/repairing the disk arrays of three servers. With ZFS, this time is cut in half.
  • Ari has a single disk workstation. She buys a new disk and plugs it in. ZFS automatically adds the new disk space into the pool. Her home directory is mirrored, while her OS and temp space is striped automatically in the background.
  • Power user Petra has a four-disk RAID5 workstation. Parity calculations make this a fairly slow set-up because of the number of writes of small files. She upgrades to ZFS and sees performance benefits, because small files are mirrored instead of included in parity calculations.
  • Jack has three disks of different sizes. Configuring a sensible partition and RAID set-up is completely trial and error. ZFS abstracts the three disks into one pool of space, and gives the best balance of performance and security. Jack declares that some media directories do not need to be fault-tolerant, and ZFS transparently stripes them across all the disks.

Scope

Server use case is the most obvious.

Design

ZFS can be used as a drop-in POSIX replacement for existing file systems. However, it provides powerful capabilities which can permeate Ubuntu. E.g. each user is allocate a file-system, cloned from an original: unchanged files are only stored once. This paradigm would suit LTSP, where each user would have filesystem. This sounds more a bit more radical than it is.

Implementation

  • Port ZFS to Linux. N.b. people are working on a BSD port already.
  • ZFS will be available in OS X 10.5 ("Leopard") and will be the default filesystem for new systems.
  • Integrate already ported http://www.wizy.org/wiki/ZFS_on_FUSE

  • Investigate use of ZFS features to replace clunky old linux tools. e.g. Version Control could be done at least partly by the FS ZFS can look back in time to see previous state, so could this be used for reconciling changes, e.g. for Unison or iFolder style file syncing for people who work in several locations.

Code

Data preservation and migration

Outstanding issues

BoF agenda and discussion

References

[http://www.opensolaris.org/os/community/zfs/ opensolaris ZFS community]

[http://en.wikipedia.org/wiki/ZFS wikipedia entry on ZFS]

News and blogs articles

[http://www.internetnews.com/dev-news/article.php/3565221 internetnews] "We would not at all be surprised to see ZFS start to show up in other operating systems at some point in the not-too-distant future," Weinberg said.

[http://blogs.sun.com/roller/page/jwadams?entry=an_initial_encounter_with_zfs a sun blog]

[http://node-0.mneisen.org/2006/12/31/zfs-unter-ubuntu-kubuntu-610-edgy-eft/ German blogpost on how to compile and install ZFS on (K)Ubuntu.] Just follow the supplied command line instructions if you do not speak German :-D ]

Comments

[wiki:JoeyStanford Joey Stanford]: +1. It should also be noted that ext2 and ext3 will no longer under active development as new development is scheduled for [http://lkml.org/lkml/2006/6/28/454 ext4].

Brit Butler: +1. It's pretty clear that Sun has got a terrific filesystem on their hands and that many people are interested in moving in this direction. I realize there are still large issues with getting ZFS on Linux but there's also nothing better than our community for making it happen.


CategorySpec

ZFS (last edited 2019-01-22 22:16:08 by seth-arnold)