TimeVault

Differences between revisions 1 and 59 (spanning 58 versions)
⇤ ← Revision 1 as of 2007-06-21 15:29:29 →
Size: 4409
Editor: 216
Comment: Initial port of docs.html to Wiki
← Revision 59 as of 2009-05-22 08:40:59 → ⇥
Size: 9910
Editor: c-98-212-129-47
Comment: Updated menu path for Ubuntu 9.04
Deletions are marked like this. Additions are marked like this.
Line 4: Line 4:
 * '''Created''': [[Date(2007-06-09T20:00:00Z)]] by A. Bashi
 * '''Contributors''': JaneWeideman, MatthiasKlose		  * '''Created''': <<Date(2007-06-09T20:00:00Z)>> by A. Bashi
Line 7: Line 6:
Line 14: Line 14:
Quick Start
== Screenshots ==

Here: TimeVault/ScreenShots

== TimeVault Snapshot ==

A snapshot is a copy of a directory at a certain point in time. Snapshots don't use space for the files that haven't changed but instead simply add a link to the same data on disk. When a file (link) is deleted, the link count for the data on disk is decremented. When it reaches 0 the data is marked free in the allocation table and new data can then be written to those blocks. As most allocations of data have only 1 file that links to them data is usually freed when its file is deleted.

Only files are hard-linked this way. In most filesystems directory hardlinks are disallowed because they can cause endless recursion. Apple specifically modified their system by adding mandatory recursion detection to prevent infinite loops, ostensibly purely for their backup software.

[[http://arstechnica.com/reviews/os/mac-os-x-10-5.ars/14| This]] is a page about Time Machine and covers how it works. TimeVault has a similar method of creating snapshots.

== Documentation ==

===== 0.7.4 Documentation: =====

(Possibly still some issues in this!)
Line 18: Line 35:
 * You must select a backup directory from System->Administration->Manage Snapshots.
 * You have to log out of and back into GNOME to have the Previous Versions option appear in file properties in Nautilus. Alternately, you can restart Nautilus from the command line: 'nautilus -q'.		  * You must configure TimeVault, the backend will start automatically but the notifier needs to be started: To get the notifier app to start in your systray whenever you log in you can add it to your startup programs in System->Preferences->Startup Applications(System->Preferences->Sessions in the tab Startup Programs if your Ubuntu is older than 9.04) you should have an option to add a program click that, name it TimeVault and put:{{{
'/usr/bin/timevault-notifier'}}}
in the command box, you can put whatever you like in the comments box. After you have done this if you log out and back in again you should magically have the TimeVault notifier app sitting in your systray. Right click on it and go to preferences to configure TimeVault as you wish.
Line 21: Line 39:
== What is a Snapshot ==

A snapshot is a copy of a directory at a certain point in time. Snapshots don't use space for the files that haven't changed but instead simply increment the link count for them. On Linux, when a file is deleted, the link count is decremented, but it isn't actually removed until the link count hits zero.

Only files are hard-linked this way. Directories have to be recreated for technical reasons (since a hard-link is actually the file it points to, you would find yourself in a completely different path if directories were hard-linked).
Line 28: Line 41:
To configure TimeVault, go to the System->Administration menu and select 'Manage Snapshots'. When you start the TimeVault Notifier for the first time the tray icon will be greyed out and if you hover over it it will say that it is unconfigured. Listed below are the steps needed to configure TimeVault:
Line 30: Line 43:
There are four tabs: General, Include, Exclude, and Management.
General		 There are three tabs: General, Include and Exclude.
=== General ===
Line 34: Line 47:
 * Allow users readonly access to snapshots: If this box is not checked, only the superuser (root, or a sudoer) can access the snapshots. If it is, then users will be able to browse snapshots according to their rights. So a user that does not have access to a directory or file will not have access to the snapshot of the directory or file. Note that only read access is granted - snapshots cannot be changed except by the root user (in the Management tab)  * Show snapshot notifications: This switch toggles on and off whether snapshot notifications are to be shown or not.
Line 39: Line 52:
Backup paths are listed here. Every directory that you want to be snapshotted should be added to this list.  * Backup paths are listed here. Every directory that you want to be snapshotted should be added to this list.
Line 43: Line 56:
Paths or patterns to exclude from the snapshots should be listed here  * Paths or patterns to exclude from the snapshots should be listed here
Line 45: Line 58:
=== Management ===
ToDo		 When you have finished making the necessary changes press the save button and TimeVault should pop up a message saying {{{
You have included new directories in the TimeVault. File signatures will now be computed.}}}
Click ok in this box and TimeVault should be active and will start watching your files.
Line 48: Line 62:
== Snapshot Backend == == TimeVault Development ==
Line 50: Line 64:
The snapshot backend initially utilized was rsnapshot. While rsnapshot is an excellent enterprise-level piece of software, it has significant complexity that would make configuration by a casual or beginning user uncomfortable (as it might someone one who is unaccustomed to rsync command-line parameters and cron files). The plethora of features available in rsnapshot and other snapshot software would also remain unused in such a simple interface, and additional layers of software would have to be built on top of rsnapshot (or any of the other many snapshot solutions available - see below). [[TimeVault/Restructure]] (you should check this page for development information)
Line 52: Line 66:
So the approach taken in TimeVault is to implement the snapshot mechanism from scratch (as explained in this seminal paper by Mike Rubel). Alpha versions are available at https://launchpad.net/timevault/+download . Once feature-adding is stopped, it will enter Beta stage after some respectable period of use by Alpha-testers.

Discuss icon and look-and-feel issues in [[TimeVault/Branding]].

Post ideas and wishlist/feature-requests in [[TimeVault/Ideas]].

Ports to Non-Ubuntu systems or a WM other than GNOME are discussed in [[TimeVault/Ports]].

== Change Log ==
=== Version 0.6.7 ===
 * Automatic reconnect to server (if restarted) from status icon
 * Changed debug message system to allow for varying degrees or verbosity (use --verbose multiple times for more debug information)
 * Added metadata saving system (this one is a pretty big change), so that file changes from apps like Beagle and Tracker (who ultimately churn through hundreds of MB a day) are not saved.
 * Added metadata caching dictionary so it needs to be queried only once for changed files per session. Dictionary auto-trims after 32K entries.
 * Added better information printout in Nautilus' property pages
 * Added Snapshot Browser handling for metadata (checkbox to hide metadata and show real changes only)
 * Fallback mechanism for determining if a file is text in the absence of a mime-type (e.g. extensionless files).
 * Diff viewer in Snapshot Browser

=== Version 0.6.8 ===
 * Snapshot picker improvements - filter by dir or file, or by right-clicking on row
 * Improved use of SQLite API (i.e. rewrite all queries so parameters are escaped by DB engine) - avoids SQL injection through crafting malformed filenames

=== Version 0.6.9 ===
 * Internal versioning - not for release

=== Version 0.6.10 ===
 * Relatively seemless configuration reload: Reload configuration when changed without restarting Nautilus, Notifier, or server
 * More config options: Enable/disable nautilus integration from TimeVault Prefs dialog.
 * Pending Snapshots Browser: When notifier is clicked, list pending snapshots (abort snapshot functionality not yet implemented)
 * Change visualization of different snaptypes in picker so that the total height is still logarithmic(base 1.5), but the ratio is linearly proportional - this makes it easier to see what's happening
 * Pending files are resumed if the server restarts
 * Gracefully add metafile/baselines to snapshotter queue (v. <0.6.10 would do so more aggressively, thereby causing unacceptably long lags when initiating a metadata sigs for newly added paths)
 * Fixed a bug where files with carriage returns in them (yes - believe it or not they exist) can be backed
 * Efficiency improvements.

=== Version 0.6.11 ===
 * Fixed Baseline generation bug where you could not generate them from the Prefs screen
 * Added a Take Baseline button in Prefs->Include
 * Added a Real and Meta filters in Snapshot Browser

=== Version 0.6.12 ===
 * Compatibility changes for Python 2.4 on Gentoo

=== Version 0.6.13 ===
 * Take Baseline button properly connected in Prefs->Include

=== Version 0.6.14 ===
 * Small bug fixes

=== Version 0.7.0 ===
 * Alpha Version for bug testers (new point release - no compatibility with saved snapshots from 0.6 branches). If someone really needs a 0.6 => 0.7 importer written, please let me know.
 * Revision of underlying layers (Config, Daemon)
 * Efficiency improvements
 * Live update on Upcoming Snapshots browser
 * Paths restructured to allow hiding of snapshots taken by others
 * Some parts are bound to be broken

=== Version 0.7.4 ===
 * New config file (1.4) format.
 * Nautilus integration currently not working.
 * Baselines currently not working.
 * Preliminary snapshot expiry support?


== FAQ ==
Check [[TimeVault/FrequentlyAskedQuestions]] to see if your question has been answered.

If not, please add it to the list of questions. Another place to look is on the forum post for [[http://ubuntuforums.org/showthread.php?t=474973|TimeVault]] or on [[https://launchpad.net/TimeVault|Launchpad]]
Line 61: Line 143:
 * rsnapshot  * rsnapshot
 * [[http://www.nongnu.org/rdiff-backup/|rdiff-backup]]
 * [[http://flyback-project.org/|flyback]]
 * [[ http://gitorious.org/projects/timevault | KDE TimeVault ]]

== Contributors ==
==== Testing and Bug Triage ====
 * KieranHogg
 * ThomasWolfe2
 * TomVetterlein

==== Logo Design ====
 * DavidEvans

==== Documentation ====
 * ThomasWolfe2
 * TomVetterlein

== Volunteers ==
If you are interesting in helping out, please volunteer here under the appropriate category or create one that suites you.

==== Testing and Bug Triage ====
 * JensTimmerman

==== Graphics ====

==== Documentation ====


== Meetings ==

 * Here are the notes from our IRC meetings: [[TimeVault/MeetingNotes]]
Line 65: Line 178:
 * Make a ToDo list  * KDE Backup System: I (AndrewStromme) have a [[ http://gitorious.org/projects/timevault | KDE project ]] with the TimeVault core ideas. Additionally, I have some [[ http://blog.chatonka.com/2009/02/timevaultng-structure-document-draft/ | Blog posts ]] regarding it.
 * KDE-port: I've done the GUI for KDE port, the code obviously needs a fair few changes but it's on the back boiler for a few weeks. -- KieranHogg

  • Launchpad Entry: https://launchpad.net/timevault

  • Created: 2007-06-09 by A. Bashi

  • Packages affected: timevault

  • Depends: python, python-gnome2, python-glade2, gksu, sudo, nautilus, gnome-terminal, python-nautilus, python-notify, python-gobject, python-gobject-dev, libpango1.0-0

Overview

TimeVault is a simple front-end for making snapshots of a set of directories. Snapshots are a copy of a directory structure or file at a certain point in time. Restore functionality is integrated into Nautilus - previous versions of a file or directory that has a snapshot can be accessed by examining the properties and selecting the 'Previous Versions' tab.

Snapshots are protected from accidental deletion or modification since they are read-only by default. The super-user can delete intermediate snapshots to save space, but files and directories that existed before or after the deletion will still be accessible.

Screenshots

Here: TimeVault/ScreenShots

TimeVault Snapshot

A snapshot is a copy of a directory at a certain point in time. Snapshots don't use space for the files that haven't changed but instead simply add a link to the same data on disk. When a file (link) is deleted, the link count for the data on disk is decremented. When it reaches 0 the data is marked free in the allocation table and new data can then be written to those blocks. As most allocations of data have only 1 file that links to them data is usually freed when its file is deleted.

Only files are hard-linked this way. In most filesystems directory hardlinks are disallowed because they can cause endless recursion. Apple specifically modified their system by adding mandatory recursion detection to prevent infinite loops, ostensibly purely for their backup software.

This is a page about Time Machine and covers how it works. TimeVault has a similar method of creating snapshots.

Documentation

0.7.4 Documentation:

(Possibly still some issues in this!)

After installing the .deb file, there are two steps required to complete installation:

  • You must configure TimeVault, the backend will start automatically but the notifier needs to be started: To get the notifier app to start in your systray whenever you log in you can add it to your startup programs in System->Preferences->Startup Applications(System->Preferences->Sessions in the tab Startup Programs if your Ubuntu is older than 9.04) you should have an option to add a program click that, name it TimeVault and put:

    '/usr/bin/timevault-notifier'

in the command box, you can put whatever you like in the comments box. After you have done this if you log out and back in again you should magically have the TimeVault notifier app sitting in your systray. Right click on it and go to preferences to configure TimeVault as you wish.

Configuration

When you start the TimeVault Notifier for the first time the tray icon will be greyed out and if you hover over it it will say that it is unconfigured. Listed below are the steps needed to configure TimeVault:

There are three tabs: General, Include and Exclude.

General

  • Enable Automated Snapshots: By default, TimeVault will not automatically take snapshots of your system. Checking this box will tell it to automatically take daily snapshots.

  • Show snapshot notifications: This switch toggles on and off whether snapshot notifications are to be shown or not.
  • Snapshot Root Directory: Where to store the snapshots. Ideally, you should pick a directory that is mounted onto another physical drive from the data you are backing up. This is so that, in the case of a hard-disk failure, the backups will remain intact (unless the drive containing the backups fails, in which case the current version of the data remains).

Include

  • Backup paths are listed here. Every directory that you want to be snapshotted should be added to this list.

Exclude

  • Paths or patterns to exclude from the snapshots should be listed here

When you have finished making the necessary changes press the save button and TimeVault should pop up a message saying 

You have included new directories in the TimeVault. File signatures will now be computed.

Click ok in this box and TimeVault should be active and will start watching your files.

TimeVault Development

TimeVault/Restructure (you should check this page for development information)

Alpha versions are available at https://launchpad.net/timevault/+download . Once feature-adding is stopped, it will enter Beta stage after some respectable period of use by Alpha-testers.

Discuss icon and look-and-feel issues in TimeVault/Branding.

Post ideas and wishlist/feature-requests in TimeVault/Ideas.

Ports to Non-Ubuntu systems or a WM other than GNOME are discussed in TimeVault/Ports.

Change Log

Version 0.6.7

  • Automatic reconnect to server (if restarted) from status icon
  • Changed debug message system to allow for varying degrees or verbosity (use --verbose multiple times for more debug information)
  • Added metadata saving system (this one is a pretty big change), so that file changes from apps like Beagle and Tracker (who ultimately churn through hundreds of MB a day) are not saved.
  • Added metadata caching dictionary so it needs to be queried only once for changed files per session. Dictionary auto-trims after 32K entries.
  • Added better information printout in Nautilus' property pages
  • Added Snapshot Browser handling for metadata (checkbox to hide metadata and show real changes only)
  • Fallback mechanism for determining if a file is text in the absence of a mime-type (e.g. extensionless files).
  • Diff viewer in Snapshot Browser

Version 0.6.8

  • Snapshot picker improvements - filter by dir or file, or by right-clicking on row
  • Improved use of SQLite API (i.e. rewrite all queries so parameters are escaped by DB engine) - avoids SQL injection through crafting malformed filenames

Version 0.6.9

  • Internal versioning - not for release

Version 0.6.10

  • Relatively seemless configuration reload: Reload configuration when changed without restarting Nautilus, Notifier, or server

  • More config options: Enable/disable nautilus integration from TimeVault Prefs dialog.

  • Pending Snapshots Browser: When notifier is clicked, list pending snapshots (abort snapshot functionality not yet implemented)
  • Change visualization of different snaptypes in picker so that the total height is still logarithmic(base 1.5), but the ratio is linearly proportional - this makes it easier to see what's happening
  • Pending files are resumed if the server restarts

  • Gracefully add metafile/baselines to snapshotter queue (v. <0.6.10 would do so more aggressively, thereby causing unacceptably long lags when initiating a metadata sigs for newly added paths)

  • Fixed a bug where files with carriage returns in them (yes - believe it or not they exist) can be backed
  • Efficiency improvements.

Version 0.6.11

  • Fixed Baseline generation bug where you could not generate them from the Prefs screen

  • Added a Take Baseline button in Prefs->Include

  • Added a Real and Meta filters in Snapshot Browser

Version 0.6.12

  • Compatibility changes for Python 2.4 on Gentoo

Version 0.6.13

  • Take Baseline button properly connected in Prefs->Include

Version 0.6.14

  • Small bug fixes

Version 0.7.0

  • Alpha Version for bug testers (new point release - no compatibility with saved snapshots from 0.6 branches). If someone really needs a 0.6 => 0.7 importer written, please let me know.

  • Revision of underlying layers (Config, Daemon)
  • Efficiency improvements
  • Live update on Upcoming Snapshots browser
  • Paths restructured to allow hiding of snapshots taken by others
  • Some parts are bound to be broken

Version 0.7.4

  • New config file (1.4) format.
  • Nautilus integration currently not working.
  • Baselines currently not working.
  • Preliminary snapshot expiry support?

FAQ

Check TimeVault/FrequentlyAskedQuestions to see if your question has been answered.

If not, please add it to the list of questions. Another place to look is on the forum post for TimeVault or on Launchpad

Other snapshot solutions include:

Contributors

Testing and Bug Triage

Logo Design

Documentation

Volunteers

If you are interesting in helping out, please volunteer here under the appropriate category or create one that suites you.

Testing and Bug Triage

Graphics

Documentation

Meetings

ToDo

  • KDE Backup System: I (AndrewStromme) have a KDE project with the TimeVault core ideas. Additionally, I have some Blog posts regarding it.

  • KDE-port: I've done the GUI for KDE port, the code obviously needs a fair few changes but it's on the back boiler for a few weeks. -- KieranHogg

CategorySpec

TimeVault (last edited 2011-08-28 18:08:15 by 105)