TimeWarp Configuration items

TimeWarp is highly configurable - including coarse grained and find grained filtering, performance tuning, and various options to assist with development and debugging.

TimeWarp configuration comes in two flavours - system wide configuration, and per-user configuration.

System wide configuration

System wide configuration is via the file /etc/conf/TimeWarp.conf file, although it will also be possible to start the TimeWarp Suite with a alternative cofig file. System wide configuration contains global configuration for backup destinations, source directories, global filters, as well as advanced options to tune performance.

User Specific Config

If the system wide configuration enables User config see the USER_CONFIGoption then the application will look for user specific configuration at <User Home>/.TimeWarp/TimeWarp.conf. A user can optionally create a configuration file, at set their own source directories, and filters for their files. If a user does not create a config file, then the global settings apply

Configurable items

The Following items are proposed to be configurable :

Minimum Set

These two paramters must be set in the system-wide configuration file

• BACKUP_ROOT must be a single existing directory which has read/write permissions for the user running TimeWarp.

• SOURCE DIRS is a comma separated list of directories or files. For all entries which are directories, then all files in that directory and all sub-directories are watched for changes - symlinks are not followed.

The user specific configuration file can also optionally contain the SOURCE_DIRS settings, but cannot contain the BACKUP_ROOT setting.

Filters

This group of parameters provide fine grained filter over the contents which are to be backed up. Any of these files in the user specific configuration file apply only to the contents of SOURCE_DIRS for that user.

The filtering settings are applied in the following ordering - Each time a file changes it is considered against these attributes, and as soon as a file is excluded then that file is not considered further

• EXCLUDE_PATH is a comma separated list of full paths to directories which are excluded. These are abolute paths - not wildcards.
• MIN_SIZE and MAX_SIZE define a range of file sizes in bytes which will be backed up.
• HIDDEN_FILES Whether hidden files, or files which are in hidden directories are excluded.
• EXLUDE_SUFFIX is a comma separated list of wildcard strings. The file name suffix (i.e. after the final "." is compared against each entry in this setting, and is not backed up if it matches one or more entry.
• EXCLUDE_MATCH is a comma separated list of wildcard strings. Each full path of each file that changes is compared against each entry in this setting, and is not backed up if it matches one or more entry.

• EXCLUDE_MATCH & EXCLUDE_SUFFIX are similar to EXCLUDE_MATCH and EXCLUDE_SUFFIX except that files which match are included and items which don't match are excluded.

Conflicts between system and user local config

Given the ability to create system and user configurations, it seems to appear possible to create conflicts such that files included by the system config is excluded by the user, or vice versa.

These conflicts can only occur if the system wide configuration and the user local configurations define SOURCE_DIR entries which overlap.

If the overlapping directory are outside the users default /home then the system wide settings take priority.

If the overlapping directories are within the users default /home then the user local settings take priority.

This allows a single advanced user to define his own backup settings for his own /home area without impacting the backup settings for any other user /home which uses the system wide settings.

Allowing User system Configuration

Setting this item to True in the system-wide configuration file will make TimeWarp look for user specific configuration files.

Limiting user system configuration

TimeWarp provides two different ways to limit which users can have user specific configurations. These item should be set in the system-wide configuration file.

• UID_LIST : A comma separated list of UIDs, which can range from 0 and must be unsigned integers. Only users with UIDs in this list will have their user specific configuration file used, all other files will be ignored.

• UID_RANGE : A range specifier (two integers separated by a colon) which specifies the minimum and maximum UID for users whose user specific configuration file will be used. Users who UID is outside the specified range while not have their configuration file used.

Advanced Settings

Performance Tuning

The following settings define various parameters which affect the performance of the application

Processes, CPU and Bandwidth

Timwearp is designed to ensure that it does not overload either the system it is executing on, or the network it is storing the archives on. To this end a number of paramters can be set :

MAX_ARCHIVERS : This sets the number of archive processes that will run on the system - on a busy system this value can be set lower - at the expense of less reponsive backups - i.e. file changes will wait longer to be recorded.

MAX_CPU : This limits the maximum amount of CPU that the archivers consume as they archive the file changes - as a % of the total CPU_TIME available. Due to the nature of the archiving and measuring process, the CPU usage will be bursty, but the CPU_TIME limit will be met on average.

MAX_BANDWIDTH : This limits the maximum amount of DATA that the archivers can transfer as they archive the file changes - as a bytes per second. Due to the nature of the archiving and measuring process, the CPU usage will be bursty, but the MAX_BANDWIDTH limit will be met on average.

Scheduling Strategies

The following values are envisaged :

• OLDEST_FIRST : The file with the longest time on the QUEUE gets archived first
• LARGEST_FIRST : The file with the largest on disk size gets archived first

The scheduler will fetch at most SCHEDULER_BATCHSIZE items from the queue, and wait for all of them to complete their archive operations before fetching another batch.

The SCHEDULER_BATCHSIZE can be changed to allow optimisation especially on busy systems. A low value would allow the system to react very quickly to new items entering the queue, with the downside of more queries on the database. A high value will make the system less responsive to rapid changes but would reduce the number of fetches on the database.

It would make no sense to set this value to lower than MAX_ARCHIVERS.

DWELL_TIME : This parameter prevents a rapidly changing file from creating too many archive entries. an update to a file will replace an earlier entry in the archive if the eariler entry is less than DWELL time seconds old. Changing this file will refuce the number of backups and thus save on resources if the system has files which frequently change.

Optional File locations

These options change where in the filesystem TimeWarp looks for or stores various items