TimeWarp Archiver

The Archiver is a separate process of which one or more will run when TimeWarp is executing. The number of Archiver processes is configurable.

The Achiver process has the simple task of storing the original file ito the relevant Archive, including recording :

• Change in file contents
• Change in file permissions/ownership
• File Deletions
• File Moves.

See Archive File Specification for details of the archive file format.

Interface with the Scheduler

The interface with the Scheduler process will be defined as soon as is sensible - the exact format is a work in progress.

Scheduler to Archiver

Data items in the Inteface - proposed

• DB Item Id
• Type of Update
  • STAT : Change to file permissions/Owership
  • UPDATE : Change to file content
  • DELETE : File Deletion
  • MOVE FROM : File moved or renamed from this name
  • MOVE TO : File moved to this name
• Path : Absolute path name of the file

Archiver to Scheduler

• DB Item Id
• Completion Status
  • COMPLETE - Archive operation completed successfully
  • FAILED - Archive operation failed
• Archive Size - for UPDATE operations, the data size that was added to the Archive
• CPU Time - the CPU time spent on this archive operation

Archive Size and CPU Time are optionally for use by the Schedule to provide the back-off algorithm (i.e. the scheduler can limit the amount of CPU time (and data bandwidth)) used over time - probably not used in the Alpha code.

ArchiverClass

Most of the functionality of the Archive process is provided by the Archiver class. This class also contains methods to allow recovey of files from the Archive, which although not usd in the Archiver process, does allow the same class to be reused where ever the Archives need to be accessed.

A full spec with Doc strings will be included here when ready.

Integrity Maintenance

The Archiver process also has the task of maintaining the integrity of invidual archive files and tryig to ensure that should the Archive process be interrupted during an archive attempt, that the data is recoverable.

For any update to an archive File the following pocess attempts to maintain integrity

1. Check that a Backup file exists - if it does delete the main archive file and skip to step c.
2. Archive file to be modified is moved to a new name in the same folder (creatig a backup)- this should be an atomic operation.
3. The Backup is copied to the original file name.
4. The Archive is modified as appropriate
5. The Back up is deleted - this should also be an atomic operaton
6. The Archive opeation is marked as complete.

If an archive operation is interrupted for any reason on or after item c (and before f), a backup of the un-modified Archive file is retained, and is automatically recovered when that archive operation is re-queued. An interruption after f will mean that whe the "incomplete" archive operation is re-queued, then the Archive file gets a double entry for that update - that is a small risk of occuring