TransparentServicePackMaker

Differences between revisions 1 and 6 (spanning 5 versions)
Revision 1 as of 2007-05-11 14:42:08
Size: 7000
Editor: dyn233056
Comment: Initial creation
Revision 6 as of 2008-08-06 16:25:14
Size: 7968
Editor: localhost
Comment: converted to 1.6 markup
Deletions are marked like this. Additions are marked like this.
Line 8: Line 8:
This spec covers the creation of an easy to use tool to create ''service packs'' (collections of updates, new programs, etc.) which require nothing more than the current APT system to install (hence "transparent", since the packs are treated as regular packages). This spec covers the creation of an easy to use tool to create ''service packs'' (bundles of packages which may contain updates, new programs, etc.) which require nothing more than the current APT system to install (hence "transparent", since the packs are treated as regular packages).
Line 12: Line 12:
It is often useful to install a defined set of packages on a system, or multiple systems, with or without Internet access. The idea of letting users and developers create ''Service Packs'' (UbuntuServicePacks for example), large collections of software installations/updates, addresses this, but current proposals for implementing such a system (like OfflineUpdateSpec) are limited due to the reliance on a service pack installation tool. Allowing service packs to be installed with nothing more than the regular APT system (using tools like Gdebi) would instantly make such packs available for all Ubuntu (or Debian-based) users, and potentially other distros, without creating yet another application installation format/method. Currently packaged software is either offered in a repository (causing unfamiliar setup for those who are unfamiliar with the concept, plus the need for a working connection to the repository), as individual package files (causing unneeded headaches when installing, especially graphically, as packages must be installed in the correct order, existing packages may get overwritten, etc.) or as a massive package incorporating what should be split into several packages (in an attempt to overcome the previous problem, but introducing new ones like difficulty replacing individual parts). Thus a method of distributing multiple packages, like a local repository, needs to be given an easy installation method, like a single package.

It is often useful to install a defined set of packages on a system, or multiple systems, with or without Internet access. The idea of letting users and developers create ''Service Packs'' (UbuntuServicePacks for example), large collections of software installations/updates like those mentioned above, addresses this, but current proposals for implementing such a system (like OfflineUpdateSpec) are limited due to the reliance on a service pack installation tool, and applications like APTonCD offer more complexity than is needed to achieve the goals of this spec (although their underlying infrastructure could be reused). Allowing service packs to be installed with nothing more than the regular APT system (using tools like Gdebi) would instantly make such packs available for all Ubuntu (or Debian-based) users, and potentially other distros, without creating yet another application installation format/method, thus making the system promising to third party vendors who can support many Debian-derived distributions with one easy to install file.
Line 16: Line 18:
* John has installed Ubuntu on 50 isolated machines for his employees, however the software installed by default is not ideal for their jobs. John creates a service pack called "work-applications" which contains all of the extra programs needed by his employees, then installs it onto each system by double clicking the created Debian package to launch Gdebi.  * John has installed Ubuntu on 50 isolated machines for his employees, however the software installed by default is not ideal for their jobs. John puts the needed packages into a service pack called "work-applications" which is then populated by any dependancies needed to get those packages running. He then installs it onto each system by double clicking the created Debian package to launch Gdebi.
Line 18: Line 20:
* Lynn's mother wants to keep in contact with her daughter when she moves away, so Lynn installs Ubuntu for her and sets up email and instant messaging applications. Lynn's mother doesn't understand the updates or installation tools, so Lynn posts her occasional service packs on CD containing all of the security and speed updates Lynn's mother could do with, without any extra programs or features which would confuse her.  * Lynn's mother is given a computer running Ubuntu by her daughter before she moves away to manage her finances and print leaflets her bakesales, but due to financial reasons she does not have an Internet connection. Lynn posts her occasional service packs on CD containing all of the updates since the previous pack was sent.
Line 20: Line 22:
* Dan works in a professional multimedia company and needs to install certain specialist applications on his workforce's Ubuntu computers. Due to the size of these programs, and the differing needs of his team, he makes different service packs called "graphical-tools" "audio-tools" and "video-tools" which all rely on the service pack "standard-setup", thus everyone gets the standard tools, each gets their own specialist tools, and no space is wasted with unneccessary programs installed or bloated service packs.  * Dan works in a professional multimedia company and needs to install certain specialist applications on his workforce's Ubuntu computers. Due to the size of these programs, and the differing needs of his team, he makes different service packs called "graphical-tools" "audio-tools" and "video-tools" which all rely on the service pack "standard-setup", thus everyone gets the standard tools, each gets their own specialist tools, and no space is wasted with unneccessary programs installed or bloated service packs.
Line 22: Line 24:
* Gemma has made a game and wants to package it for Ubuntu. The game depends on some obscure libraries, and some non-standard but widely used libraries. She puts packages of all of these libraries, along with her game, into a service pack so users installing it will definetly have all of the required libraries, however any newer versions of the widespread libraries already installed will not get overwritten by those in the pack.  * Gemma has made a game and wants to package it for Ubuntu. The game depends on some obscure libraries. She puts a package of her game into a service pack which is then populated with these libraries, so she can be sure users installing it will have all of the libraries needed without worrying about overwriting currently installed packages.

 * Gemma distributes her game on CDs, including distribution-specific packages. She finds that many Debian-derived distributions such as Ubuntu can be catered for in one service pack instead of presenting the user with individual packages for each of these distributions.
Line 26: Line 30:
A new tool will be developed which allows easy creation of service packs. For the service packs to be seen as an APT source there may need to be some work done on adding temporary sources. A new tool will be developed which allows easy creation of service packs.
Line 30: Line 34:
The tool will easily allow users to create "update" service packs by including all of the non-default (ie. not installed through k/x/ed/ubuntu-minimal/desktop meta-packages) packages installed (refetching them if needed, taking from /var/chache/apt/archives if not), along with any extra packages specified via a text box and collating them into a folder as an APT repository (if the user does not tick the include non-default packages box then only these 'extra' packages will be included). A meta-package will then be created in the same directory level as the repository folder which depends on everything inside. A ".hidden" file may also be created to hide the folder, thus leaving only the meta-package visible (however, users may think that the meta-package is the entire pack and thus move/copy it without its accompanying repository. This could be solved by distributing the folder and meta-package in an archive, or else just do away with the .hidden file altogether. This decision needs discussion). The tool will let users specify packages for the Service Pack to depend on and easily allow users to create "update" service packs by including all of the packages installed on the system, thus fulfilling Lynn's use case (size can be kept down by specifying a base package to depend on, like ubuntu-desktop, or by depending on a previous update pack). The tool will also allow users to specify packages based on their name or a path to a package file, and find out any other packages needed to get them installed, thus fulfilling Gemma's use case. A meta-package will then be created which depends on everything inside. Finally a 'setup' package is made which handles installing the whole thing. This setup packae is the only part users need to interact with, the rest is stored in a folder called 'data'.
Line 32: Line 36:
When a user double clicks the meta-package the Gdebi tool appears and allows them to install the pack just like any other package, although any packages not currently installed or available as a newer version in a current repository are taken from the included reopsitory folder. If the service pack meta-package is removed then so is any additional software it installed (the standard behaviour for tools like aptitude, which I think is now in apt-get and anyway is beyond the scope of this spec). If any software in the pack is removed then so is the pack.

An "Advanced" tab in the creation tool allows users to specify exact versions for the included packages if they want to, as well as any service pack dependencies (to let Dan's "video-tools" require the "standard-setup" pack). These dependencies just add the meta-package of the pack being depended on as a dependency of the new pack's meta-package, thus the existing system does not need modifying.
When a user double clicks the setup package the Gdebi tool appears like normal and allows them to install the pack just like any other package, this then sets up the repository and installs the packages inside if they are not already installed/available in a cofigured repository. The setup package is then removed as it is set to conflict with the meta-package. If the service pack meta-package is removed then so is any additional software it installed (the standard behaviour for tools like aptitude, outside the scope of this spec). If any software in the pack is removed then so is the pack, due to dependancy reasons.
Line 38: Line 40:
The tool should probably be made in Python with GTK/QT frontends, since the tool will only be run occasionally, and never as a daemon, thus Python's interpreted nature won't cause any system slowdown. The tool should probably be made in Python with GTK/QT frontends, since the tool will only be run occasionally, and never as a daemon, thus Python's interpreted nature won't cause any system slowdown. The tool APTonCD, which emerged at around the same time the idea for this spec did, does some of the jobs proposed here, but also does others besides and is complex to use. The underlying code could be used to implement the proposed tool, however, since its capabilities are similar.
Line 40: Line 42:
I (["Warbo"]) am working on a basic prototype of this tool, although help would be appreciated and also it is made in Java, since I am not particularly good in Python yet. The GUI is usable (although the "Advanced" settings will be accessed [when implemented] via a button, as the RAD tool used did not handle tabs well), although the file reading (for dependencies) and writing (to make the packs and packages) need to be implemented correctly.

(Link to current prototype will be put here shortly, once it is uploaded somewhere)

The current plan, until the APT source adding issue is resolved, is to create 2 packages, the first of which has no dependencies and adds a file to sources.list.d and the second of which, the main meta-package, removes this after installation (and possibly conflicts with the first package, thus causing its removal).
I ([[Warbo]]) am experimenting with a prototype of this tool. A GUI has been designed and package handling code made, thus showing the idea as feasable. The only outstanding issue is making the setup package set up and install the service pack (the code is written, but doesn't run on package installation), also current methods employed do not exist cross-distro (mainly the use of postpone). This work-in-progress can be downloaded [[http://www.freewebs.com/chriswarbo/Temporary/TransparentServicePackMaker.tar.bz2|here]] (this depends on postpone, from [[http://www.df7cb.de/projects/postpone/|here]] for non-Gutsy users, and reprepro which has been packaged for a while).
Line 48: Line 46:
The way to add APT sources should be addressed. The point of the tool is for easy, (apparently) one-file installation of large software collections, however this one file will not install without APT using its accompanying repository as a source, yet the system cannot be changed (to add the source) without installing something (hence the current solution above). Currently the setup package does not set up and install the generated sevice pack. This is due to dpkg lock issues, since the installation scripts inside work properly when run from a terminal. If this issue persists then the setup package could be replaced by a setup script (note that APT is used throughout, this is not yet-another-install-tool, the script would just automate adding a local repository and installing a package).

The code used in the prototype is not very reliable or reusable. The final implementation should be done using code from APTonCD.
Line 52: Line 52:
["Warbo"]: Please add any thoughts on the tool itself, ideas for overcoming the source issue, whether using a .hidden file would be a good idea or anything else :) [[Warbo]]: Please add any thoughts on the tool itself, ideas for overcoming the source issue or anything else :)

[[Warbo]]: APT-on-CD is now incredibly similar to the Service Pack tool I am making... Perhaps adapting APT-on-CD would be a better course of action?

Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.

Summary

This spec covers the creation of an easy to use tool to create service packs (bundles of packages which may contain updates, new programs, etc.) which require nothing more than the current APT system to install (hence "transparent", since the packs are treated as regular packages).

Rationale

Currently packaged software is either offered in a repository (causing unfamiliar setup for those who are unfamiliar with the concept, plus the need for a working connection to the repository), as individual package files (causing unneeded headaches when installing, especially graphically, as packages must be installed in the correct order, existing packages may get overwritten, etc.) or as a massive package incorporating what should be split into several packages (in an attempt to overcome the previous problem, but introducing new ones like difficulty replacing individual parts). Thus a method of distributing multiple packages, like a local repository, needs to be given an easy installation method, like a single package.

It is often useful to install a defined set of packages on a system, or multiple systems, with or without Internet access. The idea of letting users and developers create Service Packs (UbuntuServicePacks for example), large collections of software installations/updates like those mentioned above, addresses this, but current proposals for implementing such a system (like OfflineUpdateSpec) are limited due to the reliance on a service pack installation tool, and applications like APTonCD offer more complexity than is needed to achieve the goals of this spec (although their underlying infrastructure could be reused). Allowing service packs to be installed with nothing more than the regular APT system (using tools like Gdebi) would instantly make such packs available for all Ubuntu (or Debian-based) users, and potentially other distros, without creating yet another application installation format/method, thus making the system promising to third party vendors who can support many Debian-derived distributions with one easy to install file.

Use Cases

  • John has installed Ubuntu on 50 isolated machines for his employees, however the software installed by default is not ideal for their jobs. John puts the needed packages into a service pack called "work-applications" which is then populated by any dependancies needed to get those packages running. He then installs it onto each system by double clicking the created Debian package to launch Gdebi.
  • Lynn's mother is given a computer running Ubuntu by her daughter before she moves away to manage her finances and print leaflets her bakesales, but due to financial reasons she does not have an Internet connection. Lynn posts her occasional service packs on CD containing all of the updates since the previous pack was sent.
  • Dan works in a professional multimedia company and needs to install certain specialist applications on his workforce's Ubuntu computers. Due to the size of these programs, and the differing needs of his team, he makes different service packs called "graphical-tools" "audio-tools" and "video-tools" which all rely on the service pack "standard-setup", thus everyone gets the standard tools, each gets their own specialist tools, and no space is wasted with unneccessary programs installed or bloated service packs.
  • Gemma has made a game and wants to package it for Ubuntu. The game depends on some obscure libraries. She puts a package of her game into a service pack which is then populated with these libraries, so she can be sure users installing it will have all of the libraries needed without worrying about overwriting currently installed packages.
  • Gemma distributes her game on CDs, including distribution-specific packages. She finds that many Debian-derived distributions such as Ubuntu can be catered for in one service pack instead of presenting the user with individual packages for each of these distributions.

Scope

A new tool will be developed which allows easy creation of service packs.

Design

The tool will let users specify packages for the Service Pack to depend on and easily allow users to create "update" service packs by including all of the packages installed on the system, thus fulfilling Lynn's use case (size can be kept down by specifying a base package to depend on, like ubuntu-desktop, or by depending on a previous update pack). The tool will also allow users to specify packages based on their name or a path to a package file, and find out any other packages needed to get them installed, thus fulfilling Gemma's use case. A meta-package will then be created which depends on everything inside. Finally a 'setup' package is made which handles installing the whole thing. This setup packae is the only part users need to interact with, the rest is stored in a folder called 'data'.

When a user double clicks the setup package the Gdebi tool appears like normal and allows them to install the pack just like any other package, this then sets up the repository and installs the packages inside if they are not already installed/available in a cofigured repository. The setup package is then removed as it is set to conflict with the meta-package. If the service pack meta-package is removed then so is any additional software it installed (the standard behaviour for tools like aptitude, outside the scope of this spec). If any software in the pack is removed then so is the pack, due to dependancy reasons.

Implementation

The tool should probably be made in Python with GTK/QT frontends, since the tool will only be run occasionally, and never as a daemon, thus Python's interpreted nature won't cause any system slowdown. The tool APTonCD, which emerged at around the same time the idea for this spec did, does some of the jobs proposed here, but also does others besides and is complex to use. The underlying code could be used to implement the proposed tool, however, since its capabilities are similar.

I (Warbo) am experimenting with a prototype of this tool. A GUI has been designed and package handling code made, thus showing the idea as feasable. The only outstanding issue is making the setup package set up and install the service pack (the code is written, but doesn't run on package installation), also current methods employed do not exist cross-distro (mainly the use of postpone). This work-in-progress can be downloaded here (this depends on postpone, from here for non-Gutsy users, and reprepro which has been packaged for a while).

Outstanding Issues

Currently the setup package does not set up and install the generated sevice pack. This is due to dpkg lock issues, since the installation scripts inside work properly when run from a terminal. If this issue persists then the setup package could be replaced by a setup script (note that APT is used throughout, this is not yet-another-install-tool, the script would just automate adding a local repository and installing a package).

The code used in the prototype is not very reliable or reusable. The final implementation should be done using code from APTonCD.

BoF agenda and discussion

Warbo: Please add any thoughts on the tool itself, ideas for overcoming the source issue or anything else Smile :)

Warbo: APT-on-CD is now incredibly similar to the Service Pack tool I am making... Perhaps adapting APT-on-CD would be a better course of action?


CategorySpec

TransparentServicePackMaker (last edited 2008-08-06 16:25:14 by localhost)