MaverickFinishUpstart

Differences between revisions 6 and 12 (spanning 6 versions)
⇤ ← Revision 6 as of 2010-12-08 09:58:25 →
Size: 10090
Editor: 89
Comment:
← Revision 12 as of 2010-12-15 09:22:10 → ⇥
Size: 20140
Editor: 92
Comment: pause(2) is not used for debug stanza. "manual" can be used in any type of job configuration file (.conf or .override).
Deletions are marked like this. Additions are marked like this.
Line 5: Line 5:
 * '''Contributors''': JamesHunt  * '''Contributors''': JamesHunt, ScottJamesRemnant
Line 11: Line 11:
 1. Introduce Manual Stanza
Line 12: Line 13:
 1. Add start_on and stop_on D-Bus properties
Line 13: Line 15:
 1. Socket activation
 1. D-Bus activation
 1. Session support
 1. Chroot session support
 1. User session support
Line 17: Line 25:
 1. Support has been added for "override" files. These ".override" files -- which are not created by default -- allow tools to modify the behaviour of any job by overriding the job configuration file in various ways.  1. Upstart now supports a new "manual" stanza that if specified in a job configuration file causes Upstart to forget any previous "start on" stanza specified in the file. This allows system administrators to disable jobs from automatically started without excessive modification.
 1. Support has been added for "override" files. These ".override" files -- which are not created by default -- allow tools to modify the behaviour of any job by overriding the job configuration file in various ways. This is particularly useful to add the "manual" stanza defined previously.
 1. External tools may now query Upstart jobs for the conditions that will cause them to start and to stop.
Line 19: Line 29:
 1. It is now possible to activate Upstart jobs on socket connection
 1. It is now possible to activate Upstart jobs as a result of D-Bus service activation
 1. Upstart now supports multiple collections of independent job configurations
 1. Upstart now supports and can supervise jobs defined in an independent chroot to the main system
 1. Upstart now supports jobs run on behalf of non-root users
Line 24: Line 39:
This feature will make it significantly easier to debug both Upstart and
 any child processes it creates.		 This feature will make it significantly easier to debug both Upstart and any child processes it creates.

=== Introduce Manual Stanza ===

This feature will make it significantly easier to disable jobs from being automatically stated.
Line 33: Line 51:

=== Add start_on and stop_on D-Bus properties ===

This feature provides a basis for building the visualization feature on top of without requiring extraction or re-use of the configuration parser, and it also allows external bridges to use the set of matched events to determine what to listen for.
Line 44: Line 66:
=== Socket activation ===

This feature is considered to be one of the more compelling features of systemd, most people don't realise that Upstart was designed to support it and that it's a trivial addition even if we haven't found a cause to use it yet. Demonstrating this would be a good thing.

=== D-Bus activation ===

Since many jobs are likely to move to systemd as they have Fedora upstreams, we should have a simple way to replace them with equivalent Upstart jobs.

=== Session support ===

This feature provides the basis for building the chroot and user job support into Upstart.

=== Chroot session support ===

Ubuntu systems are often run in chroots, even if the chroot doesn't need full startup support, it is convenient to be able to start and stop the services in the chroot.

=== User session support ===

Ubuntu systems contain many user services, as well as the user desktop sessions, Upstart should be able to manage those as well.
Line 48: Line 90:
 * Jennifer is an application developer and wants to single-step a
   daemon process in a production-like environment to help her
   understand a bug reported by one of her users.

 * Donald has written a new daemon and is trying to understand why it
   doesn't run as he expects.		  * Jennifer is an application developer and wants to single-step a daemon process in a production-like environment to help her understand a bug reported by one of her users.

 * Donald has written a new daemon and is trying to understand why it doesn't run as he expects.

=== Introduce Manual Stanza ===

 * Sam is a system administrator who does not want a particular service to start when the server starts.
Line 57: Line 100:
 * Ben is a system administrator who managers a large group of database
   servers. The vendor has provided a buggy upstart job configuration
   file which needs to be worked around to avoid database servers from
   failing to start. Ben cannot change the existing ".conf" file since
   server security auditing software would flag this as an error.

 * Hermione wants to write a simple tool that allows a job to be
   disabled without modifying the existing ".conf" files.

 * Rafael wants to be able to change the run-levels a job runs in
   without modifying the existing ".conf" file.		  * Ben is a system administrator who managers a large group of database servers. The vendor has provided a buggy upstart job configuration file which needs to be worked around to avoid database servers from failing to start. Ben cannot change the existing ".conf" file since server security auditing software would flag this as an error.

 * Hermione wants to write a simple tool that allows a job to be disabled without modifying the existing ".conf" files.

 * Rafael wants to be able to change the run-levels a job runs in without modifying the existing ".conf" file.

=== Add start_on and stop_on D-Bus properties ===

 * Michael is a Linux distribution engineer who wishes to generate a dotty graph of their boot sequence.

 * Janine is a software developer who wishes to add a bridge between Upstart and their project, but to do that she needs to know what conditions to match to query her project and develop the bridge.
Line 71: Line 114:
 * Finn is an administrator and wants a log of all jobs showing when they
   ran and how long they took to run.

 * Lauren is a developer and wants to know how long the server
   application she has written actually ran before it crashed and what
   other jobs were running at the time of the crash.		  * Finn is an administrator and wants a log of all jobs showing when they ran and how long they took to run.

 * Lauren is a developer and wants to know how long the server application she has written actually ran before it crashed and what other jobs were running at the time of the crash.

=== Socket activation ===

 * Ted is a desktop developer, the latest graphical server relies on socket activation to start X11 support when required

=== D-Bus activation ===

 * Lucy is a plumbing engineer, several of her packages with Fedora upstreams have switched to relying on the init daemon to perform D-Bus activation, and she wishes a trivial way to convert to Upstart

=== Session support ===

 * Scott is an Upstart developer and wishes to implement chroot support and user session support.

=== Chroot session support ===

 * Colin is a build daemon administrator, build daemons run inside chroots and need managed services

=== User session support ===

 * Martin is a desktop developer, the desktop session contains services that require as much supervision as those running as the system.

 * Michelle is a server developer, she'd like Upstart to run the service as a particular user.
Line 86: Line 148:

=== Introduce Manual Stanza ===

A new job configuration file stanza "manual" will be added which takes
no parameters. If specified, this stanza makes Upstart ignore any "start
on" stanza found in the corresponding ".conf" file. If manual is
specified a job can only be started manually by an administrator using
start or initctl(8).
Line 94: Line 164:
Override files accept the same syntax as job configuration files with
one addition a new stanza called "manual". This stanza will only be
recognized in an override file (it will be an error for it to exist in a
".conf" file) and if specified makes Upstart ignore any "start on"
stanza found in the corresponding ".conf" file. If manual is specified a
job can only be started manually by an administrator using start or
initctl(8).		 Override files accept the same syntax as job configuration files.
Line 109: Line 173:
=== Add start_on and stop_on D-Bus properties ===

New "start_on" and "stop_on" properties will be added to the com.ubuntu.Upstart0_6.Job D-Bus interface. These properties will be defined as an array of variants, consisting of the event operator tree in Reverse Polish Notation.

A variant in the array may be one of:

 * a string containing the word "AND" or "OR" representing an operator
 * a structure consisting of a string and an array representing an event match, where the string is the name of the event, and the array is the arguments to the match.

Thus the job configuration:
{{{
start on started dbus and started apache
}}}

would be represented by the property value:
{{{
[["started", "dbus"], ["started", "apache"], "AND"]
}}}
Line 114: Line 197:
 * a parser or tool that generates [[http://www.graphviz.org/|graphviz /
 * dotty]] diagrams.		  * a parser or tool that generates [[http://www.graphviz.org/|graphviz / dotty]] diagrams.

=== Socket activation ===

This is largely an external bridge component; on startup it iterates the jobs (and is notified of job creation) and retrieves the start_on and stop_on properties, looking for "socket" events. It uses the match parameters of these events to determine the set of sockets to create and listen on.

These sockets are placed in an epoll level trigger, when they trigger an event is emitted passing the socket and equivalent match parameters. This will result in Upstart starting the service and passing the socket. Since the socket is level triggered, it will not trigger again until the socket is accepted and a new connection arrived (and the service may already be running).

Upstart requires a minor patch to pass the file descriptor received in the D-Bus message as part of the event and to the job.

=== D-Bus activation ===

This builds on the existing systemd D-Bus patch, replacing the SystemD activation components with an event that is emitted back over D-Bus to Upstart containing the well-known name requiring activation.

=== Session support ===

This is an additional property on Events and Jobs within Upstart, that points to a Session which is NULL by default. When non-NULL, this Session can specify additional properties for the job and additional matching.

Events only affect jobs with the same Session pointer as them, and job Events are generated with the same Session as their job. Likewise all initctl commands only affect jobs with the same Session pointer as them.

Configuration can be parsed from additional sources, and assigned to the Session of that source.

=== Chroot session support ===

The Session class will have a chroot field, when present jobs will be first chrooted into this path, before the path-related features of the job configuration are applied.

When receiving communication via D-Bus connection, the "root" of the communicating process id is examined to determine the chroot that this is coming from and the appropriate Session looked up and applied to all Events and Commands.

Configuration is parsed for a chroot on Session creation, which is generally the first time an event is emitted within that chroot.

=== User session support ===

The Session class will have a user field, when present jobs will have a PAM session created for that user before applying further configuration.

When receiving communication via D-Bus connection, the "user" of the communicating process id is examined to determine the user session that this is coming from and the appropriate User looked up and applied to all Events and Commands.

Configuration is parsed for a user on Session creation, which is generally the first time an event is emitted by that user.
Line 129: Line 247:
 1. Log a message stating including the PID of the child process)
    stating that it intends to pause the proces
 1. Pause the child process using pause(2).		  1. Log a message (including the PID of the child process) stating that it intends to pause the process.
 1. Pause the child process.

=== Introduce Manual Stanza ===

At the point the manual stanza is specified, Upstart will free any existing start_on tree and set the value of that property to NULL - it need not have any effect or record outside of the config parser.
Line 136: Line 257:
 * If a ".override" file exists without a corresponding ".conf" file,
   and then a ".conf" file is created, the ".conf" file must still be
   parsed first.		  * If a ".override" file exists without a corresponding ".conf" file, and then a ".conf" file is created, the ".conf" file must still be parsed first.

=== Add start_on and stop_on D-Bus properties ===

 * Add the D-Bus property definition to the XML files
 * Add the implementation functions to the job_class.c file, this is a readonly property so only requires the getter
Line 143: Line 267:
 * Write a parser to generate summaries and dot language output.  * Utilise the start_on and stop_on properties of jobs to generate summaries and dot language output.

=== Socket activation ===

 * Pass file descriptors received from D-Bus messages into the Event and into the Job
 * Implement external bridge

=== D-Bus activation ===

 * Patch required to D-Bus only

=== Session support ===

 * The initctl command gains a {{{--global}}} flag that disables the session support for that invocation, this is primarily intended to allow listing of global jobs from inside a chroot or as a user.

=== Chroot session support ===

 * Within a chroot, initctl will no longer by default allow control of jobs from outside.

=== User session support ===

 * When run as an ordinary user, initctl will by default show both system jobs and user jobs.
Line 154: Line 299:
=== Introduce Manual Stanza ===

No forward migration issues perceived. However, note that the new manual
stanza will introduce a backwards compatibilty issue should you try to
use a job configuration file containing the "manual" stanza with older
versions of Upstart. The solution is to remove the "manual" stanza.
Line 159: Line 311:
=== Add start_on and stop_on D-Bus properties ===

No forward migration issues perceived. However tools should expect the property to not exist if they encounter an older release of Upstart.
Line 160: Line 316:

No migration or compatibility issues perceived.

=== Socket activation ===

Software can be migrated to use Upstart by using a match such as:
{{{
start on socket PROTO=inet PORT=80
}}}

And retrieving the value of the {{{UPSTART_FD}}} environment variable that contains the listening socket, and accept the connection.

Inet-compatibility shall also be provided through:
{{{
start on socket PROTO=inet PORT=80 ACCEPT=y
}}}

Which instructs the bridge to call {{{accept()}}} and pass the resulting socket rather than the listening one.

=== D-Bus activation===

Software can be migrated to use Upstart by replacing the existing D-Bus activation configuration file with an Upstart job using a match such as:
{{{
start on dbus-activation com.ubuntu.SomeServer
}}}

=== Session support ===

No migration or compatibility issues perceived.

=== Chroot session support ===

No migration or compatibility issues perceived.

=== User session support ===

Summary

  1. Introduce Debug Stanza
  2. Introduce Manual Stanza
  3. Support for Override Files
  4. Add start_on and stop_on D-Bus properties
  5. Visualization of Jobs/Events
  6. Socket activation
  7. D-Bus activation
  8. Session support
  9. Chroot session support
  10. User session support

Release Note

  1. Upstart now supports a new "debug" stanza that if specified in a job configuration file causes Upstart to leave the child process in a paused state. This allows developers to attach a debugger to the child to trace its behaviour.
  2. Upstart now supports a new "manual" stanza that if specified in a job configuration file causes Upstart to forget any previous "start on" stanza specified in the file. This allows system administrators to disable jobs from automatically started without excessive modification.
  3. Support has been added for "override" files. These ".override" files -- which are not created by default -- allow tools to modify the behaviour of any job by overriding the job configuration file in various ways. This is particularly useful to add the "manual" stanza defined previously.
  4. External tools may now query Upstart jobs for the conditions that will cause them to start and to stop.
  5. It is now possible to visualize the flow of jobs and events as an aid to analysing system behaviour.
  6. It is now possible to activate Upstart jobs on socket connection
  7. It is now possible to activate Upstart jobs as a result of D-Bus service activation
  8. Upstart now supports multiple collections of independent job configurations
  9. Upstart now supports and can supervise jobs defined in an independent chroot to the main system
  10. Upstart now supports jobs run on behalf of non-root users

Rationale

Introduce Debug Stanza

This feature will make it significantly easier to debug both Upstart and any child processes it creates.

Introduce Manual Stanza

This feature will make it significantly easier to disable jobs from being automatically stated.

Support for Override Files

Although Upstart has a well-defined configuration syntax it is useful for system administrators and tools to be able to modify the default behaviour of a job without disrupting the provided job configuration file. Overrides provide this facility.

Add start_on and stop_on D-Bus properties

This feature provides a basis for building the visualization feature on top of without requiring extraction or re-use of the configuration parser, and it also allows external bridges to use the set of matched events to determine what to listen for.

Visualization of Jobs/Events

Upstart processes jobs and events in a very efficient manner with jobs (tasks) often completing before the adminstrator has had a chance to run "initctl status/list". This can cause confusion since users sometimes wonder why their jobs haven't started, not realizing that they have already run. Similarly, events are processed rapidly. What is needed is a way to log and view the flow of jobs, events and state changes in an easy and flexible manner.

Socket activation

This feature is considered to be one of the more compelling features of systemd, most people don't realise that Upstart was designed to support it and that it's a trivial addition even if we haven't found a cause to use it yet. Demonstrating this would be a good thing.

D-Bus activation

Since many jobs are likely to move to systemd as they have Fedora upstreams, we should have a simple way to replace them with equivalent Upstart jobs.

Session support

This feature provides the basis for building the chroot and user job support into Upstart.

Chroot session support

Ubuntu systems are often run in chroots, even if the chroot doesn't need full startup support, it is convenient to be able to start and stop the services in the chroot.

User session support

Ubuntu systems contain many user services, as well as the user desktop sessions, Upstart should be able to manage those as well.

User stories

Introduce Debug Stanza

  • Jennifer is an application developer and wants to single-step a daemon process in a production-like environment to help her understand a bug reported by one of her users.
  • Donald has written a new daemon and is trying to understand why it doesn't run as he expects.

Introduce Manual Stanza

  • Sam is a system administrator who does not want a particular service to start when the server starts.

Support for Override Files

  • Ben is a system administrator who managers a large group of database servers. The vendor has provided a buggy upstart job configuration file which needs to be worked around to avoid database servers from failing to start. Ben cannot change the existing ".conf" file since server security auditing software would flag this as an error.
  • Hermione wants to write a simple tool that allows a job to be disabled without modifying the existing ".conf" files.
  • Rafael wants to be able to change the run-levels a job runs in without modifying the existing ".conf" file.

Add start_on and stop_on D-Bus properties

  • Michael is a Linux distribution engineer who wishes to generate a dotty graph of their boot sequence.
  • Janine is a software developer who wishes to add a bridge between Upstart and their project, but to do that she needs to know what conditions to match to query her project and develop the bridge.

Visualization of Jobs/Events

  • Finn is an administrator and wants a log of all jobs showing when they ran and how long they took to run.
  • Lauren is a developer and wants to know how long the server application she has written actually ran before it crashed and what other jobs were running at the time of the crash.

Socket activation

  • Ted is a desktop developer, the latest graphical server relies on socket activation to start X11 support when required

D-Bus activation

  • Lucy is a plumbing engineer, several of her packages with Fedora upstreams have switched to relying on the init daemon to perform D-Bus activation, and she wishes a trivial way to convert to Upstart

Session support

  • Scott is an Upstart developer and wishes to implement chroot support and user session support.

Chroot session support

  • Colin is a build daemon administrator, build daemons run inside chroots and need managed services

User session support

  • Martin is a desktop developer, the desktop session contains services that require as much supervision as those running as the system.
  • Michelle is a server developer, she'd like Upstart to run the service as a particular user.

Assumptions

Design

Introduce Debug Stanza

A new job configuration file stanza "debug" will be added which takes no parameters.

Introduce Manual Stanza

A new job configuration file stanza "manual" will be added which takes no parameters. If specified, this stanza makes Upstart ignore any "start on" stanza found in the corresponding ".conf" file. If manual is specified a job can only be started manually by an administrator using start or initctl(8).

Support for Override Files

Recognize files of the form /etc/init/<job>.override. If present, these files will be parsed after the corresponding /etc/init/<job>.conf. Any stanza specified in the ".override" file will replace any specified stanza in the corresponding ".conf" file.

Override files accept the same syntax as job configuration files.

If the log priority is set appropriately or if the --verbose option has been passed to init, Upstart will display a log message when it runs a job whose environment has been modified by an override file.

Like ".conf" files, any changes to ".override" files will be automatically detected and applied to all relevant future jobs.

Add start_on and stop_on D-Bus properties

New "start_on" and "stop_on" properties will be added to the com.ubuntu.Upstart0_6.Job D-Bus interface. These properties will be defined as an array of variants, consisting of the event operator tree in Reverse Polish Notation.

A variant in the array may be one of:

  • a string containing the word "AND" or "OR" representing an operator
  • a structure consisting of a string and an array representing an event match, where the string is the name of the event, and the array is the arguments to the match.

Thus the job configuration: 

start on started dbus and started apache

would be represented by the property value: 

[["started", "dbus"], ["started", "apache"], "AND"]

Visualization of Jobs/Events

initially, the following forms of visualization will be provided:

  • a structured log file (ASCII)

  • a parser or tool that generates graphviz / dotty diagrams.

Socket activation

This is largely an external bridge component; on startup it iterates the jobs (and is notified of job creation) and retrieves the start_on and stop_on properties, looking for "socket" events. It uses the match parameters of these events to determine the set of sockets to create and listen on.

These sockets are placed in an epoll level trigger, when they trigger an event is emitted passing the socket and equivalent match parameters. This will result in Upstart starting the service and passing the socket. Since the socket is level triggered, it will not trigger again until the socket is accepted and a new connection arrived (and the service may already be running).

Upstart requires a minor patch to pass the file descriptor received in the D-Bus message as part of the event and to the job.

D-Bus activation

This builds on the existing systemd D-Bus patch, replacing the SystemD activation components with an event that is emitted back over D-Bus to Upstart containing the well-known name requiring activation.

Session support

This is an additional property on Events and Jobs within Upstart, that points to a Session which is NULL by default. When non-NULL, this Session can specify additional properties for the job and additional matching.

Events only affect jobs with the same Session pointer as them, and job Events are generated with the same Session as their job. Likewise all initctl commands only affect jobs with the same Session pointer as them.

Configuration can be parsed from additional sources, and assigned to the Session of that source.

Chroot session support

The Session class will have a chroot field, when present jobs will be first chrooted into this path, before the path-related features of the job configuration are applied.

When receiving communication via D-Bus connection, the "root" of the communicating process id is examined to determine the chroot that this is coming from and the appropriate Session looked up and applied to all Events and Commands.

Configuration is parsed for a chroot on Session creation, which is generally the first time an event is emitted within that chroot.

User session support

The Session class will have a user field, when present jobs will have a PAM session created for that user before applying further configuration.

When receiving communication via D-Bus connection, the "user" of the communicating process id is examined to determine the user session that this is coming from and the appropriate User looked up and applied to all Events and Commands.

Configuration is parsed for a user on Session creation, which is generally the first time an event is emitted by that user.

Implementation

UI Changes

Should cover changes required to the UI, or specific UI that is required to implement this

Code Changes

Introduce Debug Stanza

If the debug stanza is specified, Upstart will:

  1. Log a message (including the PID of the child process) stating that it intends to pause the process.
  2. Pause the child process.

Introduce Manual Stanza

At the point the manual stanza is specified, Upstart will free any existing start_on tree and set the value of that property to NULL - it need not have any effect or record outside of the config parser.

Support for Override Files

  • Add inotify watches for ".override" files.
  • If a ".override" file exists without a corresponding ".conf" file, and then a ".conf" file is created, the ".conf" file must still be parsed first.

Add start_on and stop_on D-Bus properties

  • Add the D-Bus property definition to the XML files
  • Add the implementation functions to the job_class.c file, this is a readonly property so only requires the getter

Visualization of Jobs/Events

  • Increase default and verbose logging.
  • Utilise the start_on and stop_on properties of jobs to generate summaries and dot language output.

Socket activation

  • Pass file descriptors received from D-Bus messages into the Event and into the Job
  • Implement external bridge

D-Bus activation

  • Patch required to D-Bus only

Session support

  • The initctl command gains a --global flag that disables the session support for that invocation, this is primarily intended to allow listing of global jobs from inside a chroot or as a user.

Chroot session support

  • Within a chroot, initctl will no longer by default allow control of jobs from outside.

User session support

  • When run as an ordinary user, initctl will by default show both system jobs and user jobs.

Migration

Introduce Debug Stanza

No forward migration issues perceived. However, note that the new debug stanza will introduce a backwards compatibilty issue should you try to use a job configuration file containing the "debug" stanza with older versions of Upstart. The solution is to remove the "debug" stanza.

Introduce Manual Stanza

No forward migration issues perceived. However, note that the new manual stanza will introduce a backwards compatibilty issue should you try to use a job configuration file containing the "manual" stanza with older versions of Upstart. The solution is to remove the "manual" stanza.

Support for Override Files

No migration or compatibility issues perceived: older versions of Upstart only consider ".conf" files.

Add start_on and stop_on D-Bus properties

No forward migration issues perceived. However tools should expect the property to not exist if they encounter an older release of Upstart.

Visualization of Jobs/Events

No migration or compatibility issues perceived.

Socket activation

Software can be migrated to use Upstart by using a match such as: 

start on socket PROTO=inet PORT=80

And retrieving the value of the UPSTART_FD environment variable that contains the listening socket, and accept the connection.

Inet-compatibility shall also be provided through: 

start on socket PROTO=inet PORT=80 ACCEPT=y

Which instructs the bridge to call accept() and pass the resulting socket rather than the listening one.

=== D-Bus activation===

Software can be migrated to use Upstart by replacing the existing D-Bus activation configuration file with an Upstart job using a match such as: 

start on dbus-activation com.ubuntu.SomeServer

Session support

No migration or compatibility issues perceived.

Chroot session support

No migration or compatibility issues perceived.

User session support

No migration or compatibility issues perceived.

Test/Demo Plan

It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to http://testcases.qa.ubuntu.com/Coverage/NewFeatures for tracking test coverage.

This need not be added or completed until the specification is nearing beta.

Unresolved issues

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

BoF agenda and discussion

Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected. 

Please document the outcome of this session at:
https://wiki.ubuntu.com/UDSProceedings/N/PackageSelectionAndSystemDefaults#Finish Upstart

Last UDS (recap): Stable version of Upstart in Ubuntu now for a few releases, working out reasonably well, but there are a number of things we need to fix (the fact that mountall is needed, user services, etc.).  Used last UDS to get a sense of what complaints were from various people and groups, and made sure that appropriate bugs were filed.

Upstart's design is too simple.  The goal of the next version is to fix problems based on deployment experience while retaining the "Upstartishness", and reach an elegant, simple design that we don't need to change again: i.e. 1.0.

https://bugs.edge.launchpad.net/upstart/+bugs
https://bugs.edge.launchpad.net/upstart/+bug/447654

BUG: upstart events can trigger apparent deadlocks

Biggest upstart bug:
  job maurice has "start on A and B"
  then emit event A
  process emitting event A hangs until event B occurs

The issue is when we use or:

  start on A and (B or C)

If you now emit A, then B, then C, the C will block waiting for a _second_ A.  

(Diagram: http://bit.ly/ddfeGG)

The solution is to make states from events.  And then jobs can wait on states in addition to events.

BUG: upstart keeps files open on /

BUG: pid tracking can be defeated which leads to upstart breakage

we should be using the proc connector to track children, this would resolve the tracking issue

BUG: service which is slow to start can appear started

Jobs which are starting slowly can appear started to a subsequent start of that job which makes that second start return immediately when the service is not actually started. This for example can trigger gdm to start too early.

This can be fixed by queuing new events against a job when a job is transitioning.

ISSUE: its very hard to find out why jobs are running

We want to be able to generate a dependency graph from a boot to find out why jobs have run.  A solution here would also allow an interactive boot.

ISSUE: chroots do not work, as you talk to the 'wrong' upstart

likely solution, tell upstart about 'chroots' so that it can track them and use the right job tree
 * if upstart is explicitly told about chroot/etc/init, then automatically start jobs in that chroot at boot
 * otherwise, assume /etc/init relative to /proc/PID/root when event received

Proposed Changes:
 - add the concepts of states which are based on events and persist beyond the event, which jobs can depend on
 - child tracking should use proc connector
 - events should be queued against jobs when the job is transitioning
 - overrides to local configuration of jobs without editing them
 - new hook on starting * to allow tracking of job dependancies
 - upstart will know about chroots and make itself available in there if it is going to start jobs automatically; and use the root directory for local start within a chroot to start the right jobs

Need to get buy in for these features

CategorySpec

FoundationsTeam/Specs/MaverickFinishUpstart (last edited 2010-12-15 09:22:10 by 92)