JGuru
Wise Old Owl
For years, most Linux distributions have been using an init daemon based on the one found in Unix System V. The init daemon is spawned by the kernel itself, and tasked with booting the rest of the system, starting all other processes, and taking care of them when they need to be stopped or when they die. While the System V init setup has worked well for Linux in the past, it hasn't aged well -- which is why we're replacing the aging init system with Upstart in Ubuntu 6.10, codenamed Edgy Eft.
Upstart is designed to be a replacement for System V Init (sysvinit), with a different purpose than other init replacements that have been developed. The problem that has been facing us as we've attempted to make things "just work" in Ubuntu is that modern desktop computers and servers are very different beasts from those in use ten years ago.
In the past, boot-time operations, such as checking and mounting the filesystem, were relatively simple. Only a limited arrangement of hardware was supported, and connection or disconnection of devices while the power was on was not possible. The kernel always knew the number and type of disks, so the user-space boot process could simply check and mount them without difficulty.
Dealing with modern hardware:
Today we have to deal with a very different world. Hardware, such as disk drives and input devices, can be connected to buses that support a near-unlimited number of devices in arbitrarily complicated topologies.
To make matters more interesting, these devices can be plugged in and removed at any time, and are not required to power up until they are going to be used. While this means that the kernel now receives interrupts when hardware changes, the initial discovery of connected hardware is more problematic; a probe can be sent, but there's no point at which we know we have received all the responses.
When we bring network filesystems into the equation, the problem gets even more difficult. Not only do we have to wait for responses from the server, the network interfaces themselves need to be configured; this may involve uploading firmware into the card, negotiating with a wireless access point, requesting an IP address from a server on the network, and logging in to a Web page to gain Internet access.
Some solutions do exist for these problems. For example, Ubuntu probes for likely hardware on which it will find the root filesystem, and then sits in a tight loop until the expected hardware appears. These kinds of tricks rely both on waiting in the boot process and knowing the hardware that we're waiting for. As users demand that their computer boots ever faster, waiting in the boot process is simply untenable. Knowing in advance the hardware we're looking for might seem prescribed, yet a lot of the more interesting features we'd like to introduce break this assumption.
We would like certain daemons, such the HP Linux Printing and Imaging System, to be only started then the associated hardware is actually available. This way users who have the hardware benefit, and users who don't are not penalized. More interesting ideas include only running podcast daemons while an iPod is connected, and so on.
The Linux kernel has become increasingly better at supporting hot-pluggable hardware, and with the introduction of udev and the kernel driver-core, user-space is now provided with information about the connected hardware, as well as notification of connection and disconnection. Yet, despite this, we still rely on a static set of shell scripts for the boot process which make assumptions about what hardware is available at what point in the boot process.
Defining events:
Events themselves are simple strings, with optional environment variables to convey more detailed information to the jobs handling the event; though we are considering adding arguments or an origin for the purposes of identifying which device was added, or which job was stopped, etc. Any part of the system can sent an event by using the "initctl" tool, or by communicating with init directly over a Unix domain socket.
Jobs are defined by files in the /etc/event.d directory, and the simplest can be just the name of a binary, or some shell code to run. The more complex can include shell code to be executed before the job is started and other shell code for after the job has stopped, as well as resource limits and environment settings. Jobs can be started when any of a list of named events occur, with multiple running instances either permitted or disallowed. They can be allowed to terminate normally themselves, stopped when another list of named events occur, or respawned on termination until any event in that list occurs.
A goal of the project is to simplify how the system is configured. Right now, one has to configure jobs to be run on startup, hourly, when a network device is up, when the AC power is reconnected, on shutdown, or when the system is suspended in several different places.
Because events can be received from any other process on the system, we intend to modify other daemons so that instead of running directories of scripts themselves, they send an event to init so all such jobs can be configured in /etc/event.d. Potential plans for the future include running jobs on behalf of users, time-based events so that daemons such as cron, anacron, and atd could be replaced and even, potentially, the ability to replace the inetd daemon.
It's our hope that with Upstart we can solve a lot of the problems that prevent Linux from "just working," without any ugly workarounds or race conditions, as well as make the job of sysadmins much easier. To find out more, visit the upstart Website, which has packages for Ubuntu and Debian, source code, mailing lists, and bug tracking for upstart.
Upstart is suitable for deployment in any Linux distribution as a replacement for sysvinit. One should be aware that, like other replacements, it has its own native configuration and will not run your existing init scripts unless configured to do so. The recommended migration plan, which Ubuntu is following, is to run the existing System V rc script and let that take care of the legacy init scripts.
A set of example jobs is available from the Website to do this, though they will require modification for other distributions. Once up and running, you can then migrate init scripts to upstart jobs one by one, as time permits, without dropping compatibility with other applications.
Read more about it here
Read about Upstart here
Upstart is designed to be a replacement for System V Init (sysvinit), with a different purpose than other init replacements that have been developed. The problem that has been facing us as we've attempted to make things "just work" in Ubuntu is that modern desktop computers and servers are very different beasts from those in use ten years ago.
In the past, boot-time operations, such as checking and mounting the filesystem, were relatively simple. Only a limited arrangement of hardware was supported, and connection or disconnection of devices while the power was on was not possible. The kernel always knew the number and type of disks, so the user-space boot process could simply check and mount them without difficulty.
Dealing with modern hardware:
Today we have to deal with a very different world. Hardware, such as disk drives and input devices, can be connected to buses that support a near-unlimited number of devices in arbitrarily complicated topologies.
To make matters more interesting, these devices can be plugged in and removed at any time, and are not required to power up until they are going to be used. While this means that the kernel now receives interrupts when hardware changes, the initial discovery of connected hardware is more problematic; a probe can be sent, but there's no point at which we know we have received all the responses.
When we bring network filesystems into the equation, the problem gets even more difficult. Not only do we have to wait for responses from the server, the network interfaces themselves need to be configured; this may involve uploading firmware into the card, negotiating with a wireless access point, requesting an IP address from a server on the network, and logging in to a Web page to gain Internet access.
Some solutions do exist for these problems. For example, Ubuntu probes for likely hardware on which it will find the root filesystem, and then sits in a tight loop until the expected hardware appears. These kinds of tricks rely both on waiting in the boot process and knowing the hardware that we're waiting for. As users demand that their computer boots ever faster, waiting in the boot process is simply untenable. Knowing in advance the hardware we're looking for might seem prescribed, yet a lot of the more interesting features we'd like to introduce break this assumption.
We would like certain daemons, such the HP Linux Printing and Imaging System, to be only started then the associated hardware is actually available. This way users who have the hardware benefit, and users who don't are not penalized. More interesting ideas include only running podcast daemons while an iPod is connected, and so on.
The Linux kernel has become increasingly better at supporting hot-pluggable hardware, and with the introduction of udev and the kernel driver-core, user-space is now provided with information about the connected hardware, as well as notification of connection and disconnection. Yet, despite this, we still rely on a static set of shell scripts for the boot process which make assumptions about what hardware is available at what point in the boot process.
Defining events:
Events themselves are simple strings, with optional environment variables to convey more detailed information to the jobs handling the event; though we are considering adding arguments or an origin for the purposes of identifying which device was added, or which job was stopped, etc. Any part of the system can sent an event by using the "initctl" tool, or by communicating with init directly over a Unix domain socket.
Jobs are defined by files in the /etc/event.d directory, and the simplest can be just the name of a binary, or some shell code to run. The more complex can include shell code to be executed before the job is started and other shell code for after the job has stopped, as well as resource limits and environment settings. Jobs can be started when any of a list of named events occur, with multiple running instances either permitted or disallowed. They can be allowed to terminate normally themselves, stopped when another list of named events occur, or respawned on termination until any event in that list occurs.
A goal of the project is to simplify how the system is configured. Right now, one has to configure jobs to be run on startup, hourly, when a network device is up, when the AC power is reconnected, on shutdown, or when the system is suspended in several different places.
Because events can be received from any other process on the system, we intend to modify other daemons so that instead of running directories of scripts themselves, they send an event to init so all such jobs can be configured in /etc/event.d. Potential plans for the future include running jobs on behalf of users, time-based events so that daemons such as cron, anacron, and atd could be replaced and even, potentially, the ability to replace the inetd daemon.
It's our hope that with Upstart we can solve a lot of the problems that prevent Linux from "just working," without any ugly workarounds or race conditions, as well as make the job of sysadmins much easier. To find out more, visit the upstart Website, which has packages for Ubuntu and Debian, source code, mailing lists, and bug tracking for upstart.
Upstart is suitable for deployment in any Linux distribution as a replacement for sysvinit. One should be aware that, like other replacements, it has its own native configuration and will not run your existing init scripts unless configured to do so. The recommended migration plan, which Ubuntu is following, is to run the existing System V rc script and let that take care of the legacy init scripts.
A set of example jobs is available from the Website to do this, though they will require modification for other distributions. Once up and running, you can then migrate init scripts to upstart jobs one by one, as time permits, without dropping compatibility with other applications.
Read more about it here
Read about Upstart here
