“For years, most Linux distributions have been using an init daemon based on the one found in Unix System V. The init daemon is spawned by the kernel itself, and tasked with booting the rest of the system, starting all other processes, and taking care of them when they need to be stopped or when they die. While the System V init setup has worked well for Linux in the past, it hasn’t aged well – which is why we’re replacing the aging init system with Upstart in Ubuntu 6.10, codenamed Edgy Eft.”
Replacing init with Upstart
35 Comments
2006-10-04 9:53 pm
Devon
This sounds like the way modern hardware was really supposed to work. Plug in a mouse, USB and mouse modules and supporting software load up. Plug in a printer, and the relevent software loads up for that.
The days of creating a static setup before you ever turn on the PC are over. The hardware manufacturers are dumping DB25 and DB9 ports for wonderful, hotpluggable USB and other similar interfaces, and its about time Linux did too.
-
2006-10-04 10:48 pm
sbergman27“””The days of creating a static setup before you ever turn on the PC are over. The hardware manufacturers are dumping DB25 and DB9 ports for wonderful, hotpluggable USB and other similar interfaces, and its about time Linux did too.”””
Linux already works like this.
You obviously do not run Linux. But try this. Plug in a ps2 mouse and keyboard and boot an Ubuntu Live CD. It does not have to be all that recent a version.
After it is up, plug in a usb mouse and a usb keyboard and start typing. You can unplug the old keyboard and mouse and toss them in the trash if you like.
If you have a usb stick, plug it in. Click the icon that pops up on your desktop and save or read a few files from it.
I’m fairly unimpressed by the dynamic device thing. We already do that.
The lack of parallelism in our boot process… is embarassing. I’m excited about that part.
However, it should be noted that knot3 did not have upstart, and I did look over a benchmark of boot times of knot3 with and without upstart.
The improvement over Dapper was dramatic in both cases. But there was only a 5 second difference between sys5init and upstart.
But that is to be expected, since upstart is still running the sys5 initscripts.
It will be in the next release that we really begin to see the potential of the upstart boot process.
Edited 2006-10-04 22:50
-
2006-10-04 11:15 pm
philip.mayAfter it is up, plug in a usb mouse and a usb keyboard and start typing. You can unplug the old keyboard and mouse and toss them in the trash if you like.
I think you misinterpreted what the previous poster was saying, it’s not a case of hot plugging usb devices which we all know linux does very well these days but a case of the modules being dynamically loaded when those devices are plugged in, I may be wrong but my understanding is that those modules are currently loaded at boot regardless of whether or not the related devices are connected or even going to be connected.
-
2006-10-05 2:39 am
bhearsumThat’s actually not the case. Hotplug can load and unload device drivers as well as do other things.
-
2006-10-05 7:45 am
RenatoRamtry to watch /var/log/messages when you plug some hardware in
The modules are loaded on demand, and this has been the case… well, forever.
I remember that the ppa module for my ancient PARALLEL iomega Zip was loaded on the fly in the 2.2 series kernel. That’s… 8 years ago?
-
2006-10-05 1:22 am
phoenixOdd, this is how things work on FreeBSD 5+, thanks to the devd notification daemon.
Linux (or, I guess distros) needs to unify things a bit into a more comprehensive, cohesive, whole. Things like hotplug, udev, acpi-support, pcmcia management scripts, and all the other byzantine “hotplug” device management stuff needs to be brought together into something along the lines of FreeBSD’s devd and devfs.
-
2006-10-05 8:31 am
leechI has been. Udev has merged in hotplug, it also does automatically load up the modules required by whatever is plugged in, at least that’s been my experience.
I do want to give FreeBSD a shot again sometime in the future. Last time I tried it out, I couldn’t get my soundcard working, and if I tried loading the module myself, it’d just lock the system. But this was wih 4.2 (I think). I’ve been spoiled by apt-get though…
So if I go the FreeBSD route, I’m going to have to try the Debian version.
Edited 2006-10-05 08:32
2006-10-04 10:41 pm
Kokopelli
Upstart might speed up boot, but that is not the reason it was made. It seems this fact keeps getting lost everytime upstart is mentioned.
That said depending on what services you have starting there may be an improvement.
Browser: Mozilla/4.0 (compatible; MSIE 4.01; Windows CE; PPC; 240×320)
2006-10-04 10:50 pm
jayc
Considering upstart is relatively new, I doubt they’ve had time to fully optimize the boot process. You know the whole saying about premature optimization.
I don’t see how it can’t do anything but speed it up. For example, for the past few releases the HP stuff has been started every time no matter if you have the HP all-in-one printer. upstart should provide the infrastructure to be able to only start it when the hardware is plugged in.
2006-10-05 2:38 am
bhearsum
What advantage does this have over SysV with hotplug? This thing sounds more like hotplug than an init system…
-
2006-10-05 2:43 pm
LunitikIt’s not a bad way to look at it…
This basically says “hey do we see a printer? yeah? ok lets start cups”… if the answer is no, cups doesn’t start…
Same goes for things like mdadm etc… it has a wider scope than hotplug, but if you understand that, think of this as hotplug for system services and you won’t be far off the mark.
2006-10-05 3:07 am
elsewhere
Is this any different from some of the “transparent” things that are going on with HAL / DBUS etc.? I’m thinking as an example, networkmanager. NM is made “aware” of network events, the availability of an interface, the availability of prefered wireless networks, etc. and make automated configuration changes on the fly based on the available information. Is Upstart a collection of existing technologies to build a common framework, or is it something that’s supposed to replace the existing combination of services?
From a booting process, does Upstart do anything that different than an experienced user would do with tweaking their setup/startup services? It seems to me that no matter how you try and adjust the bootup process, parallel processes etc., the bottleneck is an always will be the hdd transfer rate. Shaving a few seconds is nice and all, but can there really be an impact without somehow multiplying disk throughput that you couldn’t otherwise achieve by judiciously selecting your startup services?
Don’t get me wrong, I’m not knocking Upstart, I don’t really know enough about it to have an opinion one way or the other, and even if it only simplifies things for users without resorting to manual tweaking etc. I would see that as a “good” thing. I’m just trying to figure out how it meshes with current state of your average mainstream distro framework.
The article helped, certainly more info than I’ve come across before. Prior to that I thought it was just an attempt at another initNG. Looks intriguing, but I’m just wondering if it’s evolution or revolution?
-
2006-10-05 3:54 am
grat*IF* I understand it correctly, the goal of Upstart is to effectively combine init + udev, and throw in a dependency system for good measure.
On the plus side, this would mean that services load at the right time, (ie, in the correct order) and only if the device that requires that service is present.
So, for example, ntp, sshd, nfs, smb services wouldn’t load until a network device was active– and of course, as soon as the device goes active, those services would start, regardless of what else might be running/starting at the time. It’s a definite improvement of the current “hurry up and wait” syndrome that can occur in a non-tuned environment.
From what I’ve seen on Edgy, the biggest time consumer during bootup on my system(s) is the reiser disk check (long time SuSE user, my /home is almost invariably reiserfs).
My guess? Evolution, not revolution. There’s quite a bit of work in progress by various OS/distros to improve the init system. Eventually, they’ll coalesce into a Better Solution.
-
2006-10-05 3:34 pm
GregM> From what I’ve seen on Edgy, the biggest time consumer during bootup on my system(s) is the reiser disk check
BTW, there’s a patch for reiserfs3 that drastically cuts mount time. Expect to see it in the next mainline kernel release.
2006-10-05 7:27 am
2006-10-05 8:22 am
Kokopelli
Modules are loaded on demand in Linux but that is not all that upstart is about.
How about loading Mysql only if apache is running and stopping it when not.
Loading cups when a printer is attached.
Easier monitoring of running services and restarting them when they fail.
This kind of thing could be done for the most part in scripts but upstart simplifies the process lifecycle a great deal.
Further current inits are sequential and rigid in nature. Upstart is event based. So, as an example, you can have samba start when you attach a USB drive. This can occur at any time though, it is not dependent on the drive being attached or available during init. In a way upstart has more in common with inet than the traditional sys v init.
Browser: Mozilla/4.0 (compatible; MSIE 4.01; Windows CE; PPC; 240×320)
-
2006-10-05 2:35 pm
OokazeHow about loading Mysql only if apache is running and stopping it when not
I use simpleinit-msb since 2001, and since then, I can do that already.
I can’t stop mysql when apache is not running, as Apache won’t notify you it isn’t running (obviously).
Only monitoring can do that, and if monitoring, you don’t need Upstart.
But stopping mysql when stopping Apache is possible.
Loading cups when a printer is attached
This is a stupid thing to do. Cups is there to notify you when configured printers are attached or not, or allow you to use other alternative ways of printing.
There’s no point in loading Cups only when a printer is attached.
Contrary to what the article says, Upstart is not good at all except in a PC dedicated to a nomad desktop.
On a server, you need to have several levels of boot control (without network, single user, …). Launching everything available always is not a good thing at all.
Not knowing that some services couldn’t be run is not good either.
Easier monitoring of running services and restarting them when they fail
This is not easier monitoring at all, and restarting a service when it fails is a basic feature of any init system.
This kind of thing could be done for the most part in scripts but upstart simplifies the process lifecycle a great deal
No it doesn’t, they talk about scripts (or binaries) for Upstart too.
Further current inits are sequential and rigid in nature. Upstart is event based. So, as an example, you can have samba start when you attach a USB drive. This can occur at any time though, it is not dependent on the drive being attached or available during init. In a way upstart has more in common with inet than the traditional sys v init
How about Samba being started before the USB drive is plugged ? These servers work far better in standalone mode. They are daemon, which means “Disk And Execution MONitor”. There is no point in running daemons when some events occur, they’re supposed to react to events already.
You’ll tell me about inetd type daemon, but these are daemons too.
For example, there’s no point in running ntpd by event, ntpd will work far better the longer it runs, even if it lose network connectivity from time to time.
2006-10-05 8:42 am
JCooper
Taking an example from the article, this Upstart system is not just hotplug. Yes we have had dynamic *module* loading for some time, but this is so much more than that. I see it as having two advantages. The first is it is modern – by having dependencies based on events, not “after loading a and b, you need to load c, d, e and h” but instead “after loading fire off event-x and let other subsystems react to that”.
The second advantage is a byproduct of the above. Let’s step back to the article example – a network adapter. Yes, if you currently plug one in (for arguments sake) the module is loaded. But what if it’s already plugged in, but couldn’t get an IP address. Sure NetworkManager (which is AwesomeWare IMO) will handle the notifications in user-space when a connection is available… but Upstart, behind the scenes, will send an event stating there is a usable connection available. That event can spawn all sorts of things – your update manager waking up, apache loading, mysql loading, ftp server starting, mail server starting etc. Those in turn may fire off further events which subsequently will fire off more – causing all services dependent on having a live IP connection being activated automatically.
Upstart, as I understand it, is so much more than just hotplug and “my USB disk appears on my desktop when I plug it in” and “I can use this USB mouse immediately after plugging it in”. It is an entire event-driven subsystem that will load and unload dependent services on the fly.
Another design goal of Upstart is to be completely backwards compatible (Knot 3 for e.g. keeps the same init scripts) which means, for the short term, it is easily adoptable, unlike the other alternatives.
Edited 2006-10-05 09:00
-
Ookaze
The premises of the article are plain wrong. All the deficiencies they talk about are being resolved or are already resolved in Udev. They talk of cold boot problems, but if the kernel doesn’t solve them, Upstart won’t either.
I see it as having two advantages. The first is it is modern – by having dependencies based on events, not “after loading a and b, you need to load c, d, e and h” but instead “after loading fire off event-x and let other subsystems react to that”
There are already other init systems doing that on Linux. It doesn’t change anything except being able to parallelize some launchers. But you still have dependancies in the event based process. It’s actually the same thing, just less rigid.
The second advantage is a byproduct of the above. Let’s step back to the article example – a network adapter. Yes, if you currently plug one in (for arguments sake) the module is loaded. But what if it’s already plugged in, but couldn’t get an IP address
The module is still loaded … No change there.
Sure NetworkManager (which is AwesomeWare IMO) will handle the notifications in user-space when a connection is available… but Upstart, behind the scenes, will send an event stating there is a usable connection available. That event can spawn all sorts of things – your update manager waking up, apache loading, mysql loading, ftp server starting, mail server starting etc.
FYI, I was doing the same thing with DHCP, rp-pppoe, or hotplug before, and with udev after that. That’s how I was managing regular disconnections by my ISP with my P75 200, and this was 2 years ago ! So there’s nothing new there. After that, I even used it on the Thinkpad 380XD that replaced my P75 200 (this was my front firewall/router/proxy to Internet BTW). The Thinkpad used one native ethernet interface, and one PCMCIA. Everything worked like described there in Upstart.
Those in turn may fire off further events which subsequently will fire off more – causing all services dependent on having a live IP connection being activated automatically
But project Utopia already does all of this. It even configures your cards name. I don’t have any eth0 on my PCs for example. All my interfaces have custom names, created by udev.
Upstart, as I understand it, is so much more than just hotplug and “my USB disk appears on my desktop when I plug it in” and “I can use this USB mouse immediately after plugging it in”. It is an entire event-driven subsystem that will load and unload dependent services on the fly
But project Utopia does that already, except it doesn’t unload drivers (as kernel devs says there’s no point in doing this). But you can launch any script you want when your link is up or down. So really, I don’t see the point of Upstart.
-
2006-10-05 3:53 pm
TerracottaYou missed the: it’s completely backwards compatible, so transition in the short term is easy, not so with Utopia, if I’m not mistaken.
2006-10-05 3:32 pm
Lousewort
In my mind, the seperation between responsibilities of the kernel and those of software has always been quite clear; kernel talks to hardware, software talks to kernel. Kernel is therefore responsible for devices.
Hot plug is quite easily supported via loadable modules. Notification of events such as plugged devices is these days quite nicely supported by dbus/hal. But I have a question about that. Should dbus/hal not rather be done as modules? Hah! I hear you say; they must run in user space, not kernel space! But then are we not simply speaking of a need for a different kind of (that is, user space) module?
Buggering around with something like init which has worked well for simply ages, and making the whole shananigan dependent on the vagaries of various distro producers is just plain idiotic. Init works well. It does what it was always intended to do. Leave it alone.
Regarding serial loading of services- well, that’s a feature, not a bug, and hardly ’embarassing’. It is more efficient to load one service at a time; the fact that some rc.xx scripts depend on other rc.xx scripts is not the fault of init, and paralellising the load process only introduces a need for somehow controlling the order in which these inter-dependent processes get loaded. That just leads to complications, yet more incomprehensible system maintenance and pity the poor sould who needs to find and correct a dependency-related problem.
If the line is drawn in the sand regarding kernel responsibilities and application responsibilities, life will remain unambiguous and uncomplicated.
If something works, dont try to fix it.
-
Lunitik
This has NOTHING to do with hardware.
This has to do with services that start when you boot your machine.
Init has worked for ages, sure. Its been mostly unchanged since the 70’s or something. Today’s systems are far different than the systems back then however.
There is, for instance, no point trying to start web services (mysql, apache etc) if the system doesn’t find a network connection. Init would still try to start these services, which will hold up the start up process, because they won’t be able to start.
This is the kind of use-case upstart has in mind.
-
Lousewort
Ah, but EVERYTHING to do with hardware _state_
And there I completely disagree with you.
I have a LOT of uses for msql and yes, even apache if my server machine boots up while the switch is down-
because of a power failure for example,
or on an ADSL connected box
or even on a generally disconnected machine.
How can you make up the rules when to load my hardware and non-hardware related software and services? Is it the distro’s task to do this?
Edited 2006-10-06 04:47
-
2006-10-06 7:55 am
Buffalo SoldierHow can you make up the rules when to load my hardware and non-hardware related software and services?
Forgive me if I’m wrong. Just a typical linux desktop user, not a system admin or something powerful like that. So my understanding of this matter could be wrong.
But from what I read, upstart is not an object/person that rules what is loaded or not loaded on your system. But it’s a tool that lets a system loads:
a) some default rules, and
b) that rules is changeable by the system admin.
Examples of the type of users/admin that upstart is going to cater for given in the wiki (https://wiki.ubuntu.com/ReplacementInit):
– Corey is the administrator of a number of servers, and has problems with certain daemons that frequently crash. He would prefer the daemons to be automatically restarted if this happens, to avoid loss of service.
– Ryan is a database administrator. He wishes the database to be automatically backed up whenever the server is shutdown, whether for upgrade or system reboot. There is currently no way for him to set a task to be run when a service is stopped.
– David is a system administrator. He needs to be able to tell which services failed to start on boot, examine why, and see which services are currently running.
– Thomas is a system administrator. He frequently gets frustrated that there is no consistency to how tasks are added to the system. A script to perform a task at shutdown must be written and activated completely differently to one performed when the system is started. (edgy+1)
From my point of view, you sir fits the scenerio. But like I said earlier, I could be wrong.
-
Does the LSB say anything about this?
Since when is the LSB actually relevant?
Since upstart is designed to be compatible with ‘old’ init scripts, it should be LSB compatible I think.
This could be the reason why Ubuntu Edgy Eft boots so much quicker than Fedora Core 6 pre-release. Another reason could be that Fedora has almost twice as many startup scripts.
http://djst.org/blog/2006/10/03/fedora-core-6-vs-ubuntu-610/
Booting takes exactly as much with upstart on my hardware (Celeron 2.6MHz, 512M RAM) as it did with init.
I doubt this is the effect of upstart, since its main goal is not to speed things up.
However they use dash for initscripts now in edgy. The corresponding spec claims that using dash instead of bash saves you 30 seconds. However there are no figures how long the boot took overall.
My experience differ. I haven’t seen a big boost in bootup speed when I switched one box from dapper to edgy. I don’t have fc as comparions.
I doubt this is the effect of upstart, since its main goal is not to speed things up.
However they use dash for initscripts now in edgy. The corresponding spec claims that using dash instead of bash saves you 30 seconds.
What spec? Doesn’t sound like a reasonable claim.
IIRC in the dev tests it was 3.0 sec, not 30.
I apologize. I should have provided the link earlier. https://wiki.ubuntu.com/DashAsBinSh
I apologize. I should have provided the link earlier.