Submitted by Lennart Poettering of Red Hat, PulseAudio maintainer has blogged in detail about the impact of Real-Time Group scheduling in 2.6.25 kernel. The Real time patches come from -rt patchset maintained by Ingo Molnar of Red Hat which aims to make Linux the first general purpose operating system with hard real time features.
Nowhere in the article I have seen the phrase “hard real-time” please let’s not abuse phrases. Hard real-time means deadlines are “guaranteed” to be met at any circumstance and Linux is not capable of doing this below the microsecond level. In fact its not just Linux but the combination of hardware and the OS that needs to have this support, in order to achieve hard real-time. For example unavoidable hardware exceptions with higher priority would give enough non-determinism to reduce your system to “soft real-time”.
I would say merely the article’s summary is, “Linux can control real-time tasks (implying deadline-sensitive) with higher precision.”
Take a look at
http://kerneltrap.org/node/11752
http://rt.wiki.kernel.org/
Hard real time cannot be achieved with the vanilla kernel but this patchset can enable that. Vendors are already shipping products based on this.
http://www.redhat.com/mrg/
Yes all those links show that linux can work with *soft* real-time capability. *Hard* real-time is different as previously mentioned.
This is also why for instance linux is usually multiplexed with another RTOS for such delicate operations (e.g. modem communication on mobile phones).
Some of the documents I refer specifically use the word hard real time. As an example
https://wiki.ubuntu.com/RealTime/Feisty
Red Hat MRG product has specific guarantees below certain milli seconds for specific transactions. Refer to their whitepapers for more details.
I guess your definition of hard real time is different then which is ok as long as you clearly specify what your definition is or provide some reference as to to what you believe is a commonly acceptable definition.
Within specific industries, they start out with a definition or avoid the term and just specific the guarantees just to avoid this problem. That for example guarantees the max time slice among other things.
Edited 2008-04-19 00:29 UTC
Regardless of the document’s use (or misuse) of the term “hard realtime”, it’s a well understood fact that hard realtime means guaranteed deadlines; unless the Linux kernel provides microsecond accuracy, it’s “soft realtime”. This isn’t a fuzzy definition — or a minor distinction — hard realtime guarantees are required particularly for systems where subtle variations in performance or latency can have tragic consequences (eg. flight control systems, etc).
The vanilla kernel doesn’t but we are talking about -rt patchset specifically and you can read about their goals in the kernel wiki and read about the exact guarantees within specific products offered by vendors using this patchset.
I was under the impression that hard real-time meant just guaranteed deadlines with no drop in service quality, nothing to with whether it guaranteed one or several microseconds.
You are correct. Hard real time does not mean fast. Merely a guarantee to do a particular transaction within a particular time slice. In fact, enabling hard real time can make your general performance go down in preference to less spikes.
You contradict yourself, here. If hard real time means guaranteed deadlines (and it does) what do “microseconds” have to do with it? The guarantee could be that it will definitely do something within one millisecond, or one year, but if it can be even one Planck Time over that, it fails the test. The definition of hard real time is, in fact, so pedantic, that I think I can say with reasonable certainty, that nobody, except maybe God, legitimately needs it.
In short, I think there’s a Universal market of maybe 1 real time computer. And you can quote me on that. 😉
Edited 2008-04-19 19:52 UTC
Try to remember that the next time you’re riding on a plane and you’re approaching for landing. Or, when you need to stop suddenly in your car. Or, when a nuke plant operator needs to regulate cooling. Because those are scenarios where it’s needed.
Edited 2008-04-19 23:12 UTC
Where what is needed? Please be *very* specific in your answer.
BTW, what are the actual real time requirements of the nuclear power plant cooling system? Why should it require microsecond precision?
Edited 2008-04-19 23:38 UTC
Response time. Would you really like a latency of even a second or two for an aircraft after the pilot initiates a given action? It could kill the plane.
You should know this already. It may not necessarily be any one given action that requires microsecond precision; rather, it’s often dependent events that require high precision synchronization (ie. emergency systems) where several things need to happen at once when a dangerous condition is detected (eg. dropping of control rods, pumps need to be activated, power needs to be reduced, etc). It’s academic.
If your safety-critical system relies on precision timing in software – you haven’t designed it properly.
For example – control rods in a nuclear core should be physically “floating” in the coolant … such that if the coolant supply in the core drains for any reason the control rods automatically lower with the coolant level … and automatically shutting down the reaction as a result.
There are a number of such designs:
http://www.popularmechanics.com/science/research/1282196.html
It is called “failsafe”. Software failures … such as not responding in time for whatever reason … are prcisely the sort of things failsafe mechanisms should be designed to avoid.
http://www.indopedia.org/Safety_engineering.html
“Safety engineers also identify different modes of safe operation: A “probabilistically safe” system has no single point of failure, and enough redundant sensors, computers and effectors so that it is very unlikely to cause harm (usually “very unlikely” means less than one human life lost in a billion hours of operation). An “inherently safe” system is a clever mechanical arrangement that cannot be made to cause harm- obviously the best arrangement, but this is not always possible. For example, “inherently safe” airplanes are not possible. A “fail-safe” system is one that cannot cause harm when it fails. A “fault-tolerant” system can continue to operate with faults, though its operation may be degraded in some fashion.”
One should avoid software as a safety measure in a design wherever possible.
The safest designs are “inherently safe”. All nuclear reactors should be so designed, due to the catastrophic level of damage that could occur should they fail.
I shudder to think that anyone would contemplate relying on the correct operation of software … especially with critical timings involved also … in order to move the control rods of a nuclear reactor to safety.
Edited 2008-04-21 09:39 UTC
That doesn’t change the fact that Linux doesn’t do hard real-time. You can argue whether it’s needed or not as much as you like but at the end of the day Linux does not fulfill the criteria.
It does however change the facts surrounding what you might actually need hard real time for.
Anything where your requirement is safety critical … don’t design it with software in the loop.
Anything which requires low latency and no gaps in service … such as perhaps an audio card, a video card or a network card … put in a buffer and design for “gapless” servicing of the buffer.
You don’t need hard real time for a desktop or for a server.
The only things you might actually need “hard real time” for are things like stepper motor controllers and control loading controllers.
Go for Wind River or something like that … although I hear that even Wind River have now gone to Linux + real-time patches.
http://www.windriver.com/products/platforms/real-time_core/
Edited 2008-04-21 12:03 UTC
> If your safety-critical system relies on precision timing in software –
> you haven’t designed it properly.
Why? Because your obviously-favored Linux cannot meed that demand? Don’t worry, other OSes can.
> For example – control rods in a nuclear core should be physically
> “floating” in the coolant
Actually, a nuclear reactor should be protected by as many safety measures as ever possible, because failure is catastrophic. Reliable software is just one of those safety measures. Using unreliable software in that place is dangerously stupid.
> One should avoid software as a safety measure in a design wherever
> possible.
The article you linked doesn’t mention that anywhere. Anyways, using software as a safety measure *in addition to all others* is perfectly sensible, because it makes the system more secure, not less.
> Anything where your requirement is safety critical … don’t design it
> with software in the loop.
Actually, I don’t really know how I could reply to that. Maybe you should try going to a plane manufacturer and tell them to remove all security-relevant, software-driven parts from their planes. Then you’d probably see how insane that request is.
> You don’t need hard real time for a desktop or for a server.
Of course not. But it seems very much to me like you are extending that statement to embedded systems without giving any reasoning or evidence.
I would bet any amount of money the Ubuntu page is wrong.
“Hard real time” isn’t a vague descriptive phrase. It’s an actual concept used to describe ultra-strict time constraints where something not happening exactly when it needs to has severe consequences like an explosion or people dying. It’s used in stuff like nuclear power plants, avionics, etc.
Soft real time systems are those that have strict timing requirements, but missing a deadline isn’t too big a deal. Playing an MP3, for example, requires refilling the audio buffer before the old one is consumed, but it’s not the end of the world if it doesn’t happen.
It takes a whole lot of work to make a large hard real time system, and it just doesn’t make sense to put in the effort for a desktop system.
yess, well… Linux isnt just for desktops, even though this is one of its best uses
But Ubuntu is aimed squarely at the desktop.
If anybody is going to make a hard real time version of Linux, it’s probably not going to be them.
‘hard’ real time is not so well defined as you might imagine.
A truly deterministic implementation cannot have such niceties as processor cache, interrupts or file i/o as you cannot accurately predict their latencies.
A practical ‘hard’ real time system still has to allow for missed deadlines, or have some kind of priority assignment for real time threads. This opens up the possibility of priority inversion. (see Mars Rover Pathfinder.)
So there are always compromises and each implementation is specific. The Linux RT patches provide one implementation, with all the problems associated with running on commodity hardware, networking, file io etc. Whether it fits your needs is another matter.
> A truly deterministic implementation cannot have such niceties as
> processor cache, interrupts or file i/o as you cannot accurately predict
> their latencies.
That’s exactly what made me wonder how Linux could achieve determinism on unpredictable hardware. Or is this patchset only for predictable hardware?
> A practical ‘hard’ real time system still has to allow for missed
> deadlines
See Redeeman’s comment. “Really bad things” happen when a deadline is missed, so this is not an option. To be more exact, a hard real-time system can include soft real-time components as long as they don’t affect the deadlines of the “hard” part, so only the deadlines of the “soft” parts may be missed.
> or have some kind of priority assignment for real time threads.
Only if it doesn’t prevent – or actually helps – achieving the deadlines.
> The Linux RT patches provide one implementation, with all the
> problems associated with running on commodity hardware,
> networking, file io etc. Whether it fits your needs is another matter.
Sorry to say it, but this sounds like a lot of talk to avoid telling that it simply isn’t hard real-time.
These sorts of threads always get caught up in the “what is hard realtime” question, when the more interesting question is “When is hard realtime (vs soft realtime) really needed?”. When you get right down to it, hard realtime, far from desirable, is really a last resort, and should be avoided if possible, if performance is of any concern at all.
I’ve spent hours today with Mandriva 2008 ‘s 2.6.24-rt kernel, but I could not gain anything from it. Real-time audio is as far from Linux as Tokyo from Jerusalem! I know what I’m talking, since I’ve even recompiled the kernel (took 2 hours), but I cannot use the great Ardour in Mandriva.I have a good and fast PC. I don’t understand why they even are packing applications which shall not work? Ubuntu Studio is closer to the goal. But I’m only dreaming of 10 stereo tracks with latency less than 5ms.
I would recommend giving 64Studio a try. Probably tuned better than Mandrake/iva would be even with this real-time patch.
Sometimes the problem with latency is overly conservative IDE settings ( you may have to get familiar with hdparm to change this) or excessive CD-ROM drive polling.
There are some good docs floating around on this but might be quicker to take 64Studio for a spin.
http://www.64studio.com/
2 HOURS to recompile the kernel – wow, there’s now way it should take that long with recent hardware. I do remember when you could spend that much time at it but that was LONG ago !!!!
If it took 2 hours I seriously doubt your PC is fast and good.