Linux kernel 2.6 introduces improved IO scheduling that can increase speed — “sometimes by 1,000 percent or more, [more] often by 2x” — for standard desktop workloads, and by as much as 15 percent on many database workloads, according to Andrew Morton of Open Source Development Labs. This increased speed is accomplished by minimizing the disk head movement during concurrent reads, says Yahoo!News.
The New Linux Speed Trick
About The Author
Eugenia Loli
Ex-programmer, ex-editor in chief at OSNews.com, now a visual artist/filmmaker.
Follow me on Twitter @EugeniaLoli
Almost a nice article, should’ve been a little longer or gone a little deeper.
I never really grasped this scheduler thing and the article was very promising, but when the introduction was over and when it should’ve gotten interesting, it was finished. I hope i didn’t miss the page 2 link, but i can’t find it
I copied one big file, rendered a 3gb mpeg video and played mp3 (xmms) with a 2.6.4 kernel at the same time.
all enabled: preemption, ide-dma, played music with alsa devices on a sblive (no onboard!) and the sound stuttered…
Hmm, this was on 2.4.x series too and I hoped so much it will be gone. Heavy harddisc usage and playing sound is not a solved problem with 2.6.x
Ok, it’s better than 2.4.x, but sorry to say this, not so good as it was on Win2000.
It’s only my opinion and impressions, not a flame war. I love my Linux but I must know what it can do and what’s not so good to have a workaround (“do not copy to much and playing music at the same time”)
Uptime of my Internet-Router:
admin@server:~$ uptime
21:55:58 up 451 days, 3:07, 5 users, load average: 0.00, 0.00, 0.00
yes, very true, also i have had some acpi problems, where after going to sleep things didn’t wake up properly. i like the 2.6 kernel but, i think it was declared as stable too soon.
That kind of test is not very scientific. You have to make reproducible tests that yield measurable results.
Things that are wrong with the test:
Files are randomly placed in the disk, it’s not the same to have all files near in the plate than the opposite.
The block size can be different.
The available ram can be different.
The CPU usage can be different.
Background tasks can be different.
Disk cache issues.
Etc.
Hi
It just means that developers wont add random features into this series without major benefits. Its well known that the first few releases are not meant for production use.
With no distribution releasing a stable product based on 2.6 you cannot claim it thats it not as good as some other product.
If you got problems in mandrake final or fedora core 2 because of 2.6 issues then its time to make a reality check
regards
Jess
I think the new antipatory policy is designed for better I/O throughput, not for responsiveness. So everything (copy a big file, etc.) should get done more quickly than before, just not necessarily in order or with any task having a higher I/O priority than any other.
Hi
Yes. So sometimes nice values get ignored. some people have complained about this and the new set of patches in -ck tree
> I have been doing controlled tests for the past three months
disc-I/O _and_ sound output? What you’re using to make this operations realtime?
With some server benchmarks I can’t hear anything what could be “bad”, but when I listen to music, I can hear it.
That’s the difference. Ok, maybe 2.6 kernel is 1000 times better in some situations than 2.4.
But sound should get a higher realtime priority than some “stupid” disc copy.
> You have to make reproducible tests
It’s reproducible. Let’s start 2 or 3 big (around 1 gb from my tv-card) copy jobs from one partition to the other at the _same_ harddisc when you play music.
Ok, I don’t have this situation every day so it isn’t a heavy problem for me.
My opinion is, that disc i/o takes to much system power on Linux, and ide is configured properply on my system:
bash-2.05b# hdparm -tT /dev/hda
/dev/hda:
Timing buffer-cache reads: 128 MB in 0.17 seconds =744.30 MB/sec
Timing buffered disk reads: 64 MB in 1.12 seconds = 56.90 MB/sec
I think the posters point was that Windows could do all that without stuttering, and 2.6 still couldn’t. He wasn’t trying to be *scientific*. This is a common argument that people like to use for some reason.
I once had the exact same argument with a friend of mine over Linux vs BSD. I commented to him that for some reason FreeBSD (this was before 2.6 and 5.x) *felt* faster than Linux on the same hardware. Sure, it was entirly subjective, and not scientific, but I wasn’t trying to prove anything. His response was exactly like yours. It didn’t matter that I felt FreeBSD *felt* faster, because it wasn’t scientific.
Well, it mattered to ME, and also many other FreeBSD users who experienced the same sensation. Why? Who knows. Better VM maybe? The point is that if many people can notice a difference, then there is usually a reason, even if that reason ends up being mass delusion.
I don’t know why you feel the need to defend Linux so much, but your comment did nothing to change the fact that 2.6 had sound stuttering for the original poster, and Windows whatever didn’t.
You can love Linux and still see it’s limitations at the same time you know.
Nonsense. I have been doing controlled tests for the past three months. Your claims cannot be verified or reproduce, thus they amount to FUD.
Holy cow man. Consider this:
Person 1: Wow, I love it in this city, but sometimes I find it a little chilly.
Person 2: Nonsense. I have been doing controlled tests for the past three months [at my lab in the upper part of the city]. Your claims cannot be verified or reproduced, thus they amount to [FEAR, UNCERTIANLY, and DENIAL].
Person 1: *wide eyed* Umm… well, I guess you could come stay at my house [by the water ] and see for yourself if you want. *thinks to himself, what’s this guy’s problem?*
Really man. Get a grip. There is no need of such fanatical defense of Linux. Everything has some faults. I’m sure there are places where Windows skips and Linux doesn’t.
Hi
“I don’t know why you feel the need to defend Linux so much, but your comment did nothing to change the fact that 2.6 had sound stuttering for the original poster, and Windows whatever didn’t. ”
When we are talking about the scheduler the poster should post relevant stuff. Sound stuterring is not reproducible at all in my machine and its definitely not the scheduler in question. its not 1000 times slower and hence the other one has defended it.
got it?
regards
Jess
I copied one big file, rendered a 3gb mpeg video and played mp3 (xmms) with a 2.6.4 kernel at the same time.
You did all that and it stuttered? Wow.
One of Linuxs biggest performance issues (for me) has been the FACT that ever since I’ve used Linux (1996) I’ve tried many distros and no matter what – –
Whenever playing media (TV Card, Movies, Music etc.), you are grounded to that media only. I cannot play media and multitask.
Linux stutters and jerks. Its done it for years and as far as I can see, shows no hope of improvement.
You can get all technical and split all the hairs you want, but in my experience and real world use – – Linux cannot handle doing more than one big task at once if one media related. Though, the same machine running Windows or BeOS has no problems doing it.
Linux can’t even resize/move a live TV output window (bt848) without choppy redraw; at least as of last year – – I’m promised by Linux users that changed as of IMPROVEMENT X 1.X, so I’ll go and install distro X as recomennded only to find little or no impovement. This happens every new point x upgrade to distro/major window manager/kernel with no end in sight. I’m starting to think that these linux users have never used BeOS or Windows (since 95b) and feel that Linuxs poor performance in media/multitasking is somehow “normal” or acceptable.
As far as I am concerned, I have to say that 2.6 is kind of disapointmenent on *my* computer.
There is still this very annoying problem that when I am doing heavy disk operations, there are drop outs in xmms, etc… In fact, I don’t see much improvement in *my* experience. But this is on laptop, with a very slow hard drive.
Still, I don’t think it is normal to have dropouts on audio when I download a file from my local network. Heck, this problem doesn’t even happen on windows on the same machine !
Hi
can you somebody start talking about the scheduler enhancements and finally bring some relevant stuff back into this thread?
regards
Jess
Have you considered that it might have been a problem due to you utilising everything on 1 IDE Drive, and yes Windows will suffer the same state. It has nothing to do with the way the Linux Kernel is giving priority to IDE transfere but there being a lack of buffering control in xmms and that you’re trashing a poor data transfere mechanism (IDE Drive).
Try using 2 drives on separate IDE chains and have one for your intensive Video work and the other for your system and music. You’ll find that you’ll have no studdering in sound with this kind of separation, or you could update your systems drives to Serial ATA but to be honest you would either want a system with SATA built in on a Hyper Transport bus backbone or use a PCI-66 SATA controller on a board that supports PCI-66 cause PCI-33 ain’t gonna cut it.
For those proponents of BeOS that claim it can be done, sure but file transfering was a dog under BeOS speed wise.
Any Media content creation professional would have a workstation utilising separate work dedicated drives most likely SCSI based (although SATA is a decent alternative). Just standard practice be it under Mac OS, Windows or Unix.
Next.
you know what I mean, thx!
From what I understand, the way Windows and Mac OS achieve video and audio that doesn’t stutter under loads is by preserving priority levels in the scheduler specifically for multimedia applications. This is the kind of optimization that Eugenia routinely talks about (app_server dev’s cube next to kernel dev’s cube) that Linux kernel hackers hardly worry about. I’m pretty sure it would not be a “huge” deal to implement this. Of course, I could be dead wrong and Linux may already have this, but perhaps it just isn’t utilized properly? … anyone know for sure?
funny how the linux guys in the last 6 months really started getting mean when someone said anything about there little kernel.. funnier part is when someone in here posts “Oh yeah all you have to do is get the -ck kernel” its all in there. how do people get in such a hype about a kernel that has like 10+ CURRENT branches avalible! add to that the millions of other x.x.x files you have to have (dependency hell) arg, what a headache. add to that a well-over aged window server (x11) with its 10,000 window “managers” or “desktop enviroments” (diffrence being how much crap you have to install with/before either or) Hopefully with all the BS from sun/ibm recently one of them will go threw and attempt to fix the mess that is referred to as “linux”. to much choice in software can be bad. belive it…ok i’m done, mod me down
Honestly I think the sound stuttering problem is driver related, and is not the IO scheduler’s fault. I’m running an Athlon 1ghz with a Soundblaster Live!, and I haven’t experienced any stuttering – I just tested it by ripping/encoding an audio CD, having my BT client check integrety of a file, while watching a movie and playing an mp3. No skipping, but it looks like frames were being dropped from the video in some spots.
I’d imagine that somewhere someone is having a huge headache meshing the IO scheduler and pre-emptive task scheduler.
Maybe we’re finally seeing an Achilles heel to having multi-threaded disk access.
there was at one time a disk io scheduler that was designed to give a consistent data stream rate to an application rather than try and do as much io as fast as possible.
i have no idea what it was called, nor how long ago this was. what i can remember of it was that it would have ensured that xmms was getting a stream of a set speed reguardless of other disk io going on (unless there were lots and lots of independant streams needed, as it set up a minimum speed for each stream, so file copy and massive file operations could be insanely slow depending on what was going on)
a) Make sure your X server has a niceness of 0. (Check that your “/etc/X11/Xwrapper.config” has the line “nice_value=0” instead of something like “nice_value=-10”.)
b) Make sure you have enough memory since the virtual memory system in 2.6 seems to be seriously fsck’d up. Whenever I run out of physical memory I get 1-10 second pauses when playing music.
> Have you considered that it might have been a problem due
> to you utilising everything on 1 IDE Drive,
Are you kidding me? If one cp process can read >5 MiB/s then surely xmms should be able to read 20 KiB/s at the same time, don’t you think?
Unless you run a large web server with thousands of concurrent accesses there should be no problem what so ever for xmms to read the mp3, and unless you have thousands of processes fighting for CPU time there should be no problem for xmms to decode the mp3 in realtime. (There could be congestion on the bus so that the sound card is starving, but somehow other operating systems manage to handle it fairly well so this shouldn’t be a problem either. Besides, this usually results in partial looping instead of stuttering.)
> and yes Windows will suffer the same state.
Apparently you haven’t tried Linux 2.6 and windows 2k/xp on the same computer. I have tried this on several very different computers, and windows, although still quite bad, is an order of magnitude better than linux 2.6 on this.
“(There could be congestion on the bus so that the sound card is starving, but somehow other operating systems manage to handle it fairly well so this shouldn’t be a problem either. Besides, this usually results in partial looping instead of stuttering.) ”
Incorrect, I’ve watched Windows choke on MP3’s while running a file search countless times. It does it on my machine, Maxtor DiamondMax 9 w/KT400 chipset (stock VIA sound) and an XP 1800+ w/512MB PC-2700.
I’ve got the same issue, rhythmbox and muine will stutter under heavy disk I/O. xmms I don’t remember skipping, but it’s possible. it was really bad for me when I first put Linux on here (1800mhz athlon xp, kt400 chipset i think) but a week later when I realized i was running default ide drivers, after compiling 2.6.0 w/ the proper modules, things were much better. but then again just yesterday, it happened again. i was amazed.. thinking wow- i thought we were past that now.
i’ve been using linux since about 94-95, some old Slackware distro on 30-40 something floppies. i could overload it then, and, apparently even now. even though it has improved in a lot of ways like X’s responsiveness, et cetera.
i guess it’s kind of like designing an engine. you’ve got to aim for raw speed or torque..
> a) Make sure your X server…
Thanx, I will check it!
> b) Make sure you have enough memory…
Are 1 GB RAM(!) enough? And a P4 2.4Ghz? And a really fast (160GB, DMA-Mode, around 55MB/sec) Harddrive? And Slackware 9.1 with 2.6.5 running?
Hopefully nice X helps a bit, but don’t think it’s to much disc I/O usage related.
Hi folks,
so sometimes for your workload it seems 1000 times slower. Which scheduler were you using? Did you try with the other one? (Isn’t that the point?)
whats amazing is i have YET to get it to stutter, except when pan eats all RAM and SWAP and the entire system crawls to a stop. (this is a pain obvisouly). but under normal circumstances, which is a rather large load. ill copy a 4 gig file across partitions, 650 meg files from the network, sometimes both at the same time, playing music, or watching a video file and its nothing but perfect.
i am quite amazed because 2.4 was less than perfect, not horrible by any means, but it would stutter.
my biggest complaint is that i wish there is a way to stop a program from eating all the RAM + SWAP. some sort of maximum based on currently used maybe? but then again the only problem i have is really with Pan and the way it handles the newsgroup threads (500,000 posts+)
every once in a while fam will go insane an eat the processor, in the past i would notice this happeniong because things would feel slower. but since 2.6, im amazed, (and ALMOST annoyed becaused the last time it happened, fam was running for 20 hours at 99%, and i didnt realize it till i used top ).
one complaint i do have is related to the MM-sources, after a few days, all the RAM is eaten up and the system gets to be sluggish, but in the development sources its not a problem. anyone else have this issue?
Unless you run a large web server with thousands of concurrent accesses there should be no problem what so ever for xmms to read the mp3, and unless you have thousands of processes fighting for CPU time there should be no problem for xmms to decode the mp3 in realtime.
Actually you are testing ONE audio player and drawing conclusion about the WHOLE OS. I remember there being a huge thread once on linux-kernel where the internal design of XMMS was critiqued. I would suggest the testers to try it with another player like for instance AlsaPlayer ( http://www.alsaplayer.org ). It might not look sexy but the engine is supposed to be a lot better at stutter prevention than XMMS.
-fooks
whats amazing is i have YET to get it to stutter, except when pan eats all RAM and SWAP and the entire system crawls to a stop. (this is a pain obvisouly). but under normal circumstances, which is a rather large load. ill copy a 4 gig file across partitions, 650 meg files from the network, sometimes both at the same time, playing music, or watching a video file and its nothing but perfect.
Ditto here. My P4 M 2Ghz, 512 MB laptop cries for mercy once its RAM is full and the swap kicks in. How? Simple, fire up DCGUI-QT and connect to 15 or more hubs, each having a 1000 users or more and multi-download a couple of the latest movies. DCGUI-QT eats all my CPU by now. Next fire up Juk and listen to your favorites, which, at this time of writing, is Coldplay. Also, do not forget to frantically browse the Web with Firefox/Opera or whatever. Finally, do an ’emerge sync’ and the poo hits the fan: my machine crawls to a halt and music stutters….
Should I be surprised?
forgot to mention that I’m using gentoo-dev-sources: 2.6.3-r1. Any Gentoo buddies out there that know about some kernel optimized for multimedia?
this is all pretty strange to me…I am only running kernel 2.4.22 (installed using debian thru mepis automatically) on a shuttle sffpc with a gig’o’ram and p4 2.6 Ghz w/ht and no stuttering at all when using an audigy2…disk transfers aren’t as fast as I’d like but there’s no noticable slowdowns that I can recall. I even game on this thing.
foo
“can you somebody start talking about the scheduler enhancements and finally bring some relevant stuff back into this thread?”
Well this has to do with scheduling, caching and so, hasn’t it ? I don’t draw any conclusion, I specifically said that it didn’t work on MY computer. I expected the end of this problem on linux, and it is still here.
“Actually you are testing ONE audio player and drawing conclusion about the WHOLE OS.”
Same goes for alsaplayer, in fact, even if I nice it with a priority of -10. If I, for example, copy one big file from one partition to the other on the same drive, there are big audio drops. and I have more than enough ram (512 Mo, more than 300 Mo free once I deduced buffer and cache when I try it right now). Moreoever, alsaplayer interface sucks , for now.
Even when I am using jack as a server for alsaplayer, with realtime enabled, I have dropout. But it may come from my awful audio chipset. I am still investigating for the causes…
Still, under XP, I don’t have this problem. So it seems raisonable to suspect a problem from linux/driver on that matter.
I have shuttering sound too (not very often), and have an SMP system. In my case, it has nothing to do with Disk IO or CPU usage (I have a lot of disk IO sice I use software RAID 5). My guess is that X and xmms are responsible for the mess. When I scroll with firefox, or do some other stuff, sound shutters. Other apps don’t have any problem with sound.
First it is important to say that in an overloaded system no process can receive the amount of cpu needed.
The solution is to increase priority of important processes
The kernel has an autohint system to auto increase priority of interactive apps but but it is better to improve priority by hand.
Kde can put arts server on realtime priority and so it can eliminate stuttering.
Then there is chipset problem.
And acpi kernel implementation problem.
It is incredible but on high loads I have less problem on my old notebook with apm that on my new athlon with acpi and via chipset.
I have also a dual pentium III with Serverworks chipset and it is incredibly responsive: in the future I will buy only old dual cpu servers on ebay: are better than cheap but fast mono cpu via based computer
I too experienced stuttering sound with Linux 2.6.X. I noticed it while ripping a audio-cd with Sound-Juicer and listening to music with rhythmbox. In that case I assume it’s a problem of gstreamer framework, as it is still in development. When I ripped the audio-cd with cdparanoia, sound didn’t stutter. With XMMS I had no problem at all, regardless if I used Sound-Juicer or cdparanoia. That is with an Athlon XP 1800+ on Gigabyte GA7-DX mainboard and TerraTec DMX XFire 1024 soundcard w/ alsa drivers.
I forgot to mention that I haven’t tested this under greater system load than the one mentioned above…
on my relatively old system (pII-333, 440bx) the windows winamp would stutter even if i moved windows about the screen. using official drivers et al.
linux 2.4 and now 2.6 don’t fdo that, which is very nice.
i’m certainly “noticing / feeling” that 2.6 is snappier and has a lighter touch… almost feels like i have multiple separate computers inside my box, each dedicated to their own tasks!
http://bulk.fefe.de/scalability/
I have now a fibre channel controller with cpu on board linked to 15k hard disks.
Last day a kopete process gone mad and eat all memory including 1gb of swap: I did notice almost any slowdown….
At work I am (=have to) running Windows2000.
Because of a virus in our corporate network, everybody had to do a complete virus check of its computer this morning.
I can assure you that with the virus scanner running over the harddisk for 1,5 hours (which just finished) it is nearly impossible to get any other work done (with much switching between different programs). I cannot recall any comparable experience on my home machine with Debian and a pre-emptive 2.4.25 kernel.
Another point with windows 2000 is that after logging into the desktop, the system is unusable for serveral more minutes (probably while services and programs in the background are loading) while when I am booting into my Debian desktop (fluxbox) within 25 seconds the system is completely there.
I find my linux desktops more responsive than my windows 2000 desktop.
– However, it was different when running Debian with a standard kernel on a 486 – the cron jobs could make the system very irresponsive.
I don’t think I’ve seen any system that can cope with heavy hard disk activity and still play MP3s without stuttering. Even with Windows XP running on a 3Ghz Pentium 4 with 1Gb RAM I’ve heard stuttering when playing MP3s in foobar2000 or winamp. Just copying some large files between different hard drives can cause a few clicks in the MP3 playback.
Personally I use a passively cooled 200Mhz PC as an MP3 player, that way I don’t have to worry about causing it to stutter.
>> Have you considered that it might have been a problem due
>> to you utilising everything on 1 IDE Drive,
>Are you kidding me? If one cp process can read >5 MiB/s >then surely xmms should be able to read 20 KiB/s at the >same time, don’t you think?
The CPU isn’t the bottleneck though. It’s the one IDE drive. Even if the CPU can handle it, the data doesn’t just materialise out of thin air, it has to come over from your hard drive, which was suggested perhaps has to do with “you utilising everything on 1 IDE drive”
Running a lowly Duron 750 with 512Mb RAM. Quite happily listen to radio using MPlayer and do some pretty intensive GIS stuff with GRASS.
I forgot to talk about cfq which is the new i/o scheduler that will be added to anticipator and deadline. It improves very much responsiveness but the version I have tried makes all things a bit slower.
Obviously they have opened a new door with interchangeable schedulers.
With Juk or Rythmbox i find the sound stuttering alot,although I think it’s artsd and esound that is the problem(xmms skips easily with arts-output), but with xmms (alsa, but using oss-output..alsa skips too much..and with buffersize set to 10000(the same as I found out that is the only reasonable size, both in linux and windows) it doesn’t skip at all, even when I copy a lot of large files around. The system is a 600MHz pIII with 512MB cs2 sdram and an WDC WD84AA, connected to an onboard VIA-ide controller.. using 2.6.5 with default scheduler and CONFIG_PREEMPT=y
My main development machine is an Asus A7M266D, 2xMP2400, 1GB, PCI64x SCSI, GF5900 (v44.96) running a heavily modified RH9 with 2.8/1.2 split 2.6.5 kernel.
I’ve got two sound cards, the on-board C-Media (used for my PVR system) and an SB-Live! card (use for 4.1 sound).
It’s not uncommon that I work/debug/compile on the machine (with xmms playing music) while someone else is watching a DVD (using xine) in my living room. (and all of this while I run two copies of seti@home in the background.)
No shutters, no dropped frames, nothing.
Both sound cards are using arts (with a very low latency setup… games you know
) over alsa.
Oh… for a while I tried doing the same with Windows XP pro. The machine BSODed constantly.
Cheers,
Gilboa
I agree with these posts, Linux stills leaves to desire at the user experience level, it is improving but not as completely smooth as on Windows.
On windows (under normal conditions) for the audio to jump you need to do something really nasty, on Linux it happens most of the time you make the computer access three or more different files.
Anyway I’m not concerned about it, Linus is really keen on the Desktop use of Linux and I’m sure they’ll manage to fix this problems.
>>> Have you considered that it might have been a problem due
>>> to you utilising everything on 1 IDE Drive,
>>
>> Are you kidding me? If one cp process can read 5 MiB/s
>> then surely xmms should be able to read 20 KiB/s at the
>> same time, don’t you think?
>
> The CPU isn’t the bottleneck though. It’s the one IDE
> drive. Even if the CPU can handle it, the data doesn’t
> just materialise out of thin air, it has to come over
> from your hard drive, which was suggested perhaps has
> to do with “you utilising everything on 1 IDE drive”
Apparently you didn’t understand what you quoted me saying. If both cp and xmms read from the same partition and cp manages to read over 5 MiB/s then there should be no problem what so ever for xmms to read 20 KiB/s. (Both of them should need only extremely little CPU time, since they are mostly waiting on I/O.)
I mean, listen to what you’re saying. C’mon! There is NO WAY a modern HD would be so busy that you couldn’t get 20 KiB/s from it even in moderately heavy use. That is, unless the I/O system is completely fsck’d up.
> I can assure you that with the virus scanner running over
> the harddisk for 1,5 hours (which just finished) it is
> nearly impossible to get any other work done (with much
> switching between different programs). I cannot recall any
> comparable experience on my home machine with Debian and a
> pre-emptive 2.4.25 kernel.
You should’ve been in my shoes a month ago before I got more memory on my computer at work. I was running kde, mysql, jboss and intellij idea on 256MiB, and then when a garbage collection started on the JVM running idea it took over an hour for it to finish. The system was almost completely frozen for over an hour. The mouse pointer would hang for 2-10 seconds every few seconds. Let me tell you, if you are programming and are “in the zone” then after a one hour system lockup you are so far from the zone that the zone looks like a dot.
This is the only time I’ve experienced a >1h GC, but 20min GCs weren’t unusual.
No matter how busy a system is it should still have time to update the mouse pointer at least once every 100ms. 10s is two orders of magnitude greater than that minimum requirement.