Linked by Thom Holwerda on Mon 11th May 2009 20:43 UTC
Windows SuperFetch is a technology in Windows Vista and onwards that is often misunderstood. I decided to delve into this technology to see what it is all about, and to dispel some of the myths surrounding this feature.
E-mail Print r 3   · Read More · 79 Comment(s)
Thread beginning with comment 362926
To read all comments associated with this story, please click here.
segedunum
Member since:
2005-07-06

I've seen an awful lot of sites trying desperately to dispel 'myths' about Superfetch, and Prefetch in Windows XP, but it doesn't dispel them at all because any such system has to make assumptions about what your most used applications are. It really is debatable how much of a performance improvement there is using it, and I've seen no benchmarks to suggest that any improvement exists - at all. I've only seen articles dispelling 'myths' about it.

Install new apps and use them or use a reasonable variety of apps and what Windows thinks that you're going to be using is just wasting memory and eventually ends up having to be unloaded, just slowing down the current app you want to run. This can result in the very well known paging to and from disk people see as the cache is reformed - the very thing fetching is supposed to minimise. You see, Superfetch constantly has a need to use 'all' your memory because that's how it's designed. To make it work you need a ton of it and you *need* 64-bit Windows to ensure this paging situation does arise. Kind of pointless really. This situation gets far, far worse when you start using more memory intensive applications. You don't get anything for free. If you're a gamer, turn it off immediately.

Strangely, you don't see it if you turn the system off or if you use the sensible Windows Server as a workstation, which sensibly doesn't use it. Each application is treated equally. Superfetch works better the more memory you have? Well, duh. I'd love to be able to see into the future and preload for every application I'm going to be using before I do, but that's just a tad impractical. I just want the app I'm running now to run well, not just load faster.

Reducing load times? Bah. Software vendors have been preloading for years with all those pointless system tray things at startup. It might make their applications load faster but it will probably be to the detriment of what you happen to want to run.

To really make this kind of thing work then the algorithms that predict what you'll be needing need to get far better. Until they get it right just turn it off or install Windows Server and use it as a workstation. It just tries to be too clever for its own good.

Reply Score: 3

bob_bipbip Member since:
2009-04-28

I've seen an awful lot of sites trying desperately to dispel 'myths' about Superfetch, and Prefetch in Windows XP, but it doesn't dispel them at all because any such system has to make assumptions about what your most used applications are. It really is debatable how much of a performance improvement there is using it, and I've seen no benchmarks to suggest that any improvement exists - at all. I've only seen articles dispelling 'myths' about it.

my benchmark in xp:
after deleting all prefetch file in the prefetch folder, winxp take way less time to boot.
i user my xp only to play games (for work, linux is fine too). i didn't notice a slowdown in my start up time for games app.
so, winxp take less time to go from computer off to ingame without prefetch ;)

Reply Parent Score: 1

Karitku Member since:
2006-01-12

Did you even bother to read article or the link that Thom posted? Apperently no. Superfetch and application loading in tray is totally 2 different thing. You make false claims that Superfetch would make other apps to page stuff that isn't true since Superfetched material is always dumped away if memory is needed. I never seen any claims, other than forum trolls, that would prove that using Superfetch hinders gaming performance. Show us the proofs!

Reply Parent Score: 1

segedunum Member since:
2005-07-06

Superfetch and application loading in tray is totally 2 different thing.

No they arent, and lots of people seem to be hypnotised into believing that they are. The intended effect of using the two is the same - preload all or part of an application into memory to make it load faster. The only difference with Superfetch is that it is a more universal way of managing it for all applications you might use. It does not make the application run faster once it is running and Superfetch itself cannot guarantee that any given application will load or even run faster - just what it thinks you'll be using. There's only so many ways of cutting that.

You make false claims that Superfetch would make other apps to page stuff that isn't true since Superfetched material is always dumped away if memory is needed.

Superfetch relies on building a cache of memory that expands to the total amount of memory that you have. Once a cache of a large amount applications has been built up after a reasonable amount of usage (which is where the 'myth' that this gets worse over time comes from) that hits the limit of your installed system RAM then you really start to see the effect of memory management and paging happens as things get moved around. This is why you need several gigabytes of memory to make it work.

Superfetch is a process, albeit a low priority one, that moves memory and rebuilds the cache both at startup and as it's running. Memory management is expensive, especially once you start hitting certain limits. To think otherwise and to think that Superfetch is 'free' is stupid. To cultivate this image by dispelling 'myths' is even more stupid.

I never seen any claims, other than forum trolls, that would prove that using Superfetch hinders gaming performance. Show us the proofs!

Sorry, but I'm afraid you can't just start demanding proof in response to me asking for benchmarks and proof of whether Superfetch actually works for people. The subject is Superfetch therefore show me that Superfetch works. You can't just throw something in and say "Prove that it doesn't work". Doesn't work like that.

I have yet to see that this is anything other than a another pointless technology by Microsoft to expand memory requirements in a pretty pointless way.

Reply Parent Score: 1

superstoned Member since:
2005-07-07

Sorry, but I think you don't get it.

Superfetch loads things in memory it thinks you'll need. If you don't need it, no harm done - 'unloading' doesn't exist. Apps start as fast as without superfetch OR faster. Never slower. Same story with the linux memory management, expect that the linux one is less smart.

The only 2 ways in which superfetch could technically be a disadvantage are these:
- if the IO priority system is screwed, the reading superfech does COULD interfere with other activity. Depends on the quality of the kernel's management of priorities. No idea if this is an issue at all, I doubt it, but it is possible.
- superfech uses resources while your PC wouldn't have been doing anything if it weren't running. Bad for powerusage = environment and probably slightly bad for the lifespan of your hardware.

It might be true that the first issue I noted is actually real - but then the MS engineers writing this would've been rather stupid to forget about making the priority system work properly. Afaik that's not the case, so superfetch never slows anything down (but might not speed anything up either, depending on usage paterns and available memory).

The second - well, it matters on laptops and environmentalists I suppose. But that's about it...

Reply Parent Score: 2

WereCatf Member since:
2006-02-15

The only 2 ways in which superfetch could technically be a disadvantage are these:
- if the IO priority system is screwed, the reading superfech does COULD interfere with other activity. Depends on the quality of the kernel's management of priorities. No idea if this is an issue at all, I doubt it, but it is possible.
- superfech uses resources while your PC wouldn't have been doing anything if it weren't running. Bad for powerusage = environment and probably slightly bad for the lifespan of your hardware.


I can think of atleast one case where SuperFetch is a disadvantage: if you are playing a game the game will most likely read its files in more-or-less sequential order, and disk cache will load the parts of the files it assumes will be needed next. But if SuperFetch is caching something in the background while you're playing then it'll invalidate the disk cache causing slightly longer loading times.

Now, I'm not saying that's a big issue though. It's most likely negligible enough for most people not to care. I just mentioned it for the sake of argument.

Reply Parent Score: 3

segedunum Member since:
2005-07-06

Superfetch loads things in memory it thinks you'll need. If you don't need it, no harm done - 'unloading' doesn't exist.

Managing memory in any way is expensive. You have to manage that cache. Nothing is ever cost-free.

Same story with the linux memory management, expect that the linux one is less smart.

This is nothing like Linux memory management. Linux traditionally keeps a cache because memory is wasted if it isn't used. Old pages are paged to disk cache regularly. So far, so good. What Superfetch does on top of that is to 'intelligently' preload applications into a cache it thinks that you'll need at startup and as it's running. That's an expensive operation from an I/O point of view that is not guaranteed at all to help you at any given time or any given application you might be using. Preload for Linux faces all the same unreproduceable issues and questions of whether it makes a real difference to a user's overall usage. Hell, Linux's caching alone still has some extreme corner cases.

With Linux, if you load something once and then again and it is still in the cache then great. Things will be faster. However, it does not attempt to constantly fill and manage the cache based on its idea of what your usage pattern is. That's the key difference. In theory, it would be great if you could get that right which is why people think this is such a great idea but in practice things are very different.

Besides, with any form of caching the central point stands and this certainly stands with an additional service over and above caching such as Superfetch. You need plenty of 'free' memory over and above what applications you use to make it work. It's fine when you can put 'free' memory to work but it's not fine when you don't have it which is often the case on desktop systems. Whereas the cache itself might be relatively cost free given the benefits, if you start sticking a service on top that decides what should constantly be in the cache then you get a whole bunch of real unknowns.

It's not really the technological idea that is at fault with Superfetch. It's the idea that it knows what your usage pattern is, it knows what you'll be running in the future and the algorithms responsible for deciding it.

Reply Parent Score: 2

JeffS Member since:
2005-07-12

"Superfetch loads things in memory it thinks you'll need. If you don't need it, no harm done - 'unloading' doesn't exist. Apps start as fast as without superfetch OR faster. Never slower. Same story with the linux memory management, expect that the linux one is less smart."

Nonsense. The way Linux uses caching is much, much smarter, IMHO. It does moderate caching, leaving much of the system RAM free. This means less disc I/O, less flushing, less paging, but also faster load time for common apps, and faster access time for common data.

SuperFetch just tries to use as much unused RAM as possible. While the idea "unused RAM is wasted RAM" sounds decent in theory, in practice it's not. Trying to purposely use up unneeded RAM just causes more activity, more disc I/O, more paging, and more flushing (when less common stuff is started). It also drains battery life on laptops faster (more stuff going on).

And of course, no caching is a bad idea, too. Caching, in a lot of areas, has proven to have speed benefits.

Thus, the middle ground is the way to go. Linux takes the middle ground. SuperFetch takes the extreme ground of caching and using as much memory and disc I/O as possible.

And the funny thing is, Linux, with it's more moderate caching, and it's utilization of less overall resources. is much much much faster, and loads apps like Firefox, Eclipse, OpenOffice, QtCreator, NetBeans, GIMP - all cross platform apps) a gazillion times faster than Vista with Superfetch.

Reply Parent Score: 2

malxau Member since:
2005-12-04

...If you don't need it, no harm done - 'unloading' doesn't exist. Apps start as fast as without superfetch OR faster. Never slower.


This story is full of comments by people indicating that it can be slower. Note that if an app requests 200Mb RAM, the memory manager still needs to walk along the standby list discarding prefetch data page-by-page, so it's more accurate to say "SF data is not written back to the pagefile", but it still needs to be unloaded.

if the IO priority system is screwed, the reading superfech does COULD interfere with other activity. Depends on the quality of the kernel's management of priorities. No idea if this is an issue at all, I doubt it, but it is possible.


If the priority system were not screwed, it could still be bad. In theory, low priority IOs should be issued to disk on a 1-in-20 basis when normal priority IO exists. If your normal pri IO is sequential, that low priority IO can still trigger a seek and slow it down.

And yes, IO priorities are far from perfect. In Vista, no priority boosting was performed if a regular priority thread was waiting on activity performed by a low priority thread. It could stall on a collided pagefault (needs the data prefetch is fetching) or on any lock in the IO path (prefetch has a lock held, issued low pri IO that takes seconds to complete, and the foreground apps wait for the lock.)

In Win7 boosting is implemented and will be performed if a normal pri thread waits on many common locks (but not all) held by a low pri thread.

superfech uses resources while your PC wouldn't have been doing anything if it weren't running. Bad for powerusage = environment and probably slightly bad for the lifespan of your hardware.


That's inherent in the design of SF (use background cycles to load things.)

It might be true that the first issue I noted is actually real - but then the MS engineers writing this would've been rather stupid to forget about making the priority system work properly.


(Disclaimer: I am an MS engineer who worked on making the priority system work properly, but have not worked on SF itself.)

To be fair to the SF crew, Vista was finished under considerable time pressure. Changing every lock in the IO path to be low priority aware is a huge job. They were faced with the choice of shipping with no IO priority, shipping with IO priority in the disk scheduler that can invert in higher levels, or spending years making it perfect. They chose the middle option, presumably as a compromise. It still gives some benefit, but also has some inversion problems. Their choice doesn't make them stupid - it made sense at the time. Resources are always finite.

Reply Parent Score: 1