Linked by Thom Holwerda on Wed 28th Feb 2007 19:29 UTC
Jeff Atwood explains why Vista uses so much memory. "You have to stop thinking of system memory as a resource and start thinking of it as a a cache. Just like the level 1 and level 2 cache on your CPU, system memory is yet another type of high-speed cache that sits between your computer and the disk drive. And the most important rule of cache design is that empty cache memory is wasted cache memory. Empty cache isn't doing you any good. It's expensive, high-speed memory sucking down power for zero benefit. The primary mission in the life of every cache is to populate itself as quickly as possible with the data that's most likely to be needed - and to consistently deliver a high 'hit rate' of needed data retrieved from the cache."
Permalink for comment 217393
To read all comments associated with this story, please click here.
To read all comments associated with this story, please click here.
Member since:
2005-07-08
I don't even think Linux had [a disk cache] when Windows XP first came out because they were still dealing with the 2.4 kernel.
That's the funniest thing I've read so far today. Linux and every other major UNIX-like system back to at least SVR4 and beyond has had a disk cache that consumes any otherwise unused physically-resident virtual memory allocated to the kernel. Now, to be fair, Linux 2.4 implemented disk caching through the buffer cache, which uses disk blocks as the fundamental unit of storage. In Linux 2.6, the disk cache is implemented mostly through the page cache, which uses larger and more well-behaved (with respect to alignment) virtual memory pages as the fundamental unit of storage. The buffer cache is still used for caching inherently block-sized storage objects such as inodes and other filesystem metadata objects.
The kernel does have a hard limit on virtual address space with comes into play on 32-bit systems (or 64-bit systems running a 32-bit OS). This limit must be set no later than boot-time (usually it is set at install-time) and depends on how much physical memory you have installed on your system. Basically, if you have 4GB of memory installed, and you're running a 32-bit OS, the kernel can only allocate 1-3GB of this memory no matter how small the memory requirements of userspace might be. If it's set to allocate 3GB, you run the risk of starving your userspace applications of virtual memory, causing them to crash. This was a bug that some users hit while running the 32-bit Vista installer on systems with 3GB+ of memory.
These techniques are not new. There are classic OS textbooks that describe these mechanisms in great detail. In fact, I'm sure that any Windows OS that had support for NTFS must have also featured a disk cache, since the basic design consideration for NTFS is that reads should usually be satisfied from the disk cache. When the disk cache hit-rate drops, read performance on NTFS becomes unacceptable, nearly as bad as FAT32.
The fact of the matter is that physical memory is a resource. It's obviously not just a cache. Even on systems with pervasive support for pageable kernelspace (i.e. AIX), some portions of physical memory may never be paged to disk. But more importantly, the system must make the best use of its resources on all levels of the storage hierarchy. For example, local disk storage should be used as a cache for network resources such as NFS volumes. If the NFS volume is large enough, the cachefs volume should expand to fill your unused and unreserved space. Obviously some space will need to remain reserved for paging, dumping the kernel (if so configured), and for emergency maintenance by the superuser. But any other unused space is wasted if there's uncached data on a more distance level of the storage hierarchy.
Edited 2007-02-28 22:33