
Being a BeOS user (a purely desktop system) and because I code under Linux, I see
XFree86 (v4.1 on my machine) as a user and as a developper. And this is where the problem lies. My Gnome or KDE desktops are slow in comparison with other operating systems, but XFree86, the 'engine' behind these desktops, proves me that it's not. Let's look at what I have in front of me: a dual Pentium III at 933Mhz with 512MB of memory, a Radeon 32 AIW, a modified Mandrake 8.0 powered by kernel 2.4.18.
Yet another good article
This one full of good technical stuff
I wish it was longer though
As for drivers being a issue in XFree. Having a better driver is part of the solution but if XFree had more optomization then every system would benefit including those with bad or good drivers.