The users' main gripe (except the involved installation process) is that Gentoo takes too long to compile either itself (during installation) or when later compiling big packages (eg. KDE/Gnome) or when updating the system. Installing the system, using the Stage 1 tarball, can take from 24 hours up to 2 days, depending on the power of your system. Also, (un)fortunately, most packages have new ebuilds in the Portage tree every few days or weeks, with fixed bugs. This is of course a good thing. But it is not when you need to wait hours to recompile from scratch all these packages every few days. And when "updating world", most of the times it would take a while. This is a tiring process.
Purists will say that they just leave the compilation processes taking place overnight, but let's face it, this is hardly an elegant solution, it does not solve the problem, it just hides it (in the dark of the night). Developers will argue that Portage is great for solving dependancies automatically, and I will agree with this.
However, on my personal experience with Gentoo since this April, there were more than 10 distinct cases where updating the system or libraries it broke other packages or even the system. And at the end of the day, Portage is not an easy to use system. Sure, "emerge -u world" or "emerge kde" sounds like a piece of cake, and it is a piece of cake. But when you actually get to use the system, you will see that you need to get your nose in several /etc/ scripts or Portage's own command line arguments. Many times I had to edit portage-related ebuild scripts or configuration files (both on /etc/ and /usr/portage/ and elsewhere) and when it comes to the cryptic USE command, it just becomes too complicated and too involved for the plain user. It is just more than I would wish to do with my system.
Today, it is possible to download binary ebuilds, however there are not many such binaries uploaded. Still, 99% of the cases deal with source recompilation on Gentoo. And while some will argue that this is Gentoo's strength, I would argue that this is absolutely and utterly tiring for a desktop machine, and simple users might end up leaving the distribution because of this reason.
I would like to suggest that the Gentoo maintainers and third party Gentoo devs distribute binaries for all software they enter in the portage tree. It would be nice if the software is categorized first by platform, then by the Gentoo version and then by CPU architecture. In other words, the long and painful job of creating packages will get in the developer's shoulders, not on the user's. Gentoo should only support i586, i686, PIII, Athlon, AthlonXP and Pentium4 architectures by utilizing the mcpu and march GCC flags. Users for the rest (and least used) CPU architectures (like K6, Athlon-MP, PPro, Cyrix etc) could have two options: either use the default i586 build which works on all modern CPUs, or simply resort in the traditional Portage method, the recompilation, if they want to squeeze every bit of performance out of their machines.
The power of Gentoo should be in the way it handles the packages, not so much on how fast it makes the system. Becoming just 2% or 3% or even 10% faster than itself when built for another CPU architecture, or than Mandrake or Red Hat, does not justify the cost of losing users.
Portage is a great package management system, it is considered as the next generation of Debian's and FreeBSD's package managers. However lacks a good front-end to make its most advanced usage (which is often required) easier (there are five different GUI front-ends, each one with its own problems), it often breaks stuff, and it does not provide an easy way to exclude a specific package when globally updating the system. If Gentoo "sponsors" and supports one of the 5 gui front ends and complete it, change Portage to be as simple as "Install/Uninstall Packages" and "Update System" instead of all this jibberish and confusing UI, it would be best for the further acceptance of Gentoo by everyday users, and not just a bunch of geeks and developers. OSNews recently hosted an excellent article on what kind of integration and ease of use a modern package system should provide.
Gentoo developers, this is not about giving up your beliefs, it is about providing the best for your users. Sometimes "best" is not what would actually be best, but what would be hassle-free for the user. Microsoft built an empire with this way of thinking.
On the other hand, distributing binaries built for different architectures, it ensures bugs on the applications themselves, because GCC creates different binary code for each time of them. And developers do not always have handy machines with all these different CPUs, to test with. This is why Gentoo would need a "beta test" cycle before everything goes to the main Portage tree (and I am not talking about beta testing via editing configuration files in order to allow emerging of "hidden" ebuilds).
Personally, I haven't updated my Gentoo box for a month now. I am fed up with all these compilations taking place almost daily (I like trying lots of applications...), and especially now that GCC 3.x is the default compiler and it is way slower in compilation times than GCC 2.9x (but creates faster code), it is just a no-go option for Gentoo to remain my main Linux distribution (I have 4 others installed currently). I am a geek, and I am a developer (Gentoo's main userbase). However, after a while, I just want to use the OS, and not let it get in my way with broken libraries or packages (my last 'update world' broke my ALSA sound - haven't bothered to fix it completely yet, I got sound only via the terminal and not on KDE/Gnome).
I do like Gentoo, I love its community and its helpfull developers and I do like it being fast and all. But I believe that my next main Linux distro will be Red Hat 8, if Gentoo Linux won't offer a "user-oriented" hassle-free distribution in time. If this is not the scope of Gentoo Linux, well, no hard feelings, I would perfectly understand and part ways under the best of terms. But Portage's automatic dependancy solver is still sweet and hard to part ways.