Linked by Jordan Spencer Cunningham on Mon 28th Sep 2009 18:21 UTC, submitted by Michael
Benchmarks Phoronix was kind enough to add a deliciously lengthy nine-page compare and contrast between FreeBSD 8 and Ubuntu 9.10 to their arsenal of articles. "Canonical will be releasing Ubuntu 9.10 at the end of next month while the final release of FreeBSD 8.0 is also expected within the next few weeks. With these two popular free software operating systems both having major updates coming out at around the same time, we decided it warranted some early benchmarking as we see how the FreeBSD 8.0 and Ubuntu 9.10 performance compares. For looking more at the FreeBSD performance we also have included test results from FreeBSD 7.2, the current stable release. In this article are mostly the server and workstation oriented benchmarks with the testing being carried out on a dual AMD Opteron quad-core workstation."
Order by: Score:
Pointless comparisson
by sbenitezb on Mon 28th Sep 2009 19:02 UTC
sbenitezb
Member since:
2005-07-22

Everyone knows Linux is optimized for speed. I would like a benchmark for serving under heavy load.

Reply Score: 3

RE: Pointless comparisson
by SReilly on Mon 28th Sep 2009 19:35 UTC in reply to "Pointless comparisson"
SReilly Member since:
2006-12-28

Everyone knows Linux is optimized for speed. I would like a benchmark for serving under heavy load.

Ehm, correct me if I'm wrong but isn't all optimization basically about speed? You can optimize for I/O, filesystem, multimedia, database and/or other applications but aren't all these optimizations always about speed?

You're going to have to be more specific. Optimizing for heavy load is no good if it crawls. If fact, a machine that is slow under heavy load is not optimized at all.

Reply Score: 7

RE[2]: Pointless comparisson
by project_2501 on Mon 28th Sep 2009 21:18 UTC in reply to "RE: Pointless comparisson"
project_2501 Member since:
2006-03-20

Let me explain the difference.

System A runs fast. 80 miles/hour. Its a porshe.
System B runs slower. 50 miles/hour. Its a truck.

Thing is, add more users, System A quickly slows to a crawl.

System B meanwhile doesn't slow down by much. Carrying 50 users it still maintains 49 miles/hour.

Speed itself doesn't complete the picture. Scalability adds the additional dimension.

Reply Score: 5

RE[3]: Pointless comparisson
by SReilly on Mon 28th Sep 2009 21:57 UTC in reply to "RE[2]: Pointless comparisson"
SReilly Member since:
2006-12-28

I can see where you are coming from and would agree, if we where talking about auto mobiles, but we aren't. We are talking essentially about calculations per second.

I agree that scalability is a factor when dealing with heavy loads but if your system is slow to begin with, it makes no difference how many more users you throw at it. It still crawls. Sure, it might crawl less than a system that is not optimized for high I/O but it still crawls.

All systems have limits, and some of these limits can be increased at the expense of others. This is a standard issue when dealing with system optimization. The bottoms line though is always that if one part of the system is significantly below par, it will drag the rest down.

We are not talking about different types of fuel, they both run on electricity in the end. We are also not talking about different types of engines, they where tested on the same hardward. I have always had the impression that FreeBSD scales horizontally (as in SMP) better than Linux, but frankly what does that serve you if your system crawls?

Reply Score: 2

RE[4]: Pointless comparisson
by project_2501 on Mon 28th Sep 2009 22:37 UTC in reply to "RE[3]: Pointless comparisson"
project_2501 Member since:
2006-03-20

It all depends .. as it always does, on what you wnat to do with your system.

And this is why any serious investor in such systems always carries out tests before committing.

Reply Score: 2

RE[5]: Pointless comparisson
by SReilly on Tue 29th Sep 2009 13:41 UTC in reply to "RE[4]: Pointless comparisson"
SReilly Member since:
2006-12-28

As a Unix system engineer, I install big iron serves as a job. All of these installations include optimization and I'm sorry to say this so bluntly, it never just "depends". In fact, if I where to say that to a customer, they would tell me to get lost.

When a customer buys a system, they are always for a particular task, be that database, java applications, backup infrastructure, the list goes on. Every time we optimize for a task, it is always on the basis of speed.

If a customer wants a system to be good at heavy user loads, then we optimize for I/O and filesystem speed and try to keep the amount of background processes to a minimum. With java applications, it's all about cutting the system down to the minimum and making sure the app server has as many resources as possible. Believe me, java app servers are hungry. As for databases, it's about filesystem block size more than anything.

What do all of these have in common? Speed. There is no way that your system is optimized if it's slow. If your system runs dog slow to begin with, it is not going to be any faster under high load.

I have been dealing with high end Unix systems for the past seven years and that is no rule of thumb. That is a fact. Yes, I can throw a thousand users at a Solaris or AIX box, but I can do that just as easily with a Linux box. And it'd be half the price on a Linux machine to boot.

Reply Score: 3

RE[4]: Pointless comparisson
by vivainio on Mon 28th Sep 2009 23:16 UTC in reply to "RE[3]: Pointless comparisson"
vivainio Member since:
2008-12-26

I have always had the impression that FreeBSD scales horizontally (as in SMP) better than Linux, but frankly what does that serve you if your system crawls?


Funny, I always had the exact opposite impression (BSD = bad SMP).

Reply Score: 2

RE[5]: Pointless comparisson
by maaxx on Tue 29th Sep 2009 01:43 UTC in reply to "RE[4]: Pointless comparisson"
maaxx Member since:
2007-11-06

"I have always had the impression that FreeBSD scales horizontally (as in SMP) better than Linux, but frankly what does that serve you if your system crawls?


Funny, I always had the exact opposite impression (BSD = bad SMP).
"

Not quite. Since 7.x it has got significantly better. I can only see SMP even more improved in 8.0.

Reply Score: 1

RE[5]: Pointless comparisson
by strcpy on Tue 29th Sep 2009 04:01 UTC in reply to "RE[4]: Pointless comparisson"
strcpy Member since:
2009-05-20

Funny, I always had the exact opposite impression (BSD = bad SMP).


Funny, I always had the exact opposite impression (Linux = BAD).

Not only was the comparison pointless, but our comments appear to be pointless too.

Reply Score: 1

RE[6]: Pointless comparisson
by segedunum on Tue 29th Sep 2009 09:57 UTC in reply to "RE[5]: Pointless comparisson"
segedunum Member since:
2005-07-06

Funny, I always had the exact opposite impression (Linux = BAD). Not only was the comparison pointless....

Your comment was pointless because you added nothing whatsoever to the discussion. The OP was talking and expressing an impression of SMP and then you steamed in for some reason to tell us that Linux was bad.

If you have nothing to add to the thread of discussion........keep your mouth shut.

Reply Score: 4

RE[7]: Pointless comparisson
by Melicerte on Tue 29th Sep 2009 10:46 UTC in reply to "RE[6]: Pointless comparisson"
Melicerte Member since:
2006-08-29

"Funny, I always had the exact opposite impression (Linux = BAD). Not only was the comparison pointless....

Your comment was pointless because you added nothing whatsoever to the discussion. [...]
"
Well, I disagree with you. I also have the feeling the the comparison is pointless actually. And that's IMO, adding something to the discussion.

Reply Score: 1

RE[5]: Pointless comparisson
by segedunum on Tue 29th Sep 2009 10:09 UTC in reply to "RE[4]: Pointless comparisson"
segedunum Member since:
2005-07-06

Funny, I always had the exact opposite impression (BSD = bad SMP).

Historically yes, until Linux's not-so-fair scheduler came in. Things have probably improved since then (last year), but FreeBSD's SMP system is a lot better than it was. With all these multi-core machines around it has to be these days.

Reply Score: 2

RE[3]: Pointless comparisson
by tony on Mon 28th Sep 2009 21:58 UTC in reply to "RE[2]: Pointless comparisson"
tony Member since:
2005-07-06

Let me explain the difference.

System A runs fast. 80 miles/hour. Its a porshe.
System B runs slower. 50 miles/hour. Its a truck.

Thing is, add more users, System A quickly slows to a crawl.

System B meanwhile doesn't slow down by much. Carrying 50 users it still maintains 49 miles/hour.

Speed itself doesn't complete the picture. Scalability adds the additional dimension.


I don't know if that actually works as a comparison. We're talking about systems with the same hardware, running different tasks. It'd be like how fast could a single 2-seater Porsche get a person from point at to point B, versus how fast a 2-seater Porsche could get a family of 5 to soccer practice, pick up the groceries, etc. How well you handle a large number of tasks with limited resources (disk, IO, memory, CPUs), etc.

Reply Score: 2

RE[4]: Pointless comparisson
by rirmak on Wed 30th Sep 2009 12:27 UTC in reply to "RE[3]: Pointless comparisson"
rirmak Member since:
2009-06-23

The vehicle analogy referred to the two operating systems, not to the hardware they used. That's what the article was about, remember? ;)

Reply Score: 1

RE[3]: Pointless comparisson
by Kebabbert on Sat 3rd Oct 2009 18:45 UTC in reply to "RE[2]: Pointless comparisson"
Kebabbert Member since:
2007-07-27

There are some indications that Linux doesnt really utilize many CPUs that good, as for instance Solaris does. But Linux may be faster att small loads, with few threads. But when trying to use many CPUs on a large machine, Linux will have problems spreading the load. Whereas Solaris has no such problems. Solaris has run on 106 CPU machines S15k for long. I dont know about FreeBSD scalability though.

In short, I think scalability counts. Linux may win over Solaris on small loads, but I believe Linux will have problems on large machines. (Caveat, doing other things than a highly specialized task such as number crunching which Linux does now on large clusters. Such Linux clusters never handle many users, only number crunch with a tailored Linux kernel)

Reply Score: 2

RE[2]: Pointless comparisson
by sbenitezb on Tue 29th Sep 2009 04:02 UTC in reply to "RE: Pointless comparisson"
sbenitezb Member since:
2005-07-22

There's a world of difference when an OS is heavily optimized for speed and only gets a marginal speed gain (as shown by the benchmark) leaving predictability and stability aside. FreeBSD may not be so fast in paper, but I bet mere milliseconds don't really matter at all if your data is at risk or your server stops responding when you load it.

Reply Score: 2

RE[3]: Pointless comparisson
by segedunum on Tue 29th Sep 2009 11:06 UTC in reply to "RE[2]: Pointless comparisson"
segedunum Member since:
2005-07-06

...I bet mere milliseconds don't really matter at all if your data is at risk or your server stops responding when you load it.

If you have reason to believe that a FreeBSD system would prevent all that better than anything else, as well as some benchmarks and some way of actually measuring that then feel free to share them. You'd make quite a bit of money. Otherwise, it's all shotting in the dark.

When a set of benchmarks come out telling you that one system is slower than another in some way you generally get some smart Alec who likes to point out that the apparently slower system is 'optimised' for 'something else'. That something else is generally always completely immeasurable.

Reply Score: 2

RE[4]: Pointless comparisson
by strcpy on Tue 29th Sep 2009 18:07 UTC in reply to "RE[3]: Pointless comparisson"
strcpy Member since:
2009-05-20

When a set of benchmarks come out telling you that one system is slower than another in some way you generally get some smart Alec who likes to point out that the apparently slower system is 'optimised' for 'something else'. That something else is generally always completely immeasurable.


And then there is always that smart Linux troll.

Reply Score: 2

running FreeBSD8-RC1
by Nagilum on Mon 28th Sep 2009 19:56 UTC
Nagilum
Member since:
2009-07-01

from /usr/src/UPDATING:
NOTE TO PEOPLE WHO THINK THAT FreeBSD 8.x IS SLOW:
FreeBSD 8.x has many debugging features turned on, in
both the kernel and userland. These features attempt to detect
incorrect use of system primitives, and encourage loud failure
through extra sanity checking and fail stop semantics. They
also substantially impact system performance. If you want to
do performance measurement, benchmarking, and optimization,
you'll want to turn them off. This includes various WITNESS-
related kernel options, INVARIANTS, malloc debugging flags
in userland, and various verbose features in the kernel. Many
developers choose to disable these features on build machines
to maximize performance. (To disable malloc debugging, run
ln -s aj /etc/malloc.conf.)

Unfortunately Phoronix neglected to publish the kernelconfig they used and also how you compile your software is up to you. So if you like to use a newer gcc on FreeBSD for your ports you are free to do so..

Edited 2009-09-28 20:01 UTC

Reply Score: 9

RE: running FreeBSD8-RC1
by kenji on Mon 28th Sep 2009 20:29 UTC in reply to "running FreeBSD8-RC1"
kenji Member since:
2009-04-08

+1

That was what I was thinking, but you beat me to it.

FreeBSD is so endlessly tweakable that certainly better performance could be achieved.

Reply Score: 1

Comment by SJ87
by sj87 on Tue 29th Sep 2009 04:29 UTC
sj87
Member since:
2007-12-16

Testing unstable releases is pointless anyways IMO. Never understood why the guys at Phoronix just can't wait that extra couple of weeks to do a proper performance test.

Reply Score: 2

RE: Comment by SJ87
by Doc Pain on Tue 29th Sep 2009 18:08 UTC in reply to "Comment by SJ87"
Doc Pain Member since:
2006-10-08

Testing unstable releases is pointless anyways IMO.


Page 1 of the article mentiones: "With FreeBSD 8.0 we were using the AMD64 DVD of the first release candidate using a stock installation."

So what FreeBSD are they using? FreeBSD/AMD64 8.0 RC1. It's the first release candidate. It still contains many debugging information because it's not considered a "ready" release yet.

Furthermore, they are using a "stock installation" which implies that they are using the default GENERIC kernel, as far as it seems. So possible optimization and customization did not take place. You can achieve real improvements by setting your system configuration.

Reply Score: 2

RE[2]: Comment by SJ87
by strcpy on Tue 29th Sep 2009 18:20 UTC in reply to "RE: Comment by SJ87"
strcpy Member since:
2009-05-20

Yes you can.

Personally I have deep dissatisfaction with the FreeBSD way of doing certain things, namely buttons here and there; like switching one or two would magically improve anything by a factor.

Ferrari is a Ferrari. You can not make a sport-car by tuning, so to speak.

The other point: while I do not disagree that these "debugging options" would not be useful and a good thing from software engineering perspective, maybe it would be time to turn them off already in the RC phase. I don't know, maybe turn these on prior to the release candidates, but it is nevertheless boring to read the same comment about "massive debugging options" over and over again.

But I won't disagree that those benchmarks that this site keeps pushing are pointless, and done by clueless people. Just like the catalogue-like charts for Windows performance that every consumer hardware review just has to publish.

Reply Score: 2

RE[3]: Comment by SJ87
by Doc Pain on Tue 29th Sep 2009 18:40 UTC in reply to "RE[2]: Comment by SJ87"
Doc Pain Member since:
2006-10-08

Personally I have deep dissatisfaction with the FreeBSD way of doing certain things, namely buttons here and there; like switching one or two would magically improve anything by a factor.


Not magically, not by a factor (or order of magnitute); but fact is that there are still many things enabled in the RC that are for debugging use primarily, so the end user does not benefit from them, and they slow down certain things a bit.

Ferrari is a Ferrari. You can not make a sport-car by tuning, so to speak.


I've seen racing tractors recently. :-)

The other point: while I do not disagree that these "debugging options" would not be useful and a good thing from software engineering perspective, maybe it would be time to turn them off already in the RC phase. I don't know, maybe turn these on prior to the release candidates, [...]


Well, that could be an option, but keep im mind what a RC is: a release candidate, nothing more, nothing less. It's still in testing, while most of the features are already in a "fixed state". If you're seeking for the "bleeding edge" of FreeBSD, you would use -CURRENT; in this development branch, things can even be inserted experimentally, and removed from the system if they aren't good. A RC is of much higher quality, but it's not quite ready.


[...] but it is nevertheless boring to read the same comment about "massive debugging options" over and over again.


Not massive, but still existent.

Furthermore, the history of FreeBSD - at least in my own individual way of view, as shown that there are further improvements to be expected after 8.0-RELEASE, for example in 8.1.

Reply Score: 2

A more interesting comparison
by fridder on Tue 29th Sep 2009 14:32 UTC
fridder
Member since:
2007-11-03

Indeed these tests are a bit, well, pointless however the so called "big operating system comparison" could yield some fascinating results. One thing that would be very interesting, that I doubt Phoronix will test, is a performance comparison between FreeBSD's ZFS and Opensolaris's ZFS. Can't wait till 8.0-RELEASE to test that myself.

Reply Score: 1