Linked by Thom Holwerda on Wed 17th Nov 2010 23:10 UTC, submitted by Debjit
Internet Explorer There's a bit of a ruckus on the web about how Microsoft was supposedly cheating when it comes to Internet Explorer 9's performance on benchmarks. Digitizor, as well as some enterprising readers over at HackerNews, came to the conclusion that Microsoft included code in IE9 specifically to ace the SunSpider benchmark. I was ready to write a scathing article about this, - until I loaded up the IEBlog. As it turns, it's not cheating, it's not a bug - it's an actual piece of smart code optimisation other browsers don't have yet.
Order by: Score:
its Microsoft
by Nex6 on Wed 17th Nov 2010 23:20 UTC
Nex6
Member since:
2005-07-06

naaaahhh, its Microsoft they must have been cheating. it can not be any other way... /end_sarcasm

Reply Score: 8

RE: its Microsoft
by Kroc on Wed 17th Nov 2010 23:34 UTC in reply to "its Microsoft"
Kroc Member since:
2005-11-10

Being closed source and bugs being hidden behind the most hideous of login walls doesn’t help with transparency when it comes to this problem. They did the right thing in explaining clearly on the blog though.

Reply Score: 3

RE: its Microsoft
by kwanbis on Thu 18th Nov 2010 00:05 UTC in reply to "its Microsoft"
kwanbis Member since:
2005-07-06

Incredible how people are!

As if Microsoft was ever convicted for bad businesses practices, for breaking monopoly laws, or for screwing with "parteners".

</sarcasm>

Edited 2010-11-18 00:08 UTC

Reply Score: 3

RE[2]: its Microsoft
by Hiev on Thu 18th Nov 2010 00:07 UTC in reply to "RE: its Microsoft"
Hiev Member since:
2005-09-27

Or for humans righs violations ... oh ,sorry that was Nokia.

http://www.eff.org/deeplinks/2010/10/saharkhiz-v-nokia

Reply Score: 2

RE[2]: its Microsoft
by modmans2ndcoming on Thu 18th Nov 2010 01:17 UTC in reply to "RE: its Microsoft"
modmans2ndcoming Member since:
2005-11-09

as if they weren't punished for such crimes and have changed such practices in the 15 years since committing them.

[/sarcasm]

Reply Score: 2

RE[3]: its Microsoft
by kwanbis on Thu 18th Nov 2010 05:17 UTC in reply to "RE[2]: its Microsoft"
kwanbis Member since:
2005-07-06

So people should forget about 20 years of wrongdoing and just be happy ... Yeah, as if you can tell an old dog new tricks ...

Reply Score: 0

RE: its Microsoft
by Valhalla on Thu 18th Nov 2010 02:03 UTC in reply to "its Microsoft"
Valhalla Member since:
2006-01-24

Well it's certainly not hard to understand how people could get that notion if they had ever written code.

Adding either of these two, a 'true' and a 'return' which both does nothing in the context of this code and thus would certainly be optimized away as dead code suddenly prevents this dead code elimination optimization that before either of these (again, doing nothing) changes was able to optimize away the entire function.

http://people.mozilla.com/~sayrer/2010/sunspider/diff1.html
http://people.mozilla.com/~sayrer/2010/sunspider/diff2.html

But even so I don't think they are cheating, I believe this is due to the 'frailty' of their dead code eliminator, which as proven by this needs more work.

I've seen alot weirder things than this in my days as a programmer and I can't see why the IE devs would cheat this way if they were interested in cheating (which I don't think they are, I may have issues with Microsoft as a company but I have nothing but respect for their developers and no self-respecting dev would do something like hardcoding a dead code elimination, not even with Ballmer breathing down their necks).

Reply Score: 3

...
by Hiev on Wed 17th Nov 2010 23:34 UTC
Hiev
Member since:
2005-09-27

Good news, I'm glad most common browsers have now a very fast javascript engine, it is time to eat all those javascript books getting dust on the shelf.

Reply Score: 2

RE: ... (lots of OT)
by reez on Wed 17th Nov 2010 23:48 UTC in reply to "..."
reez Member since:
2006-06-28

Yeah and also many not so common browsers use these engines.

Recently found Conkeror, which looks damn nice to me. Efficient (in the vi(m) way), fast, lightweight _very_ (easily) extensible and compatible with Firefox extensions. Oh and it's a xulrunner application, which means it is easy to hack, since you start it isn't used as a precompiled application.

Yeah, I know there is vimperator, but it's different. Pretty nice concept IMO. Oh and I tried an web application (mix of HTML, JS an plugins), which didn't work on any of the big browsers -didn't try IE though - and it worked.

Still using Firefox, since I currently don't have the time to readjust.

Sorry for the off topic. Just wanted to mention it, because it's a nice alternative browser which I didn't know and is different from other alternative browsers, like Arora or Midori.

Reply Score: 2

RE[2]: ... (lots of OT)
by TheGZeus on Thu 18th Nov 2010 15:56 UTC in reply to "RE: ... (lots of OT)"
TheGZeus Member since:
2010-05-19

Conk is awesome. I've been using it for a couple years now.
Interestingly enough, vimperator is a fork of Conkeror, from back when Conkeror was an FF extension. Probably little to no code in common any more, though they have gotten ideas from each other.
The biggest difference is, like the vim/emacs difference, one uses a config format, the other uses a full turing-complete language.

Edited 2010-11-18 16:02 UTC

Reply Score: 2

It isn't cheating, but
by lemur2 on Wed 17th Nov 2010 23:43 UTC
lemur2
Member since:
2007-02-17

It clearly isn't cheating, but it does mean that the Sunspider benchmark isn't terribly useful for trying to assess how fast IE9 is going to be for real-world javascript application, which can't be simply optimised out (not run).

This is a particularly interesting point to note considering that, very shortly after Fireox 4.0b7 has been released containing the new Jaegermonkey engine, Microsoft have just rushed out a new beta of IE9.

http://arstechnica.com/microsoft/news/2010/11/internet-explorer-9-p...

Just three weeks after the release of Platform Preview 6, Microsoft has released a seventh preview of Internet Explorer 9. The highlight of this new release is not new standards compliance but rather performance.

Improved performance has been a key goal of Internet Explorer 9's development. The new browser version relies heavily on GPU acceleration to provide high-speed graphics rendering and animation, and it includes a new JavaScript engine, codenamed Chakra, that compiles JavaScript in the background to achieve JavaScript performance that's tens or hundreds of times faster than Internet Explorer 8.

Throughout the development process, Microsoft has emphasized that the focus is on real-world website performance, not on popular but narrow benchmarks like the SunSpider JavaScript benchmark. Internet Explorer 9's SunSpider performance has improved over the course of its development, but the company says that this has been a side-effect of its real-world focused improvements rather than a deliberate objective.


OK then, we can all agree with that. So what do Microsoft do next? They proceed use the flawed Sunspider benchmark, and ONLY the Sunspider benchmark, to compare their new preview of IE9 against other browsers.

So where is the compariosn using Google's v8bench benchmark? More interestingly, where is the comparison using Mozilla's Kraken benchmark, which is the one benchmark amongst them all that actually claims that "the focus is on real-world website performance, not on popular but narrow benchmarks like the SunSpider JavaScript benchmark"?

http://blog.mozilla.com/blog/2010/09/14/release-the-kraken-2/
More than Sunspider, V8, and Dromaeo, Kraken focuses on realistic workloads and forward-looking applications. We believe that the benchmarks used in Kraken are better in terms of reflecting realistic workloads for pushing the edge of browser performance forward. These are the things that people are saying are too slow to do with open Web technologies today, and we want to have benchmarks that reflect progress against making these near-future apps universally available.


So, Microsoft, we are wondering ... how does IE9 perform in comparison to other browsers when measured on a benchmark that actually tries to assess "real-world website performance"? Since there IS a benchmark which actually claims measuring this performance as its primary aim, and since Microsoft say that that is what they are aiming at, then why didn't they use the Kraken benchmark?

Just saying.

Edited 2010-11-17 23:48 UTC

Reply Score: 7

RE: It isn't cheating, but
by siimo on Wed 17th Nov 2010 23:53 UTC in reply to "It isn't cheating, but"
siimo Member since:
2006-06-22

Well Kraken is actually quite new and SunSpider has been recognised as "the" benchmark to use by everyone for a while now. So I wouldn't go blaming Microsoft because they are using the most popular benchmark.

There is nothing stopping you or any other person to run these tests in other benchmarking suites and post a review, of course.

Reply Score: 4

RE[2]: It isn't cheating, but
by lemur2 on Thu 18th Nov 2010 00:20 UTC in reply to "RE: It isn't cheating, but"
lemur2 Member since:
2007-02-17

Well Kraken is actually quite new and SunSpider has been recognised as "the" benchmark to use by everyone for a while now. So I wouldn't go blaming Microsoft because they are using the most popular benchmark. There is nothing stopping you or any other person to run these tests in other benchmarking suites and post a review, of course.


Why not? Why shouldn't I criticise Microsoft, using Microsoft's own words?

http://blogs.msdn.com/b/ie/archive/2010/09/14/performance-what-comm...
WebKit SunSpider
WebKit SunSpider is one of the most commonly cited benchmarks and has become a focus for the industry. Given that SunSpider focuses on JavaScript performance and not real world scenarios that matter to customers this is unfortunate. SunSpider is developed by the Apple WebKit team with the goal of measuring real world JavaScript performance. There are 26 test cases which attempt to measure the performance of individual JavaScript features including arrays, math, and regular expressions. The WebKit SunSpider tests exercise less than 10% of the API’s available from JavaScript and many of the tests loop through the same code thousands of times. This approach is not representative of real world scenarios and favors some JavaScript engine architectures over others.


From the horse's mouth, so to speak.

As you say, Kraken is fairly new, but here is what Microsoft had to say (before Kraken was available) about its predecessor, Dromaeo.
Mozilla Dromaeo
Mozilla’s Dromaeo is one of the best engineered JavaScript and DOM benchmarks available today.


What can I say, other than question why Microsoft are publishing results ONLY from Sunspider, when they are fully aware that it is nonsense to do so? Especially when you consider that in the exact same breath, Microsoft had just said that real world performance was what actually mattered?

Just saying.

Edited 2010-11-18 00:23 UTC

Reply Score: 2

RE[3]: It isn't cheating, but
by WorknMan on Thu 18th Nov 2010 00:31 UTC in reply to "RE[2]: It isn't cheating, but"
WorknMan Member since:
2005-11-13

What can I say, other than question why Microsoft are publishing results ONLY from Sunspider, when they are fully aware that it is nonsense to do so? Especially when you consider that in the exact same breath, Microsoft had just said that real world performance was what actually mattered?


LOL, reminds me of the old days when CPU manufacturers swore up and down that megahertz didn't matter, until/unless they had the chip with the highest number of mhz, at which time it became a marketing bullet point.

Reply Score: 3

RE[3]: It isn't cheating, but
by tomcat on Thu 18th Nov 2010 21:21 UTC in reply to "RE[2]: It isn't cheating, but"
tomcat Member since:
2006-01-06

Look, you're simply a BIGOTED HYPOCRITE. When it served your purposes to use SunSpider to bash Microsoft, it was the greatest thing since sliced bread. Now that Microsoft actually did work to optimize its code to improve its performance on SunSpider, suddenly the test isn't useful to you anymore as a propaganda tool. LOL!

Reply Score: 0

RE[4]: It isn't cheating, but
by lemur2 on Thu 18th Nov 2010 21:44 UTC in reply to "RE[3]: It isn't cheating, but"
lemur2 Member since:
2007-02-17

Look, you're simply a BIGOTED HYPOCRITE. When it served your purposes to use SunSpider to bash Microsoft, it was the greatest thing since sliced bread.


Quote please.

Now that Microsoft actually did work to optimize its code to improve its performance on SunSpider, suddenly the test isn't useful to you anymore as a propaganda tool. LOL!


It wasn't me who bashed the usefulness of Sunspider. I have had zero comment on it, I haven't even looked at the code. This very sub-thread, which I started, even has the words "It isn't cheating" in the title.

It was Microsoft engineers who bashed Sunspider. Here once again is what Microsoft had to say about it:
http://blogs.msdn.com/b/ie/archive/2010/09/14/performance-what-comm...
WebKit SunSpider
WebKit SunSpider is one of the most commonly cited benchmarks and has become a focus for the industry. Given that SunSpider focuses on JavaScript performance and not real world scenarios that matter to customers this is unfortunate. SunSpider is developed by the Apple WebKit team with the goal of measuring real world JavaScript performance. There are 26 test cases which attempt to measure the performance of individual JavaScript features including arrays, math, and regular expressions. The WebKit SunSpider tests exercise less than 10% of the API’s available from JavaScript and many of the tests loop through the same code thousands of times. This approach is not representative of real world scenarios and favors some JavaScript engine architectures over others.


Let's stick to the facts, please. We wouldn't want anyone to get all hysterical and start sprouting unwarrented insults that are clearly wrong, or libelous even.

Oh, wait ... too late.

Edited 2010-11-18 21:47 UTC

Reply Score: 2

v RE[5]: It isn't cheating, but
by Hiev on Fri 19th Nov 2010 00:57 UTC in reply to "RE[4]: It isn't cheating, but"
RE[6]: It isn't cheating, but
by lemur2 on Fri 19th Nov 2010 01:42 UTC in reply to "RE[5]: It isn't cheating, but"
lemur2 Member since:
2007-02-17

It was MS? Aren't you saying in this topic SunSpider is flawed? sure MS has bashed sunspider, and so do you. You are so pathetic. Oh and here is the quote you pathological hypocrite: They proceed use the flawed Sunspider benchmark


My my, testy, aren't we? What an angry little critter.

I called it flawed because Microsft themselves did. I am quoting Microsoft.

I haven't evaluated javascript benchmarks, but I have read a Microsoft analysis of them. Here it is, so that you too may read it:
http://blogs.msdn.com/b/ie/archive/2010/09/14/performance-what-comm...

Quote from Microsoft regarding Sunspider flaws: "This approach is not representative of real world scenarios and favors some JavaScript engine architectures over others."

Edited 2010-11-19 01:48 UTC

Reply Score: 2

RE[7]: It isn't cheating, but
by Hiev on Fri 19th Nov 2010 02:15 UTC in reply to "RE[6]: It isn't cheating, but"
Hiev Member since:
2005-09-27

Haha, try to fix it now.
To late.

Reply Score: 1

RE[4]: It isn't cheating, but
by Junius on Fri 19th Nov 2010 14:06 UTC in reply to "RE[3]: It isn't cheating, but"
Junius Member since:
2009-10-25

This is OT so I'll keep it short 'n' sweet. I've been a long time lurker on osnews and never have I seen someone presume they have the right to talk to another human being like that.

Yes you have a difference of opinion, yes you have some good points - I presume (I don't care to read anything else you write). But with that attitude you just lost the argument.

Please for the sake of a fantastic tech community and news resource, try to treat people with respect, or if that's beyond you. Just go away.

...

"LOL!"

Reply Score: 1

RE[5]: It isn't cheating, but
by tomcat on Sat 20th Nov 2010 01:39 UTC in reply to "RE[4]: It isn't cheating, but"
tomcat Member since:
2006-01-06

Thanks for the comment, lemur3, or lemur4, or lemur5, or whoever you are. I could care less.

Reply Score: 2

RE[5]: It isn't cheating, but
by lemur2 on Sun 21st Nov 2010 09:47 UTC in reply to "RE[4]: It isn't cheating, but"
lemur2 Member since:
2007-02-17

This is OT so I'll keep it short 'n' sweet. I've been a long time lurker on osnews and never have I seen someone presume they have the right to talk to another human being like that.

Yes you have a difference of opinion, yes you have some good points - I presume (I don't care to read anything else you write). But with that attitude you just lost the argument.


tomcat cheers for Microsoft. When it happens that he goes off the rails like that, one can be sure that one has made a significant and convincing point that tomcat cannot debate with civility. Ad hominem attacks (a.k.a attack the messenger, not the message) in response are his only recourse.

Indeed one doesn't have to read tomcats points (if indeed one can find any) to know that when he posts in that way, he has already lost the argument.

I would like to post a note of appreciation for your words, and also to let you know that you are not really OT in posting them. Some people are swayed by strong language and vitriol, thinking that there must be some reason behind such anger. There is, but the reason is simply an attempt to discredit that other party in the debate, it has nothing to do with the validity or otherwise of the actual points made in the debate. Anyway, it can't but help to point these things out, so that perhaps people won't be swayed by them.

Reply Score: 2

RE[6]: It isn't cheating, but
by tomcat on Mon 22nd Nov 2010 18:55 UTC in reply to "RE[5]: It isn't cheating, but"
tomcat Member since:
2006-01-06

lemur2 does nothing but criticize Microsoft.


Fixed it for you.

Reply Score: 2

RE: It isn't cheating, but
by Hiev on Thu 18th Nov 2010 00:01 UTC in reply to "It isn't cheating, but"
Hiev Member since:
2005-09-27

im glad you ask, I just made a benchmark with kraken of IE9 P7 vs Chrome and the winner was IE9 PV7.

Edited 2010-11-18 00:02 UTC

Reply Score: 2

RE: It isn't cheating, but
by galvanash on Thu 18th Nov 2010 00:22 UTC in reply to "It isn't cheating, but"
galvanash Member since:
2006-01-25

It clearly isn't cheating, but it does mean that the Sunspider benchmark isn't terribly useful for trying to assess how fast IE9 is going to be for real-world javascript application, which can't be simply optimised out (not run).


I completely disagree. Have you seen much real world JavaScript code??? I think Microsoft is just scratching the surface with dead code removal. Look out for the upcoming announcement of stupid code removal, which will revolutionize the internet by actively avoiding execution of about the 80% or so of JavaScript code which does nothing useful anyway.

Reply Score: 6

RE[2]: It isn't cheating, but
by lemur2 on Thu 18th Nov 2010 00:29 UTC in reply to "RE: It isn't cheating, but"
lemur2 Member since:
2007-02-17

"It clearly isn't cheating, but it does mean that the Sunspider benchmark isn't terribly useful for trying to assess how fast IE9 is going to be for real-world javascript application, which can't be simply optimised out (not run).
I completely disagree. Have you seen much real world JavaScript code??? I think Microsoft is just scratching the surface with dead code removal. Look out for the upcoming announcement of stupid code removal, which will revolutionize the internet by actively avoiding execution of about the 80% or so of JavaScript code which does nothing useful anyway. "


You might well disagree with me, but Microsoft's own engineers happen to agree with me. See for yourself:

http://blogs.msdn.com/b/ie/archive/2010/09/14/performance-what-comm...
http://www.osnews.com/permalink?450394

It is only the Microsoft PR people (who presumably are desparate to spread false impressions) who would put up results from Sunspider ONLY, after having just said in the very sentence before that what really mattered was real-world performance.

It is pretty funny, really.

Just saying.

Edited 2010-11-18 00:30 UTC

Reply Score: 1

RE[3]: It isn't cheating, but
by galvanash on Thu 18th Nov 2010 01:07 UTC in reply to "RE[2]: It isn't cheating, but"
galvanash Member since:
2006-01-25

My post was a joke... Sorry if that wasn't obvious.

That aside, if you want a serious response. I don't care what Microsoft's PR people say, just as I don't generally care what any PR people say - their job is to present a version of the truth that suits their employers desired outcome.

Just to push the point, benchmarking a product in order to give developers feedback to improve it is a useful exercise. The minute you (as in the software developer) publish the benchmark you are in PR territory and caveat emptor to the reader.

I'm not saying that all PR in the software world is veiled deception, but it sure as hell can be, and it doesn't matter who the company is. There isn't much use in pointing fingers at them when they twist the truth, you may as well blame a skunk for stinking...

And no, I don't think it is particularly funny or surprising that some actual developers at MS don't agree with how PR characterizes their work - that is probably true of any software company. You may not like MS, but they employ quite a lot of intelligent developers, and most intelligent people who work hard at what they do don't like their work being distilled down to sound bites and promoted like it was a soft drink - it cheapens what is otherwise a satisfying experience (i.e. trying to write better software). So I would expect any good software company to have a few developers that don't tow the PR party line - it is a sign of health if you ask me.

Its the companies that manage to present a single unified message to the outside world without a single voice of dissent in sight which you should be frightened of...

Reply Score: 5

RE[4]: It isn't cheating, but
by lemur2 on Thu 18th Nov 2010 01:46 UTC in reply to "RE[3]: It isn't cheating, but"
lemur2 Member since:
2007-02-17

My post was a joke... Sorry if that wasn't obvious. That aside, if you want a serious response. I don't care what Microsoft's PR people say, just as I don't generally care what any PR people say - their job is to present a version of the truth that suits their employers desired outcome. Just to push the point, benchmarking a product in order to give developers feedback to improve it is a useful exercise. The minute you (as in the software developer) publish the benchmark you are in PR territory and caveat emptor to the reader. I'm not saying that all PR in the software world is veiled deception, but it sure as hell can be, and it doesn't matter who the company is. There isn't much use in pointing fingers at them when they twist the truth, you may as well blame a skunk for stinking... And no, I don't think it is particularly funny or surprising that some actual developers at MS don't agree with how PR characterizes their work - that is probably true of any software company. You may not like MS, but they employ quite a lot of intelligent developers, and most intelligent people who work hard at what they do don't like their work being distilled down to sound bites and promoted like it was a soft drink - it cheapens what is otherwise a satisfying experience (i.e. trying to write better software). So I would expect any good software company to have a few developers that don't tow the PR party line - it is a sign of health if you ask me. Its the companies that manage to present a single unified message to the outside world without a single voice of dissent in sight which you should be frightened of...


Sorry, you are right, I didn't pick up on the joke in your post. I read it too quickly. Pardon me.

Fair enough, yours is a pretty reasonable attitude actually. So too is the apparent attitude of the Microsoft developers.

One small part of my complaint is that this is not what ordinary people get to hear.

A larger part of my complaint is that websites such as Ars Technica should know better, and they should present material like this (performance comparisons) with a bit of technical balance to it, and do some investigations of their own, and not simply regurgitate what Microsoft marketing PR has to say. We all know how reliable Microsoft PR and marketing is liable to be ... which is to say, not one bit reliable.

The most significant part of what I would say about all this is for people to take this as a lesson learned. There are a few useful "rules to live by" that can be gleaned from this:
(1) Do not believe what mainstream media is trying to tell you without examining it at least a little bit for yourself. "Follow the money" is a good rule of thumb to use to sort out what is really going on.
(2) Look at whatever Microsoft marketing say, and then examine what they do NOT say about a topic. The latter examination is most likely to uncover some actual truth.
(3) If someone makes a counter-claim about a talking point that Microsoft marketing are pushing, do not discount out of hand what the naysayers have to say. Microsoft marketing and PR most definitely has an agenda. "Follow the money" once again is a good rule of thumb.

Reply Score: 2

RE: It isn't cheating, but
by google_ninja on Fri 19th Nov 2010 19:22 UTC in reply to "It isn't cheating, but"
google_ninja Member since:
2006-02-05

The thing is, if a javascript engine is in the realm of V8, it is way more then fast enough. If it is 0.8x as fast, or 2x as fast is actually pretty irrelevant, because it will be more then fast enough for anything that a webpage will throw at it.

The major issue nowadays is dom manipulation (and style recalculation) speeds. Sun Spider numbers are for browser developers ego more then anything else when you are talking about modern browsers. Sun Spider like tests still make sense for the server side implementations though.

Reply Score: 2

Of Course.
by Tuishimi on Thu 18th Nov 2010 00:00 UTC
Tuishimi
Member since:
2005-07-06

Reminds me of a story back in my days at DEC.

Another software engineer came over to my office all upset... I asked why and he said "I've written the same exact program in 4 languages, compiled it and ran it..."

"Yes, so?"

"The results make NO SENSE!"

So I went over to watch and he ran the test (just a simple loop/timer test). He wrote a program in C, BASIC, FORTRAN and COBOL. We made an assumption that since the compiler groups share technologies, that they should all pretty much compile down to the same machine code and the execution time should be similar. We expected BASIC and COBOL to take longer simply because they always load some extra memory management stuff ahead of the game (pre-allocate memory for string handling and such). But the results (and this is just shooting out of my butt because this was 20 years ago) were something like:

COBOL : 1.2 seconds
BASIC : 2.0 seconds
C : 0.5 seconds
FORTRAN: 0 seconds

He was upset because 0 was impossible. So he went into the machine code and found that the FORTRAN compiler was so good at optimizing looping and math code that it simply determined the code was unused and optimized it out. The program did nothing so the run time was zero.

It was pretty funny.

The point behind the story being large companies in the habit of producing compilers that other businesses depend on have dedicated engineers always finding ways to improve the performance. IBM, Intel, MS, DEC, etc. should be expected to implement optimizations like that.

Reply Score: 6

RE: Of Course.
by imaginant on Thu 18th Nov 2010 14:52 UTC in reply to "Of Course."
imaginant Member since:
2010-02-26

I'm not a coder, but am interested: How important or valuable is "dead code elimination" in day-to-day use. Apparently it is not included in other browsers. I would guess that it is not that much of a speed booster since I cannot imagine that Microsoft coders are that much smarter that all other coders (a practical statement, not a slur on MS coders). You can see how this would indicate Microsoft's intentions. Would someone help me understand, please?

Reply Score: 1

RE[2]: Of Course.
by Tuishimi on Thu 18th Nov 2010 15:48 UTC in reply to "RE: Of Course."
Tuishimi Member since:
2005-07-06

All I know is that it is or at least WAS a common practice back in the day of mainframes. Compilers were written to be highly optimizing which included being able to identify unused or non-functional (code that doesn't contribute to the data end result in any way) and not include it when translating to machine code.

This was a practice in place decades ago.

I do not see why it should indicate their intentions other than to add optimizations that could possibly speed things up. I liken it more to a developer sitting there one day and having light bulb go on in his head and saying "HEY! I've got an idea, why don't we do THIS!"

But since we cannot know this, it could very well be that they analyzed the tests the competition uses and realized they could speed it up that way...

Reply Score: 3

RE[2]: Of Course.
by Tuishimi on Thu 18th Nov 2010 15:52 UTC in reply to "RE: Of Course."
Tuishimi Member since:
2005-07-06

Also, I should note... the test code that was written in my above example was an increment algorithm.

The programs basically were:

var x = 0;

while (x < 1000000) do
x = x + 1
end

print x

...the FORTRAN compiler recognized the loop, actually performed the calculation and placed it in the code as:

print 1000000

That's a nice optimization when it saves tons of CPU cycles in a program. Another thing to note is that DEC wrote one of the best FORTRAN compilers in existence and it was used in scientific and government circles for large calculations.

Reply Score: 3

RE[2]: Of Course.
by gilboa on Fri 19th Nov 2010 12:58 UTC in reply to "RE: Of Course."
gilboa Member since:
2005-07-06

Given the fact that at least as far as I can see, this is strictly a compile time optimization (even if it being done by JIT), the compiler can't really notice that:
for (count=0;count<x;count++); y+=count
... is dead code when x=0 (resulting in y=y), as long as x isn't constant (read: calculated or given as input).

As such, dead code removal is more-or-less limited to bad-programmer-prevention - read: cleaning up after sloppy programmers.
I doubt that it has much effect in real life programming. (Though I'm a C programmer, so I can't really comment about JS programmers...)

- Gilboa

Reply Score: 2

RE[3]: Of Course.
by flynn on Fri 19th Nov 2010 16:42 UTC in reply to "RE[2]: Of Course."
flynn Member since:
2009-03-19

As such, dead code removal is more-or-less limited to bad-programmer-prevention - read: cleaning up after sloppy programmers.
I doubt that it has much effect in real life programming. (Though I'm a C programmer, so I can't really comment about JS programmers...)

Dead code can sometimes occur as a result of other optimizations. You could have every optimization that might produce dead code try to clean it up in some ad hoc way, but it's a lot easier and cleaner to simply make a global dead code elimination pass.

Here is a (somewhat contrived) example for constant propagation:

x = 5;
y = x + z;

The compiler can propagate the constant and rewrite y to be:

y = 5 + z;

at which point the assignment to x becomes dead.

Reply Score: 1

RE[4]: Of Course.
by gilboa on Fri 19th Nov 2010 21:51 UTC in reply to "RE[3]: Of Course."
gilboa Member since:
2005-07-06

True, but such optimization will usually remove a single line. While it is possible that such an optimization will remove a big chunk of code (E.g. for/while/if/etc), my gut tells me that it'll be very rare.

- Gilboa

Reply Score: 2

RE[5]: Of Course.
by flynn on Sat 20th Nov 2010 00:13 UTC in reply to "RE[4]: Of Course."
flynn Member since:
2009-03-19

True, but such optimization will usually remove a single line. While it is possible that such an optimization will remove a big chunk of code (E.g. for/while/if/etc), my gut tells me that it'll be very rare.

- Gilboa

Often dead code elimination results in a cascade of dead code, so some if statements might end up being eliminated. Loops will almost never be eliminated because doing so might change the behavior of the program.

while(true) { }

is pure dead code, but eliminating it will change the behavior of the program and make a non-terminating program terminate.

Reply Score: 1

RE[6]: Of Course.
by gilboa on Sat 20th Nov 2010 15:23 UTC in reply to "RE[5]: Of Course."
gilboa Member since:
2005-07-06

Actually while (true) must -not- be deleted by dead code removal.

At times, endless while look is actually quite useful. (E.g. waiting for debugger to connect).

- Gilboa

Reply Score: 2

RE[3]: Of Course.
by draburn on Mon 22nd Nov 2010 00:10 UTC in reply to "RE[2]: Of Course."
draburn Member since:
2010-03-05

Given the fact that at least as far as I can see, this is strictly a compile time optimization (even if it being done by JIT), the compiler can't really notice that:
for (count=0;count<x;count++); y+=count
... is dead code when x=0 (resulting in y=y), as long as x isn't constant (read: calculated or given as input).


Well, it can notice that,
- given x <= 1, the loop is dead code (noop)
- given x > 1, the whole loop is equivalent to: y += x*(x-1)/2.

Hence, a really good compiler should probably replace the loop by a proper
if (x>1) y += x*(x-1)/2

Edit: what I'm trying to prove here is that, when different compiler optimizations start to work together, the results can be amazing and really difficult to predict. Put it in other way, you don't need *bad* code nor a silly programmer to produce code that can be significantly optimized...

Edited 2010-11-22 00:14 UTC

Reply Score: 1

RE: Of Course.
by StaubSaugerNZ on Mon 22nd Nov 2010 09:27 UTC in reply to "Of Course."
StaubSaugerNZ Member since:
2007-07-13

Th sensible conclusion is that the benchmark's test makes no sense. It is too trivial to be representative of any significant proportion of real-world execution scenarios (which cannnot be eliminated to zero cycles).

Reply Score: 2

RE[2]: Of Course.
by Tuishimi on Mon 22nd Nov 2010 15:42 UTC in reply to "RE: Of Course."
Tuishimi Member since:
2005-07-06

I don't disagree.

Reply Score: 2

It's not actually dead code analysis
by quodlibetor on Thu 18th Nov 2010 00:26 UTC
quodlibetor
Member since:
2009-03-07

If you look at the first comment on the hacker news post that you link to--or various other places around the web--you see that people have actually looked at the idea that it's dead code analysis.

One of many things that have been done to test is just adding a "true;" statement into the function body that is being tested, (that's also a noop, basically the epitome of dead code) with that "dead code" inserted the Chakra Javascript engine does the full test. (Similar other meaningless changes to the code also result in Chakra running the full test.)

What this means is that if IE is actually doing dead code analysis, it's doing it incredibly narrowly. Which is what people have been saying since the beginning. It's still probably cheating, even though Microsoft decided to talk about it.

Reply Score: 4

galvanash Member since:
2006-01-25

To be fair, I would be careful of trying to characterize this as cheating. Since it is closed source, we can't see what the engine is actually doing.

I'm just saying it is entirely plausible that adding a true statement is pushing some internal threshold for this optimization over some edge. The point being that since JavaScript is a dynamic language dead code elimination is quite a bit more complex than with static languages and their implementation may just be initially very simple-minded so as not to cause issues during runtime code changes.

I do think it would be possible to determine with some thorough testing whether or not they are actually cheating here. If they are proven to be cheating shame on them. I just don't think that a few iterations of salting the code with some true or return statements is thorough.

Just saying, don't attribute to malice which can be easily explained by ignorance...

Reply Score: 3

Hiev Member since:
2005-09-27

I don't think is cheating, I ran the kraken bentchmark and It was faster than Chrome.

Edited 2010-11-18 01:28 UTC

Reply Score: 2

galvanash Member since:
2006-01-25

Again, to be fair, that may be true but it doesn't in any way indicate that they were not cheating, as dead code elimination generally only optimizes poorly thought out code - most of the time it shouldn't have any positive effect at all because the dead code usually indicates an error on the programmers part (properly written code generally doesn't contain any dead code paths).

Some thorough testing will determine whether they were cheating easily enough... Since they seem to be sticking by their guns I expect they will be vindicated after further review. If not, like I said shame on them.

Reply Score: 2

Hiev Member since:
2005-09-27

Is not cheating, is smart way of optimization. Bypassing useless code is a well known algoritm.

Edited 2010-11-18 01:58 UTC

Reply Score: 2

Delgarde Member since:
2008-08-19

Is not cheating, is smart way of optimization. Bypassing useless code is a well known algoritm.


Agreed. The question is whether the browser itself is optimizing out dead code - or whether it's been written to recognize the specific example of dead code in this particular benchmark.

Being closed-source, there's no way of knowing, but the claimed fragility in the face of minor changes to the test suggests cheating is possible. Of course, it just could be that the optimizer just isn't that good yet - it just happens to work well in this case because the benchmark doubles as a test-case for the engine...

Reply Score: 3

lemur2 Member since:
2007-02-17

I don't think is cheating, I ran the kraken bentchmark and It was faster than Chrome.


Are you taking a leaf out of Microsoft's book here, and only talking about results where IE9 came out ahead?

I could do the same if you like. Here you go then, here is a nice graph to chew on:
http://blog.mozilla.com/dmandelin/files/2010/11/kraken-1103.png

This is the kind of graph that you WON'T see (I'd wager) in the mainstream media.

Reply Score: 1

Hiev Member since:
2005-09-27

I trust the benthmark I can run, not the betchmarks a well known anti-ms troll like you put. And my bentchmar concurr with kraen and sunspider, IE9 P7 is faster than Chrome.

Edited 2010-11-18 02:02 UTC

Reply Score: 1

lemur2 Member since:
2007-02-17

I trust the benthmark I can run, not the betchmarks a well known anti-ms troll like you put. And my bentchmar concurr with kraen and sunspider, IE9 P7 is faster than Chrome.


Here is some help for running the Kraken benchmark for yourself:

http://krakenbenchmark.mozilla.com/

You have probably already found that link. OK, apart from IE9 preview 7, here is another link that you might need:

http://www.mozilla.com/en-US/firefox/all-beta.html

Let us know how it turns out.

PS: As far as trusting benchmarks one can run oneself, it transpires that IE9 P7 doesn't run on my machine, it doesn't install properly. In fact, the installer doesn't even get as far as the first "next" button. I am running Arch Linux, which runs Firefox, Chrome and Opera really nicely. Do you have any suggestions?

Edited 2010-11-18 02:37 UTC

Reply Score: 1

Hiev Member since:
2005-09-27

IE9 wins again.

Reply Score: 2

lemur2 Member since:
2007-02-17

IE9 wins again.


I doubt it. I doubt it very much indeed.

Reply Score: 2

Tuishimi Member since:
2005-07-06

Kraken on IE9 Platform Preview 1.9.8023.6000
=============================================
http://krakenbenchmark.mozilla.com/kraken-1.0/results.html?%7B~...


Kraken JavaScript Benchmark Results Content Version: kraken-1.0Run Again

(You can bookmark this results URL for later comparison.)
To compare to another run, paste a saved result URL in the text field below and press enter:

===============================================RESULTS (means and 95% confidence intervals)-----------------------------------------------Total: 14450.0ms +/- 0.4%----------------------------------------------- ai: 1079.2ms +/- 1.3% astar: 1079.2ms +/- 1.3% audio: 6502.2ms +/- 0.7% beat-detection: 1333.0ms +/- 0.5% dft: 2817.2ms +/- 1.2% fft: 1304.8ms +/- 0.9% oscillator: 1047.2ms +/- 0.7% imaging: 4843.6ms +/- 0.5% gaussian-blur: 2799.3ms +/- 0.4% darkroom: 1125.7ms +/- 0.8% desaturate: 918.6ms +/- 1.1% json: 226.4ms +/- 0.8% parse-financial: 100.8ms +/- 1.2% stringify-tinderbox: 125.6ms +/- 1.4% stanford: 1798.6ms +/- 0.3% crypto-aes: 353.1ms +/- 0.6% crypto-ccm: 263.5ms +/- 0.8% crypto-pbkdf2: 909.7ms +/- 0.4% crypto-sha256-iterative: 272.3ms +/- 0.7%

Reply Score: 2

Tuishimi Member since:
2005-07-06

Kraken on Chromium 9.0.579.65687
================================
http://krakenbenchmark.mozilla.com/kraken-1.0/results.html?%7B~...

===============================================
RESULTS (means and 95% confidence intervals)
-----------------------------------------------
Total: 13804.0ms +/- 0.7%
-----------------------------------------------

ai: 914.7ms +/- 0.6%
astar: 914.7ms +/- 0.6%

audio: 5035.8ms +/- 0.9%
beat-detection: 1320.0ms +/- 0.6%
dft: 1881.6ms +/- 1.7%
fft: 1245.5ms +/- 1.1%
oscillator: 588.7ms +/- 3.0%

imaging: 5769.0ms +/- 0.5%
gaussian-blur: 2381.8ms +/- 0.4%
darkroom: 1679.9ms +/- 1.1%
desaturate: 1707.3ms +/- 0.8%

json: 469.9ms +/- 0.8%
parse-financial: 216.8ms +/- 1.3%
stringify-tinderbox: 253.1ms +/- 1.1%

stanford: 1614.6ms +/- 1.3%
crypto-aes: 291.7ms +/- 4.7%
crypto-ccm: 244.1ms +/- 1.3%
crypto-pbkdf2: 799.3ms +/- 0.8%
crypto-sha256-iterative: 279.5ms +/- 1.1%

Reply Score: 2

Tuishimi Member since:
2005-07-06

The comparison of the two:
----------------------------

TEST COMPARISON FROM TO DETAILS

====================================================================== ==============

** TOTAL **: 1.047x as fast 14450.0ms +/- 0.4% 13804.0ms +/- 0.7% significant

====================================================================== ==============

ai: 1.180x as fast 1079.2ms +/- 1.3% 914.7ms +/- 0.6% significant
astar: 1.180x as fast 1079.2ms +/- 1.3% 914.7ms +/- 0.6% significant

audio: 1.29x as fast 6502.2ms +/- 0.7% 5035.8ms +/- 0.9% significant
beat-detection: 1.010x as fast 1333.0ms +/- 0.5% 1320.0ms +/- 0.6% significant
dft: 1.50x as fast 2817.2ms +/- 1.2% 1881.6ms +/- 1.7% significant
fft: 1.048x as fast 1304.8ms +/- 0.9% 1245.5ms +/- 1.1% significant
oscillator: 1.78x as fast 1047.2ms +/- 0.7% 588.7ms +/- 3.0% significant

imaging: *1.191x as slow* 4843.6ms +/- 0.5% 5769.0ms +/- 0.5% significant
gaussian-blur: 1.175x as fast 2799.3ms +/- 0.4% 2381.8ms +/- 0.4% significant
darkroom: *1.49x as slow* 1125.7ms +/- 0.8% 1679.9ms +/- 1.1% significant
desaturate: *1.86x as slow* 918.6ms +/- 1.1% 1707.3ms +/- 0.8% significant

json: *2.08x as slow* 226.4ms +/- 0.8% 469.9ms +/- 0.8% significant
parse-financial: *2.15x as slow* 100.8ms +/- 1.2% 216.8ms +/- 1.3% significant
stringify-tinderbox: *2.02x as slow* 125.6ms +/- 1.4% 253.1ms +/- 1.1% significant

stanford: 1.114x as fast 1798.6ms +/- 0.3% 1614.6ms +/- 1.3% significant
crypto-aes: 1.21x as fast 353.1ms +/- 0.6% 291.7ms +/- 4.7% significant
crypto-ccm: 1.079x as fast 263.5ms +/- 0.8% 244.1ms +/- 1.3% significant
crypto-pbkdf2: 1.138x as fast 909.7ms +/- 0.4% 799.3ms +/- 0.8% significant
crypto-sha256-iterative: *1.026x as slow* 272.3ms +/- 0.7% 279.5ms +/- 1.1% significant

Reply Score: 2

Tuishimi Member since:
2005-07-06

And HOLY MOLEY! Here is the Firefox 4 B7 results
=================================================
http://krakenbenchmark.mozilla.com/kraken-1.0/results.html?%7B~...


===============================================
RESULTS (means and 95% confidence intervals)
-----------------------------------------------
Total: 6969.8ms +/- 1.4%
-----------------------------------------------

ai: 1817.5ms +/- 4.0%
astar: 1817.5ms +/- 4.0%

audio: 2580.0ms +/- 0.8%
beat-detection: 716.6ms +/- 1.1%
dft: 552.3ms +/- 1.9%
fft: 572.4ms +/- 2.2%
oscillator: 738.7ms +/- 0.6%

imaging: 1534.9ms +/- 0.6%
gaussian-blur: 654.5ms +/- 0.3%
darkroom: 298.5ms +/- 0.7%
desaturate: 581.9ms +/- 1.3%

json: 222.5ms +/- 0.8%
parse-financial: 148.1ms +/- 1.1%
stringify-tinderbox: 74.4ms +/- 0.9%

stanford: 814.9ms +/- 1.7%
crypto-aes: 211.3ms +/- 0.8%
crypto-ccm: 140.9ms +/- 0.5%
crypto-pbkdf2: 335.6ms +/- 4.5%
crypto-sha256-iterative: 127.1ms +/- 1.8%

Reply Score: 2

galvanash Member since:
2006-01-25

Since the back and forth on this were bugging me...

Just ran these, and I don't much care what wins since I prefer Chrome mostly for reasons other than performance (I just like the UI).


13" Macbook Pro/Windows 7 Ultimate - 64-bit
2.4Ghz Core2 Duo/4GB Memory

Kraken 1.0 Benchmark Results

Chrome 7.0.517.44
18887.1ms

IE9 B2 9.0.7930.16406
61780.8ms

IE9 P7 9.0.8023.6000
18594.9ms

Firefox 4 B7
9115.5ms

Note: FF doesn't report a build number in the about box, 2.0.0.3960 is the file version as reported by the properties on firefox.exe, which may not mean anything useful. But it is B7 I just downloaded and installed it)

So yes, IE9 P7 does beat the production release of Chrome by a hair, however is gets smoked quite badly by FF B7. Granted results are going to probably differ some based on platform, but on a Core2 Duo IE9 gets it's ass kicked severely in this benchmark by FF. To be fair it is Mozilla's benchmark, so I'm not terribly surprised. But the fact that IE9 P7 is on the same level as Chrome (and have roughly tripled performance compared to their beta in a fairly short time) leads me to believe they are definitely on the right track.

That said, the UI in FF B7 feels sluggish compared to Chrome... Ive only used it for a few minutes, but I already don't like it much. Hopefully that will improved over time.

Edited 2010-11-18 05:37 UTC

Reply Score: 3

Tuishimi Member since:
2005-07-06

Yeah, my scores are posted above, with a lot of cruft, sorry. Basically the same results on a 6 Core AMD with 8 gigs of RAM...

Chrome still feels snappiest to me... FF B7 next followed by IE9. It's probably all just some weird perception thing since page rendering times between the three cannot be vastly different.

I am using FF B7 to write this post.

But as an aside, I thought FF was going to use process isolation for tabs like Chrome in FF4? It isn't, apparently. Comparatively speaking, FF B7 uses about 10000K more of RAM than Chrome 9...

Reply Score: 2

Tuishimi Member since:
2005-07-06

And one final note, IE9 uses about half the RAM of both Chrome 9 and FF 4 (for same number of tabs, same sites).

HOWEVER... Font rendering on IE9 and FF5 is the same, and I prefer the rendering of Chrome, which is crisper.

Edited 2010-11-18 05:46 UTC

Reply Score: 3

gilboa Member since:
2005-07-06

Why not answer to the point (e.g. by posting your benchmark results) instead of engaging in 3 y/o like personal attacks against the OP?

He posted results. You have contrary results? Post them!

- Gilboa

Reply Score: 2

The next step is obvious.
by Almafeta on Thu 18th Nov 2010 02:21 UTC
Almafeta
Member since:
2007-02-22

We need to not just benchmark for making good code fast.

We need to make benchmarks that test making terrible code at least functional. Sisyphean, but think of the rewards if it works!

Reply Score: 2

RE: The next step is obvious.
by Tuishimi on Thu 18th Nov 2010 05:51 UTC in reply to "The next step is obvious."
Tuishimi Member since:
2005-07-06

Facebook?

Reply Score: 2

RE[2]: The next step is obvious.
by sorpigal on Thu 18th Nov 2010 15:30 UTC in reply to "RE: The next step is obvious."
sorpigal Member since:
2005-11-02

It's too bad geocities is offline. That was always a great place to find crap code, whether Javascript or HTML, for real-world-crap testing.

Reply Score: 2

RE[3]: The next step is obvious.
by Tuishimi on Thu 18th Nov 2010 15:52 UTC in reply to "RE[2]: The next step is obvious."
Tuishimi Member since:
2005-07-06

Ha ha! I was going to write that FIRST... ;) I figured I'd take the next best and still active thing.

Reply Score: 2

RE[3]: The next step is obvious.
by Almafeta on Thu 18th Nov 2010 23:34 UTC in reply to "RE[2]: The next step is obvious."
Almafeta Member since:
2007-02-22

It's too bad geocities is offline. That was always a great place to find crap code, whether Javascript or HTML, for real-world-crap testing.


There's always Myspace.

Reply Score: 2

not cheating but certainly dishonest
by TechGeek on Thu 18th Nov 2010 02:36 UTC
TechGeek
Member since:
2006-01-14

It may not be cheating but its certainly dishonest. Microsoft knew that they weren't running the full test they way it was designed to be run. Yet they tout their results without telling anyone that the test is now meaningless because they added a short cut to their code. Its like a marathon runner taking a taxi to the finish line. Nice and classy Microsoft.

Reply Score: 1

Delgarde Member since:
2008-08-19

It may not be cheating but its certainly dishonest. Microsoft knew that they weren't running the full test they way it was designed to be run. Yet they tout their results without telling anyone that the test is now meaningless because they added a short cut to their code. Its like a marathon runner taking a taxi to the finish line. Nice and classy Microsoft.


Not at all - this isn't a race, with rules about following the correct route. If their runtime can correctly recognise that a bunch of code has no effect, then skipping it is absolutely the right thing to do - in compiled languages, this kind of thing is normal.

The only concern with this is whether it skips the test because the runtime itself is smart enough to recognize the test code as unnecessary, or because a Microsoft engineer recognized the test code as unnecessary and wrote the runtime to skip it.

Reply Score: 2

Comment by _xmv
by _xmv on Thu 18th Nov 2010 10:05 UTC
_xmv
Member since:
2008-12-09

So basically it's a bug in Sunspider ;)

Reply Score: 2

RE: Comment by _xmv
by TheGZeus on Thu 18th Nov 2010 16:01 UTC in reply to "Comment by _xmv"
TheGZeus Member since:
2010-05-19

Basically. Looks like a 'naive' benchmark.

Reply Score: 2

That still doesn't explain it.
by Timmmm on Thu 18th Nov 2010 11:27 UTC
Timmmm
Member since:
2006-07-25

Maaaybe the "true;" caused the dead-code detection to fail, although I can't see how.

But how on earth can adding "return;" to the end of a function change anything at all? It shouldn't change the AST at all.

Still highly suspicious.

Reply Score: 2

update
by jacquouille on Thu 18th Nov 2010 15:24 UTC
jacquouille
Member since:
2006-01-02

Rob Sayre has posted a new entry on his blog,

http://blog.mozilla.com/rob-sayre/2010/11/17/dead-code-elimination-...

Reply Score: 3

Not right!
by bjossir on Thu 18th Nov 2010 15:43 UTC
bjossir
Member since:
2007-01-14

Can anyone explain to me how the redundant "return" at the end of the function can cause the DCE to fail.

This is too funny ;)

Reply Score: 1

Don't buy it...
by TemporalBeing on Thu 18th Nov 2010 18:43 UTC
TemporalBeing
Member since:
2007-08-22

While I can buy that MS did put a dead-code eliminator in there, the changes that were introduced that broke IE9 should not have - the optimizer should have determined that it was still dead code that didn't do anything, and thus it should have continued to run at the same level. Thereby, either Microsoft is cheating or their optimizer is broken if it exists at all.

Reply Score: 2

RE: Don't buy it...
by Valhalla on Thu 18th Nov 2010 21:59 UTC in reply to "Don't buy it..."
Valhalla Member since:
2006-01-24

Thereby, either Microsoft is cheating or their optimizer is broken if it exists at all.


Well, again I can see that one would be able to draw that conclusion from what we've seen, but guys over at hackerne.ws have been able to create other code snippets different from cordicsincos() in which IE9 optimizes away the entire function, and it fails to do so also in those cases when a true or return is added so logically there really must be a dead code elimination optimization in there, but it's obviously very fragile or very specifically tuned.

But EITHER WAY, it doesn't matter because it says absolutely nothing about real world performance because there's no reason for a real world program to call a function that does nothing or returns nothing of interest, which means it won't be able to optimize it away. The snippets shown on msdn are not real world examples, and while there are certainly often occurences of code that is never called in programs, for performance gains to be had they would actually have to be called, and for these gains to be of any potential it would have to be called very often.

So for this optimization to be of use in the 'real world' someone would have had to write a program that continously call an expensive function that in the end does nothing or returns nothing that will be used, which would then mark it as a candidate to be removed as 'dead code'. In short, you won't see this optimization yeilding any major gains outside synthetic benchmarks, which is probably why no other browser has yet bothered with implementing it.

Reply Score: 2

RE[2]: Don't buy it...
by lemur2 on Thu 18th Nov 2010 23:13 UTC in reply to "RE: Don't buy it..."
lemur2 Member since:
2007-02-17

"Thereby, either Microsoft is cheating or their optimizer is broken if it exists at all.
Well, again I can see that one would be able to draw that conclusion from what we've seen, but guys over at hackerne.ws have been able to create other code snippets different from cordicsincos() in which IE9 optimizes away the entire function, and it fails to do so also in those cases when a true or return is added so logically there really must be a dead code elimination optimization in there, but it's obviously very fragile or very specifically tuned. But EITHER WAY, it doesn't matter because it says absolutely nothing about real world performance because there's no reason for a real world program to call a function that does nothing or returns nothing of interest, which means it won't be able to optimize it away. The snippets shown on msdn are not real world examples, and while there are certainly often occurences of code that is never called in programs, for performance gains to be had they would actually have to be called, and for these gains to be of any potential it would have to be called very often. So for this optimization to be of use in the 'real world' someone would have had to write a program that continously call an expensive function that in the end does nothing or returns nothing that will be used, which would then mark it as a candidate to be removed as 'dead code'. In short, you won't see this optimization yeilding any major gains outside synthetic benchmarks, which is probably why no other browser has yet bothered with implementing it. "

Without examining it in detail, this seems to me a very good reason why Microsoft has been able to just slightly edge out other browsers (by the barest of margins) on the Sunspider benchmark, yet still manage to get soundly beaten on other benchmark tests which reportedly (and according to Microsoft's own engineers) include more real-world-like javascript code.

Essentially, as I understand it, if the javascript code in question actually does something, the code won't be dead code, and Microsoft's optimization won't be able to eliminate it.

Paraphrased: As far as I can see, this optimization is only useful for the purpose of obtaining a better score on weak benchmarks like Sunspider (as opined by Microsoft's engineers).

Figuring out exactly how "obtaining a better score on weak benchmarks like Sunspider" helps users in any way, or is anything other than pure PR deception, is an exercise I will leave up to the readers.

Reply Score: 2

RE[3]: Don't buy it...
by Valhalla on Fri 19th Nov 2010 00:14 UTC in reply to "RE[2]: Don't buy it..."
Valhalla Member since:
2006-01-24

Essentially, as I understand it, if the javascript code in question actually does something, the code won't be dead code, and Microsoft's optimization won't be able to eliminate it.

Well, the code may do something, but what it does is useless since it's not used in anyway by the program. It's akin to someone standing in the corner and counting to a million for no reason other than having to do just that before being allowed to move on.

This happens in synthetic benchmarks, but not in real world programs, unless someone actually wants their code to run slower for some obscure reason.

Reply Score: 2

RE[4]: Don't buy it...
by lemur2 on Fri 19th Nov 2010 01:47 UTC in reply to "RE[3]: Don't buy it..."
lemur2 Member since:
2007-02-17

"Essentially, as I understand it, if the javascript code in question actually does something, the code won't be dead code, and Microsoft's optimization won't be able to eliminate it.
Well, the code may do something, but what it does is useless since it's not used in anyway by the program. It's akin to someone standing in the corner and counting to a million for no reason other than having to do just that before being allowed to move on. This happens in synthetic benchmarks, but not in real world programs, unless someone actually wants their code to run slower for some obscure reason. "

Precisely. Spot on. Exactly so.

So, follow that observation through. Since dead code is typically found in (flawed) synthetic benchmarks but not so much in real world code, what exactly is the purpose of dead code optimization? Is it perhaps to be able to run synthetic benchmarks tolerably well, even when one's browser doesn't do so well for real world code? Maybe?

Reply Score: 1

RE[5]: Don't buy it...
by saynte on Fri 19th Nov 2010 03:23 UTC in reply to "RE[4]: Don't buy it..."
saynte Member since:
2007-12-10

It is very normal for any compiler to perform dead code elimination, it is a very standard technique. It isn't at all shocking that Microsoft has also implemented it.

Reply Score: 1

It's broken
by indech on Thu 18th Nov 2010 20:12 UTC
indech
Member since:
2005-12-06
The smell of cat piss is not yet gone.
by another_sam on Thu 18th Nov 2010 20:26 UTC
another_sam
Member since:
2009-08-19

Thom did you consider last update on
http://digitizor.com/2010/11/17/internet-explorer-9-caught-cheating...
? They re-responsed Microsoft.

from digitizor.com:

Microsoft has updated their blog post addressing this issue. (Read here.) They attribute this to dead code elimination. They did not include any explanation as to why the dead code elimination fails miserably on adding “true” or “return“, which changes nothing in the actual function, or even why it fails when the for statement is replaced by while as demonstrated in Hacker News. It could be a bug or them over-training the dead code elimination method for SunSpider though.

Reply Score: 1