Linked by Brooss on Tue 15th Mar 2011 23:32 UTC
Benchmarks A comment on the recent article about the Bali release of Googles WebM tools (libvpx) claimed that one of the biggest problems facing the adoption of WebM video was the slow speed of the encoder as compared to x264. This article sets out to benchmark the encoder against x264 to see if this is indeed true and if so, how significant the speed difference really is.
Thread beginning with comment 466396
To view parent comment, click here.
To read all comments associated with this story, please click here.
RE: WebM vs H.264 benchmark
by lemur2 on Wed 16th Mar 2011 12:27 UTC in reply to "WebM vs H.264 benchmark"
lemur2
Member since:
2007-02-17

I personally am much more interested in the quality aspect than in encoder speed. Unfortunately, there seems to be no quality benchmark anywhere that is actually fair to both parties; either it tries to pose H.264 as superior and thus uses inferior settings for WebM, or vice versa.


Here is a decent attempt at quality comparison done in May 2010:

http://compression.ru/video/codec_comparison/h264_2010/index.html

In June 2010 they produced this update which included VP8:

http://compression.ru/video/codec_comparison/h264_2010/vp8_vs_h264....

Considering only quality per bit, and ignoring encoder speed, this SSIM metric comparison puts WebM quality behind that of x264 but ahead of XviD h264. The VP8 "best" preset just overtakes the x264 high-speed preset, all other presets of x264 are progressively slightly better.

In June 2010 the only version of WebM was the launch version of libvpx, the reference codec.

Since that time there have been two subsequent releases of libvpx.

http://www.osnews.com/story/24502/WebM_Project_Releases_New_Version...

These WebM releases were announced here (Aylesbury):
http://blog.webmproject.org/2010/10/vp8-codec-sdk-aylesbury-release...
and here (Bali):
http://blog.webmproject.org/2011/03/vp8-codec-sdk-bali-released.htm...

http://www.osnews.com/permalink?465497

Bali: "Best" mode average quality improved 6.1% over Aylesbury using the SSIM metric.
Aylesbury: "Best" mode average quality improved 6.3% over launch release using the SSIM metric.

Total improvement from launch release to Bali release = 1.061 * 1.063 = 1.127843. I rounded this out to 12.8%.


The most recent release of the libvpx reference code for WebM is 12.8% better quality than the version shown in the graphs plotted in June 2010 at compression.ru.

Hope this helps.

Edited 2011-03-16 12:34 UTC

Reply Parent Score: 2

RE[2]: WebM vs H.264 benchmark
by WereCatf on Wed 16th Mar 2011 12:42 in reply to "RE: WebM vs H.264 benchmark"
WereCatf Member since:
2006-02-15

The most recent release of the libvpx reference code for WebM is 12.8% better quality than the version shown in the graphs plotted in June 2010 at compression.ru.

Hope this helps.


No, it doesn't. Percentages do not say anything about how it actually looks like to the eye and thus referring to them as some sort of a holy bible doesn't really tell me much.

Reply Parent Score: 2

RE[3]: WebM vs H.264 benchmark
by lemur2 on Wed 16th Mar 2011 12:53 in reply to "RE[2]: WebM vs H.264 benchmark"
lemur2 Member since:
2007-02-17

"The most recent release of the libvpx reference code for WebM is 12.8% better quality than the version shown in the graphs plotted in June 2010 at compression.ru.

Hope this helps.


No, it doesn't. Percentages do not say anything about how it actually looks like to the eye and thus referring to them as some sort of a holy bible doesn't really tell me much.
"

This observation is perfectly correct. In "blind testing", a group of people are shown different versions of a video, without being told anything else about them, and they are asked which version they prefer.

Blind testing of WebM launch release showed that people were simply unable to pick the difference between h264 and WebM. Some preferred one, other preferred the other, and still others said they couldn't pick between them.

"The eye" effectively can't pick these small differences in measured video quality.

PS: It was you who asked for video quality benchmark results, not I.

Edited 2011-03-16 12:59 UTC

Reply Parent Score: 2