Linked by Brooss on Wed 23rd Mar 2011 23:14 UTC
Benchmarks A new set of x264 and vpxenc encoder benchmarks have been published. The new benchmarks address many of the concerns raised in the comments about the methodology used in the previous article, such as using SSIM for quality measurement. Theora is also included in these tests.
Thread beginning with comment 467592
To read all comments associated with this story, please click here.
Good set of benchmarks
by WereCatf on Wed 23rd Mar 2011 23:42 UTC
WereCatf
Member since:
2006-02-15

I found this quite enlightening. It still doesn't answer every question I might have, but then again, you can't expect a benchmark to ever answer every possible question.

As for the results: vpxenc is still under heavy development but it's very promising to see that as of now it matches x264 baseline in SSIM-quality. Considering how young the encoder still is this promises lots of optimization potential is still left, both quality- and speedwise!

What's interesting is how poorly Theora performs. Many enthusiasts have been hailing Theora as the best open video codec for a good long while now, but it gets totally trumped on by both the contenders here even though the encoder is already quite well-established.

Anyways, I have to congratulate vpxenc devs on a good job and I hope to see more improvements coming soon. It is awesome to have an actually great open video codec that challenges H.264 quality-wise!

Reply Score: 5

RE: Good set of benchmarks
by lemur2 on Thu 24th Mar 2011 02:10 in reply to "Good set of benchmarks"
lemur2 Member since:
2007-02-17

I found this quite enlightening. It still doesn't answer every question I might have, but then again, you can't expect a benchmark to ever answer every possible question. As for the results: vpxenc is still under heavy development but it's very promising to see that as of now it matches x264 baseline in SSIM-quality. Considering how young the encoder still is this promises lots of optimization potential is still left, both quality- and speedwise!


Exactly what these benchmarks mean is subject to interpretation. In reality, what they mean is that in order to encode WebM to the same quality as x624, it will take longer. Alternatively, they mean that if you are only prepared to devote the same period of time to encoding, then the video produced by x264 within your time budget will be a little better quality per bit than WebM.

In real life, since encoding is done only very rarely by the vast majority of people, the latter comparison doesn't come in to play, and the former comparison is the only thing that has any practical importance.

For all practical intents and purposes, all this means is that it will take you a little longer (and cost you infinitely less) to encode your video clips in WebM to the same quality as you would have had if you were using x264 legally.

If you install a copy of the new Firefox 4 browser you can see for yourself that the WebM video clips themselves do not lack for quality:

http://www.mozilla.com/en-US/firefox/4.0/whatsnew/

Reply Parent Score: 7

RE[2]: Good set of benchmarks
by rr7.num7 on Thu 24th Mar 2011 03:54 in reply to "RE: Good set of benchmarks"
rr7.num7 Member since:
2010-04-30

[q]

In real life, since encoding is done only very rarely by the vast majority of people, the latter comparison doesn't come in to play, and the former comparison is the only thing that has any practical importance.

For all practical intents and purposes, all this means is that it will take you a little longer (and cost you infinitely less) to encode your video clips in WebM to the same quality as you would have had if you were using x264 legally.


You are absolutely right. But you know how it is, some "pro" content providers complain because they're too lazy because of that little extra time or extra step in converting the output from their video editing software.

I say: who the fuck cares? As a software developer I wish I could choose the language/frameworks/tools for every project I work in (that would make my job way easier) but often I can't. So what? That's why it's called a job.

As a content consumer I really hope WebM becomes the dominant format of the web. If this means professional filmmakers' work will be 2, 3, 4 times harder, then so be it. To all of them: grow up, do your jobs and get over it.

Reply Parent Score: 4

RE: Good set of benchmarks
by Brooss on Thu 24th Mar 2011 18:18 in reply to "Good set of benchmarks"
Brooss Member since:
2010-11-13

What's interesting is how poorly Theora performs. Many enthusiasts have been hailing Theora as the best open video codec for a good long while now, but it gets totally trumped on by both the contenders here even though the encoder is already quite well-established.

It has been suggested that the bitrate used in these tests it too low for a fair comparison with Theora. E.g. bitrates below a certain point cause Theora to preform much worse than it would above that point. I don't know if this is true for this test but it is possible Theora would be more competitive at higher bitrates.

Reply Parent Score: 1