Linked by Brooss on Tue 15th Mar 2011 23:32 UTC
Benchmarks A comment on the recent article about the Bali release of Googles WebM tools (libvpx) claimed that one of the biggest problems facing the adoption of WebM video was the slow speed of the encoder as compared to x264. This article sets out to benchmark the encoder against x264 to see if this is indeed true and if so, how significant the speed difference really is.
Thread beginning with comment 466455
To view parent comment, click here.
To read all comments associated with this story, please click here.
RE: Nice
by galvanash on Wed 16th Mar 2011 22:28 UTC in reply to "Nice"
galvanash
Member since:
2006-01-25

I still need to do some more tests to be completely fair, but just to give a taste of what I finding so far:

I used this video as source: http://media.xiph.org/video/derf/y4m/sintel_trailer_2k_480p24.y4m

This is the Sintel trailer at 854x480. Its about 52 seconds long. I used the exact same encoder binaries as the linked article, although my machine is different (Phenom II X3 720 with 4GB memory, Windows 7 64-bit).

I compared x264 "baseline/slow" to webm "good/cpu-used=0" using a variable bitrate of 1000Kbs. Since multi-threading affects webm quality adversely, and x264 is mulththreaded by default, I am choosing to force x264 to run single threaded by using --threads 1 for this comparison. This is as apples-to-apples as I think you can get for a single threaded comparison (Ill do a multithreaded comparison later). I'm reporting PSNR, but frankly I don't thing it is at all relevant at this bitrate.

x264

Commandline: x264.exe -o sintel_trailer_2k_480p24.mp4 sintel_trailer_2k_480p24.y4m --profile baseline -B 1000 --preset slow --threads 1 --psnr

Avg PSNR: 52.375
FPS: 15.04
Filesize: 5436Kb
Actual Bitrate: 851.94Kbps

webm

Commandline: vpxenc.exe -o sintel_trailer_2k_480p24.webm sintel_trailer_2k_480p24.y4m --good --cpu-used=0 --target-bitrate=1000 --end-usage=0 -p 1 --psnr

Avg PSNR: 50.862
FPS: 6.43
Filesize: 5017Kbs
Actual Bitrate: 766.710Kbps

Conclusion

Quality is very comparable subjectively, x264 is a tad better imo (but not much). When I get all of this together I will post the actual output videos so people can look at them.

Filesize is close enough since both stayed well under the target. Webm is a bit smaller, but in this case that isn't actually a good thing. Both had plenty of bitrate to work with, and both ended up not needing anywhere near all of it (I used a high target because frankly that is what people normally do at these low resolutions).

note: I have noticed that webm when run with the "--good --cpu-used=0" with a high bitrate cap tends to undershoot much more than when run using "--best" (but it is also twice as slow when run that way). I prefer using best because it tends to get much better quality (i.e. higher bitrate) when given alot of headroom, but I didn't use it here because I did not want to be accused of an unfair comparison.

The big difference, as I have stated before, is in speed. x264 is over twice as fast (and frankly it is being hobbled here since running it designed to be run multi-threaded)... I'm still in the same ballpark speedwise as the original article, but the gap has widened a wee bit (i.e. roughly 60% slower). Well see how that pans out at HD resolutions.

Reply Parent Score: 3

RE[2]: Nice
by galvanash on Wed 16th Mar 2011 23:32 in reply to "RE: Nice"
galvanash Member since:
2006-01-25

Same setup, only change is the source is 1280x720 and the bitrate is set to 2000.

x264

Commandline: x264.exe -o sintel_trailer_2k_720p24.mp4 sintel_trailer_2k_720p24.y4m --profile baseline -B 2000 --preset slow --threads 1 --psnr

Avg PSNR: 53.647
FPS: 7.23
Filesize: 10702KB
Actual Bitrate: 1678.26Kbps

webm

Commandline: vpxenc.exe -o sintel_trailer_2k_720p24.webm sintel_trailer_2k_720p24.y4m --good --cpu-used=0 --target-bitrate=2000 --end-usage=0 -p 1 --psnr

Avg PSNR: 52.067
FPS: 3.35
Filesize: 10118KB
Actual Bitrate: 1548.36Kbps

Conclusion

Encoding speed relative to x264 stays about the same (this is much better than previous versions of webm at this resolution). Quality is a mixed bag. Webm looks much better during low motion scenes (detail is higher), but suffers during high motion (smearing). Overall quality is about the same imo.

Ill do a full battery of tests offline (multi-threaded too) and put a comparison together with graphs and links to the output files. Will probably take me a few days.

Reply Parent Score: 2

RE[3]: Nice
by lemur2 on Thu 17th Mar 2011 01:27 in reply to "RE[2]: Nice"
lemur2 Member since:
2007-02-17

Quality is a mixed bag. Webm looks much better during low motion scenes (detail is higher), but suffers during high motion (smearing). Overall quality is about the same imo.


When people see high motion in real life, they actually perceive it as a blur. It is perhaps a mistake to demerit WebM for having this characteristic.

When h264 videos have to make compromises on quality per bit, which happens in high motion scenes, the compressed video exhibits artefacts ... little extraneous bits that aren't there in the original scene. The human eye doesn't do anything similar when people are looking at scenes real life.

Just saying.

Edited 2011-03-17 01:28 UTC

Reply Parent Score: 2