The sweet smell of competition is lingering in the air. That sweet smell which indicates that somewhere in the vicinity a company is working on actually improving a product so we can all benefit. This time around, it’s Adobe, delivering the first Flash 10.2 beta. Prime feature? Complete hardware acceleration of the entire video pipeline – fully cross platform, cross-form factor. Cross-platform! There’s a catch, though.
Us Windows users have been able to enjoy a perfectly performing Flash video player for a long time now (I barely notice a difference between Flash video and vanilla H264), and Mac OS X users, too, have been able to somewhat enjoy some form of hardware accelerated video (much more limited than on Windows, though). Linux users have been left in the cold by Adobe – until now, that is.
Adobe didn’t just enable some basic form of hardware acceleration – no, this isn’t scalpel work, but full-on chainsaw action. Adobe’s new Stage Video technology enables not just hardware acceleration when it comes to decoding H264; in fact, it accelerates the entire video pipeline, including color conversion, scaling, and blitting.
“Working together with hardware vendors has helped us take advantage of the GPU to offload not only H.264 hardware decoding (introduced in Flash Player 10.1) but the rest of the video rendering pipeline, including color conversion, scaling, and blitting,” writes Tom Nguyen, product manager of Flash Platform Runtimes, “How efficient is hardware acceleration in Flash Player 10.2 beta? Using Stage Video, we’ve seen laptops play smooth 1080p HD video with just over 0% CPU usage.” They’re not kidding.
Of course, there’s a catch. Well, two catches, but one of them isn’t such a big deal. First, Stage Video is an API, and developers need to actually add support for it into their video player SWFs. I’m sure all the major Flash video players will be updated quickly enough, so this isn’t too big of a deal.
The second catch is a bigger problem. On Linux, Stage Video makes use of NVIDIA’s VDPAU, and while this is an entirely open standard, it has so far not been adopted by AMD and Intel. As such, you’re going to need an NVIDIA chipset, running the proprietary NVIDIA driver, in order to take advantage of this on Linux. Frame compositing on Linux is done via OpenGL.
I don’t blame Adobe for this – the Linux graphics stack is a mess, and they need to make some choices. It sucks for non-NVIDIA users, but alas.
I don’t understand, though. Couldn’t they have done the same or similar that Gnash did for 0.8.8 as mentioned in this Phoronix article? http://www.phoronix.com/scan.php?page=news_item&px=ODUzOQ
They _could_ have. They still _might_.
The real tragedy is that AMD, Intel, and Nvidia haven’t cooperated yet to create a standard framework… Not a surprise, mind you.
Pretty cool stuff, but I’m not going to go with a wrapped plugin solution again (Pure 64Bit Linux here) to try out the 32bit Beta…
The 64Bit version of Flash 10.1 exists and seems to work at least as well as the 32Bit version, when are they going to stop maintaining them separately?
EDIT: Actually, the 64Bit version is at 10.2, but doesn’t have the hardware accel yet… HoHum.
Edited 2010-12-03 00:13 UTC
Yes, same for me. Also I would rather have a bugfixed 64bit version than this acceleration stuff. It runs fast enough for me, but the wrong use of memcpy produces sound errors on quite a few webpages. It would be a matter of s/\<memcpy\>/memmove/g to fix that and it would be much appreciated! Hm, is there a way to patch the .so in order to bend the references to memcpy to memmove?
There’s some tutorials out there on how to create a .so that’ll work with LD_PRELOAD… you could replace the memcopy function with memmove… you’d probably need to use nspluginwrapper to run the plugin instead of the plugin directly running in your browser afterwards (so you don’t have to LD_PRELOAD your replacement function for the whole browser)
Works just as fine as 10.1 (or worse) for me on OS X 10.4.6.
That mean instantly crashes for 10.2 (only tried once) and crashes all the time on non-youtube videos (read porn) and eventually on your everyday banner in 10.1.
Versions before that worked (except for general flash crashes) but then they didn’t had any GPU acceleration.
I’ve turned acceleration off though in the settings menu.
Video Decode support through XvMC, VDPAU and VA-API APIs using the 3D engine (as opposed to the UVD dedicated hardware) is a work-in-progress for the open source Gallium3D drivers for AMD/ATI chipsets R300 through R800.
http://wiki.x.org/wiki/RadeonFeature
It is not possible at this time for the open source drivers to utilise the UVD hardware which is in R600/700, Evergreen and N.Islands chipsets because AMD/ATI have not released the programming information for this part of their chipsets a yet. As I understand it, there is significant doubt that ATI/AMD will ever be able to release this information due to concerns that this will also give clues to the DRM provisions which are also part of UVD. If that is the case, the video decode work which is underway using the 3D engine (shaders) might become the only option for the open source drivers, even for chipsets which include UVD.
Edited 2010-12-03 00:46 UTC
Sounds awesome, reminds me very much of Light Spark:
http://sourceforge.net/apps/trac/lightspark
Which is using a combination of LLVM and GLSL to hardware accelerate video play back plus a whole host of other Flash functions. I agree, the best thing that the *NIX world can do is build on top of GLSL and updating Mesa to OpenGL 4.1 compliance than trying to deal get access to proprietary video decoding features.
With that being said, the one thing Apple has done well is build on top of GLSL really cool features such as Core Animation etc. which provide a high level API than having to leave up to third parties to redesign the wheel.
Apple is the one the lit the unintentional fire under Adobe to prove it or lose it.
You’re absolutely right. And now that hardware acceleration for Flash is possible, I’m sure Apple will proudly announce Flash on the iPhone and iPad any day now.
</sarcasm>
Unfortunately the frameworks that Apple provide for flash acceleration (or any other video format) are really crappy as seen in QTX, nowhere near the flexibility and capability of things like VDPAU or DXVA2
Edited 2010-12-03 01:58 UTC
Adobe doesn’t use QTX for Flash – they utilise Core Animation, Core Audio, VDPAU which is called VideoDecodeAcceleration.framework – the framework is merely the nVidia framework relabelled and provided on Apple. There is also more to hardware acceleration than just having the right API’s on the system the browser also has to support ‘Pepper Extensions’ to NPAPI:
https://wiki.mozilla.org/NPAPI
And go down to “Accepted NPAPI Specifications” where so far only Safari and Chrome have actually implemented such features hence the reason why the performance of Flash on Firefox is craptacular on Mac OS X but works beautifully with Safari and Chrome.
I didn’t mean to imply that they did use QTX merely that QTX uses the same framework as flash. But I wonder if the framework has been neutered, as I can only accelerate a VERY limited set of h264 vids in OSX versus VDPAU.
I would argue that flash performance is fairly craptacular in Safari (on OSX) too at least in comparison to windows. It is better than it used to be but still not good as of 10.1, haven’t tried 10.2 because adobe fail with 64 bit. The flash codebase must be a wretched tangle if they can’t recompile the source to 64 bits very easily.
Edited 2010-12-03 12:00 UTC
They’re hardware limitations and nothing to do with VPAU – the following lists the limitations:
http://en.wikipedia.org/wiki/Nvidia_PureVideo
There is little Apple can do in terms of the limitations apart from maybe creating a whole new framework that is based on top of GLSL rather than tapping into the video decoding part of the GPU. It would be more complex but it would also probably support a lot more video formats, sizes on more GPU’s.
Regarding the codebase, it wouldn’t surprise me if it was like a horror film – At some point I think Adobe need to clear out the crud, break backwards compatibility because it isn’t just Adobe suffering but many developers having to work around bugs at the same time; it doesn’t help anyone in the long run to have perpetual backwards compatibility if it comes at the expense of improvements in the future.
Unfortunately files that work using VDPAU in linux don’t work in OSX. These are only h264( I’m not including VC-1 and divx which are also accelerated under vdpau but not in osx). VDPAU works on everything thrown at it, QTX chokes on most things (in terms of bitstream acceleration).
Seeing as flash seemed to have a similar discrepancy between DXVA2 and the accel framework on osx suggests to me there is something wrong with the framework on osx. It would be interesting to see what the differences are between all the platforms now that bitstream acceleration is available to all.
Apple really needs to improve the acceleration framework and QTX. They really ought to do it fast, especially after they coasted on the old QT framework since the early 90s piling crap on top of it much the same way that adobe is doing to flash now.
Edited 2010-12-03 15:42 UTC
VDPAU is only available on a small subset of video cards on Mac OS X with their justification that their desktops are already fast as it is and thus don’t require the additional acceleration provided by VDPAU. I don’t agree with their position but that is their position none the less.
Regarding QTX, I’m assuming you’re talking about QuickTime X the application and not QtKit the framework (QuickTime X is based on QtKit, the old framework wouldn’t have been updated for hardware acceleration) – it depends on whether they’re still using their internal private API or whether they’ve moved to the VDPAU. It is all speculation at this point until someone at Apple gives us the heads up on what is happening but I wouldn’t be surprised if the QtKit hasn’t been moved over yet.
The hardware acceleration that people see in QuickTime X only came about with the launch of the early 2009 MacBook Pro/MacBook which had the 9400M GPU and preloaded with Mac OS X 10.5.6 but it was not until 10.6.4 that the VDPAU framework was added which I speculate was developed by by nVidia rather than Apple themselves given that it is pretty much mirrors VDPAU. Given that Apple doesn’t develop the nVidia drivers themselves I’d say it is more likely than not that Apple got nVidia to develop a framework that taps into their drivers. With the gap between the Video Acceleration Framework making its appearance (and stabilising) and the delivering of hardware acceleration in QtKit (and in turn QuickTime X) I’d say that the QtKit is probably still using the private framework that originally appeared in 10.5.6.
It makes sense for them to use VDPAU for QtKit but it depends on whether they had enough time to make it so – and that is where the problem lies. As for VDPAU I really question whether having a vendor specific framework that taps into vendor specific hardware is actually useful in the long run when it would be better for Apple to recreate VDPAU using GLSL which would make it hardware agnostic. As long as the underlying hardware supports for example GLSL 1.30 then it will accelerate video decoding – yes I’m sure it would be complex in the initial design and implementation but if it means that Apple can design a framework once and for it to last over multiple generations of GPU hardware then it would be a move in the right direction.
Windows is a different kettle of fish because right now they’re working to replace their spaghetti of API’s with a single framework called ‘Media Foundation’:
http://en.wikipedia.org/wiki/Media_Foundation
According to the wikipedia article (which one takes with a grain of salt) it’ll eventually replace a whole host of API’s as the foundation features expand. I am assuming that with such a hardware agnostic framework that what Microsoft is doing is using their shader language as the basis of DXVA2 rather than it tapping into a specific part of the GPU. It will be interesting though what Windows 8 will deliver in terms of further Media Foundation development – whether there it will be completed or whether, like so many other frameworks, stay in a state of limbo until someone can be bothered finishing off an otherwise great idea on paper.
I think you’re confusing the framework with the player. The framework itself has already been updated – QtKit which was provided in 10.5 and further enhanced in 10.6 is the completely Cocoa based QuickTime framework but the problem is that many developers aren’t using it because they fear if they do then they’ll cease supporting 10.4 customers.
Apple for example still uses the old QuickTime framework when it comes to iTunes for example which is one of the reasons why it hasn’t become 64bit yet. Apple’s professional tools are still using the old QuickTime carbon framework – again it is very difficult to get third parties moving forward when Apple isn’t making the first move especially if you have a product that competes with Apple – if you move before Apple and thus you throw away 10.4 compatibility and Apple keeps to the old carbon API (thus supporting 10.4 customers) you could potentially lose customers.
As for accelerating the framework, the QtKit framework has already been highly optimised but the question is whether third parties are using the QtKit framework or whether they’re using the old carbon one. I also note in my reply to the first paragraph that we don’t know the internals of QtKit and what role VDPAU plays or whether they’re still using an internal private API.
Edited 2010-12-04 02:34 UTC
I believe most devs are using the old carbon framework because the new one doesn’t have many features beyond playing a video and outputting sound (is there even a public api for the new framework?).
DXVA2 can use the gpu video acceleration hardware not just shaders. However since it builds a graph of the pipeline you can put shader functions to modify the presentation (I don’t think it even needs to be done inside the dxva, possibly only the compositor). The thing is due to BS hollywood policies to view h264 in windows drivers need to couple decoding to presentation thus destroying all the flexibility of DXVA2 (at least on most video players attempting to use bitstream acceleration, dunno about flash).
There is a huge difference between the QuickTime X player and the QuickTime framework – please spend some time learning the difference *please*.
Here is some documentation:
http://developer.apple.com/library/mac/#documentation/Cocoa/Concept…
According to that documentation it doesn’t seem ‘limiting’. Considering it was only made available in 10.5 maybe it was a matter of waiting till it made sense to move – if you’re going to move to a new framework you might as well do it at the same time as you move to 64bit thus hitting two birds with one stone.
The QuickTime X player is limiting on Snow Leopard but that has more to do with Apple choosing only to do a very simple player instead of something covered all the bases. What will Lion do? it depends on whether Apple see’s merit in extensively adding more features to QuickTime or whether at best the player remains very basic.
I never said Quicktime X player; It is just a player that uses QTKit. I’m talking about Quicktime X as opposed to Quicktime 7. QTKit is built on top of these and provides a simpler api to both.
If you choose to use QTX you forfeit things like saving and editing and if QTX fails it falls back to QT7. Incidentally this appears to be the reason that no media editing app uses QTX.
You told me to read the documentation and so I did; Did you? It appears that you are mistaken on what QTX is…
Apologises for my previous post – I should have read the documentation rather than relying on second hand information I heard through the grape vine. Maybe this explains why there has been such a long time between Apple’s professional tools being released – that there is a whole lot of features that need to be added to QtKit before they can migrate across.
I’ve read the documentation and would it be correct to assume that QtKit is abstracted above QuickTime C based API? I’ve had a look at the framework reference itself and it makes sense if you’re interested in abstracting as high as you can and not want to deal with the details but the current Kit as it stands doesn’t seem to allow the kind of low level I think you mentioned in a previous post.
From the vantage point of a lay man it seems like a bit of a clusterfuck – if you want a feature complete low level API you have to use the QuickTime framework in C which only supports 32bit or you have to give up that those low level features if you want 64bit support. I can only hope that maybe Lion will bring about a completing of QTKit so that developers are able to move their code fully to the new API and thus make their products 64bit rather than being stuck in a state of limbo of having to use an IPC between 32bit QuickTime and their 64bit project like what Adobe Media Encoder CS5 does.
Not a shot in hell.
As much as Linux fans hate Flash I think it will end up working towards their benefit through Android tablets.
$200 Android tablets with Flash and decent hardware would fly off the shelves.
Mystified.
Could you explain by what process that helps linux fans?
I’d presume he’s referring to Android using the Linux kernel, not that that helps desktop Linux much.
Linux fans have been cheering on Android, maybe you don’t read tech news.
One thing has nothing to do with the other. And Flash still performs poorly on mobile devices such as the Sansung Tab. And that’s got a two core CPU, and a small screen. How many phones are using 10.1? Three, four?
You think it’s just Apple? While they’re the ones who are willing to say things publicly, most other manufacturers aren’t exactly hurrying to support it.
No. Apple is the one responsible for the lack in OS X to begin with and later yes for bitching about Flash performance. But it’s hard to see how it wasn’t because that bitching somehow benefited them and still how it mostly wasn’t their own fault.
Easier to admit than that your own product suck I guess.
Don’t know why they don’t have flash on the iPhone and iPad but maybe that could somehow be the reason.
No …Flash has some serious problems. They have just about fixed their video after a looooong time blaming others. The reason it still is worse on osx than windows IS Apples fault though.
While my current laptop has an NVIDIA card the constant lack of native AMD64 builds is really getting my goat.
10.1 was the best Adobe have EVER done with flash on Linux and as much as i’d like it to go away it’s not happening any time soon.
Well I just installed the 10.2 beta on my Ubuntu Lucid laptop, Intel GM965, Core 2 Duo @ 2.1Ghz, 3 GB DDR2, using Chromium. I’m sure there’s little or no VA support for this chipset, but I figured I’d give it a shot.
The driver definitely provides a higher, more stable frame rate, and screen tearing is almost unnoticeable- for once Hulu felt like it was a local video. Hulu specifically is a terribly performing video player compared to the Youtube one, and all it took was scrolling a window (not the player window) to disrupt the smoothness of the video.
I popped open the stock Youtube player to enjoy some HD video tests. 1080p videos were choppy, but local 1080p plays slow and out of sync on this machine so I wasn’t surprised.
I tried 720p. First off, this thing was a CHAMP when I went full screen and let go of my mouse. Video played at a wonderful frame rate with no interruptions and nice fluid, smooth motion. Moving the mouse or doing anything outside the player lessened performance somewhat, but not enough to bother someone whose attention was turned to, say, a web page on another monitor.
Most of the speed gain will be felt when Flash apps are ported to use the new Stage APIs, so I checked out the patched Youtube video player that Adobe provides as a demo. I really didn’t notice much of a difference between the two players however, I would bet that’s because of the VA support (or lack thereof) in my graphics hardware.
Finally the full screen multi-monitor stuff. Nope, none of it works at all in my testing. Full screen videos behave exactly the same as 10.1: videos always show on the primary monitor and clicking on *anything* not part of the video window will result in returning to windowed mode (oh, and the small player stays visible on your other monitor, just froze on the current frame).
Did I mention that CPU usage is as high as ever, using about 146% (that means 1 full core and 1/2 of the other) for chromium while watching a video. I think it’s safe to assume Flash was running on the fully utilized core.
For those with VA-capable video cards I’m sure this is a boon, for those with integrated graphics it is a moderate plus, and, at least for those multi-monitor dudes on Linux (myself included), Hulu popout will remain very useful at least for the time being.
If they ever come out with a decent intel atom netbook with a nvidia ion chipset that fits in my budget this would be good news.
It’s nice to see Adobe showing linux users some love though, not complaining by any means…
Keep up the good work Adobe, but please, can you do anything for intel gma video cards to help them run faster under linux. my netbook works great for everything except flash using Ubuntu netbook remix
What’s your budget? The 1201N from Asus fits your specs and is $484 from Amazon:
http://www.amazon.com/Seashell-1215N-PU17-BK-12-1-Inch-Netbook-Batt…
Is that too much?
The GMA 950 is still a very common gpu and in a lot of Core 2 Duo notebooks, he shouldn’t have to buy a new computer for smoother Flash.
The 950 GMA has no video decoding capabilities whatsoever, so there’s nothing Adobe, or anyone, can do here. Well, the OpenGL-based colorspace conversion, compositing and blitting will help, but the video itself will need to be decoded in software. What can help here is a multithreaded decoder, because the Atom has hypertheading, but even ffmpeg-mt can just barely decode 720p.
In Linux, only Intel HD graphics (Core i3/i5 integrated) has hardware h264 decode via VAAPI. Hardware h264 decode on Intel 4500HD has been promised for Q3 of this year, but it’s December and nothing has happened yet.
Edited 2010-12-03 19:35 UTC
It has video decoding, but not h.264.
that’s out of my price range. If I ever do get a Ion based netbook it’ll have to be under 200 dollars. Honestly my Acer Aspire One ZG5 works great for me for everything but flash, just hoping that Adobe could maybe make their optimizations work for any video card insteads of just specific ones.
Despite Adobe’s promise to ship a *BSD build years ago.
I’d love to see this myself given my recent tinkering with FreeBSD. It’s a great OS and it’s a shame that it gets treated like it does.
Oh, but they did ship a build for the most popular flavor of BSD around… Mac OS X
Edited 2010-12-03 13:34 UTC
OS X is not *BSD – it’s not even close. It’s just used some of the building bricks. OS X is no more BSD than GNU/Linux. Or put differently: OS X is not *BSD.
There are some video sites I literally can’t switch to fullscreen because the damn Flash plugin takes up so much CPU power its audio/video go off sync and responsiveness goes to hell (and they’re not even high-definition video). I’ve been waiting for Linux hardware acceleration for a long time now, and now, there’s another reason to be glad to be an nVidia user. Hopefully it gets picked up by the other two major companies soon enough though. The only problem: What the hell ever happened to the 64-bit Flash plugin for Linux?
Now hopefully Flash goes extinct before Adobe adds yet another major feature to the program, because given their past history they’ll release it for Windows exclusively for at least three years before adding it to the Linux version…
Edited 2010-12-03 07:01 UTC
You probably don’t want to read about Flash 11 then.
64-bit Adobe Flash “Square” Preview Release 3 [Nov 30, 2010]
http://labs.adobe.com/downloads/flashplayer10_square.html
Edited 2010-12-03 18:36 UTC
Shame Adobe hasn’t got its house in order.
This is still based on flash 10.1
Nope, sure don’t.
Cool – thanks! For some reason, even a Google search didn’t help this time.
Tip: it’s an annoying workaround, but at least for YouTube on my computer, the FLV files get dumped in /tmp as /tmp/Flash* (where the * is some alphanumeric string which appears to be random). You can open those files in your favorite native video player and play them full screen with no issues.
That does work (I used that method to save SWF games and cartoons and stuff), but like you said it is annoying.
My preference for Flash videos is actually to use the Firefox extension Video DownloadHelper to save a version of the video in a standard MPEG-4 container to watch offline. It makes saving embedded Flash video files easy. YouTube has plain .mp4 versions of most of their videos, and I’ll take those over Flash files–I want to avoid anything Flash if possible, I’ll take a “proper” video format any day. Of course, the .flv files can be saved as well if a particular video (or site) has no .mp4 version.
It’s become more annoying lately (&fmt=18 has to manually be added to the URL for the MP4 file to be shown if it is available), but luckily it still works. And all 720p/1080p videos seem to be MP4 files by default.
It sure is nice being able to save these videos for later use; who knows if YouTube will decide to delete them, and while I have cable, its max speed is 80-90KB/s (yeah, cheap service, regular costs like $30 more per month). And not everyone has access to the Internet–being able to put them on a DVD or USB flash memory stick to watch on any MPEG-4-capable video player is very useful. It’s a breach of service I’m sure, but oh well–if they don’t want the videos to be seen, then they shouldn’t be hosting them for the public to see on the Internet.
Edited 2010-12-04 01:03 UTC
I pretty much do the same, though I use the Greasemonkey script here: http://userscripts.org/scripts/show/62634 which gives you a informative menu http://s3.amazonaws.com/uso_ss/icon/62634/large.png?1270376010 to choose from when downloading. No URL stuff needs to be added. I don’t know how Video DownloadHelper extension operates these days, but I recall back when I used it that it contacted it’s own site whenever you wanted to save a video, which I found kind of annoying.
Oh gees ! I’ve been moaning against Adobe & Flash litterally for years. It makes my system crawl, it’s a proprietary piece of software, they took ages before supporting 64 bits Linux (and still partly do).
But what a relief. Finally, I will be able to switch HD movies fullscreen without making everything crawl.
BTW, they chose to use VDPAU. OK, that may be a safe choice, as Nvidia drivers are definitely the best. But I thought there was the VA API (video acceleration) which is more universal ? (and there’s a VA->VDPAU wrapper anyway)
OK, now waiting for the 64 build. Hope it won’t take 10 more years.
Edited 2010-12-03 10:34 UTC
S3 also uses VDPAU, though (if i recall correctly) it’s only in their 400+ series.
Still no 64bit!
Try the Adobe Flash Square preview. I’ve been running it for a few months on an Intel GMA965 and able to watch ESPN3, Hulu, Youtube videos fullscreen with no stuttering.
Interesting that the latest build claims to be:
“Shockwave Flash 10.3 d162”
Sure it is better, but it is still not using video hardware acceleration..which is a pile of Fail. I suppose maybe adobe thought that we would need help heating our houses this winter…
I am not so sure what the author meant by declaring the Linux graphics stack a mess.
Actually it has never been more straight forward.
DDX and the Xorg driver and bam, works out of the box complete with MESA support.
mplayer guys have been doing this for years now.
They do not complain either, it just works even at 1080p on my workstation with a ATI 3870 card (laptop).
Works great.
-Hack
They do not complain either, it just works even at 1080p on my workstation with a ATI 3870 card (laptop).
In software, yes, the discussion is about hardware decoding.
Yep we all love how flashplayer turns our linux lappys into goddamn helicopters with 100% CPU Usage while this issue never existed on windows.
Finally linux users can enjoy livestream.com too.
…and it’s so fast I can’t even see it.
J/K.