OSNews

I remember not so long back having loads of issues with gstreamer, but since i'm running the 7.* development series things are working great and it largely replaced xine-based media on my system.

Most people will only see this glory when 8.0 comes out, but it will be very good.

to the global domination of free desktops.

Wow... GStreamer sure is starting to rock.

the lack of a system like gstreamer is what drove me away from developing for linux. however, the lack of a c++ api for gstreamer made me choose the BeOS Media Kit instead, knowing that it would somehow probably make it to other platforms as well.
personal issues made me stop developing my application before it reached alpha stage, but if I ever decide to start developing it again I might perhaps take a second look at gstreamer if someone would just build a c++ wrapper around it.

gstreamer is very important for linux as this will make developing media apps as easy as on Windows or BeOS, so perhaps we will finally see more of them popping up in the next few years. Windows is still a bad system for music production, and will probably never be a good one since MS doesn't care about that market. Linux can be tweaked to perform a lot better in those situations, and will hopefully never have the audio interface hell that windows has.

So are there any commercial counterparts that really offer anything simmilar to gstreamers goal?

In other words, can we expects a big nasty blood fest in a few years?

This is great article.
OS news has been producing some great articles lately and this one is right up there. Great work.

i suppose you have read the article before you posted.
so you may have noticed, that there is a c++ wrapper above gstreamer, it is in kde cvs...

well, maybe you do not consider qt-c++ (moc and stuff) as c++, but you may really look at the api before (which you obviously did not, as nothing forces you to write qt-c++).

maybe you only want a stl-based wrapper. i can understand this. afaik, this is not yet done. and propably will not be in the foreseeable future.
and why would you want this. if you do want gui apps, you will have to choose a graphics lib, assuming you do not want xlib (or something exiotic like fresco), you will have to choose either gtk-- or kdelibs, well, you do have a gstreamer api for that language flavor.

so what do you think is missing?

~ibotty

I am sure, you mean by 0.7.* and 0.8.0.. ;-)

and so far, none of it working right-out-of-the-box-with-all-kind-of-(streaming media) as wmp does.
seems there has to come a vendor of a proprietary player first (real/helix) to finally make that happen...

sometimes, the fragmentation of linux-projects makes one mad (from a users perspective). gstremaer might be a nice framework for future media applications, but i for one would prefer a player that just works and integrates properly for a start!

:-(

At least visit their site before making uninformed comments. 2 clicks off the main page and you'll find C++ bindings under development for gstreamer. http://www.gstreamer.net/bindings/c++/

What about GStreamer and web cameras? Are these in GStreamer's domain (regarding capturing/playingback and/or streaming video)? I am writing this because I need an ov511 driver for FreeBSD 5.2 and the only available is one named "vid" but is just a frame grabber and not a video capture driver (so it can't be used with Gnomemeeting). Apparently they can't port the ov511 driver to FreeBSD because FreeBSD doesn't have a Video4Linux infrastructure. So, is Gstreamer sitting on top of V4L or can it replace it and so these stuff can be more OS agnostic?

Additionally, where is freedesktop.org's HAL fitting into the whole Gstreamer picture here?

quote: "and so far, none of it working right-out-of-the-box-with-all-kind-of-(streaming media) as wmp does."

First of all wmp is just a player. Gstreamer also has _editing_ capabilities. There are already a couple of nice video editors available that use gstreamer.

Secondly I think mplayer beats the sh*t out of wmp regarding supported formats. mplayer plays everything wmp does, plays divx/xvid without having to install 3rd party plugins AND plays quicktime/realmedia out of the box. Beat that (Xine is also very good btw)

I agree with the author that gstreamer has a lot more potential than mplayer, because well.. it's just a mediaplayer and nothing more *g*

quote: "sometimes, the fragmentation of linux-projects makes one mad (from a users perspective). gstremaer might be a nice framework for future media applications, but i for one would prefer a player that just works and integrates properly for a start! "

I believe projects like Gstreamer make it possible to reach a level of integration that commercial software can never reach. There are already great gstreamer based players out there (totem for gnome). I personally believe gstreamer is not yet ready for (my|the) desktop, but it's definately usable and getting better every day. If you want a nice integrated player _today_ that supports all your video and audio formats without hassle, stick with mplayer or xine for now. There are nice frontends available for both that integrate perfectly with the gnome and kde desktops (just take a look at freshmeat).

I love GStreamer. I love Ruby. guys, life's just so sweet sometime, thanks!

In the article claims that swfdec (swfdec.sf.net) is the only open source decoder of swf. What about GPL Flash (http://www.swift-tools.net/Flash/)?? It's way better (and older) and support interactive stuff (buttons etc.)

If you actually read what I wrote you would see that I said it is the only swf decoder available under a license as liberal as the LGPL.

what will happen when MAS will become 'the facto' in the X world? (will be included by Xouvert R2 and is sponsored by X.org and FreeDesktop.org)

MAS is a competitor of GStreamer?

I *love* the concept behind Gstreamer. I had to implement a server-based multi-client audio/video multimedia-thingie for use in a museum at work. While trying the different opportunities i stumbled across gstreamer and realized that it just was what i needed. however, back then i had problems implementing it due to weird errors and time constraints. in order to at least have something valuable for the customer to see i did a quick & dirty flash/xml clientserver thingie.
great to see gstreamer mature, now if i've to work on the before-mentioned project again i'll definately try gstreamer again.

Well there is some overlap. MAS is however more aimed at being a sound/media server so the overlap doesn't need to be that big in practice.

That said I don't think many in the GNOME and KDE community takes MAS serious anymore after their abysmal track record for the last years.

"Secondly I think mplayer beats the sh*t out of wmp regarding supported formats. mplayer plays everything wmp does, plays divx/xvid without having to install 3rd party plugins AND plays quicktime/realmedia out of the box. Beat that (Xine is also very good btw)"

mplayer doesn't allow for fullscreen-video on my ati pro 8 mb notebook (at least i don't know how to achieve it)
besides, the last time i checked the mplayer-plug for mozilla, it rquired a good amount of tweaking to get it work with the (embedded) gui.

as for xine, their mozilla-plug is afaik still in development (since ages) and from my experience, very unstable/unreliable.
also, xine is a resource-hog - with some divx, i can't get fullscreen-video due to processor (pIII-600). with wmp und xp and the same movie - no problem.

it seems one of the downsides of oss-development that people are eager to code (new) exiting stuff, pursuing new approaches (resulting in functional overlapping projects and rel. thin and shattered manpower), while "filling the gaps" has a very low priority - doesn't make fun.
this is understandable, but from a users perpective - quite enerving!

Some areas I think should be covered wider in article but were not:

Personally, I consider one of the strongest points of GStreamer to be its total independence of formats supported, and also ability to add / remove them through plugins, and not by recompiling. This means there is nothing like "list of formats supported by GStreamer", as supporting is just a matter of writing appropriate plugin(s). As of today, it would be even impossible to say exactly what is supported by GStreamer, as every once in while someone on mail list announces plugin we weren't aware of . This is unlike how for example MPlayer handles that, and is more similar to what i.e. DirectShow does. Also, core GStreamer is very much independent of what plugins do, as set of features provided by core library is surprisingly small, although generic.
That separation allowed for example to implement aforementioned tagging almost exclusively by means of plugins, with very little extra provisions from core. Also, flexibility of architecture is clearly shown by GStreamer's first DVD-Nav capable app ever, which was gst-launch - generic commandline tool for constructing pipelines . Navigation support was achieved solely by work on plugins, without even touching gst-launch itself. Next one, gst-player (GStreamer's GTK player) gained it by changing only the 40 lines necessary because it catches mouse events.
This abstraction is lifted even higher by libgstplay, hacked on by Julien Moutte, that allows apps to request convenient media playback virtually without knowing anything about the whole process.
All that makes possible what was traditionally weak point ot Linux and Free OS's in general - powerful multimedia handling without the need to reinvent the wheel everytime someone needs it.

Gstreamer is perhaps the media framework open source Unix needs to turn it into a formidable media platform. Already open source Unix has the best media players available, Mplayer and Xine.

With Gstreamer's editing capabilities, I see professional media developers flocking to *nix when Gstreamer matures and when media applications begin to take advantage of its robustness and flexibility.

Who says open source doesn't innovate? That being said, the last time I used Gstreamer via Totem as a front end, I couldn't play DVDs or CDs or a number of media formats-- real player quicktime.

That was a major turnoff. It's always a good idea to begin supporting basic infrastructures used by one and all, if only to garner the attention and support of users and potential developers, before concentrating on advanced features useful to a niche few.

Based on my experience, I immediately concluded Gstreamer wasn't as mature and as useful as Mplayer and Xine were, of course until I read this article and revisited Gstreamer's website. I'm positive several other users, supporters and potential developers made such quick and haste judgments based on their experiences if they were similar to mine.

So to the Gstreamer developers, great work, but if users can't play their CDs or DVDs using a Gstreamer front end, you encourage misconceptions and false notions about Gstreamer's usefulness and maturity to them. Now I know better, but I've been a victim of such shortsightedness and so are many others.

Well, after reading the comment titled "Additional notes" by Mathrick, perhaps I didn't have the appropriate plug ins installed to playback CDs and DVDs. So, I ask humbly, are there plug ins available to playbacks DVDs and CDs?

So, out of interest, if I wrote a decoder for, say GIF files, and an encoder for, say PNG files, could the GSTreamer framework allow me to write an image conversion application (which could later be extended simply by adding more plugins)?

If so, this reminds me of the old Amiga datatypes mechanism...

"Datatypes is basically an extension of normal shared libraries (or DLLs) to provide generic data handling capabilities. With this facility, any datatypes-aware program--whether viewer, web browser, or image editor--can be extended after the fact, simply by adding the appropriate datatype for whatever new format comes along."

Sweet!

yeah, I've should have taken a better look at it. sorry, I was so tired when I wrote the post that I didn't bother. at the time when I considered using gstreamer it didn't have c++ support afaik. but that was some time ago. good to see that it's getting along.

It looks cool but its not GPL - its LGPL right ? I would think that a industry standard for linux would need to be truely GPL in order for widespread support from developers out there.

Stop trolling. If it isn't the 12.*.*.* trolls out is the comcast trolls at it again asking stupid questions regarding licenses. Here is a hint sonny, jump over to gnome.org and count how many libraries use LGPL then make that same comment again.

Please don't forget that GStreamer exists for over 4 years now. It still has NO stable API and is not very well implemented into GNOME either. I think that after 4 years of hacking on GStreamer it should have reached a point where the API is stable and where it's integrated correctly into GNOME (and not half). Maybe a hint for the developers to work towards an serious API freeze pretty soon and make it become stable and reliable.

So, out of interest, if I wrote a decoder for, say GIF files, and an encoder for, say PNG files, could the GSTreamer framework allow me to write an image conversion application

Yes, exactly.
In fact you application would be a simple one line command line

gst-launch filesrc location=file.gif ! gifdec ! pngenc ! filesink file.png

perhaps I didn't have the appropriate plug ins installed to playback CDs and DVDs. So, I ask humbly, are there plug ins available to playbacks DVDs and CDs?
Yes, there are cdplayer and dvd{read,nav}src. cdplayer isn't anything new, and what's interesting is dvdnavsrc as it's what gives navigational capabilities. Everything is basically there, now only playback smoothness needs to be polished (currently navigation events may block playback for a short while)

What about GStreamer and web cameras?

Yes. Web cams show up as video4linux (or v4l2) devices. GStreamer will support these, but the current support is sorely lacking. Encoder plugins or whatever they're called is one of their goals, though, so this should be improved greatly by 1.0.

I only wish the .net bindings were in better shape (i.e. buildable). That would really make using this framework a pleasure.

"mplayer doesn't allow for fullscreen-video on my ati pro 8 mb notebook"

...

"also, xine is a resource-hog - with some divx, i can't get fullscreen-video due to processor (pIII-600). with wmp und xp and the same movie - no problem."

synergy, that's not a problem with MPlayer or xine, but with your video driver. You need hardware acceleration for fullscreen video where the scaling is done in hardware, instead of the slow CPU-based scaling. It works well with the NVIDIA cards, ATI now also offers Linux drivers, but their older models might require the GATOS driver for XVideo support (that's overlay in XFree86 - the hardware video acceleration).

Greate article Christian, what about integrating animated SVG into GStreamer?

The paragraph about the competition isn't very complete, I think. He says that GStreamer has been doing software audio mixing for some time, so that would make esd (and asd) competition, as well as aRtsd. aRtsd also does MIDI sequencing (I believe), and as that's a stated goal of GStreamer, that would be competition on that front as well. Plus, ALSA now has its dmix capability which allows sofware mixing to happen at a lower level (either in the kernel or the ALSA library).

He also doesn't talk at all about network transparency, which is the point of MAS, so I imagine that GStreamer does that as well.

Basically, there's a lot of overlapping competition in two areas: per-soundcard, software audio mixing and network-transparent audio. Personally, I feel that ALSA's dmix is the best solution for the former (and I imagine that GStreamer will be able to take advantage of that in its ALSA output plugin), and I also think that network-transparent audio is a solution looking for a problem. I don't think that any setup that has thin-client workstations won't be that concerned about having its users play mp3s.

But it seems that GStreamer does other things, too, but as he stated, user playback hasn't been a priority for them (and it is for most users, which is why we're all still using XMMS...)

Great article, but all the way through I kept thinking
"this is like virtualdub!" (except better, of course).
How come there isn't a virtualdub under linux?

The technical reason is that VDub is depending heavily on VfW (Video for Windows) and that technology just isn't portable If you meant 'a VDub like application' then I suggest you take a look at avidemux2 http://avidemux.tuxfamily.org/

The idea of people working on an NLE for Unix is exciting. I hope they will have a serious look at the only 2 professional NLEs currently available (on Mac/Windows): Avid and Final Cut Pro. And that they will spend a lot of time talking with professional film editors.

The paragraph about the competition isn't very complete, I think. He says that GStreamer has been doing software audio mixing for some time, so that would make esd (and asd) competition,

No, as much as SDL isn't competiting with OpenGL

as well as aRtsd. aRtsd also does MIDI sequencing (I believe), and as that's a stated goal of GStreamer, that would be competition on that front as well.

AFAICT, aRtsd is kind of failed experiment, and also is good suited for audio synthesis, but not so good as general media processing framework, at least not w/o serious hacking. GStreamer already has that.

Plus, ALSA now has its dmix capability which allows sofware mixing to happen at a lower level (either in the kernel or the ALSA library).

See esd comment above

He also doesn't talk at all about network transparency, which is the point of MAS, so I imagine that GStreamer does that as well.

It is entirely possible (see Ramon's rt(s)p work), but not essential, and definitely not something worth making central point in design. GStreamer just allows that, but doesn't impose it itself. Also there is gst MAS plugin, despite MAS being (IMO) vaporware.

Basically, there's a lot of overlapping competition in two areas: per-soundcard, software audio mixing and network-transparent audio. Personally, I feel that ALSA's dmix is the best solution for the former (and I imagine that GStreamer will be able to take advantage of that in its ALSA output plugin)

I don't see any reason not use it

and I also think that network-transparent audio is a solution looking for a problem. I don't think that any setup that has thin-client workstations won't be that concerned about having its users play mp3s.

Raw audio streams on network doesn't sound too good either . But as w/ everything in GStreamer - you *can* do it, but don't *have* to. GStreamer can transparently add network transparency ;P

But it seems that GStreamer does other things, too, but as he stated, user playback hasn't been a priority for them (and it is for most users, which is why we're all still using XMMS...)

Not really true. Playback *is* important, but it's not *the* ultimate goal. Having framework that allows for playback, as well as encoding and transformation, is, however. And currently, playback is good enough to be used on daily basis. I myself use gst-launch as my primary video player (no seeking, no pausing, nothing more than video window and sound output and it works really good.

Well with animation being added to librsvg now I guess we will get a librsvg based plugin as soon as that is done.

Hopefully someone also makes a KSVG plugin since that would enable displaying SVG animations right away.

NMM is really good. I used it. They have lots of plugins, and they are doing big steps foreward. And it's possible to use it via network.

*Another* framework?

I agree Linux is really lacking a standard audio framework, esd and artsd really sucks. But now we have JACK, MAS -- and GStreamer!

I would be much more interested in how these compare for features:

* JACK has been worked on a lot to minimize latency. How does GStreamer compare in this regard? It is probably the most important aspect of media production. Perhaps there are befinits of co-operation with the kernel scheduler here?

* GStreamer sounds like it has a lot going for it in transcoding different codecs. Would an application like VideoLan become extremely simple when built on this? Did you guys look at their code and/or design, which is really proven in practice?

* MAS has the X Consorium's blessing. This may be more important than ever now when X seems to be undergoing some political changes. When/if Xouvert takes over, will they choose MAS, JACK, GStreamer or something else?

* What is the general consensus among the KDE people? They tend to be less kind of company-ish and a rather pragmatic bunch.

In 5-6 years time there would be a fully functional media playing and editing framework for Linux that just works and is well integrated with the desktops and APIs.

Until then, either use Mac OS X or Windows or compromise.

*Another* framework?

I agree Linux is really lacking a standard audio framework, esd and artsd really sucks. But now we have JACK, MAS -- and GStreamer!

GStreamer is *media processing* framework, not audio output one. In fact, gstreamer can use any of these for sound manipulation / output. You should think of these project as being complementary, not directly competing

I would be much more interested in how these compare for features:

* JACK has been worked on a lot to minimize latency. How does GStreamer compare in this regard? It is probably the most important aspect of media production. Perhaps there are befinits of co-operation with the kernel scheduler here?

I don't really know much about RT, but AFAIK there is currently no hard RT support in gst, and adding it would require some (minor, mostly advertising RT capabilities) tweaking on core, and heavy hacking on plugins that want to be RT safe. Note also that GStreamer itself doesn't introduce any latency and is capable of using JACK. Andy Wingo did some work on RT cleaniness so provided you don't use any unsafe elems in pipeline, it should preserve RT.

* GStreamer sounds like it has a lot going for it in transcoding different codecs. Would an application like VideoLan become extremely simple when built on this? Did you guys look at their code and/or design, which is really proven in practice?

Yes, it could be much simplified. But then, VLC is bit of framework itself, so switching over to GStreamer would be invalidating much of its code. As a sidenote, GStreamer is also proven in practice

* MAS has the X Consorium's blessing. This may be more important than ever now when X seems to be undergoing some political changes. When/if Xouvert takes over, will they choose MAS, JACK, GStreamer or something else?

http://www.mediaapplicationserver.net/public-mhonarc/mas-devel/

IMHO project with 30 mails since 2002 is dead and already stinking. AFAICT MAS has been just-around-the-corner ever since its beginning.

* What is the general consensus among the KDE people? They tend to be less kind of company-ish and a rather pragmatic bunch.
That's even better. I'm no KDE guy, but from what I know KDE is currently looking into possible solutions for mmedia. And GStreamer works, is already used in JuK, and will be probably supported in amaroK player. Moreover, choosing GStreamer will definitely be step into direction of greater interoperability, as there is currently no other widely used framework with comparable capabilities. One of the main concerns seemed to be dependency on GLib, but since it's now required by Qt it shouldn't matter anymore. So from pragmatic point of view, GStreamer is only natural choice.

What is the general consensus among the KDE people? They tend to be less kind of company-ish and a rather pragmatic bunch.

There is no consensus yet among KDE developers. You can read in the mailing lists that there has been a lot of different point of views. We had at one point a lot of stability problems with aRts and very few developers except Stefen Westerfeld knew how to fix the code. My guess is that we will be prudent in dumping aRts and making bold experimentations.

My feeling is that the preferred solution for KDE would be a simple solid audio server for users like me who have basic needs (listening MP3, watching a DVD), more or less what Alsa provides + network transparency which is important for thin clients and a optional complex multimedia server used by content creating applications (music, video, etc.).

Since when did Qt *require* upon GLib? (Nothing against GStreamer - I think it'd be good fro KDE/GNOME to use the same framework)

Since when did Qt *require* upon GLib? (Nothing against GStreamer - I think it'd be good fro KDE/GNOME to use the same framework)
Ever since ATK support was added for accesibility reasons

In your competetion survey you don't mention the JACK framework. I'm coming from the music production & sound
enginieering domain and the JACK seems to be really very
prominsing. Especially as real-time constraints and latency
problems have been considered very well. And in some way
JACK seems to be a very promising standard for sound
applications in linux.

Admitted, it is currently not about multi media, but may be,
it can be integrated into gstreamer for sound/audio processing?

MIDI-Support is on the way. What do you think?

I think Windows, BeOS, and Mac OS X already have this market covered.

Have anyone taken a look at Mplayer G2? At least their plan seems to be to make a library, that should be easy to use for both a player and for instance a non linear editing program without losing the speed of the old mplayer but keeping compatibility with all formats. The old mplayer doesn't seem to be a contender to gstreamer, but the new one does. Of course it's not ready yet.

Just a little advice, and please don't take it the wrong way. The second to last paragrah has a sentence that does not make sense and it is run-on.
Quote:
"As said it has been a long and at times though journey, but I feel that we have now finally gotten to the point both with the library and with available applications using it, that I feel that we are able to deliver what I originally wanted. Just the continuation of the projects and efforts that are already underway is proof enough that we have succeeded in delivering what I was looking for those years ago."

There are others please proof read so as not to degrade the quality of your work.