Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.


This specification discusses the needs of Desktop Developers in terms of high level API's for audio and video handling.


GStreamer; provides Linux and Unix systems today with a very powerfull and full featured API for developing multimedia applications. The wide range of applications available and under rapid development currently shows that GStreamer has hit the sweet spot in terms of features and API for the development of multimedia applications. Due to this and the adoption by GNOME as its multimeda subsystem GStreamer is the de-facto multimedia framework for Unix and Linux today.

However for applications which primary purpose is not dealing with media something more high level is needed as it becomes to complex to do 'simple' stuff.

Use cases

Examples of applications which could benefit from a more high level API is Games who needs sound effects, photo album applications which want to also thumbnail video clips, general desktop system sounds. Word processors and presentation applications wanting to embed a video clip. This API could also be useful for low-end applications, like a simple desktop sound recorder (grecord from gnome-media being an example).


This higher level API should build upon and blend into the low-level GStreamer API. So that when application developers need to go beyond the functionality of the higher level objects they have a natural migration path.


  • Phonon; . is designed with some of these thoughts in mind and would probably be a good starting point.

  • The solution should be as desktop agnostic as possible
  • Easily bindable/language agnostic


  • One idea is to implement as much of a GStreamer backend to Phonon in C/glib as possible. That way Qt/Phonon API and a glib-style API can both be implemented on top of it with as little extra work possible.


Wim Taymans of Fluendo will start working on a GStreamer backend to Phonon as soon as he returns from the Mountainview conference and will use the feedback from this session as input for the design process and initial work. Anyone else wanting to contribute to the coding effort are of course welcome.

BoF agenda and discussion

  • BOF on Bluesky ideas for tighter multimedia integration in the desktop (to establish possible usecases)
  • Wim will talk about the API's offered by Phonon and GStreamer
  • Discussion on what API's we need for our usecases and to what degree the types of API offered by Phonon or GStreamer today satisfy those.
  • API integration points with solutions such as PulseAudio;

Other links


MultimediaAPI (last edited 2008-08-06 16:18:33 by localhost)