This page is here to collect together proceedings from sessions as part of the 'Multimedia' track at the Natty UDS in Orlando, Florida.

Please add proceedings by doing the following:

• Add a new section with the name of the session.
• Add the outcomes of the session as a collection of bullet points. The goal of the proceedings is to really focus on decided outcomes, so please try to keep them crisp.

Thanks!

Proceedings

Optimize JPEG Decoding

• Agreed Major Usescases for JPEG Decoder: Browser (many small images), Image Gallery (Large Images)
• Selection of Software JPEG Decoder for optimization: IJG's Libjpeg

• JPEG Interface to be exposed: libJPEG (& OpenMAX possibly)

• Combining H/W & S/W Decoder: Decided to do it with simple GLUE layer over libJPEG.

• Runtime Selection of SoC specific execution path: Agreed to have only compile time support.
• ARM+NEON Optimizations Ideas: Agreed to not do SMP-aware codec, Agreed for doing no optimization for ARM11 and older processors.

Linaro Seeds for Multimedia WG 11.05

• If it makes sense, seed(s) could be based on the upcoming developer seed
• Research should be done to find and include tools for analysis of video/audio streams/content
• nfsclient needs to be included for easy access to test content
• Support for application logging needs to be included for those applications that have a logging feature
• There should be enough content for test / validation either as part of the image or accessible
• It was suggested that the landing teams should be the team to assemble demo materials for specific boards with the help of the Multimedia WG
• There should be enough support included to support automated testing both running and reporting results

Ubuntu One integration with Shotwell

• Make Ubuntu One's publish functionality available in Shotwell's publish options
• Make Ubuntu One's mobile photos available by default in Shotwell

Cairo backend and backend selection

https://blueprints.edge.launchpad.net/linaro-graphics-wg/+spec/multimedia-linaro-cairo-1105

• cairo is not the right place to handle runtime selection. It is up to the higher-level code to decide and use the best cairo facilities.
• We should implement a GLES2 backend, by preferably making the current experimental GL backend GL/GLES2 "agnostic". We should measure and publish the performance benefits.

Linaro Seeds for Graphics WG 11.05

• developer tools should be included in a seed with all dev packages, compilers, build essentials, automake, enough to build gles, graphical applications

• alf suggestioned the following seed layout : x11-base + dev-base -> gfx-dev + gfx-packags

• There should be two flavours, dev and test/demo (meant to boot, run simple demo)
• spandex - python based framework for gles benchmarks by nokia
• packages should include enough to build gles, applications and run them dev versions of mesa, x, freeglut, extention wrangler, dri experimental (for gallium), meego-touch (maybe), cairo, cairo-perf, gtk-perk, mesa-demos
  • could have multiple window managers etc installed but allow for easy switching for the purposes of development
• size constraints are a consideration for the purposes of a head (such as alip) which has a goal to fit into 64M flash
• automated test images should support automated test running and result reporting in general
• [action] connect with team defining development seed (tgall-foo)
• [action] need to push fix for germinate for mulitple PPAs (alf)
• [action] Jesse Barker to push graphics dev packages to tgall-foo
• [action] Jesse Barker will work gles performance tools list and supply input
• [action] check / remove SOC specific headers / images from headless, move to hwpack (tgall-foo)

Qt backend and backend selection

https://blueprints.edge.launchpad.net/linaro-graphics-wg/+spec/multimedia-linaro-qt-1105

• Some runtime selection already implemented in Qt, specifically for NEON paths.
• Qt engineering opposes runtime selection of desktop OpenGL vs. OpenGL ES due to significant runtime symbol collisions. This issue will affect all toolkits targeted with similar plans, so this issue will be planned as a separate project (GL proxy layer - possibly GLEW can help here).

Skia backend and backend selection

https://blueprints.edge.launchpad.net/linaro-graphics-wg/+spec/multimedia-linaro-skia-1105

• Runtime selection of NEON will use hardware capabilities from /proc/self/auxv (recommended method), and NEON support in toolchain should also be considered for implementation.
• Proposal will be presented to upstream Skia team.
• Barring objections from the team, solution will be implemented and submitted upstream.
• GLES backend selection deemed out of scope for this topic (see Qt proceedings).

Direct rendering support for ARM-based GPUs

https://blueprints.edge.launchpad.net/linaro-graphics-wg/+spec/multimedia-linaro-dri-1105

• Kernel support for ARM-based (non-PCI-attached in general) GPUs in upstream (great news).
• Additional memory managers (most hardware vendors have their own) are potentially repellent to upstream DRM.
• Linaro graphics WG will contact maintainer for first-hand account.
• Vendors should be urged to extend existing DRM memory management framework rather than adding their own.
• No real user space DRI issues apart from dependence upon proprietary kernel interfaces.
• Any integration of vendor Xorg video driver code will be serious uphill struggle without full release (most hardware vendors maintain closed source components to their driver stacks which prevents much upstream acceptace).
• Gallium holds much promise and will continue to be tracked for performance criteria.

Optimization opportunities in compositors

https://blueprints.edge.launchpad.net/linaro-graphics-wg/+spec/multimedia-linaro-compositors-1105

• The Linaro priority list is already using OpenGL ES, and, crucially, the EGL_KHR_image_pixmap extension (avoids custom extensions or waiting for Khronos to ratify one).
• Linaro graphics WG will confirm that EGL_KHR_image_pixmap is functionally equivalent to GLX_EXT_texture_from_pixmap.
• Compiz and kwin upstreams are working on OpenGL ES backends as well.

Special Features for Touch-based Systems

The session was focused on recording brainstorms/ideas on creative ways in which we might use touch interfaces.

• using an iPad to control an Ubuntu installation (i.e., a large trackpad)
  • "lots" of hardware that can support 2-touch (1 is probably all you need for a remote) and ubuntu
    • ideacom (4 touch), joojoo (egalax), wacom, cando, mozart
  • lots of crazy ideas that are possible...
  • advanced remotes (e.g., home automation, light shows, serious multimedia)
• creating a touch app for Ubuntu to act as a remote control for TVs
• enabling auto-flip for the Dell XT2 (and similar form-factors with screens that rotate around)
• enabling auto-rotate for tablets with the appropriate hardware
  • tabuntu? lp project (tablet-screen-rotation-support)
  • support is getting better
• separate cursors for interacting with touch and mouse simultaneously
  • you can configure X to do this
  • supported via MPX in XInput 2.0
  • ideally, the cursor for the touch source would not be visible; just the mouse cursor
• Equivalent to MS Touch Pack?
  • we don't have this
  • we do have PyMT sample apps -- a good place to start
  • any ideas? share them with PyMT mail list!

Improving audio apport symptom

• Decided to go ahead and make audio apport symptom more detailed, ask more initial questions to aid triaging and in some cases help users to help themselves
• Investigate whether we can add a shortcut to the audio apport symptom from Gnome's volume control

Secure, low-latency audio that just works

• Integrate Jack and PulseAudio even more, so that PulseAudio adds ports to Jack whenver Jack is started

• Investigate feasibility of integrating a currently existing command-line prototype of recording VoIP calls

• Investigate whether we can help FFADO (FireWire Audio) to complete their work moving FFADO into standard ALSA at kernel level and how much it would take to complete the effort.

• Investigate if it's possible to make RealTimeKit hand out memlock permissions in a safe way, to keep people from giving themselves memlock permissions (which is bad from a security point of view).

Functional and performance graphics benchmarks

https://blueprints.edge.launchpad.net/linaro-graphics-wg/+spec/multimedia-linaro-benchmark-1105

• Base performance will be tracked using glmark[-es2], cairo-perf-trace, gtkperf, x11perf.
• Additional test cases (both performance and functionality) will be added based upon user experience problem reports.
• Further investigation into spandex framework for additional directed functional tests.

General Xorg Planning

• Version planning:
  • Mesa 7.10 (planned release Dec 2010)
  • Decision on staying with Xserver 1.9 or switching to 1.10 (planned release Feb 2011) postponed pending discussion with Chase Douglas about multitouch requirements.
  • libdrm / xserver-xorg-video-ati / xserver-xorg-video-intel / xserver-xorg-video-nouveau:
    • No big changes on the horizon, so plan on latest upstream releases at Feature Freeze.
• Gallium discussion:
  • Upstream has switched default r300 DRI driver from the classic r300c to the gallium r300g driver. r300g requires KMS support, so this presents problems with 3D when using usermodesetting.
  • UMS is still useful - there's an (obscure) feature missing from KMS, UMS is a useful debugging fallback and workaround for bugs in KMS.
  • Also r600c / r600g is interesting, but r600g isn't yet default upstream.
  • PoR: Support 3D by shipping both r300c+r300g and r600c+r600g with a distro-patch to select appropriately based on xorg.conf and available KMS support.
• Input redirection:
  • Compiz would like input-redirection support in the X server to support input to transformed windows (zoom, exposé, etc).
    • Currently has hacks to make at least zoom work properly, but it's not possible to make these hacks work with touch.
  • Patches which apply to current X server exist.
  • Chase to investigate patches and decide whether to push for upstream.
• xvfb discussion:
  • xvfb is possibly broken in Maverick, and this breaks build-time testsuites for some packages.
  • Security team needs this to work all the time.
  • PoR: Investigate whether xvfb tests can be added to xserver build-time checks. Talk with Debian upstream about integrating these tests into the xserver packaging.