Sound

Differences between revisions 1 and 73 (spanning 72 versions)
Revision 1 as of 2010-02-01 19:34:57
Size: 6306
Editor: unassigned
Comment: moved from private wiki; -symbolic icons changed temporarily to -panel
Revision 73 as of 2013-02-08 18:01:21
Size: 28368
Editor: mpt
Comment: clarifies when the microphone item is present
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
There should be a unified sound menu taking the place of the Gnome `mixer_applet2`. We also offer this design to the Kubuntu and KDE communities as a foundation for a possible replacement of the `kmix` applet. <<Include(Ayatana/PageTemplate/Header)>>

So that people can easily change sound volume and control music playback, there should be a unified sound menu taking the place of the Gnome `mixer_applet2` and the notification area items for Rhythmbox, Banshee, and Amarok.

{{attachment:cases.png}} {{attachment:Screenshot-2.png}}

''Errata:
 * The scrub bar should not be present, and the track-specific items should go between the track info and the playback controls.
 * See [[#microphone-item|The microphone item]] for details of when the microphone item is present.''

We also offer this design, and its associated APIs, to the Kubuntu and KDE communities as a foundation for a possible replacement of the `kmix` applet.
Line 8: Line 18:
 * A new named icon, `audio-volume-zero-panel`: a speaker with zero sound waves emanating. (Eventually this will be `audio-volume-zero-symbolic`.)
 * A new named icon, `audio-volume-muted-blocking-panel` (sound is playing, but it is muted): the same as `audio-volume-zero-symbolic`, but red. (We are aware that this color-only difference will not be noticable for monochromats and some dichromats.) (Eventually this will be `audio-volume-muted-blocking-symbolic`.)
 * A new named icon, `audio-output-none-panel` (no output devices are available): the same as `audio-volume-zero`, but in outline form. `audio-output-none-symbolic`.)
 * A new named icon, `audio-volume-low-zero-symbolic`: a speaker with zero sound waves emanating.
 * A new named icon, `audio-volume-muted-blocking-symbolic` (sound is playing, but it is muted): the same as `audio-volume-low-zero-symbolic`, but in the error color. (We are aware that this color-only difference will not be noticable for monochromats and some dichromats.)
 * A new named icon, `audio-output-none-symbolic` (no output devices are available): the same as `audio-volume-zero`, but in outline form.
 * Zero-volume and maximum-volume icons for a microphone. ''What should their names be?''
 * Custom scrub bar rendering.
 * Play, Pause, Previous, and Next button rendering.

<<Anchor(presence)>>
== Menu presence ==

{{attachment:sound-menu-setting.png}}

At the bottom left corner of the System Settings “Sound” panel should be a checkbox, “Show sound volume in the menu bar”. It should be checked by default for a new user account. Whenever it is checked, the sound menu should be [[MenuBar#system-status|present in the menu bar]].
Line 14: Line 34:
||<tablestyle="float: right; margin: 0 0 1em 1em;" style="border: none;">{{attachment:sound-menu.jpg}}||

The menu should give easiest access to the volume of the ''primary output device'' (the headphones if any are plugged in, or the main speakers otherwise). Other devices can be accessed through the
Sound Preferences window.

Except where specified, the menu should inherit all the normal behavior and keyboard navigation of a standard menu. '''Test case:''' With the volume slider item highlighted, press Page Down. The highlight should move to the bottom item in the menu. '''Test case:''' With the “Mute All” item highlighted, press Left. The sound menu should close and the menu to its left in the panel should open.
The menu should give easiest access to the volume of the [[PrimarySoundOutput|primary sound output device]] and the primary sound input device. Other devices can be accessed through the Sound panel of System Settings.

Except where specified, the menu should inherit all the normal behavior and keyboard navigation of a standard menu. '''Test case:''' With the volume slider item highlighted, press Page Down. The highlight should move to the bottom item in the menu. '''Test case:''' With the “Mute” item highlighted, press Left. The sound menu should close and the menu to its left in the panel should open.
Line 22: Line 40:
If no available output devices have been detected (yet), the title of the menu should be `audio-output-none-symbolic` (in the default theme, an outline version of the speaker icon). Whenever there is no primary sound output device (because no available output devices have been detected yet), the title of the menu should be `audio-output-none-symbolic` (in the default theme, an outline version of the speaker icon), and its accessible name should be “Volume (no devices)”.
Line 26: Line 44:
 * The title of the menu should be an icon roughly representing the current volume for the primary output device: `audio-volume-muted-panel`, `audio-volume-zero-panel`, `audio-volume-low-panel`, `audio-volume-medium-panel`, or `audio-volume-high-panel` (which, in the default theme, should be a speaker icon with zero to three sound waves). The icon should be updated even if the volume is being changed while the menu is open.

 * If the sound is muted and the computer begins “playing” sound, the icon should switch instantly to `audio-volume-muted-blocked-panel` (in the default theme, a red version of `audio-volume-muted-panel`), persist for five seconds after sound stops playing, then take one second to fade back to `audio-volume-muted-panel`. 
 * The title of the menu should be an icon roughly representing the current volume for the primary output device: `audio-volume-muted-panel`, `audio-volume-low-zero-symbolic`, `audio-volume-low-symbolic`, `audio-volume-medium-symbolic`, or `audio-volume-high-symbolic` (which, in the default theme, should be a speaker icon with zero to three sound waves). The icon should be updated even if the volume is being changed while the menu is open. The accessible name should be of the form “Volume (74%)”.

 * If the sound is muted and the computer begins “playing” sound, the icon should switch instantly to `audio-volume-muted-blocked-symbolic` (in the default theme, a red version of `audio-volume-muted-panel`), persist for five seconds after sound stops playing, then take one second to fade back to `audio-volume-muted-panel`.
Line 32: Line 50:
=== “Mute All”/“Unmute” ===

The first item, “Mute All”, makes it easy to urgently silence unwanted sound using a pointing device. If there are no available sound output devices, the item should be insensitive. Activating the item should mute all output devices, while remembering their previous volumes. After being chosen, “Mute All” should change to “Unmute”.

“Unmute” should return all output devices to the volume settings they had immediately before “Mute All” was selected. Either choosing “Unmute”, ''or'' setting the volume of any output device manually, should change “Unmute” back to “Mute All”.

=== Volume item ===
=== “Mute”/“Unmute” ===

The first item, “Mute”, makes it easy to urgently silence unwanted sound using a pointing device. If there are no available sound output devices, the item should be insensitive. Activating the item should mute all output devices, while remembering their previous volumes. After being chosen, “Mute” should change to “Unmute”.

“Unmute” should return all output devices to the volume settings they had immediately before “Mute” was selected. Either choosing “Unmute”, ''or'' changing any volume setting manually, should change “Unmute” back to “Mute”.

=== Output volume item ===
Line 43: Line 61:
 1. A borderless, mouseover-less button that sets the volume to zero (as distinct from muting it). The icon for the button should be `audio-volume-zero` (in the default theme, a speaker with no sound waves).
 1. A slider that lets you set the volume more precisely.
 1. A borderless, mouseover-less button that sets the volume to zero (as distinct from muting it). The icon for the button should be `audio-volume-low-zero` (in the default theme, a speaker with no sound waves).
 1. A slider that lets you set the volume more precisely. Whenever sound is muted, the slider should temporarily be at zero. Unmuting should return the slider to its previous position.
Line 51: Line 69:
 * Ctrl Left and Ctrl Right should decrease the volume to 0 percent or increase it to 100 percent, respectively. (This overrides the normal large-step behavior of Ctrl Left and Ctrl Right with GTK sliders.)
Line 56: Line 73:
=== “Sound Preferences…”/“Sound Settings…” ===

The menu should end with a separator, followed by a “Sound Preferences…” item that opens the Gnome Sound Preferences window. (A Kubuntu version of this menu item would be “Sound Settings…”, and would open the Sound panel in KDE System Settings.) Launch feedback for this item should be exactly the same as when you launch an application any other way.
'''Implementation:''' The Pulse``Audio async API takes care of detecting different devices and whether there is sound coming out of them. There will be need to be a connection to either Pulse or HAL to detect the addition of new devices. This could be something like plugging in a new USB headset.

<<Anchor(microphone-item)>>
=== Microphone volume item ===

||<tablestyle="float: right; margin: 0 0 1em 1em;" style="border:none;">{{attachment:microphone-slider.png}}||

If a VoIP (`media.role=phone`) or sound recording (`media.role=production`) audio stream is active, and the computer has a detected microphone, the output volume item should be followed by a microphone volume item. The microphone volume item should look and behave exactly the same as the output volume item, except that the button icons should feature a microphone rather than a speaker.

<<Anchor(compliance)>>
=== Music player section ===

Any music player that advertises itself over [[http://mpris.org/ Mpris]] should have its own section in the sound menu. Multiple compliant music players may result in multiple music sections, ordered alphabetically by application `Name`. (In future, there may even be multiple sections for the same player.)

A compliant player should not be present in the menu merely because it is installed, but should insert itself as soon as you start actively using it (''e.g.'' playing something with it for the first time, or adding music to its library). The player’s own settings interface should also have a checkbox for whether the player should be present in the sound menu right now:

    ||<tablestyle="background: #ddd; color: #000; font: caption sans-serif;" style="border: none;">☑ Show Foo``Player in the sound menu ||

''Future work: Add a “Where?” button that shows you where the sound menu is.''

A compliant player should also keep playing if you close its window while it is playing; exit if you close its window while it is not playing; and remember exact state across sessions, so that after exit and relaunch it is as if the player had never exited.

A music section should consist of [[#music-player-item|an item for the music player itself]], followed by any or all of [[#track-custom-items|track-specific custom items]], [[#playback-item|a playback item]], [[#playlist-submenu|a playlist submenu]], and [[#player-custom-items|player-global custom items]]. Some of these items are present only when the player is running.

For the purpose of the following sections that describe these items, the '''active track''' for a player is the track that is playing, that is paused, or that would play if the Play command was given (if this is known ahead of time). At any time, a player may or may not have an active track.

<<Anchor(music-player-item)>>
==== Music player item ====

This item should be present regardless of whether the player is running.

The first row of the item should consist of the application’s icon and `Name`. A player should also be able, in rare cases, to specify custom text in place of the application name (by changing the 'Identity' property string on the Mpris root interface to the full string desired).

Choosing the item should launch the player; if the player is already running, it should open or focus an existing instance. Whenever a running player does not have a sensible main window that could be opened or focused (as indicated by Mpris [[http://www.mpris.org/2.1/spec/Root_Node.html#Property:CanRaise|canRaise()]] = false), the player item should be insensitive.

{{attachment:player-item-playing.png}}

If the player is running and has an active track, the music player item should also include, below the player icon and name, the track or album art and three rows of text. If no track or album art is available, it should fall back to a generic track icon. The text should be on the leading side, and top-aligned next to the art. It should consist of the track name, artist, and album, one on each line (aligned the same way as the rest of the menu, regardless of what language each string is in). For each line, if there is not enough room in the menu to display all the text, it should be ellipsized in the middle; and if the data is not known, the line should be left blank. The player name and track data combined should be highlightable and activatable like a single normal menu item.

There should be a D-Bus API for exporting info about the playing track (not the same as the active track) for use elsewhere (for example, setting a custom IM status).

''We need precise fonts, sizes, and spacing specified for these elements.''

<<Anchor(scrub-bar-item)>>
## ==== Scrub bar item ====

## If the music player is not running, or if it is running but has no active track, this item should not be present at all.

## If there is an active track, this item should be a scrub bar, with the time elapsed at its leading end (''e.g.'' “0:31”), and time remaining in at least minutes and seconds at its trailing end (''e.g.'' “1:31:54”). There should be enough space reserved at each end of the scrub bar to show the full length of that track (so if the menu happens to be open when tracks change, the scrub bar may change length). The scrub bar and times should update live while the menu is open.

## The scrub bar item should be navigable with a pointing device or keyboard in the same way as a normal menu item, but should not be activatable. In addition:
## * The scrub bar should be manipulable with the mouse (by dragging the thumb, or clicking in the trough) regardless of whether the item is highlighted.
## * When the item is highlighted, the left and right arrow keys should scrub backwards or forwards through the track, accelerating as long as the key is held down.

==== Track-specific custom items ====

The API should let the music player provide up to three track-specific custom menu items — for example, for buying, “liking”, or “banning” the song currently playing. Any of these items may be a submenu containing other items.

<<Anchor(playback-item)>>
==== Playback item ====

The playback item should be present if the player can be controlled externally ([[http://standards.freedesktop.org/mpris-spec/2.2/Player_Interface.html#Property:CanControl|CanControl]]). The playback item should consist of a large Play/Pause button in the middle, with Back and Forward buttons (using ◀◀ Rewind and ▶▶ Fast-Forward symbols) on the sides.

If something is playing, the main button should be Pause. If nothing is playing, the main button should be Play. The Back button should be sensitive whenever the player can seek ([[http://standards.freedesktop.org/mpris-spec/2.2/Player_Interface.html#Property:CanSeek|CanSeek]]) ''or'' go to a previous track ([[http://standards.freedesktop.org/mpris-spec/2.2/Player_Interface.html#Property:CanGoPrevious|CanGoPrevious]]). And the Forward button should be sensitive whenever the the player can seek ''or'' go to a next track ([[http://standards.freedesktop.org/mpris-spec/2.2/Player_Interface.html#Property:CanGoNext|CanGoNext]]) (bug Bug:782060).

When using a pointing device, whichever button is under the pointer should highlight in a similar way to a normal menu item in the same theme. When using the keyboard, the Play/Pause button alone should highlight in the same way. Pressing the Space key should visibly activate the Play/Pause button, and the Left and Right keys should visibly activate the Back and Forward buttons respectively. None of the elements should have a visible focus ring, and Tab and Shift Tab should not do anything.

When they are present and sensitive, the functions and accessible labels of the buttons should be as follows.

 * The Play/Pause button should pause or resume the current track, depending on whether it is currently playing. Its accessible label should be “Pause” or “Play“, respectively.

 * If a track is currently playing, the Back button should rewind the track for however long it is pushed. If a track is not currently playing, ''or'' if the Back button is released within 0.5 seconds of being pushed, then the player should navigate to the beginning of the previous track if the current track is playing and at less than two seconds from its start, or the beginning of the current track otherwise. The accessible label for the button should be “Back (hold to rewind)” if a track is currently playing, or just “Back” otherwise.

 * If a track is currently playing, the Forward button should fast-forward the track for however long it is pushed. If a track is not currently playing, ''or'' if the Forward button is released within 0.5 seconds of being pushed, then the player should navigate to the beginning of the track ''after'' the next one if the current track is playing and at less than two seconds from its end, or the beginning of the next track otherwise. The accessible label for the button should be “Forward (hold to fast-forward)” if a track is currently playing, or just “Forward” otherwise.

<<Anchor(track-custom-items)>>
||<tablestyle="float: right; margin: 0 0 1em 1em;" style="border: none;">{{attachment:sound-menu-v2.jpg}}||

<<Anchor(playlist-submenu)>>
==== Playlist submenu ====

The MPRIS API should let the player expose icons and names of playlists, even when the player is not running.

The playlist item should be present in the menu only if the music player is currently exposing any playlists. If the player also indicates that it is playing (or is paused partway through) a playlist, the submenu title should be the name of the playlist. Otherwise, it should be “Choose Playlist”.

The submenu itself should contain items named for each of the playlists. Selecting an item should start playing that playlist.

<<Anchor(player-custom-items)>>
==== Player-global custom items ====

The API should let the music player provide up to three player-global custom menu items, even when the player is not running. Any of these items may be a submenu containing other items, for setting equalizer settings for example.

=== “Sound Settings…” ===

The menu should end with a separator, followed by a “Sound Settings…” item that opens the Sound panel in System Settings. Launch feedback for this item should be exactly the same as when you launch an application any other way.

(If PulseAudio is not running when you choose “Sound Settings…”, `gnome-volume-control` should try to start it. If it fails, it should invoke Apport during the development cycle, or an error alert in a stable release. ''This is outside the scope of the menu; it should be part of the specification for the Sound Settings themselves.'')
Line 63: Line 174:

== Music player integration ==

=== Registration process ===
==== For Natty ====
In order to register each player needs to implement the MPRIS2 spec. The service which controls the menu automatically detects MPRIS clients appearing on DBus and reacts accordingly.
It is essential that the implementation populates the DesktopEntry property on the MPRIS root interface. Access to the Desktop file is mandatory inorder to register successfully.

Registration will need to be disabled if a user would prefer for a certain player not to be controllable from the menu. Guidelines as to how this preference is to be represented in the UI of the client (specifically Rhythmbox, Banshee or Amarok) has been included below. We would like to provide a unified music player experience on the Ubuntu platform therefore we strongly urge application developers to follow this spec.

To remove a player from the menu the client can either use the dbus method 'blacklist-media-player(string desktop, bool blacklist)' on 'com.canonical.indicators.sound' (as of release 0.5.9) or alternatively it will need to write to the indicator-sound gsettings. The settings id is "com.canonical.indicators.sound" and the key is "blacklisted-media-players". The client should add the name of the desktop file (just the name, not the full path and minus the .desktop suffix) to this array of strings. The indicator sound service will remember each client (provided that it is not blacklisted) which has successfully registered so that when the media client is not running it will still appear in a hibernated state in the menu. Any successful changes to the blacklisted gsettings should be automatically reflected in the menu. A player can query its blacklisted state using the dbus api 'IsBlacklisted(string desktop-id)' (this will return a boolean depending on its presence in the blacklist obviously). Again alternatively you can just talk directly with gsettings. Please use the gsettings (either directly or through the dbus api) to save and fetch the state of a players permissions inorder to minimize bugs.

In order to test client registration, application developers should fetch the latest lp:indicator-sound release. As of release 0.5.3 this new way of registering has been implemented.

==== For Maverick ====

Each player should use a similar approach used with Ted Gould’s app indicators in order to enable remote control from the sound menu.

This is very straightforward. Here is an example of the mandatory override method of a Rhythmbox plugin in Vala which basically establishes Ayatana registration.

    public override void activate (RB.Shell shell) {

        debug("Rhythmbox - Hello Ayatana world\n");

        server = Server.ref_default();

        server.set("type", "music.rhythmbox");

        server.set_desktop_file("/usr/share/applications/rhythmbox.desktop");

        server.show();
 }

It is important to note that the name of the application which is the suffix on the server type string should be the first word in the filename for the desktop file. Therefore for rhythmbox which has rhythmbox.desktop as the name of its desktop file uses "music.rhythmbox". Similarly Banshee which has a desktop file called banshee-1.desktop uses "music.banshee" as its server type string. The reason for this is that the service needs to be able to identify the music player shutdown from the libindicate 'server_removed' callback which passes the server type as an identifier. The complete registration the client also needs to have an MPRIS root and player interface available.

Bindings for libindicate exist for Python, Vala, Mono and C are available. https://launchpad.net/libindicate. Please contact Conor if you need any help with this (https://launchpad.net/~cjcurran) .

Registration will need to be disabled if a user would prefer for a certain player not to be controllable from the menu. Guidelines as to how this preference is to be represented in the UI of the client (specifically Rhythmbox, Banshee or Amarok) has been included below. We would like to provide a unified music player experience on the Ubuntu platform therefore we strongly urge application developers to follow this spec.

=== Rhythmbox ===

 * Rhythmbox should be altered to comply with [[#compliance|the standard behavior]]: it should keep playing if closed while playing, it should exit if closed while not playing, and it should restore location and other state on relaunching.

 * The “Status Icon” plug-in for Rhythmbox should be replaced by notifications integrated into Rhythmbox itself. The layout of the “General” tab in Rhythmbox’s “Preferences” window should be adjusted — by changing the “Browser Views” radio buttons to a “Browser views show:” option menu, and by changing the two columns of “Visible Columns” checkboxes to three columns of “Track listings show:” checkboxes — to make room for a “Notifications for playing tracks:” option menu. This menu should contain three items: “Always”, “When Rhythmbox is hidden”, and “Never”.

 On first launch of a version of Rhythmbox that integrates notifications in this way, the configuration for the “Status Icon” plug-in (if any) should be mapped to the equivalent value for the “Notifications for playing tracks:” menu. If the plug-in had been disabled, the value of the new setting should be “Never”.

 * The “General” tab in Rhythmbox’s “Preferences” window should also include a “Show Rhythmbox in the sound menu” checkbox. The option should be off by default for a new user account, but should turn itself on automatically when you first play something or add items to the music library.

 ||<style="border: none; text-align: right; vertical-align: bottom;">{{attachment:rhythmbox-preferences-old.png}}||<style="border: none; vertical-align:middle;">↘||<|2 style="border: none; vertical-align: middle;">{{attachment:rhythmbox-preferences-new.jpg}}||
 ||<style="border: none; text-align: right; vertical-align: top;">{{attachment:rhythmbox-plug-in-old.png}}||<style="border: none;">↗||

=== Banshee ===

 * Banshee should be altered to comply with [[#compliance|the standard behavior]]: it should keep playing if closed while playing, it should exit if closed while not playing, and it should restore location and other state on relaunching.

 * Banshee should no longer have a notification area item.

 * At the end of the “Miscellaneous” section in the “General” tab in Banshee’s “Preferences” window should be a “Show Banshee in the sound menu” checkbox. The option should be off by default for a new user account, but should turn itself on automatically when you first play something or add items to the music library.
 {{attachment:banshee-preferences-old.png}}

||<tablestyle="background: #ffc; color: #000;"> Not targeted for Maverick.||

 * In the sound menu, Banshee should have one track-specific item: a “Rating” submenu, with “☆☆☆☆☆”, “★☆☆☆☆”, “★★☆☆☆”, “★★★☆☆”, “★★★★☆”, and “★★★★★” radio items, that sets the rating for the active track.

||<tablestyle="background: #ffc; color: #000;"> Not targeted for Maverick.||

 * In the sound menu, Banshee should have two player-specific items:
  * a “Repeat” submenu, with “Off”, “This Track”, and “All” radio items;
  * a “Shuffle” submenu, with “Off”, “by Song”, “by Artist”, “by Album”, “by Rating”, and “by Score” radio items.

=== Amarok ===

Whenever Amarok is running in an environment where the sound menu is present:

 * It should comply with [[#compliance|the standard behavior]]: it should keep playing if closed while playing, it should exit if closed while not playing, and it should restore location and other state on relaunching.

 * It should no longer have a notification area item.

 * In the “General” section of the “Configure - Amarok” window, the “Show tray icon” checkbox should be replaced by a “Show Amarok in the sound menu” checkbox. The option should be off by default for a new user account, but should turn itself on automatically when you first play something or add items to the music library.
 {{attachment:amarok-configure-old.png}}


=== MPD / Music Player Daemon ===

An external project exists that integrates MPD with the sound menu:
https://launchpad.net/mpd-sound-menu
 {{attachment:mpd-sound-menu.png}}

Another similar project is mpDris2:
https://github.com/eonpatapon/mpDris2

== Upstream implications ==

 * Development of an API
 * Media Player changes:
   * Support API
   * Use notification area
   * Amarok will need to detect whether the sound menu is present.

Specifications involving Free``Desktop:

 * MPRIS (see [[#mpris-amarok|the Amarok example]]) (''broken link?'')
   * Add track specific items
   * Needs to be able to query playlists (and start them)
   * The new version of MPRIS can be found here http://www.mpris.org/2.0/spec/
 * Application Shortcuts (Modify Desktop File spec)
   * http://standards.freedesktop.org/desktop-entry-spec/latest/
   * Need usecases and justification (Mostly on Messaging Menu and Sound Menu)

== Future work ==

 * Integrate Totem.
 * Show multiple sound outputs separately.
 * Sound configuration and management.
 * Consider http://pulseaudio.org/wiki/WritingVolumeControlUIs#Colouredvolumesliders
 * Should equalizer settings go into the system? (or should they be track-specific/application-specific)
 * Respect surround-sound in the menu and in the settings window
 * Fade out other audio automatically if you receive a VoIP call? (Where would a setting for this go?)

== Risks ==

 * People may be disconcerted that something appearing to be a volume menu contains items for a music player. Or they may think that the presence of those items means that the volume is specific to the music player rather than system-wide. We should perform user testing once we have a basic implementation.

== Alternative proposed designs ==

 * [[http://picasaweb.google.com/100804433705878937883/Mockups|Wyatt]]
 * [[http://www.foopics.com/showfull/c7e1ce704b9de523e51ec5d4a3569003|Thorsten Wilms]]

See also “[[http://dribbble.com/shots/358642-Music-Controller|Music controller]]” by Ryan Henricus.

== Unresolved issues ==

 * We need a high-fidelity visual mockup to help us decide the width of the track metadata and scrub bar, and therefore the overall width of the menu.
 * We need an animated mockup of each item being highlighted as you navigate through it.

=== Proposed playlists extension ===

So that people can easily change sound volume and control music playback, there should be a unified sound menu taking the place of the Gnome mixer_applet2 and the notification area items for Rhythmbox, Banshee, and Amarok.

cases.png Screenshot-2.png

Errata:

  • The scrub bar should not be present, and the track-specific items should go between the track info and the playback controls.
  • See The microphone item for details of when the microphone item is present.

We also offer this design, and its associated APIs, to the Kubuntu and KDE communities as a foundation for a possible replacement of the kmix applet.

Artwork requirements

Beyond icons that are already included in themes:

  • A new named icon, audio-volume-low-zero-symbolic: a speaker with zero sound waves emanating.

  • A new named icon, audio-volume-muted-blocking-symbolic (sound is playing, but it is muted): the same as audio-volume-low-zero-symbolic, but in the error color. (We are aware that this color-only difference will not be noticable for monochromats and some dichromats.)

  • A new named icon, audio-output-none-symbolic (no output devices are available): the same as audio-volume-zero, but in outline form.

  • Zero-volume and maximum-volume icons for a microphone. What should their names be?

  • Custom scrub bar rendering.
  • Play, Pause, Previous, and Next button rendering.

sound-menu-setting.png

At the bottom left corner of the System Settings “Sound” panel should be a checkbox, “Show sound volume in the menu bar”. It should be checked by default for a new user account. Whenever it is checked, the sound menu should be present in the menu bar.

The menu should give easiest access to the volume of the primary sound output device and the primary sound input device. Other devices can be accessed through the Sound panel of System Settings.

Except where specified, the menu should inherit all the normal behavior and keyboard navigation of a standard menu. Test case: With the volume slider item highlighted, press Page Down. The highlight should move to the bottom item in the menu. Test case: With the “Mute” item highlighted, press Left. The sound menu should close and the menu to its left in the panel should open.

Title

Whenever there is no primary sound output device (because no available output devices have been detected yet), the title of the menu should be audio-output-none-symbolic (in the default theme, an outline version of the speaker icon), and its accessible name should be “Volume (no devices)”.

If there are any available output devices:

  • The title of the menu should be an icon roughly representing the current volume for the primary output device: audio-volume-muted-panel, audio-volume-low-zero-symbolic, audio-volume-low-symbolic, audio-volume-medium-symbolic, or audio-volume-high-symbolic (which, in the default theme, should be a speaker icon with zero to three sound waves). The icon should be updated even if the volume is being changed while the menu is open. The accessible name should be of the form “Volume (74%)”.

  • If the sound is muted and the computer begins “playing” sound, the icon should switch instantly to audio-volume-muted-blocked-symbolic (in the default theme, a red version of audio-volume-muted-panel), persist for five seconds after sound stops playing, then take one second to fade back to audio-volume-muted-panel.

  • As a hidden treasure, when the pointer is over the title, whether the menu is open or not, clicking a mousewheel up or down should increase or decrease the volume respectively, to the nearest 10-percent step.

“Mute”/“Unmute”

The first item, “Mute”, makes it easy to urgently silence unwanted sound using a pointing device. If there are no available sound output devices, the item should be insensitive. Activating the item should mute all output devices, while remembering their previous volumes. After being chosen, “Mute” should change to “Unmute”.

“Unmute” should return all output devices to the volume settings they had immediately before “Mute” was selected. Either choosing “Unmute”, or changing any volume setting manually, should change “Unmute” back to “Mute”.

Output volume item

If there is a primary sound output device, the menu should contain an item for changing its volume, followed by a separator. The Up and Down keys should navigate between the volume item and other items, just as for a normal menu item. The item should highlight when navigated to, or on mouseover, just as normal menu items do, but the item should not be activatable (clicking it or pressing Enter should not close the menu).

The item should contain three individual interactive elements.

  1. A borderless, mouseover-less button that sets the volume to zero (as distinct from muting it). The icon for the button should be audio-volume-low-zero (in the default theme, a speaker with no sound waves).

  2. A slider that lets you set the volume more precisely. Whenever sound is muted, the slider should temporarily be at zero. Unmuting should return the slider to its previous position.
  3. A borderless, mouseover-less button that sets the volume to maximum. The icon for this button should be audio-volume-high (in the default theme, a speaker with three sound waves).

Whenever the volume item is highlighted:

  • The Left and Right arrow keys should instantly decrease or increase the volume, respectively, to the nearest 5 percent step.
  • The “-” and “+” keys should instantly decrease or increase the volume, respectively, to the nearest 5 percent step.
  • Rolling a mousewheel up or down should increase or decrease the volume respectively, 10% per click.
  • None of the elements should have a visible focus ring, and Tab and Shift Tab should not do anything. (There are enough other ways to activate each of the elements, and a focus ring inside part of a highlighted menu item would be ugly.)

Changing the volume with the volume item should provide the same audio feedback as changing the volume any other way (such as with the volume keys on the keyboard), but should not generate notification bubbles.

Implementation: The PulseAudio async API takes care of detecting different devices and whether there is sound coming out of them. There will be need to be a connection to either Pulse or HAL to detect the addition of new devices. This could be something like plugging in a new USB headset.

Microphone volume item

microphone-slider.png

If a VoIP (media.role=phone) or sound recording (media.role=production) audio stream is active, and the computer has a detected microphone, the output volume item should be followed by a microphone volume item. The microphone volume item should look and behave exactly the same as the output volume item, except that the button icons should feature a microphone rather than a speaker.

Music player section

Any music player that advertises itself over http://mpris.org/ Mpris should have its own section in the sound menu. Multiple compliant music players may result in multiple music sections, ordered alphabetically by application Name. (In future, there may even be multiple sections for the same player.)

A compliant player should not be present in the menu merely because it is installed, but should insert itself as soon as you start actively using it (e.g. playing something with it for the first time, or adding music to its library). The player’s own settings interface should also have a checkbox for whether the player should be present in the sound menu right now:

  • ☑ Show FooPlayer in the sound menu

Future work: Add a “Where?” button that shows you where the sound menu is.

A compliant player should also keep playing if you close its window while it is playing; exit if you close its window while it is not playing; and remember exact state across sessions, so that after exit and relaunch it is as if the player had never exited.

A music section should consist of an item for the music player itself, followed by any or all of track-specific custom items, a playback item, a playlist submenu, and player-global custom items. Some of these items are present only when the player is running.

For the purpose of the following sections that describe these items, the active track for a player is the track that is playing, that is paused, or that would play if the Play command was given (if this is known ahead of time). At any time, a player may or may not have an active track.

Music player item

This item should be present regardless of whether the player is running.

The first row of the item should consist of the application’s icon and Name. A player should also be able, in rare cases, to specify custom text in place of the application name (by changing the 'Identity' property string on the Mpris root interface to the full string desired).

Choosing the item should launch the player; if the player is already running, it should open or focus an existing instance. Whenever a running player does not have a sensible main window that could be opened or focused (as indicated by Mpris canRaise() = false), the player item should be insensitive.

player-item-playing.png

If the player is running and has an active track, the music player item should also include, below the player icon and name, the track or album art and three rows of text. If no track or album art is available, it should fall back to a generic track icon. The text should be on the leading side, and top-aligned next to the art. It should consist of the track name, artist, and album, one on each line (aligned the same way as the rest of the menu, regardless of what language each string is in). For each line, if there is not enough room in the menu to display all the text, it should be ellipsized in the middle; and if the data is not known, the line should be left blank. The player name and track data combined should be highlightable and activatable like a single normal menu item.

There should be a D-Bus API for exporting info about the playing track (not the same as the active track) for use elsewhere (for example, setting a custom IM status).

We need precise fonts, sizes, and spacing specified for these elements.

Track-specific custom items

The API should let the music player provide up to three track-specific custom menu items — for example, for buying, “liking”, or “banning” the song currently playing. Any of these items may be a submenu containing other items.

Playback item

The playback item should be present if the player can be controlled externally (CanControl). The playback item should consist of a large Play/Pause button in the middle, with Back and Forward buttons (using ◀◀ Rewind and ▶▶ Fast-Forward symbols) on the sides.

If something is playing, the main button should be Pause. If nothing is playing, the main button should be Play. The Back button should be sensitive whenever the player can seek (CanSeek) or go to a previous track (CanGoPrevious). And the Forward button should be sensitive whenever the the player can seek or go to a next track (CanGoNext) (bug 782060).

When using a pointing device, whichever button is under the pointer should highlight in a similar way to a normal menu item in the same theme. When using the keyboard, the Play/Pause button alone should highlight in the same way. Pressing the Space key should visibly activate the Play/Pause button, and the Left and Right keys should visibly activate the Back and Forward buttons respectively. None of the elements should have a visible focus ring, and Tab and Shift Tab should not do anything.

When they are present and sensitive, the functions and accessible labels of the buttons should be as follows.

  • The Play/Pause button should pause or resume the current track, depending on whether it is currently playing. Its accessible label should be “Pause” or “Play“, respectively.
  • If a track is currently playing, the Back button should rewind the track for however long it is pushed. If a track is not currently playing, or if the Back button is released within 0.5 seconds of being pushed, then the player should navigate to the beginning of the previous track if the current track is playing and at less than two seconds from its start, or the beginning of the current track otherwise. The accessible label for the button should be “Back (hold to rewind)” if a track is currently playing, or just “Back” otherwise.

  • If a track is currently playing, the Forward button should fast-forward the track for however long it is pushed. If a track is not currently playing, or if the Forward button is released within 0.5 seconds of being pushed, then the player should navigate to the beginning of the track after the next one if the current track is playing and at less than two seconds from its end, or the beginning of the next track otherwise. The accessible label for the button should be “Forward (hold to fast-forward)” if a track is currently playing, or just “Forward” otherwise.

sound-menu-v2.jpg

Playlist submenu

The MPRIS API should let the player expose icons and names of playlists, even when the player is not running.

The playlist item should be present in the menu only if the music player is currently exposing any playlists. If the player also indicates that it is playing (or is paused partway through) a playlist, the submenu title should be the name of the playlist. Otherwise, it should be “Choose Playlist”.

The submenu itself should contain items named for each of the playlists. Selecting an item should start playing that playlist.

Player-global custom items

The API should let the music player provide up to three player-global custom menu items, even when the player is not running. Any of these items may be a submenu containing other items, for setting equalizer settings for example.

“Sound Settings…”

The menu should end with a separator, followed by a “Sound Settings…” item that opens the Sound panel in System Settings. Launch feedback for this item should be exactly the same as when you launch an application any other way.

(If PulseAudio is not running when you choose “Sound Settings…”, gnome-volume-control should try to start it. If it fails, it should invoke Apport during the development cycle, or an error alert in a stable release. This is outside the scope of the menu; it should be part of the specification for the Sound Settings themselves.)

Handling upgrades

When you log in to a version of Ubuntu that implements this sound menu, and you are running a gnome-panel that includes mixer_applet2, that applet should automatically be removed from your panel configuration.

Music player integration

Registration process

For Natty

In order to register each player needs to implement the MPRIS2 spec. The service which controls the menu automatically detects MPRIS clients appearing on DBus and reacts accordingly. It is essential that the implementation populates the DesktopEntry property on the MPRIS root interface. Access to the Desktop file is mandatory inorder to register successfully.

Registration will need to be disabled if a user would prefer for a certain player not to be controllable from the menu. Guidelines as to how this preference is to be represented in the UI of the client (specifically Rhythmbox, Banshee or Amarok) has been included below. We would like to provide a unified music player experience on the Ubuntu platform therefore we strongly urge application developers to follow this spec.

To remove a player from the menu the client can either use the dbus method 'blacklist-media-player(string desktop, bool blacklist)' on 'com.canonical.indicators.sound' (as of release 0.5.9) or alternatively it will need to write to the indicator-sound gsettings. The settings id is "com.canonical.indicators.sound" and the key is "blacklisted-media-players". The client should add the name of the desktop file (just the name, not the full path and minus the .desktop suffix) to this array of strings. The indicator sound service will remember each client (provided that it is not blacklisted) which has successfully registered so that when the media client is not running it will still appear in a hibernated state in the menu. Any successful changes to the blacklisted gsettings should be automatically reflected in the menu. A player can query its blacklisted state using the dbus api 'IsBlacklisted(string desktop-id)' (this will return a boolean depending on its presence in the blacklist obviously). Again alternatively you can just talk directly with gsettings. Please use the gsettings (either directly or through the dbus api) to save and fetch the state of a players permissions inorder to minimize bugs.

In order to test client registration, application developers should fetch the latest lp:indicator-sound release. As of release 0.5.3 this new way of registering has been implemented.

For Maverick

Each player should use a similar approach used with Ted Gould’s app indicators in order to enable remote control from the sound menu.

This is very straightforward. Here is an example of the mandatory override method of a Rhythmbox plugin in Vala which basically establishes Ayatana registration.

  • public override void activate (RB.Shell shell) {
    • debug("Rhythmbox - Hello Ayatana world\n"); server = Server.ref_default(); server.set("type", "music.rhythmbox"); server.set_desktop_file("/usr/share/applications/rhythmbox.desktop"); server.show();
  • }

It is important to note that the name of the application which is the suffix on the server type string should be the first word in the filename for the desktop file. Therefore for rhythmbox which has rhythmbox.desktop as the name of its desktop file uses "music.rhythmbox". Similarly Banshee which has a desktop file called banshee-1.desktop uses "music.banshee" as its server type string. The reason for this is that the service needs to be able to identify the music player shutdown from the libindicate 'server_removed' callback which passes the server type as an identifier. The complete registration the client also needs to have an MPRIS root and player interface available.

Bindings for libindicate exist for Python, Vala, Mono and C are available. https://launchpad.net/libindicate. Please contact Conor if you need any help with this (https://launchpad.net/~cjcurran) .

Registration will need to be disabled if a user would prefer for a certain player not to be controllable from the menu. Guidelines as to how this preference is to be represented in the UI of the client (specifically Rhythmbox, Banshee or Amarok) has been included below. We would like to provide a unified music player experience on the Ubuntu platform therefore we strongly urge application developers to follow this spec.

Rhythmbox

  • Rhythmbox should be altered to comply with the standard behavior: it should keep playing if closed while playing, it should exit if closed while not playing, and it should restore location and other state on relaunching.

  • The “Status Icon” plug-in for Rhythmbox should be replaced by notifications integrated into Rhythmbox itself. The layout of the “General” tab in Rhythmbox’s “Preferences” window should be adjusted — by changing the “Browser Views” radio buttons to a “Browser views show:” option menu, and by changing the two columns of “Visible Columns” checkboxes to three columns of “Track listings show:” checkboxes — to make room for a “Notifications for playing tracks:” option menu. This menu should contain three items: “Always”, “When Rhythmbox is hidden”, and “Never”. On first launch of a version of Rhythmbox that integrates notifications in this way, the configuration for the “Status Icon” plug-in (if any) should be mapped to the equivalent value for the “Notifications for playing tracks:” menu. If the plug-in had been disabled, the value of the new setting should be “Never”.
  • The “General” tab in Rhythmbox’s “Preferences” window should also include a “Show Rhythmbox in the sound menu” checkbox. The option should be off by default for a new user account, but should turn itself on automatically when you first play something or add items to the music library.

    rhythmbox-preferences-old.png

    rhythmbox-preferences-new.jpg

    rhythmbox-plug-in-old.png

Banshee

  • Banshee should be altered to comply with the standard behavior: it should keep playing if closed while playing, it should exit if closed while not playing, and it should restore location and other state on relaunching.

  • Banshee should no longer have a notification area item.
  • At the end of the “Miscellaneous” section in the “General” tab in Banshee’s “Preferences” window should be a “Show Banshee in the sound menu” checkbox. The option should be off by default for a new user account, but should turn itself on automatically when you first play something or add items to the music library.

    banshee-preferences-old.png

Not targeted for Maverick.

  • In the sound menu, Banshee should have one track-specific item: a “Rating” submenu, with “☆☆☆☆☆”, “★☆☆☆☆”, “★★☆☆☆”, “★★★☆☆”, “★★★★☆”, and “★★★★★” radio items, that sets the rating for the active track.

Not targeted for Maverick.

  • In the sound menu, Banshee should have two player-specific items:
    • a “Repeat” submenu, with “Off”, “This Track”, and “All” radio items;
    • a “Shuffle” submenu, with “Off”, “by Song”, “by Artist”, “by Album”, “by Rating”, and “by Score” radio items.

Amarok

Whenever Amarok is running in an environment where the sound menu is present:

  • It should comply with the standard behavior: it should keep playing if closed while playing, it should exit if closed while not playing, and it should restore location and other state on relaunching.

  • It should no longer have a notification area item.
  • In the “General” section of the “Configure - Amarok” window, the “Show tray icon” checkbox should be replaced by a “Show Amarok in the sound menu” checkbox. The option should be off by default for a new user account, but should turn itself on automatically when you first play something or add items to the music library.

    amarok-configure-old.png

MPD / Music Player Daemon

An external project exists that integrates MPD with the sound menu: https://launchpad.net/mpd-sound-menu

  • mpd-sound-menu.png

Another similar project is mpDris2: https://github.com/eonpatapon/mpDris2

Upstream implications

  • Development of an API
  • Media Player changes:
    • Support API
    • Use notification area
    • Amarok will need to detect whether the sound menu is present.

Specifications involving FreeDesktop:

Future work

  • Integrate Totem.
  • Show multiple sound outputs separately.
  • Sound configuration and management.
  • Consider http://pulseaudio.org/wiki/WritingVolumeControlUIs#Colouredvolumesliders

  • Should equalizer settings go into the system? (or should they be track-specific/application-specific)
  • Respect surround-sound in the menu and in the settings window
  • Fade out other audio automatically if you receive a VoIP call? (Where would a setting for this go?)

Risks

  • People may be disconcerted that something appearing to be a volume menu contains items for a music player. Or they may think that the presence of those items means that the volume is specific to the music player rather than system-wide. We should perform user testing once we have a basic implementation.

Alternative proposed designs

See also “Music controller” by Ryan Henricus.

Unresolved issues

  • We need a high-fidelity visual mockup to help us decide the width of the track metadata and scrub bar, and therefore the overall width of the menu.
  • We need an animated mockup of each item being highlighted as you navigate through it.

Proposed playlists extension

Sound (last edited 2017-03-30 15:52:16 by mpt)