Sound

Differences between revisions 61 and 177 (spanning 116 versions)
Revision 61 as of 2011-03-16 11:45:05
Size: 28249
Editor: eth0
Comment: + media.role=production
Revision 177 as of 2016-10-24 13:11:55
Size: 43912
Editor: mpt
Comment: updates link
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
<<Include(Ayatana/PageTemplate/Header)>>

So that people can easily change sound volume and control music playback, there should be a unified sound menu taking the place of the Gnome `mixer_applet2` and the notification area items for Rhythmbox, Banshee, and Amarok.
## page was renamed from SoundMenu
Sound in Ubuntu is based on PulseAudio. On both phone and PC, its user interface includes [[#settings|a “Sound” panel in System Settings]] (which also covers vibrations on the phone), [[#notification|volume notification bubbles]], and the [[#menu|sound menu]].

''For help with using sound in Ubuntu, see [[https://help.ubuntu.com/community/Sound|Ubuntu Help online]].''

<<TableOfContents()>>

<<Anchor(primary-output)>>
== Primary sound output ==

Phones have only one hardware volume rocker. And for simplicity, prominent software UI for volume (such as [[#menu|the sound menu]]) also has a single slider. So Ubuntu needs to decide, at any time, exactly what these elements control the volume of (bug Bug:1336536). This should be handled by two elements: the primary sound output device, and the active role.

The '''primary sound output''' device is, in implementation terms, the default Pulse``Audio device (device 0). Whenever headphones or a headset are connected, they should be the primary output for as long as they are connected. Otherwise, the primary output should depend on whatever happened most recently:

 * If you manually choose an output device in the “Sound” settings panel or a similar utility, that choice should be followed.
 * If an app changes the primary output itself, that choice should be followed. For example, an app running a voice call may change primary output to the earpiece for the duration of the call, and therefore any other sounds (such as notification sounds) should also be played through the earpiece for the duration of the call.
 * If the app that set the current primary output crashes, the primary output should return to whichever existing audio output device was the primary output most recently.
 * If you connect any audio output device that is not headphones, a headset, or [[http://voices.canonical.com/david.henningsson/2012/04/14/audio-over-hdmi-and-displayport-in-ubuntu-12-04/|a secondary display]], the primary output should change to that device (bug Bug:1404729).
 * If the previous primary output disconnects, the new primary output should be whichever remaining audio output device was the primary output most recently.

''This definition does not yet take into account simultaneous use of multiple audio outputs (for example, headphones for an audio call and speakers for music).''

Each possible primary output should have a human-readable '''label''' for use in notifications and volume sliders: “Speaker”, “Headphones”, “Headset”, or “Earpiece”. (“Earpiece” will seldom be seen, because the phone screen is blank when you have the phone to your ear during a call.)

<<Anchor(role)>>
== Audio output role ==

Any time audio is played, it is output using a particular '''audio role'''. The purpose of audio roles is to minimize the number of times you need to adjust the volume, by remembering your previously chosen volume for the same kind of audio. For example, your preferred volume for alarms is likely to be fairly consistent, but different from your preferred volume for listening to conversations on speakerphone.

If there are too many audio roles, you’ll need to adjust the volume too many times when you move to a louder/quieter environment. But if there are too few audio roles, you’ll need to adjust the volume too often even in the same environment, because very different kinds of audio will use the same role.

Because devices have only one set of volume buttons/keys, there needs to be an '''active output role''', which is the role that the volume buttons/keys will adjust if you use them right now.

The active output role should be determined as follows:
 * When an alarm is sounding, the active role should be `alarm`.
 * Otherwise, if the phone is ringing, the active role should be `ringtone`.
 * Otherwise, if a phone call is in progress, the active role should be `phone`.
 * Otherwise, the active output role should be `multimedia`.

Setting the role to `alert` should be functionally identical to setting it to `multimedia`. This is so that you can both change the volume of alerts even when they are not currently playing, and change the volume of other sound effects even when they are not currently playing (bug Bug:1498466), while maintaining backwards compatibility with apps that distinguish the roles, and forward compatibility if we distinguish them in future.

<<Anchor(volume)>>
== Volume ==

Sound volume should be remembered independently for each input device, so that you don’t need to adjust it repeatedly when switching devices. For example, when you plug in a microphone, input volume should be the same as when you last used a microphone, even if Ubuntu has restarted since then.

Furthermore, sound volume should be remembered independently for each ''combination of'' primary output device and active output role (bug Bug:1485522). For example, the volume for media played through headphones, the volume for media played through speakers, and the volume for a ringtone played through speakers, should be three independent volume levels (bug Bug:1336536).

Sound volume should also be independent of which user — or whether any user — is logged in. This way, if you mute or turn down sound at the login screen, you will not be surprised or frustrated by sound playing loudly after you log in (bug Bug:840777)). And in the common case where a device has only one user, you will not be frustrated by having to set a comfortable volume twice, once before you login (for a screenreader, for example) and then again after you log in.

Sound volume can be adjusted in several ways, but they should all have the same visual feedback.

=== Mute ===

The Mute action (not to be confused with [[#silent-mode|Silent Mode]]) should idempotently set the volume of [[#primary-output|the active output role through the primary sound output]] to zero (bug Bug:1124005). That is, if you Mute and then Mute again, sound should still be muted. And if you Mute then make the volume Louder, it should end up just above zero.

(Mute could be a mode, switching between zero and whatever was the previous volume. That would make temporary muting more convenient, but it would mean pressing the Mute key would sometimes cause sound to start blaring, which would be unpleasant.)

<<Anchor(silent-mode)>>
=== Silent Mode ===

When Silent Mode is on and you are using the internal speaker, the phone should play only sounds that you have explicitly requested. For example, alarms, music, and sound from videos should play as normal, but ringtones, message notification sounds, and interface sound effects should not. When Silent Mode is on and you are not using the internal speaker (which probably means you are using headphones), all sounds should play as normal. ''For more details, see'' SilentMode.

If you enter or exit Silent Mode using a hardware switch, the phone should vibrate briefly (since the display might not be on): once when entering, and twice when exiting. There should be no such vibration if you toggle it on-screen.

=== Quieter ===

The Quieter action should lower the volume of the active output role through the primary sound output to whichever is higher: zero, or the highest multiple of 1/16 that is lower than the current volume.

=== Louder ===

The Louder action should raise the volume of the active output role through the primary sound output to whichever is lower: the maximum possible, or the lowest multiple of 1/16 that is higher than the current volume.

<<Anchor(volume-keys)>>
=== Hardware volume keys ===

A keyboard, phone, or tablet’s MUTE_KEY, VOLUME_DOWN_KEY, or VOLUME_UP_KEY, should trigger the [[#Mute|Mute]], [[#Quieter|Quieter]], and [[#Louder|Louder]] actions respectively. When the screen is off, to minimize accidental volume changes, the keys should have effect only if any audio is currently playing, and they should not turn on the screen (bug Bug:1291458).

<<Anchor(notification)>>
== Output/volume change notification ==

||<style="border:none">{{attachment:sound-volume-osd.png}}||<style="border:none">{{attachment:sound-volume-osd.mockup.jpg}}<<BR>>[[attachment:Volume - Notification v1.jpg|full-size mockup]]<<BR>>''Erratum: This mockup needs changing to name and illustrate the primary output.''||

Whenever sensible, there should be '''visible confirmation''' of a manual output device change, and both '''visible and audible confirmation''' of a manual volume change. (There should be no special notification of output role changes.)

Visible confirmation in general is sensible only when the display is on. Therefore:
 * The display should not wake up just to show an output/volume change notification.

And confirmation of manual volume changes is sensible only when nothing else is demonstrating the volume change. Therefore:
 * There should be no extra visible notification if the focused window contains a [[https://goo.gl/PVJvu2|system volume control]].
 * There should be no extra audible confirmation if sound currently playing through the `multimedia` role (bug Bug:1378961).

Apart from those cases:

 * '''Visible confirmation''' should take the form of a confirmation bubble, showing the name of the relevant role, and the icon and volume of the primary output (bug Bug:1409696). For example, when you plug in headphones, a notification bubble should appear showing the current volume for headphones — both to reassure you that sound won’t blare from the speakers, and to remind you what the current headphones volume is. The bubble’s icon should be of a muted speaker if volume is muted, or otherwise an unmuted speaker with waves representing the approximate volume. To make clearer which key was pressed, if the bubble was not already open, the gauge should show the old value for half a second before switching to the new value. If you press the reduce-volume key when the volume is already at zero, or the increase-volume key when the volume is already at maximum, the icon and gauge should flash.

 * '''Audible confirmation''' should take the form of a subtle noise playing at the new volume (bug Bug:1436801).

== Pausing media playback when headphones are disconnected ==

If audio ''is'' currently playing through the `multimedia` role, and the output role changes because headphones or a headset was disconnected, any current player should pause, to guard against music or other embarrassing audio suddenly blaring (bug Bug:1457531). (That is, an Mpris pause signal should be sent to whichever app is using the media hub.)

<<Anchor(settings)>>
== Settings ==

Sound volume, and the primary sound output, should be global settings — persisting across sessions and user accounts. For example, if you set the sound volume at the login screen, that volume should persist after you log in (bug Bug:840777), and vice versa.

Other settings should be specific to individual user accounts. For example, someone who is deaf in one ear may choose mono output, while the other users of the device use the default stereo output.

<<Anchor(phone-settings)>>
=== Phone ===

||<^ tablestyle="float:left;margin:0 1em 1em 0;" style="border:none;width:310px">{{attachment:settings-sound.phone.png}}||

The “Silent Mode” switch should be followed by a [[https://docs.google.com/document/d/1n4Vc06BXlB0YRUdr3KTUBko-RAa2tCQwX2SoE_kpwpw/edit|system volume control]] (bug Bug:1389761) for adjusting the volume of [[#primary-output|the active output role through the primary sound output]]. It should be labelled according to the primary sound output: “Speaker:”, “Headset:”, “Earpiece:”, etc.

Turning on “Dialpad tones”, “Keyboard sound”, or “Lock sound” (bug Bug:1208418) should play the relevant sound once as a preview. (For “Dialpad tones”, this should be a quick sequence of three or four dialpad tones.) “Dialpad tones” should be an exact mirror of [[https://goo.gl/LteZNn|the same setting in the “Phone” app]].

“Other vibrations” should cover all vibrations except for those relating to phone calls and messages (bug Bug:1358610).

''Currently, this screen houses the Phone and Messages sound settings, on the assumption that people wanting to change their ringtone or message tone will look here rather than inside those apps. In future, we may need to (a) give third-party communication apps access to these settings, (b) give third-party communication apps the ability to join this screen, or (c) demote the Phone and Messages sounds to the apps themselves.''

<<Anchor(ringtone)>>

||<tablestyle="clear:both" style="border:none">||

||<^ tablestyle="float:left;margin:0 1em 1em 0;" style="border:none;">{{attachment:ringtone.png}}{{attachment:phone-settings-sound-message.png}}||

''Erratum: The “Ringtone” screen should not have a “None” option.''

When you choose a sound on the “Ringtone” or “Message received” screens, it should play regardless of whether you are in Silent Mode.

The “Ringtone” screen should end with a non-scrolling Stop button, enabled whenever a ringtone is currently previewing, so that you can stop a long-playing sound. (“Message received” sounds are not long enough to warrant the same button.)

Either way, the rest of each screen should be taken up by a radio list of installed sounds of that type. The “Ringtone” list should end with a “Custom Ringtone…” item, and the “Message received” list should end with a “None” item. Choosing a sound — even the already-selected sound — should play it as well as selecting it.

||<tablestyle="clear:both" style="border:none">||

==== Custom ringtone ====

(bug Bug:1268097)

{{attachment:ringtone-custom-picker.phone.png}}{{attachment:ringtone-custom-analyzing.phone.png}}{{attachment:ringtone-custom-range.phone.png}}

Choosing “Custom Ringtone…” should open a “Choose Track” audio picker for choosing a song or other audio. If you cancel the picker, the ringtone selection should return to its previous value.

If you do choose an audio file in the picker, a “Choose Loop” dialog should then open, with a centered spinner and the word “Analyzing…” until analysis of the track is complete.

Then the dialog contents should be replaced by a visualization of the audio track, an [[https://docs.google.com/document/d/1nFm8xiYhKXXuEO_IvMXoD0lASbYzYXva1BWMVanU3iw/edit#heading=h.vkaz9spw6fdn|interval slider]] (with non-overlapping M:SS captions) for choosing which range of the audio should loop as the ringtone, and a Play/Stop button for previewing the loop. (Play/Stop, rather than Play/Pause, because usually you will want to hear the loop from the beginning.)

If the audio is a minute or shorter, the default range should be the whole file. If the audio is longer than a minute, it is probably a whole song. Song intros are often quiet and/or less recognizable than the chorus, so the default range should be [[http://ronggong1010.blogspot.co.uk/2014/11/chorus-detection-gotos-algorithm.html|the detected chorus]] of the song.

If you complete choosing a custom ringtone, it should be shown in the list immediately above the “Custom Ringtone…” item, named according to the title of the track where available (for example, the MP3 “Title” tag), or otherwise the filename. That custom ringtone should remain in the list until you ever choose a different custom ringtone.

||<tablestyle="clear:both" style="border:none">||

<<Anchor(settings-pc)>>
=== PC ===

[[SystemSettings|System Settings]] should have a “Sound” panel. Most of the settings should be categorized across four tabs: “Input”, “Output” (the default tab), “Sound Effects”, and “Applications”.

{{attachment:sound-settings-pc-volume.jpg}}

Since output volume is the most urgent setting (especially if you do not have the sound menu turned on), “Output volume:” should be at the top of the panel, above these tabs. It should be followed by checkboxes for “Mute” (bug Bug:1276063) and “Allow louder than 100%” (with a translation note advising to keep the label brief).

Whenever “Mute” is checked, the volume slider above should be insensitive. Whenever “Allow louder than 100%” is checked, its label should instead be “Allow louder than 100% (may distort sound)” (avoiding distracting mention of distortion if it is unchecked), and the slider should grow to the right (without its label moving) to include a red area for greater-than-maximum volume.

{{attachment:sound-menu-setting.png}}

At the bottom left corner of the System Settings “Sound” panel should be “Show in the menu bar:” checkboxes, “Sound volume” and “Microphone when connected”. By default in a new user’s account, “Sound volume” should be checked and “Microphone when connected” should be unchecked.

== Indicators and menus ==

<<Anchor(phone-menu)>>
=== Phone ===

||<tablestyle="float:left;margin:0 1em 1em 0" style="border:none">{{attachment:phone-sound-menu.png}}||

The indicator should be a speaker icon the same as [[#Title|the title of the PC menu]], except that whenever audio is playing, it should be preceded by a ▶ Play icon.

The indicator and the [[https://goo.gl/JZBdSx|system volume control]] (bug Bug:1389761) — identical to [[#phone-settings|its counterpart in System Settings]] — should both represent, and the slider should adjust, the volume of [[#primary-output|the active output role through the primary sound output]].

To minimize clutter, the menu should show playback controls only for those players that are currently playing, or if none are, the player that was playing most recently. The controls should not be shown for a player that cannot be controlled externally ([[http://standards.freedesktop.org/mpris-spec/2.2/Player_Interface.html#Property:CanControl|CanControl]]). Similarly, the Previous/Next buttons may or may not be present, depending on the player’s capabilities.

Whenever audio is playing and the player is providing track data, this data should be shown below the controls. If the player is not providing track data, the icon and name of the player itself should be shown instead, at the same size. Either way, this item should launch or switch to the player.

||<tablestyle="clear:both" style="border:none">||
<<Anchor(menu)>>
=== PC ===

<<Anchor(microphone-menu)>>
==== Microphone menu ====

||<tablestyle="float:left;margin:0 1em 1em 0" style="border:none;">{{attachment:microphone-menu.png}}||

Whenever “Show in the menu bar: Microphone when connected” is checked, and an external microphone is connected, a microphone menu should be present in the [[MenuBar#system-status|system status area]].

Whenever the microphone is muted, the volume slider should be insensitive.

“Sound Input Settings…” should open [[#settings-pc|System Settings “Sound” panel]] to the “Input” tab.

<<Anchor(sound-menu)>>
==== Sound menu ====

<<Anchor(presence)>>whenever “Show in the menu bar: Sound volume” is checked, a sound menu should be present in the [[MenuBar#system-status|system status area]].

This menu exists to let people easily change sound volume and control music playback. It should give easiest access to the volume of the [[#primary-output|primary sound output device]] and the primary sound input device. Other devices can be accessed through the Sound panel of System Settings.
Line 7: Line 211:
We also offer this design, and its associated APIs, to the Kubuntu and KDE communities as a foundation for a possible replacement of the `kmix` applet.

<<TableOfContents()>>

== Artwork requirements ==

Beyond icons that are already included in themes:
 * A new named icon, `audio-volume-low-zero-symbolic`: a speaker with zero sound waves emanating.
 * A new named icon, `audio-volume-muted-blocking-symbolic` (sound is playing, but it is muted): the same as `audio-volume-low-zero-symbolic`, but in the error color. (We are aware that this color-only difference will not be noticable for monochromats and some dichromats.)
 * A new named icon, `audio-output-none-symbolic` (no output devices are available): the same as `audio-volume-zero`, but in outline form.
 * Zero-volume and maximum-volume icons for a microphone. ''What should their names be?''
 * A custom ♫ icon.
 * Custom scrub bar rendering.
 * Play, Pause, Previous, and Next button rendering.

== Menu structure ==

The menu should give easiest access to the volume of the [[PrimarySoundOutput|primary sound output device]] and the primary sound input device. Other devices can be accessed through the Sound Preferences window.
''Erratum: The scrub bar should not be present, and the track-specific items should go between the track info and the playback controls.''
Line 28: Line 215:
=== Title ===

Whenever there is no primary sound output device (because no available output devices have been detected yet), the title of the menu should be `audio-output-none-symbolic` (in the default theme, an outline version of the speaker icon), and its accessible name should be “Volume (no devices)”.

If there are any available output devices:

 * The title of the menu should be an icon roughly representing the current volume for the primary output device: `audio-volume-muted-panel`, `audio-volume-low-zero-symbolic`, `audio-volume-low-symbolic`, `audio-volume-medium-symbolic`, or `audio-volume-high-symbolic` (which, in the default theme, should be a speaker icon with zero to three sound waves). The icon should be updated even if the volume is being changed while the menu is open. The accessible name should be of the form “Volume (74%)”.

 * If the sound is muted and the computer begins “playing” sound, the icon should switch instantly to `audio-volume-muted-blocked-symbolic` (in the default theme, a red version of `audio-volume-muted-panel`), persist for five seconds after sound stops playing, then take one second to fade back to `audio-volume-muted-panel`.

 * As a hidden treasure, when the pointer is over the title, whether the menu is open or not, clicking a mousewheel up or down should increase or decrease the volume respectively, to the nearest 10-percent step.

=== “Mute”/“Unmute” ===

The first item, “Mute”, makes it easy to urgently silence unwanted sound using a pointing device. If there are no available sound output devices, the item should be insensitive. Activating the item should mute all output devices, while remembering their previous volumes. After being chosen, “Mute” should change to “Unmute”.

“Unmute” should return all output devices to the volume settings they had immediately before “Mute” was selected. Either choosing “Unmute”, ''or'' changing any volume setting manually, should change “Unmute” back to “Mute”.

=== Output volume item ===

If there is a primary sound output device, the menu should contain an item for changing its volume, followed by a separator. The Up and Down keys should navigate between the volume item and other items, just as for a normal menu item. The item should highlight when navigated to, or on mouseover, just as normal menu items do, but the item should not be activatable (clicking it or pressing Enter should not close the menu).
===== Title =====

Whenever there is no [[#primary-output|primary sound output]] (for example, because no available output devices have been detected yet), the title of the menu should be an outline version of the speaker icon (`audio-output-none-symbolic`), and its accessible name should be “Volume (no devices)”.

Whenever there is a primary sound output, the title should be:

 * If sound is muted and an application tried to play sound in the past five seconds, a red speaker with a cross (`audio-volume-muted-panel`).

 * Otherwise, if sound is muted, a normal-colored speaker with a cross (`audio-volume-muted-panel`).

 * Otherwise, a speaker icon with zero to three sound waves, roughly representing the current volume for the primary output device (`audio-volume-low-zero-symbolic`, `audio-volume-low-symbolic`, `audio-volume-medium-symbolic`, or `audio-volume-high-symbolic`). The icon should be updated even if the volume is being changed while the menu is open. The accessible label should be of the form “Volume (74%)”.

As a hidden treasure, when the pointer is over the title, whether the menu is open or not, clicking a mousewheel up or down should invoke the [[#Louder|Louder]] or [[#Quieter|Quieter]] action respectively. (As usual, this should result in a notification bubble showing the volume change.)

===== “Mute” =====

The first item, “Mute”, makes it easy to urgently silence unwanted sound using a pointing device. If the [[#primary-output|primary sound output]] is null (there are no available sound output devices), the item should be insensitive. Choosing it should trigger [[#Mute|the Mute action]], ''without'' [[#notification|the usual volume notification]].

===== Output volume item =====

If there is a [[#primary-output|primary sound output]], the menu should contain an item for changing its volume, followed by a separator. The Up and Down keys should navigate between the volume item and other items, just as for a normal menu item. The item should highlight when navigated to, or on mouseover, just as normal menu items do, but the item should not be activatable (clicking it or pressing Enter should not close the menu).
Line 51: Line 238:
 1. A borderless, mouseover-less button that sets the volume to zero (as distinct from muting it). The icon for the button should be `audio-volume-low-zero` (in the default theme, a speaker with no sound waves).
 1. A slider that lets you set the volume more precisely. Whenever sound is muted, the slider should temporarily be at zero. Unmuting should return the slider to its previous position.
 1. A borderless, mouseover-less button that sets the volume to zero. The icon for the button should be `audio-volume-low-zero` (in the default theme, a speaker with no sound waves).
 1. A slider that lets you set the volume more precisely. Whenever “Allow louder than 100%” is checked, the slider should include a red area for greater-than-maximum volume (bug Bug:786525). Whenever sound is muted, the slider should temporarily be at zero. Unmuting should return the slider to its previous position.
Line 61: Line 248:
Changing the volume with the volume item should provide the same audio feedback as changing the volume any other way (such as with the volume keys on the keyboard), but should not generate notification bubbles. Changing the volume in any of these ways should ''not'' trigger [[#notification|the usual volume notification]].
Line 66: Line 253:
=== Microphone volume item === ===== Microphone volume item =====
Line 70: Line 257:
If a VoIP (`media.role=phone`) or sound recording (`media.role=production`) audio stream is active, and the computer has a detected microphone, the output volume item should be followed by a microphone volume item. The microphone volume item should look and behave exactly the same as the output volume item, except that the button icons should feature a microphone rather than a speaker. The output volume item should be followed by a microphone volume item whenever all of these are true:
 * a VoIP (`media.role=phone`) or sound recording (`media.role=production`) audio stream is active
 * the computer has a detected external microphone
 * the [[#microphone-menu|microphone menu]] is not present.

The microphone volume item should look and behave exactly the same as the output volume item, except that the button icons should feature a microphone rather than a speaker.
Line 73: Line 265:
=== Music player section ===

There should be an API that lets a music player add a section to the sound menu. Multiple compliant music players may result in multiple music sections, ordered alphabetically by application `Name`. (In future, there may even be multiple sections for the same player.)

A compliant player should not insert itself into the menu merely because it is installed, but should insert itself as soon as you start actively using it (''e.g.'' playing something with it for the first time, or adding music to its library). The player’s own settings interface should also have a checkbox for whether the player should be present in the sound menu right now:
===== Music player sections =====

Whenever you are logged in (that is, not in the standalone installer session or at the login screen), any music player that advertises itself over [[http://mpris.org/ Mpris]] should have its own section in the sound menu. Multiple compliant music players may result in multiple music sections, ordered alphabetically by application `Name`. (In future, there may even be multiple sections for the same player.)

A compliant player should not be present in the menu merely because it is installed, but should insert itself as soon as you start actively using it (''e.g.'' playing something with it for the first time, or adding music to its library). The player’s own settings interface should also have a checkbox for whether the player should be present in the sound menu right now:
Line 85: Line 277:
A music section should consist of [[#music-player-item|an item for the music player itself]], followed by any or all of [[#track-metadata-item|a track metadata item]], [[#scrub-bar-item|a scrub bar item]], [[#playback-item|a playback item]], [[#track-custom-items|track-specific custom items]], [[#playlist-submenu|a playlist submenu]], and [[#player-custom-items|player-global custom items]]. Some of these items are present only when the player is running. A music section should consist of [[#music-player-item|an item for the music player itself]], followed by any or all of [[#track-custom-items|track-specific custom items]], [[#playback-item|a playback item]], [[#playlist-submenu|a playlist submenu]], and [[#player-custom-items|player-global custom items]]. Some of these items are present only when the player is running.
Line 90: Line 282:
==== Music player item ==== ====== Music player item ======
Line 94: Line 286:
The item should consist of a ♫ icon and the `Name` of the music player. A player should also be able, in rare cases, to specify custom text in place of the application name (by changing the 'Identity' property string on the Mpris root interface to the full string desired). The first row of the item should consist of the application’s icon and `Name`. A player should also be able, in rare cases, to specify custom text in place of the application name (by changing the 'Identity' property string on the Mpris root interface to the full string desired).
Line 98: Line 290:
<<Anchor(track-metadata-item)>>
==== Track metadata item ====


If the music player is not running, or if it is running but has no active track, this item should not be present at all.

If there is an active track, the item should consist of the track or album art on the leading side, and top-aligned next to it, three rows of text. If no track or album art is available, it should fall back to a generic track icon. The text should be the track name, artist, and album, one on each line (aligned the same way as the rest of the menu, regardless of what language each string is in). For each line, if there is not enough room in the menu to display all the text, it should be ellipsized in the middle; and if the data is not known, the line should be left blank. This assemblage should be highlightable and activatable like a single normal menu item; as a hidden treasure, activating the item should copy the album art and track metadata to the clipboard.
{{attachment:player-item-playing.png}}

If the player is running and has an active track, the music player item should also include, below the player icon and name, the track or album art and three rows of text. If no track or album art is available, it should fall back to a generic track icon. The text should be on the leading side, and top-aligned next to the art. It should consist of the track name, artist, and album, one on each line (aligned the same way as the rest of the menu, regardless of what language each string is in). For each line, if there is not enough room in the menu to display all the text, it should be ellipsized in the middle; and if the data is not known, the line should be left blank. The player name and track data combined should be highlightable and activatable like a single normal menu item.
Line 120: Line 309:
====== Track-specific custom items ======

The API should let the music player provide up to three track-specific custom menu items — for example, for buying, “liking”, or “banning” the song currently playing. Any of these items may be a submenu containing other items.
Line 121: Line 314:
==== Playback item ====

The playback item should consist of a large Play/Pause button in the middle, with Back and Forward buttons (using ◀◀ Rewind and ▶▶ Fast-Forward symbols) on the sides. If something is playing, the main button should be Pause. If nothing is playing, the main button should be Play.

If the player is not running, or if it is running but has no active track, the playback item should not be present by default. However, the API should provide a way for a player to register a Play action, so that the Play button can work even when the player is not running. If this has been done, the whole playback item should be present, but the Back and Forward buttons should be insensitive.

The API should also provide a way for a player to make the Back and Forward buttons sensitive or insensitive, and to turn the Play button into a Pause button and back again.

When using a pointing device, whichever button is under the pointer should highlight in a similar way to a normal menu item in the same theme. When using the keyboard, the Play/Pause button alone should highlight in the same way. Pressing the Space key should visibly activate the Play/Pause button, and the Left and Right keys should visibly activate the Back and Forward buttons respectively. None of the elements should have a visible focus ring, and Tab and Shift Tab should not do anything.
====== Playback item ======

To minimize clutter, the menu should show playback controls only for those players that are currently playing, or if none are, the player that was playing most recently. The controls should not be shown for a player that cannot be controlled externally ([[http://standards.freedesktop.org/mpris-spec/2.2/Player_Interface.html#Property:CanControl|CanControl]]). The playback item should consist of a large Play/Pause button in the middle, with Back and Forward buttons (using ◀◀ Rewind and ▶▶ Fast-Forward symbols) on the sides if the player allows it.

If something is playing, the main button should be Pause. If nothing is playing, the main button should be Play. The Back button should be sensitive whenever the player can seek ([[http://standards.freedesktop.org/mpris-spec/2.2/Player_Interface.html#Property:CanSeek|CanSeek]]) ''or'' go to a previous track ([[http://standards.freedesktop.org/mpris-spec/2.2/Player_Interface.html#Property:CanGoPrevious|CanGoPrevious]]). And the Forward button should be sensitive whenever the the player can seek ''or'' go to a next track ([[http://standards.freedesktop.org/mpris-spec/2.2/Player_Interface.html#Property:CanGoNext|CanGoNext]]) (bug Bug:782060).

The Space key should visibly activate whichever button is highlighted. However, highlighting the Back or Forward button with a keyboard should not be possible; instead, the Left and Right keys should immediately activate Back and Forward respectively whenever any of the three are highlighted (bug Bug:733285). Apart from that, and their button appearance, each button should behave in every way like a menu item. In particular:
 * The visual style for a highlighted button should be the same as for a normal menu item in the same theme.
 * Clicking on a button, moving to a different menu item, then releasing, should activate that other item instead.
 * Conversely, clicking on a different menu item (or any menu title), moving to a button, and releasing, should activate that button (bug Bug:779782).
 * None of the buttons should have a visible focus ring, and Tab and Shift Tab should not do anything when they are highlighted.
Line 142: Line 337:
==== Track-specific custom items ====

The API should let the music player provide up to three track-specific custom menu items — for example, for buying, “liking”, or “banning” the song currently playing. Any of these items may be a submenu containing other items.
Line 147: Line 338:
==== Playlist submenu ====

''This can be implemented last, since players don’t currently provide API access to playlists. For 10.10 we aim to extend the MPRIS spec to allow playlists to be properly managed and implement playlist support for rhythmbox in its current DBus API state''
====== Playlist submenu ======
Line 158: Line 347:
==== Player-global custom items ====

||<tablestyle="background: #ffc; color: #000;"> Not targeted for Maverick.||
====== Player-global custom items ======
Line 164: Line 351:
=== “Sound Preferences…”/“Sound Settings…” ===

The menu should end with a separator, followed by a “Sound Preferences…” item that opens the Gnome Sound Preferences window. (A Kubuntu version of this menu item would be “Sound Settings…”, and would open the Sound panel in KDE System Settings.) Launch feedback for this item should be exactly the same as when you launch an application any other way.

(If PulseAudio is not running when you choose “Sound Preferences…”, `gnome-volume-control` should try to start it. If it fails, it should invoke Apport during the development cycle, or an error alert in a stable release. ''This is outside the scope of the menu; it should be part of the specification for the Sound Preferences themselves.'')

== Handling upgrades ==

When you log in to a version of Ubuntu that implements this sound menu, and you are running a `gnome-panel` that includes `mixer_applet2`, that applet should automatically be removed from your panel configuration.

== Music player integration ==

=== Registration process ===
==== For Natty ====
In order to register each player needs to implement the MPRIS2 spec. The service which controls the menu automatically detects MPRIS clients appearing on DBus and reacts accordingly.
It is essential that the implementation populates the DesktopEntry property on the MPRIS root interface. Access to the Desktop file is mandatory inorder to register successfully.
===== “Sound Settings…” =====

Whenever you are logged in (that is, not in the standalone installer session or at the login screen), the menu should end with a separator, followed by a “Sound Settings…” item that opens the Sound panel in System Settings. Launch feedback for this item should be exactly the same as when you launch an application any other way.

===== Music player integration =====

To register in the menu, each player needs to implement the MPRIS2 spec. The service which controls the menu automatically detects MPRIS clients appearing on DBus and reacts accordingly.
It is essential that the implementation populates the DesktopEntry property on the MPRIS root interface. Access to the Desktop file is mandatory to register successfully.
Line 187: Line 366:
==== For Maverick ====

Each player should use a similar approach used with Ted Gould’s app indicators in order to enable remote control from the sound menu.

This is very straightforward. Here is an example of the mandatory override method of a Rhythmbox plugin in Vala which basically establishes Ayatana registration.

    public override void activate (RB.Shell shell) {

        debug("Rhythmbox - Hello Ayatana world\n");

        server = Server.ref_default();

        server.set("type", "music.rhythmbox");

        server.set_desktop_file("/usr/share/applications/rhythmbox.desktop");

        server.show();
 }

It is important to note that the name of the application which is the suffix on the server type string should be the first word in the filename for the desktop file. Therefore for rhythmbox which has rhythmbox.desktop as the name of its desktop file uses "music.rhythmbox". Similarly Banshee which has a desktop file called banshee-1.desktop uses "music.banshee" as its server type string. The reason for this is that the service needs to be able to identify the music player shutdown from the libindicate 'server_removed' callback which passes the server type as an identifier. The complete registration the client also needs to have an MPRIS root and player interface available.

Bindings for libindicate exist for Python, Vala, Mono and C are available. https://launchpad.net/libindicate. Please contact Conor if you need any help with this (https://launchpad.net/~cjcurran) .

Registration will need to be disabled if a user would prefer for a certain player not to be controllable from the menu. Guidelines as to how this preference is to be represented in the UI of the client (specifically Rhythmbox, Banshee or Amarok) has been included below. We would like to provide a unified music player experience on the Ubuntu platform therefore we strongly urge application developers to follow this spec.

=== Rhythmbox ===

 * Rhythmbox should be altered to comply with [[#compliance|the standard behavior]]: it should keep playing if closed while playing, it should exit if closed while not playing, and it should restore location and other state on relaunching.

 * The “Status Icon” plug-in for Rhythmbox should be replaced by notifications integrated into Rhythmbox itself. The layout of the “General” tab in Rhythmbox’s “Preferences” window should be adjusted — by changing the “Browser Views” radio buttons to a “Browser views show:” option menu, and by changing the two columns of “Visible Columns” checkboxes to three columns of “Track listings show:” checkboxes — to make room for a “Notifications for playing tracks:” option menu. This menu should contain three items: “Always”, “When Rhythmbox is hidden”, and “Never”.

 On first launch of a version of Rhythmbox that integrates notifications in this way, the configuration for the “Status Icon” plug-in (if any) should be mapped to the equivalent value for the “Notifications for playing tracks:” menu. If the plug-in had been disabled, the value of the new setting should be “Never”.

 * The “General” tab in Rhythmbox’s “Preferences” window should also include a “Show Rhythmbox in the sound menu” checkbox. The option should be off by default for a new user account, but should turn itself on automatically when you first play something or add items to the music library.

 ||<style="border: none; text-align: right; vertical-align: bottom;">{{attachment:rhythmbox-preferences-old.png}}||<style="border: none; vertical-align:middle;">↘||<|2 style="border: none; vertical-align: middle;">{{attachment:rhythmbox-preferences-new.jpg}}||
 ||<style="border: none; text-align: right; vertical-align: top;">{{attachment:rhythmbox-plug-in-old.png}}||<style="border: none;">↗||

=== Banshee ===

 * Banshee should be altered to comply with [[#compliance|the standard behavior]]: it should keep playing if closed while playing, it should exit if closed while not playing, and it should restore location and other state on relaunching.

 * Banshee should no longer have a notification area item.

 * At the end of the “Miscellaneous” section in the “General” tab in Banshee’s “Preferences” window should be a “Show Banshee in the sound menu” checkbox. The option should be off by default for a new user account, but should turn itself on automatically when you first play something or add items to the music library.
 {{attachment:banshee-preferences-old.png}}

||<tablestyle="background: #ffc; color: #000;"> Not targeted for Maverick.||

 * In the sound menu, Banshee should have one track-specific item: a “Rating” submenu, with “☆☆☆☆☆”, “★☆☆☆☆”, “★★☆☆☆”, “★★★☆☆”, “★★★★☆”, and “★★★★★” radio items, that sets the rating for the active track.

||<tablestyle="background: #ffc; color: #000;"> Not targeted for Maverick.||

 * In the sound menu, Banshee should have two player-specific items:
  * a “Repeat” submenu, with “Off”, “This Track”, and “All” radio items;
  * a “Shuffle” submenu, with “Off”, “by Song”, “by Artist”, “by Album”, “by Rating”, and “by Score” radio items.

=== Amarok ===

Whenever Amarok is running in an environment where the sound menu is present:

 * It should comply with [[#compliance|the standard behavior]]: it should keep playing if closed while playing, it should exit if closed while not playing, and it should restore location and other state on relaunching.

 * It should no longer have a notification area item.

 * In the “General” section of the “Configure - Amarok” window, the “Show tray icon” checkbox should be replaced by a “Show Amarok in the sound menu” checkbox. The option should be off by default for a new user account, but should turn itself on automatically when you first play something or add items to the music library.
 {{attachment:amarok-configure-old.png}}


=== MPD / Music Player Daemon ===

An external project exists that integrates MPD with the sound menu:
https://launchpad.net/mpd-sound-menu
 {{attachment:mpd-sound-menu.png}}

== Upstream implications ==

 * Development of an API
 * Media Player changes:
   * Support API
   * Use notification area
   * Amarok will need to detect whether the sound menu is present.

Specifications involving Free``Desktop:

 * MPRIS (see [[#mpris-amarok|the Amarok example]]) (''broken link?'')
   * Add track specific items
   * Needs to be able to query playlists (and start them)
   * The new version of MPRIS can be found here http://www.mpris.org/2.0/spec/
 * Application Shortcuts (Modify Desktop File spec)
   * http://standards.freedesktop.org/desktop-entry-spec/latest/
   * Need usecases and justification (Mostly on Messaging Menu and Sound Menu)

== Future work ==
===== Future work =====
Line 285: Line 371:
 * Consider http://pulseaudio.org/wiki/WritingVolumeControlUIs#Colouredvolumesliders  * Consider [[https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/Developer/Clients/WritingVolumeControlUIs/#colouredvolumesliders|coloured volume sliders]]
Line 290: Line 376:
== Risks == ===== Session variations =====

''This is a summary only; the specifications of individual items are authoritative.''

In the '''guest session''' and the '''live session''', the indicator and menu should be present as normal.

In the '''standalone installer session''' and the '''login screen''', only the mute and volume items should be present.

===== Risks =====
Line 294: Line 388:
== Alternative proposed designs ==

 * [[http://picasaweb.google.com/100804433705878937883/Mockups|Wyatt]]
 * [[http://www.foopics.com/showfull/c7e1ce704b9de523e51ec5d4a3569003|Thorsten Wilms]]

== Unresolved issues ==

 * We need a high-fidelity visual mockup to help us decide the width of the track metadata and scrub bar, and therefore the overall width of the menu.
 * We need an animated mockup of each item being highlighted as you navigate through it.
 * How to turn the sound menu off completely? (currently gnome-volume-control won’t open if isn’t running — leave till Maverick+1?)

=== Proposed playlists extension ===
===== Alternative proposed designs =====

 * [[http://picasaweb.google.com/100804433705878937883/Mockups|Wyatt Kirby]]
 * “[[http://dribbble.com/shots/358642-Music-Controller|Music controller]]” by Ryan Henricus

<<Anchor(unknown-device)>>
=== Handling unknown audio jack devices ===

(bug Bug:1239605)

When a device is plugged into an audio jack that can’t distinguish between headphones, headsets, or microphones, Ubuntu should display an “Unknown Audio Device” dialog.

{{attachment:sound-device-unknown.png}}

If you unplug the device before responding to the dialog, the dialog should close automatically. “Headphones”, “Headset”, and “Microphone” should be buttons that close the dialog, just like the “Cancel” and “Sound Settings…” buttons do. The “Sound Settings…” button should also open the “Sound” panel of System Settings.

{{attachment:sound-device-settings.png}}

''If'' an unknown audio device has ever been connected in this Ubuntu installation (to avoid it cluttering the interface on PCs where this is not an issue), the bottom of that “Sound” panel should have a “When an unknown audio device is plugged in:” menu. The menu should have options “Treat as headphones”, “Treat as headset”, “Treat as microphone”, “Ignore”, a separator, and finally “Ask what to do”.

If, in the prompt, you choose “Use this choice for unknown devices from now on” and then any of the options (except “Sound Settings…”), that should change the setting in the menu in System Settings. (Choosing “Cancel” should change the setting to “Ignore”.)

<<Anchor(limits)>>
== High volume limits and warnings ==

[[http://www.isvr.co.uk/labtests/en50332.htm|EN 50332]] ([[http://shop.bsigroup.com/ProductDetail/?pid=000000000030281663|unfortunately not freely available]]) defines a “programme simulation noise” test signal for testing audio volume.

EU standard EN 60950-1/Al2 ([[http://www.comm-2000.com/ProductDetail.aspx?UniqueKey=16670|unfortunately not freely available]]) defines:
 * '''''acoustic output''''' as the 30-second A-weighted equivalent sound pressure level L,,Aeq,T,, when playing that programme simulation noise;
 * '''''electrical output''''' as the output through the analogue electrical output socket when playing that programme simulation noise.

It then requires that “A personal music player and earphones or headphones intended to be used with personal music players shall…”, while headphones/earphones are being used:

 a. “Protect the user from unintentional acoustic outputs exceeding” 85 dBA.

 a. “Have a standard acoustic output level not exceeding” 85 dBA, “and automatically return to an output level not exceeding those mentioned above when the power is switched off.

 a. “Provide a means to actively inform the user of the increased sound pressure when the equipment is operated with an acoustic output exceeding” 85 dBA. “Any means used shall be acknowledged by the user before activating a mode of operation which allows for an acoustic output exceeding those mentioned above. The acknowledgement does not need to be repeated more than once every 20 h of cumulative listening time.”

 a. “Have a warning as specified”, “on the equipment, or on the packaging, or in the instruction manual”, with a particular graphic and words to the effect of “To prevent possible hearing damage, do not listen at high volume levels for long periods.”.

 a. “Not exceed … acoustic output” of 100 dBA or electrical output of 150 mV.

To meet these requirements (bug Bug:1480912):

||<^ tablestyle="float:right;margin:0 0 1em 1em" style="border:none">{{attachment:sound-volume-confirmation.png}}||

(a, c) Whenever you increase volume, for the first time in 20 hours, such that acoustic output would be more than 85 dBA, an “Increase Volume” dialog should appear, with primary text “High volume can damage your hearing.” and “Cancel” and “OK” buttons. Unlike any other dialog, pressing the Volume Down button should be equivalent to choosing “Cancel”. If you choose “OK”, [[#notification|the volume change should happen as normal]]. (Normal practice would be to label the button “Increase Anyway” or similar, but that would risk the button becoming a bragging-mark like “[[https://www.google.com/search?tbm=isch&q=parental+advisory|Parental Advisory]]”.) So that both buttons do what you expect and the dialog does not disappear unexpectedly, any [[#role|role]] change should be postponed as long as the dialog is up (bug Bug:1504065).
 
(b) Whenever you start up or wake the device, if the volume is such that the acoustic output would be greater than 85 dBA, it should be reduced to 85 dBA.
 
(d) Any phone or tablet shipped with Ubuntu should include the volume warning in its packaging or manual.
 
(e) Sound output should never be greater than 100 dBA, and electric output should never be greater than 150 mV.

Sound in Ubuntu is based on PulseAudio. On both phone and PC, its user interface includes a “Sound” panel in System Settings (which also covers vibrations on the phone), volume notification bubbles, and the sound menu.

For help with using sound in Ubuntu, see Ubuntu Help online.

Primary sound output

Phones have only one hardware volume rocker. And for simplicity, prominent software UI for volume (such as the sound menu) also has a single slider. So Ubuntu needs to decide, at any time, exactly what these elements control the volume of (bug 1336536). This should be handled by two elements: the primary sound output device, and the active role.

The primary sound output device is, in implementation terms, the default PulseAudio device (device 0). Whenever headphones or a headset are connected, they should be the primary output for as long as they are connected. Otherwise, the primary output should depend on whatever happened most recently:

  • If you manually choose an output device in the “Sound” settings panel or a similar utility, that choice should be followed.
  • If an app changes the primary output itself, that choice should be followed. For example, an app running a voice call may change primary output to the earpiece for the duration of the call, and therefore any other sounds (such as notification sounds) should also be played through the earpiece for the duration of the call.
  • If the app that set the current primary output crashes, the primary output should return to whichever existing audio output device was the primary output most recently.
  • If you connect any audio output device that is not headphones, a headset, or a secondary display, the primary output should change to that device (bug 1404729).

  • If the previous primary output disconnects, the new primary output should be whichever remaining audio output device was the primary output most recently.

This definition does not yet take into account simultaneous use of multiple audio outputs (for example, headphones for an audio call and speakers for music).

Each possible primary output should have a human-readable label for use in notifications and volume sliders: “Speaker”, “Headphones”, “Headset”, or “Earpiece”. (“Earpiece” will seldom be seen, because the phone screen is blank when you have the phone to your ear during a call.)

Audio output role

Any time audio is played, it is output using a particular audio role. The purpose of audio roles is to minimize the number of times you need to adjust the volume, by remembering your previously chosen volume for the same kind of audio. For example, your preferred volume for alarms is likely to be fairly consistent, but different from your preferred volume for listening to conversations on speakerphone.

If there are too many audio roles, you’ll need to adjust the volume too many times when you move to a louder/quieter environment. But if there are too few audio roles, you’ll need to adjust the volume too often even in the same environment, because very different kinds of audio will use the same role.

Because devices have only one set of volume buttons/keys, there needs to be an active output role, which is the role that the volume buttons/keys will adjust if you use them right now.

The active output role should be determined as follows:

  • When an alarm is sounding, the active role should be alarm.

  • Otherwise, if the phone is ringing, the active role should be ringtone.

  • Otherwise, if a phone call is in progress, the active role should be phone.

  • Otherwise, the active output role should be multimedia.

Setting the role to alert should be functionally identical to setting it to multimedia. This is so that you can both change the volume of alerts even when they are not currently playing, and change the volume of other sound effects even when they are not currently playing (bug 1498466), while maintaining backwards compatibility with apps that distinguish the roles, and forward compatibility if we distinguish them in future.

Volume

Sound volume should be remembered independently for each input device, so that you don’t need to adjust it repeatedly when switching devices. For example, when you plug in a microphone, input volume should be the same as when you last used a microphone, even if Ubuntu has restarted since then.

Furthermore, sound volume should be remembered independently for each combination of primary output device and active output role (bug 1485522). For example, the volume for media played through headphones, the volume for media played through speakers, and the volume for a ringtone played through speakers, should be three independent volume levels (bug 1336536).

Sound volume should also be independent of which user — or whether any user — is logged in. This way, if you mute or turn down sound at the login screen, you will not be surprised or frustrated by sound playing loudly after you log in (bug 840777)). And in the common case where a device has only one user, you will not be frustrated by having to set a comfortable volume twice, once before you login (for a screenreader, for example) and then again after you log in.

Sound volume can be adjusted in several ways, but they should all have the same visual feedback.

Mute

The Mute action (not to be confused with Silent Mode) should idempotently set the volume of the active output role through the primary sound output to zero (bug 1124005). That is, if you Mute and then Mute again, sound should still be muted. And if you Mute then make the volume Louder, it should end up just above zero.

(Mute could be a mode, switching between zero and whatever was the previous volume. That would make temporary muting more convenient, but it would mean pressing the Mute key would sometimes cause sound to start blaring, which would be unpleasant.)

Silent Mode

When Silent Mode is on and you are using the internal speaker, the phone should play only sounds that you have explicitly requested. For example, alarms, music, and sound from videos should play as normal, but ringtones, message notification sounds, and interface sound effects should not. When Silent Mode is on and you are not using the internal speaker (which probably means you are using headphones), all sounds should play as normal. For more details, see SilentMode.

If you enter or exit Silent Mode using a hardware switch, the phone should vibrate briefly (since the display might not be on): once when entering, and twice when exiting. There should be no such vibration if you toggle it on-screen.

Quieter

The Quieter action should lower the volume of the active output role through the primary sound output to whichever is higher: zero, or the highest multiple of 1/16 that is lower than the current volume.

Louder

The Louder action should raise the volume of the active output role through the primary sound output to whichever is lower: the maximum possible, or the lowest multiple of 1/16 that is higher than the current volume.

Hardware volume keys

A keyboard, phone, or tablet’s MUTE_KEY, VOLUME_DOWN_KEY, or VOLUME_UP_KEY, should trigger the Mute, Quieter, and Louder actions respectively. When the screen is off, to minimize accidental volume changes, the keys should have effect only if any audio is currently playing, and they should not turn on the screen (bug 1291458).

Output/volume change notification

sound-volume-osd.png

sound-volume-osd.mockup.jpg
full-size mockup
Erratum: This mockup needs changing to name and illustrate the primary output.

Whenever sensible, there should be visible confirmation of a manual output device change, and both visible and audible confirmation of a manual volume change. (There should be no special notification of output role changes.)

Visible confirmation in general is sensible only when the display is on. Therefore:

  • The display should not wake up just to show an output/volume change notification.

And confirmation of manual volume changes is sensible only when nothing else is demonstrating the volume change. Therefore:

  • There should be no extra visible notification if the focused window contains a system volume control.

  • There should be no extra audible confirmation if sound currently playing through the multimedia role (bug 1378961).

Apart from those cases:

  • Visible confirmation should take the form of a confirmation bubble, showing the name of the relevant role, and the icon and volume of the primary output (bug 1409696). For example, when you plug in headphones, a notification bubble should appear showing the current volume for headphones — both to reassure you that sound won’t blare from the speakers, and to remind you what the current headphones volume is. The bubble’s icon should be of a muted speaker if volume is muted, or otherwise an unmuted speaker with waves representing the approximate volume. To make clearer which key was pressed, if the bubble was not already open, the gauge should show the old value for half a second before switching to the new value. If you press the reduce-volume key when the volume is already at zero, or the increase-volume key when the volume is already at maximum, the icon and gauge should flash.

  • Audible confirmation should take the form of a subtle noise playing at the new volume (bug 1436801).

Pausing media playback when headphones are disconnected

If audio is currently playing through the multimedia role, and the output role changes because headphones or a headset was disconnected, any current player should pause, to guard against music or other embarrassing audio suddenly blaring (bug 1457531). (That is, an Mpris pause signal should be sent to whichever app is using the media hub.)

Settings

Sound volume, and the primary sound output, should be global settings — persisting across sessions and user accounts. For example, if you set the sound volume at the login screen, that volume should persist after you log in (bug 840777), and vice versa.

Other settings should be specific to individual user accounts. For example, someone who is deaf in one ear may choose mono output, while the other users of the device use the default stereo output.

Phone

settings-sound.phone.png

The “Silent Mode” switch should be followed by a system volume control (bug 1389761) for adjusting the volume of the active output role through the primary sound output. It should be labelled according to the primary sound output: “Speaker:”, “Headset:”, “Earpiece:”, etc.

Turning on “Dialpad tones”, “Keyboard sound”, or “Lock sound” (bug 1208418) should play the relevant sound once as a preview. (For “Dialpad tones”, this should be a quick sequence of three or four dialpad tones.) “Dialpad tones” should be an exact mirror of the same setting in the “Phone” app.

“Other vibrations” should cover all vibrations except for those relating to phone calls and messages (bug 1358610).

Currently, this screen houses the Phone and Messages sound settings, on the assumption that people wanting to change their ringtone or message tone will look here rather than inside those apps. In future, we may need to (a) give third-party communication apps access to these settings, (b) give third-party communication apps the ability to join this screen, or (c) demote the Phone and Messages sounds to the apps themselves.

ringtone.pngphone-settings-sound-message.png

Erratum: The “Ringtone” screen should not have a “None” option.

When you choose a sound on the “Ringtone” or “Message received” screens, it should play regardless of whether you are in Silent Mode.

The “Ringtone” screen should end with a non-scrolling Stop button, enabled whenever a ringtone is currently previewing, so that you can stop a long-playing sound. (“Message received” sounds are not long enough to warrant the same button.)

Either way, the rest of each screen should be taken up by a radio list of installed sounds of that type. The “Ringtone” list should end with a “Custom Ringtone…” item, and the “Message received” list should end with a “None” item. Choosing a sound — even the already-selected sound — should play it as well as selecting it.

Custom ringtone

(bug 1268097)

ringtone-custom-picker.phone.pngringtone-custom-analyzing.phone.pngringtone-custom-range.phone.png

Choosing “Custom Ringtone…” should open a “Choose Track” audio picker for choosing a song or other audio. If you cancel the picker, the ringtone selection should return to its previous value.

If you do choose an audio file in the picker, a “Choose Loop” dialog should then open, with a centered spinner and the word “Analyzing…” until analysis of the track is complete.

Then the dialog contents should be replaced by a visualization of the audio track, an interval slider (with non-overlapping M:SS captions) for choosing which range of the audio should loop as the ringtone, and a Play/Stop button for previewing the loop. (Play/Stop, rather than Play/Pause, because usually you will want to hear the loop from the beginning.)

If the audio is a minute or shorter, the default range should be the whole file. If the audio is longer than a minute, it is probably a whole song. Song intros are often quiet and/or less recognizable than the chorus, so the default range should be the detected chorus of the song.

If you complete choosing a custom ringtone, it should be shown in the list immediately above the “Custom Ringtone…” item, named according to the title of the track where available (for example, the MP3 “Title” tag), or otherwise the filename. That custom ringtone should remain in the list until you ever choose a different custom ringtone.

PC

System Settings should have a “Sound” panel. Most of the settings should be categorized across four tabs: “Input”, “Output” (the default tab), “Sound Effects”, and “Applications”.

sound-settings-pc-volume.jpg

Since output volume is the most urgent setting (especially if you do not have the sound menu turned on), “Output volume:” should be at the top of the panel, above these tabs. It should be followed by checkboxes for “Mute” (bug 1276063) and “Allow louder than 100%” (with a translation note advising to keep the label brief).

Whenever “Mute” is checked, the volume slider above should be insensitive. Whenever “Allow louder than 100%” is checked, its label should instead be “Allow louder than 100% (may distort sound)” (avoiding distracting mention of distortion if it is unchecked), and the slider should grow to the right (without its label moving) to include a red area for greater-than-maximum volume.

sound-menu-setting.png

At the bottom left corner of the System Settings “Sound” panel should be “Show in the menu bar:” checkboxes, “Sound volume” and “Microphone when connected”. By default in a new user’s account, “Sound volume” should be checked and “Microphone when connected” should be unchecked.

Indicators and menus

Phone

phone-sound-menu.png

The indicator should be a speaker icon the same as the title of the PC menu, except that whenever audio is playing, it should be preceded by a ▶ Play icon.

The indicator and the system volume control (bug 1389761) — identical to its counterpart in System Settings — should both represent, and the slider should adjust, the volume of the active output role through the primary sound output.

To minimize clutter, the menu should show playback controls only for those players that are currently playing, or if none are, the player that was playing most recently. The controls should not be shown for a player that cannot be controlled externally (CanControl). Similarly, the Previous/Next buttons may or may not be present, depending on the player’s capabilities.

Whenever audio is playing and the player is providing track data, this data should be shown below the controls. If the player is not providing track data, the icon and name of the player itself should be shown instead, at the same size. Either way, this item should launch or switch to the player.

PC

Microphone menu

microphone-menu.png

Whenever “Show in the menu bar: Microphone when connected” is checked, and an external microphone is connected, a microphone menu should be present in the system status area.

Whenever the microphone is muted, the volume slider should be insensitive.

“Sound Input Settings…” should open System Settings “Sound” panel to the “Input” tab.

Sound menu

whenever “Show in the menu bar: Sound volume” is checked, a sound menu should be present in the system status area.

This menu exists to let people easily change sound volume and control music playback. It should give easiest access to the volume of the primary sound output device and the primary sound input device. Other devices can be accessed through the Sound panel of System Settings.

cases.png Screenshot-2.png

Erratum: The scrub bar should not be present, and the track-specific items should go between the track info and the playback controls.

Except where specified, the menu should inherit all the normal behavior and keyboard navigation of a standard menu. Test case: With the volume slider item highlighted, press Page Down. The highlight should move to the bottom item in the menu. Test case: With the “Mute” item highlighted, press Left. The sound menu should close and the menu to its left in the panel should open.

Title

Whenever there is no primary sound output (for example, because no available output devices have been detected yet), the title of the menu should be an outline version of the speaker icon (audio-output-none-symbolic), and its accessible name should be “Volume (no devices)”.

Whenever there is a primary sound output, the title should be:

  • If sound is muted and an application tried to play sound in the past five seconds, a red speaker with a cross (audio-volume-muted-panel).

  • Otherwise, if sound is muted, a normal-colored speaker with a cross (audio-volume-muted-panel).

  • Otherwise, a speaker icon with zero to three sound waves, roughly representing the current volume for the primary output device (audio-volume-low-zero-symbolic, audio-volume-low-symbolic, audio-volume-medium-symbolic, or audio-volume-high-symbolic). The icon should be updated even if the volume is being changed while the menu is open. The accessible label should be of the form “Volume (74%)”.

As a hidden treasure, when the pointer is over the title, whether the menu is open or not, clicking a mousewheel up or down should invoke the Louder or Quieter action respectively. (As usual, this should result in a notification bubble showing the volume change.)

“Mute”

The first item, “Mute”, makes it easy to urgently silence unwanted sound using a pointing device. If the primary sound output is null (there are no available sound output devices), the item should be insensitive. Choosing it should trigger the Mute action, without the usual volume notification.

Output volume item

If there is a primary sound output, the menu should contain an item for changing its volume, followed by a separator. The Up and Down keys should navigate between the volume item and other items, just as for a normal menu item. The item should highlight when navigated to, or on mouseover, just as normal menu items do, but the item should not be activatable (clicking it or pressing Enter should not close the menu).

The item should contain three individual interactive elements.

  1. A borderless, mouseover-less button that sets the volume to zero. The icon for the button should be audio-volume-low-zero (in the default theme, a speaker with no sound waves).

  2. A slider that lets you set the volume more precisely. Whenever “Allow louder than 100%” is checked, the slider should include a red area for greater-than-maximum volume (bug 786525). Whenever sound is muted, the slider should temporarily be at zero. Unmuting should return the slider to its previous position.

  3. A borderless, mouseover-less button that sets the volume to maximum. The icon for this button should be audio-volume-high (in the default theme, a speaker with three sound waves).

Whenever the volume item is highlighted:

  • The Left and Right arrow keys should instantly decrease or increase the volume, respectively, to the nearest 5 percent step.
  • The “-” and “+” keys should instantly decrease or increase the volume, respectively, to the nearest 5 percent step.
  • Rolling a mousewheel up or down should increase or decrease the volume respectively, 10% per click.
  • None of the elements should have a visible focus ring, and Tab and Shift Tab should not do anything. (There are enough other ways to activate each of the elements, and a focus ring inside part of a highlighted menu item would be ugly.)

Changing the volume in any of these ways should not trigger the usual volume notification.

Implementation: The PulseAudio async API takes care of detecting different devices and whether there is sound coming out of them. There will be need to be a connection to either Pulse or HAL to detect the addition of new devices. This could be something like plugging in a new USB headset.

Microphone volume item

microphone-slider.png

The output volume item should be followed by a microphone volume item whenever all of these are true:

  • a VoIP (media.role=phone) or sound recording (media.role=production) audio stream is active

  • the computer has a detected external microphone
  • the microphone menu is not present.

The microphone volume item should look and behave exactly the same as the output volume item, except that the button icons should feature a microphone rather than a speaker.

Music player sections

Whenever you are logged in (that is, not in the standalone installer session or at the login screen), any music player that advertises itself over http://mpris.org/ Mpris should have its own section in the sound menu. Multiple compliant music players may result in multiple music sections, ordered alphabetically by application Name. (In future, there may even be multiple sections for the same player.)

A compliant player should not be present in the menu merely because it is installed, but should insert itself as soon as you start actively using it (e.g. playing something with it for the first time, or adding music to its library). The player’s own settings interface should also have a checkbox for whether the player should be present in the sound menu right now:

  • ☑ Show FooPlayer in the sound menu

Future work: Add a “Where?” button that shows you where the sound menu is.

A compliant player should also keep playing if you close its window while it is playing; exit if you close its window while it is not playing; and remember exact state across sessions, so that after exit and relaunch it is as if the player had never exited.

A music section should consist of an item for the music player itself, followed by any or all of track-specific custom items, a playback item, a playlist submenu, and player-global custom items. Some of these items are present only when the player is running.

For the purpose of the following sections that describe these items, the active track for a player is the track that is playing, that is paused, or that would play if the Play command was given (if this is known ahead of time). At any time, a player may or may not have an active track.

Music player item

This item should be present regardless of whether the player is running.

The first row of the item should consist of the application’s icon and Name. A player should also be able, in rare cases, to specify custom text in place of the application name (by changing the 'Identity' property string on the Mpris root interface to the full string desired).

Choosing the item should launch the player; if the player is already running, it should open or focus an existing instance. Whenever a running player does not have a sensible main window that could be opened or focused (as indicated by Mpris canRaise() = false), the player item should be insensitive.

player-item-playing.png

If the player is running and has an active track, the music player item should also include, below the player icon and name, the track or album art and three rows of text. If no track or album art is available, it should fall back to a generic track icon. The text should be on the leading side, and top-aligned next to the art. It should consist of the track name, artist, and album, one on each line (aligned the same way as the rest of the menu, regardless of what language each string is in). For each line, if there is not enough room in the menu to display all the text, it should be ellipsized in the middle; and if the data is not known, the line should be left blank. The player name and track data combined should be highlightable and activatable like a single normal menu item.

There should be a D-Bus API for exporting info about the playing track (not the same as the active track) for use elsewhere (for example, setting a custom IM status).

We need precise fonts, sizes, and spacing specified for these elements.

Track-specific custom items

The API should let the music player provide up to three track-specific custom menu items — for example, for buying, “liking”, or “banning” the song currently playing. Any of these items may be a submenu containing other items.

Playback item

To minimize clutter, the menu should show playback controls only for those players that are currently playing, or if none are, the player that was playing most recently. The controls should not be shown for a player that cannot be controlled externally (CanControl). The playback item should consist of a large Play/Pause button in the middle, with Back and Forward buttons (using ◀◀ Rewind and ▶▶ Fast-Forward symbols) on the sides if the player allows it.

If something is playing, the main button should be Pause. If nothing is playing, the main button should be Play. The Back button should be sensitive whenever the player can seek (CanSeek) or go to a previous track (CanGoPrevious). And the Forward button should be sensitive whenever the the player can seek or go to a next track (CanGoNext) (bug 782060).

The Space key should visibly activate whichever button is highlighted. However, highlighting the Back or Forward button with a keyboard should not be possible; instead, the Left and Right keys should immediately activate Back and Forward respectively whenever any of the three are highlighted (bug 733285). Apart from that, and their button appearance, each button should behave in every way like a menu item. In particular:

  • The visual style for a highlighted button should be the same as for a normal menu item in the same theme.
  • Clicking on a button, moving to a different menu item, then releasing, should activate that other item instead.
  • Conversely, clicking on a different menu item (or any menu title), moving to a button, and releasing, should activate that button (bug 779782).

  • None of the buttons should have a visible focus ring, and Tab and Shift Tab should not do anything when they are highlighted.

When they are present and sensitive, the functions and accessible labels of the buttons should be as follows.

  • The Play/Pause button should pause or resume the current track, depending on whether it is currently playing. Its accessible label should be “Pause” or “Play“, respectively.
  • If a track is currently playing, the Back button should rewind the track for however long it is pushed. If a track is not currently playing, or if the Back button is released within 0.5 seconds of being pushed, then the player should navigate to the beginning of the previous track if the current track is playing and at less than two seconds from its start, or the beginning of the current track otherwise. The accessible label for the button should be “Back (hold to rewind)” if a track is currently playing, or just “Back” otherwise.

  • If a track is currently playing, the Forward button should fast-forward the track for however long it is pushed. If a track is not currently playing, or if the Forward button is released within 0.5 seconds of being pushed, then the player should navigate to the beginning of the track after the next one if the current track is playing and at less than two seconds from its end, or the beginning of the next track otherwise. The accessible label for the button should be “Forward (hold to fast-forward)” if a track is currently playing, or just “Forward” otherwise.

sound-menu-v2.jpg

Playlist submenu

The MPRIS API should let the player expose icons and names of playlists, even when the player is not running.

The playlist item should be present in the menu only if the music player is currently exposing any playlists. If the player also indicates that it is playing (or is paused partway through) a playlist, the submenu title should be the name of the playlist. Otherwise, it should be “Choose Playlist”.

The submenu itself should contain items named for each of the playlists. Selecting an item should start playing that playlist.

Player-global custom items

The API should let the music player provide up to three player-global custom menu items, even when the player is not running. Any of these items may be a submenu containing other items, for setting equalizer settings for example.

“Sound Settings…”

Whenever you are logged in (that is, not in the standalone installer session or at the login screen), the menu should end with a separator, followed by a “Sound Settings…” item that opens the Sound panel in System Settings. Launch feedback for this item should be exactly the same as when you launch an application any other way.

Music player integration

To register in the menu, each player needs to implement the MPRIS2 spec. The service which controls the menu automatically detects MPRIS clients appearing on DBus and reacts accordingly. It is essential that the implementation populates the DesktopEntry property on the MPRIS root interface. Access to the Desktop file is mandatory to register successfully.

Registration will need to be disabled if a user would prefer for a certain player not to be controllable from the menu. Guidelines as to how this preference is to be represented in the UI of the client (specifically Rhythmbox, Banshee or Amarok) has been included below. We would like to provide a unified music player experience on the Ubuntu platform therefore we strongly urge application developers to follow this spec.

To remove a player from the menu the client can either use the dbus method 'blacklist-media-player(string desktop, bool blacklist)' on 'com.canonical.indicators.sound' (as of release 0.5.9) or alternatively it will need to write to the indicator-sound gsettings. The settings id is "com.canonical.indicators.sound" and the key is "blacklisted-media-players". The client should add the name of the desktop file (just the name, not the full path and minus the .desktop suffix) to this array of strings. The indicator sound service will remember each client (provided that it is not blacklisted) which has successfully registered so that when the media client is not running it will still appear in a hibernated state in the menu. Any successful changes to the blacklisted gsettings should be automatically reflected in the menu. A player can query its blacklisted state using the dbus api 'IsBlacklisted(string desktop-id)' (this will return a boolean depending on its presence in the blacklist obviously). Again alternatively you can just talk directly with gsettings. Please use the gsettings (either directly or through the dbus api) to save and fetch the state of a players permissions inorder to minimize bugs.

In order to test client registration, application developers should fetch the latest lp:indicator-sound release. As of release 0.5.3 this new way of registering has been implemented.

Future work
  • Integrate Totem.
  • Show multiple sound outputs separately.
  • Sound configuration and management.
  • Consider coloured volume sliders

  • Should equalizer settings go into the system? (or should they be track-specific/application-specific)
  • Respect surround-sound in the menu and in the settings window
  • Fade out other audio automatically if you receive a VoIP call? (Where would a setting for this go?)

Session variations

This is a summary only; the specifications of individual items are authoritative.

In the guest session and the live session, the indicator and menu should be present as normal.

In the standalone installer session and the login screen, only the mute and volume items should be present.

Risks
  • People may be disconcerted that something appearing to be a volume menu contains items for a music player. Or they may think that the presence of those items means that the volume is specific to the music player rather than system-wide. We should perform user testing once we have a basic implementation.

Alternative proposed designs

Handling unknown audio jack devices

(bug 1239605)

When a device is plugged into an audio jack that can’t distinguish between headphones, headsets, or microphones, Ubuntu should display an “Unknown Audio Device” dialog.

sound-device-unknown.png

If you unplug the device before responding to the dialog, the dialog should close automatically. “Headphones”, “Headset”, and “Microphone” should be buttons that close the dialog, just like the “Cancel” and “Sound Settings…” buttons do. The “Sound Settings…” button should also open the “Sound” panel of System Settings.

sound-device-settings.png

If an unknown audio device has ever been connected in this Ubuntu installation (to avoid it cluttering the interface on PCs where this is not an issue), the bottom of that “Sound” panel should have a “When an unknown audio device is plugged in:” menu. The menu should have options “Treat as headphones”, “Treat as headset”, “Treat as microphone”, “Ignore”, a separator, and finally “Ask what to do”.

If, in the prompt, you choose “Use this choice for unknown devices from now on” and then any of the options (except “Sound Settings…”), that should change the setting in the menu in System Settings. (Choosing “Cancel” should change the setting to “Ignore”.)

High volume limits and warnings

EN 50332 (unfortunately not freely available) defines a “programme simulation noise” test signal for testing audio volume.

EU standard EN 60950-1/Al2 (unfortunately not freely available) defines:

  • acoustic output as the 30-second A-weighted equivalent sound pressure level LAeq,T when playing that programme simulation noise;

  • electrical output as the output through the analogue electrical output socket when playing that programme simulation noise.

It then requires that “A personal music player and earphones or headphones intended to be used with personal music players shall…”, while headphones/earphones are being used:

  1. “Protect the user from unintentional acoustic outputs exceeding” 85 dBA.
  2. “Have a standard acoustic output level not exceeding” 85 dBA, “and automatically return to an output level not exceeding those mentioned above when the power is switched off.
  3. “Provide a means to actively inform the user of the increased sound pressure when the equipment is operated with an acoustic output exceeding” 85 dBA. “Any means used shall be acknowledged by the user before activating a mode of operation which allows for an acoustic output exceeding those mentioned above. The acknowledgement does not need to be repeated more than once every 20 h of cumulative listening time.”
  4. “Have a warning as specified”, “on the equipment, or on the packaging, or in the instruction manual”, with a particular graphic and words to the effect of “To prevent possible hearing damage, do not listen at high volume levels for long periods.”.
  5. “Not exceed … acoustic output” of 100 dBA or electrical output of 150 mV.

To meet these requirements (bug 1480912):

sound-volume-confirmation.png

(a, c) Whenever you increase volume, for the first time in 20 hours, such that acoustic output would be more than 85 dBA, an “Increase Volume” dialog should appear, with primary text “High volume can damage your hearing.” and “Cancel” and “OK” buttons. Unlike any other dialog, pressing the Volume Down button should be equivalent to choosing “Cancel”. If you choose “OK”, the volume change should happen as normal. (Normal practice would be to label the button “Increase Anyway” or similar, but that would risk the button becoming a bragging-mark like “Parental Advisory”.) So that both buttons do what you expect and the dialog does not disappear unexpectedly, any role change should be postponed as long as the dialog is up (bug 1504065).

(b) Whenever you start up or wake the device, if the volume is such that the acoustic output would be greater than 85 dBA, it should be reduced to 85 dBA.

(d) Any phone or tablet shipped with Ubuntu should include the volume warning in its packaging or manual.

(e) Sound output should never be greater than 100 dBA, and electric output should never be greater than 150 mV.

Sound (last edited 2017-03-30 15:52:16 by mpt)