clarifies behavior of the music player item on the phone
+ bug 1240979 (After claiming to connect no audio is routed to the headset)
|Deletions are marked like this.||Additions are marked like this.|
|Line 25:||Line 25:|
|* If you connect headphones, or any audio output device [[http://www.tuicool.com/articles/Nfm2my|except a secondary display]], the primary output should change to that device.||* If you connect or pair headphones, or any audio output device [[http://www.tuicool.com/articles/Nfm2my|except a secondary display]], the primary output should change to that device (bug Bug:1240979).|
For help with using sound in Ubuntu, see Ubuntu Help online.
- Primary sound output
- Volume/output notification
- Sound menu
- Handling unknown audio jack devices
- Sound settings in general
- Unresolved issues
Primary sound output
Various interface elements need to know what the primary sound output device is. For example, this is the device:
- that should change volume when you press the Quieter/Louder keys on a PC keyboard, or the volume buttons/rocker on a phone
whose volume should be shown in the sound menu on the PC and phone.
In implementation terms, this is the default PulseAudio device (device 0).
The primary output should depend on whatever happened most recently:
- If you manually choose an output device in the “Sound” settings panel or a similar utility, that choice should be followed.
- If an app changes the primary output itself (for example, if a phone or Voip app changes primary output to the earpiece), that choice should be followed.
- If the app that set the current primary output crashes, the primary output should return to whichever existing audio output device was the primary output most recently.
If you disconnect the previous primary output, the new primary output should be whichever remaining audio output device was the primary output most recently.
This definition does not yet take into account simultaneous use of multiple audio outputs (for example, headphones for an audio call and speakers for music).
When the primary sound output changes for any reason, a confirmation bubble should appear showing the icon, and volume, of the new primary output.
Sound volume should be remembered independently for each input and output device, so that you don’t need to adjust it repeatedly when switching devices. For example, when you plug in headphones, output volume should be the same as when you last used headphones, even if Ubuntu has restarted since then.
Sound volume should also be independent of which user — or whether any user — is logged in. This way, if you mute or turn down sound at the login screen, you will not be surprised or frustrated by sound playing loudly after you log in. And in the common case where a device has only one user, you will not be frustrated by having to set a comfortable volume twice, once before you login (for a screenreader, for example) and then again after you log in.
Sound volume can be adjusted in several ways, but they should all have the same visual feedback.
The Mute action should idempotently set the volume to zero (bug 1124005) for the primary sound output. That is, if you Mute and then Mute again, sound should still be muted. And if you Mute then make the volume Louder, it should end up just above zero.
(Mute could be a mode, switching between zero and whatever was the previous volume. That would make temporary muting more convenient, but it would mean pressing the Mute key would sometimes cause sound to start blaring, which would be unpleasant.
The Quieter action should lower the volume to whichever is higher: zero, or the highest multiple of 1/16 that is lower than the current volume.
The Louder action should raise the volume to whichever is lower: the maximum possible, or the lowest multiple of 1/16 that is higher than the current volume.
Hardware volume keys
When the primary sound output changes for any reason, or the sound volume changes for any reason (unless otherwise specified), a subtle noise should play for you to gauge the volume the change in volume, and it should be shown in a confirmation bubble with a volume gauge (bug 1232633). For example, this bubble should appear if you change volume using a remote control or a voice command.
The bubble’s icon should be of a muted speaker if volume is muted, or otherwise an unmuted speaker with waves representing the approximate volume. To make clearer which key was pressed, if the bubble was not already open, the gauge should show the old value for half a second before switching to the new value. If you press the reduce-volume key when the volume is already at zero, or the increase-volume key when the volume is already at maximum, the icon and gauge should flash.
Visually, the bubble should not contain any text. But for accessibility purposes, the bubble should expose title text of the form “Sound muted” or “Volume: 60 percent”, as appropriate.
Whenever the phone is in Silent Mode, the symbol and text “This phone is in Silent Mode.” (neither of them scrolling) should appear atop the “Sound” screen, and also the “Ringtone” and “Message received” screens (to explain why you can’t hear the preview on those screens). Whenever the phone is not in Silent Mode, the symbol and text should not be present at all.
Apart from that, the “Ringtone” screen should consist of a non-scrolling “Stop Playing” button whenever you are not in Silent Mode, followed by a radio list of installed ringtones and “None”. “Stop Playing” should be sensitive whenever a sound is currently playing. The “Message received” screen should similarly contain a radio list of installed alert sounds and “None”, but without the “Stop Playing” button because the sounds are too brief for it to be useful. On both screens, choosing an item — even the already-selected item — should play the sound as well as selecting it. Similarly, turning on “Keyboard sound” or “Lock sound” should play the relevant sound once as a preview.
Indicator and menu
The indicator should be a speaker icon the same as the title of the PC menu, except that whenever audio is playing, it should be preceded by a ▶ Play icon.
Both the indicator and the volume slider should show the audio volume whenever audio is playing. Whenever audio is not playing, they should show the ringer/alert volume.
The menu should always show exactly only one set of playback controls, for the player that was playing most recently. For example, if audio is currently playing, the controls should be for that player. The Previous/Next buttons may or may not be present, depending on the player’s capabilities.
Whenever audio is playing and the player is providing track data, this data should be shown below the controls. If the player is not providing track data, the icon and name of the player itself should be shown instead, at the same size. Either way, this item should launch or switch to the player.
When Silent Mode is on and you are using the internal speaker, the phone should play only sounds that you have explicitly requested. For example, alarms, music, and sound from videos should play as normal, but ringtones, message notification sounds, and interface sound effects should not. When Silent Mode is on and you are not using the internal speaker (which probably means you are using headphones), all sounds should play as normal. For more details, see SilentMode.
System Settings should have a “Sound” panel. Most of the settings should be categorized across four tabs: “Input”, “Output” (the default tab), “Sound Effects”, and “Applications”.
Since output volume is the most urgent setting (especially if you do not have the sound menu turned on), “Output volume:” should be at the top of the panel, above these tabs. It should be followed by checkboxes for “Mute” (bug 1276063) and “Allow louder than 100%” (with a translation note advising to keep the label brief).
Whenever “Mute” is checked, the volume slider above should be insensitive. Whenever “Allow louder than 100%” is checked, its label should instead be “Allow louder than 100% (may distort sound)” (avoiding distracting mention of distortion if it is unchecked), and the slider should grow to the right (without its label moving) to include a red area for greater-than-maximum volume.
The sound menu should let people easily change sound volume and control music playback. It should give easiest access to the volume of the primary sound output device and the primary sound input device. Other devices can be accessed through the Sound panel of System Settings.
Errata: See The microphone item for details of when the microphone item is present.
See The microphone item for details of when the microphone item is present.
Except where specified, the menu should inherit all the normal behavior and keyboard navigation of a standard menu. Test case: With the volume slider item highlighted, press Page Down. The highlight should move to the bottom item in the menu. Test case: With the “Mute” item highlighted, press Left. The sound menu should close and the menu to its left in the panel should open.
At the bottom left corner of the System Settings “Sound” panel should be a checkbox, “Show sound volume in the menu bar”. It should be checked by default for a new user account. Whenever it is checked, the sound menu should be present in the menu bar.
Whenever there is no primary sound output device (because no available output devices have been detected yet), the title of the menu should be an outline version of the speaker icon (audio-output-none-symbolic), and its accessible name should be “Volume (no devices)”.
Whenever any output devices are available, the title should be:
If sound is muted and an application tried to play sound in the past five seconds, a red speaker with a cross (audio-volume-muted-panel).
Otherwise, if sound is muted, a normal-colored speaker with a cross (audio-volume-muted-panel).
Otherwise, a speaker icon with zero to three sound waves, roughly representing the current volume for the primary output device (audio-volume-low-zero-symbolic, audio-volume-low-symbolic, audio-volume-medium-symbolic, or audio-volume-high-symbolic). The icon should be updated even if the volume is being changed while the menu is open. The accessible label should be of the form “Volume (74%)”.
As a hidden treasure, when the pointer is over the title, whether the menu is open or not, clicking a mousewheel up or down should invoke the Louder or Quieter action respectively. (As usual, this should result in a notification bubble showing the volume change.)
The first item, “Mute”, makes it easy to urgently silence unwanted sound using a pointing device. If the primary sound output is null (there are no available sound output devices), the item should be insensitive. Choosing it should trigger the Mute action, without the usual volume notification.
Output volume item
If there is a primary sound output device, the menu should contain an item for changing its volume, followed by a separator. The Up and Down keys should navigate between the volume item and other items, just as for a normal menu item. The item should highlight when navigated to, or on mouseover, just as normal menu items do, but the item should not be activatable (clicking it or pressing Enter should not close the menu).
The item should contain three individual interactive elements.
A borderless, mouseover-less button that sets the volume to zero. The icon for the button should be audio-volume-low-zero (in the default theme, a speaker with no sound waves).
A slider that lets you set the volume more precisely. Whenever “Allow louder than 100%” is checked, the slider should include a red area for greater-than-maximum volume (bug 786525). Whenever sound is muted, the slider should temporarily be at zero. Unmuting should return the slider to its previous position.
A borderless, mouseover-less button that sets the volume to maximum. The icon for this button should be audio-volume-high (in the default theme, a speaker with three sound waves).
Whenever the volume item is highlighted:
- The Left and Right arrow keys should instantly decrease or increase the volume, respectively, to the nearest 5 percent step.
- The “-” and “+” keys should instantly decrease or increase the volume, respectively, to the nearest 5 percent step.
- Rolling a mousewheel up or down should increase or decrease the volume respectively, 10% per click.
- None of the elements should have a visible focus ring, and Tab and Shift Tab should not do anything. (There are enough other ways to activate each of the elements, and a focus ring inside part of a highlighted menu item would be ugly.)
Changing the volume in any of these ways should not trigger the usual volume notification.
Implementation: The PulseAudio async API takes care of detecting different devices and whether there is sound coming out of them. There will be need to be a connection to either Pulse or HAL to detect the addition of new devices. This could be something like plugging in a new USB headset.
Microphone volume item
If a VoIP (media.role=phone) or sound recording (media.role=production) audio stream is active, and the computer has a detected microphone, the output volume item should be followed by a microphone volume item. The microphone volume item should look and behave exactly the same as the output volume item, except that the button icons should feature a microphone rather than a speaker.
Music player sections
Whenever you are logged in (that is, not in the standalone installer session or at the login screen), any music player that advertises itself over http://mpris.org/ Mpris should have its own section in the sound menu. Multiple compliant music players may result in multiple music sections, ordered alphabetically by application Name. (In future, there may even be multiple sections for the same player.)
A compliant player should not be present in the menu merely because it is installed, but should insert itself as soon as you start actively using it (e.g. playing something with it for the first time, or adding music to its library). The player’s own settings interface should also have a checkbox for whether the player should be present in the sound menu right now:
☑ Show FooPlayer in the sound menu
Future work: Add a “Where?” button that shows you where the sound menu is.
A compliant player should also keep playing if you close its window while it is playing; exit if you close its window while it is not playing; and remember exact state across sessions, so that after exit and relaunch it is as if the player had never exited.
A music section should consist of an item for the music player itself, followed by any or all of track-specific custom items, a playback item, a playlist submenu, and player-global custom items. Some of these items are present only when the player is running.
For the purpose of the following sections that describe these items, the active track for a player is the track that is playing, that is paused, or that would play if the Play command was given (if this is known ahead of time). At any time, a player may or may not have an active track.
Music player item
This item should be present regardless of whether the player is running.
The first row of the item should consist of the application’s icon and Name. A player should also be able, in rare cases, to specify custom text in place of the application name (by changing the 'Identity' property string on the Mpris root interface to the full string desired).
Choosing the item should launch the player; if the player is already running, it should open or focus an existing instance. Whenever a running player does not have a sensible main window that could be opened or focused (as indicated by Mpris canRaise() = false), the player item should be insensitive.
If the player is running and has an active track, the music player item should also include, below the player icon and name, the track or album art and three rows of text. If no track or album art is available, it should fall back to a generic track icon. The text should be on the leading side, and top-aligned next to the art. It should consist of the track name, artist, and album, one on each line (aligned the same way as the rest of the menu, regardless of what language each string is in). For each line, if there is not enough room in the menu to display all the text, it should be ellipsized in the middle; and if the data is not known, the line should be left blank. The player name and track data combined should be highlightable and activatable like a single normal menu item.
There should be a D-Bus API for exporting info about the playing track (not the same as the active track) for use elsewhere (for example, setting a custom IM status).
We need precise fonts, sizes, and spacing specified for these elements.
Track-specific custom items
The API should let the music player provide up to three track-specific custom menu items — for example, for buying, “liking”, or “banning” the song currently playing. Any of these items may be a submenu containing other items.
The playback item should be present only for the most recently playing player, and only if it can be controlled externally (CanControl). The playback item should consist of a large Play/Pause button in the middle, with Back and Forward buttons (using ◀◀ Rewind and ▶▶ Fast-Forward symbols) on the sides.
If something is playing, the main button should be Pause. If nothing is playing, the main button should be Play. The Back button should be sensitive whenever the player can seek (CanSeek) or go to a previous track (CanGoPrevious). And the Forward button should be sensitive whenever the the player can seek or go to a next track (CanGoNext) (bug 782060).
The Space key should visibly activate whichever button is highlighted. However, highlighting the Back or Forward button with a keyboard should not be possible; instead, the Left and Right keys should immediately activate Back and Forward respectively whenever any of the three are highlighted (bug 733285). Apart from that, and their button appearance, each button should behave in every way like a menu item. In particular:
- The visual style for a highlighted button should be the same as for a normal menu item in the same theme.
- Clicking on a button, moving to a different menu item, then releasing, should activate that other item instead.
Conversely, clicking on a different menu item (or any menu title), moving to a button, and releasing, should activate that button (bug 779782).
- None of the buttons should have a visible focus ring, and Tab and Shift Tab should not do anything when they are highlighted.
When they are present and sensitive, the functions and accessible labels of the buttons should be as follows.
- The Play/Pause button should pause or resume the current track, depending on whether it is currently playing. Its accessible label should be “Pause” or “Play“, respectively.
If a track is currently playing, the Back button should rewind the track for however long it is pushed. If a track is not currently playing, or if the Back button is released within 0.5 seconds of being pushed, then the player should navigate to the beginning of the previous track if the current track is playing and at less than two seconds from its start, or the beginning of the current track otherwise. The accessible label for the button should be “Back (hold to rewind)” if a track is currently playing, or just “Back” otherwise.
If a track is currently playing, the Forward button should fast-forward the track for however long it is pushed. If a track is not currently playing, or if the Forward button is released within 0.5 seconds of being pushed, then the player should navigate to the beginning of the track after the next one if the current track is playing and at less than two seconds from its end, or the beginning of the next track otherwise. The accessible label for the button should be “Forward (hold to fast-forward)” if a track is currently playing, or just “Forward” otherwise.
The MPRIS API should let the player expose icons and names of playlists, even when the player is not running.
The playlist item should be present in the menu only if the music player is currently exposing any playlists. If the player also indicates that it is playing (or is paused partway through) a playlist, the submenu title should be the name of the playlist. Otherwise, it should be “Choose Playlist”.
The submenu itself should contain items named for each of the playlists. Selecting an item should start playing that playlist.
Player-global custom items
The API should let the music player provide up to three player-global custom menu items, even when the player is not running. Any of these items may be a submenu containing other items, for setting equalizer settings for example.
Whenever you are logged in (that is, not in the standalone installer session or at the login screen), the menu should end with a separator, followed by a “Sound Settings…” item that opens the Sound panel in System Settings. Launch feedback for this item should be exactly the same as when you launch an application any other way.
(If PulseAudio is not running when you choose “Sound Settings…”, gnome-volume-control should try to start it. If it fails, it should invoke Apport during the development cycle, or an error alert in a stable release. This is outside the scope of the menu; it should be part of the specification for the Sound Settings themselves.)
Music player integration
To register in the menu, each player needs to implement the MPRIS2 spec. The service which controls the menu automatically detects MPRIS clients appearing on DBus and reacts accordingly. It is essential that the implementation populates the DesktopEntry property on the MPRIS root interface. Access to the Desktop file is mandatory to register successfully.
Registration will need to be disabled if a user would prefer for a certain player not to be controllable from the menu. Guidelines as to how this preference is to be represented in the UI of the client (specifically Rhythmbox, Banshee or Amarok) has been included below. We would like to provide a unified music player experience on the Ubuntu platform therefore we strongly urge application developers to follow this spec.
To remove a player from the menu the client can either use the dbus method 'blacklist-media-player(string desktop, bool blacklist)' on 'com.canonical.indicators.sound' (as of release 0.5.9) or alternatively it will need to write to the indicator-sound gsettings. The settings id is "com.canonical.indicators.sound" and the key is "blacklisted-media-players". The client should add the name of the desktop file (just the name, not the full path and minus the .desktop suffix) to this array of strings. The indicator sound service will remember each client (provided that it is not blacklisted) which has successfully registered so that when the media client is not running it will still appear in a hibernated state in the menu. Any successful changes to the blacklisted gsettings should be automatically reflected in the menu. A player can query its blacklisted state using the dbus api 'IsBlacklisted(string desktop-id)' (this will return a boolean depending on its presence in the blacklist obviously). Again alternatively you can just talk directly with gsettings. Please use the gsettings (either directly or through the dbus api) to save and fetch the state of a players permissions inorder to minimize bugs.
In order to test client registration, application developers should fetch the latest lp:indicator-sound release. As of release 0.5.3 this new way of registering has been implemented.
- Integrate Totem.
- Show multiple sound outputs separately.
- Sound configuration and management.
- Should equalizer settings go into the system? (or should they be track-specific/application-specific)
- Respect surround-sound in the menu and in the settings window
- Fade out other audio automatically if you receive a VoIP call? (Where would a setting for this go?)
This is a summary only; the specifications of individual items are authoritative.
In the guest session and the live session, the indicator and menu should be present as normal.
In the standalone installer session and the login screen, only the mute and volume items should be present.
- People may be disconcerted that something appearing to be a volume menu contains items for a music player. Or they may think that the presence of those items means that the volume is specific to the music player rather than system-wide. We should perform user testing once we have a basic implementation.
Alternative proposed designs
See also “Music controller” by Ryan Henricus.
Handling unknown audio jack devices
When a device is plugged into an audio jack that can’t distinguish between headphones, headsets, or microphones, Ubuntu should display an “Unknown Audio Device” dialog.
If you unplug the device before responding to the dialog, the dialog should close automatically. “Headphones”, “Headset”, and “Microphone” should be buttons that close the dialog, just like the “Cancel” and “Sound Settings…” buttons do. The “Sound Settings…” button should also open the “Sound” panel of System Settings.
If an unknown audio device has ever been connected in this Ubuntu installation (to avoid it cluttering the interface on PCs where this is not an issue), the bottom of that “Sound” panel should have a “When an unknown audio device is plugged in:” menu. The menu should have options “Treat as headphones”, “Treat as headset”, “Treat as microphone”, “Ignore”, a separator, and finally “Ask what to do”.
If, in the prompt, you choose “Use this choice for unknown devices from now on” and then any of the options (except “Sound Settings…”), that should change the setting in the menu in System Settings. (Choosing “Cancel” should change the setting to “Ignore”.)
Sound settings in general
Sound volume, and the primary sound output, should be global settings — persisting across sessions and user accounts. For example, if you set the sound volume at the login screen, that volume should persist after you log in (bug 840777), and vice versa.
Other settings should be specific to individual user accounts. For example, someone who is deaf in one ear may choose mono output, while the other users of the device use the default stereo output.
- We need a high-fidelity visual mockup to help us decide the width of the track metadata and scrub bar, and therefore the overall width of the menu.
- We need an animated mockup of each item being highlighted as you navigate through it.