BrightnessAndDisplays

Revision 44 as of 2016-09-22 15:55:31

Clear message

Whenever a device’s hardware does not include brightness controls, Ubuntu should provide software UI for adjusting brightness, taking into account ambient light when known. You should also be able to configure external displays, either as mirrors or as individual spaces for windows/surfaces. The primary UI is a System Settings panel named “Brightness”, “Brightness & Displays”, or “Displays”, depending on the device.

(X) not possible (./) defined Warning /!\ undefined

Use case

Phone

Tablet

PC

see current brightness level

(./) just look at the display

adjust brightness manually

(./) System Settings

Warning /!\

Warning /!\

set automatic adjustment

(./) System Settings

Warning /!\

Warning /!\

Settings

Brightness and display settings are intertwined, so they make most sense in the same panel of System Settings. However, some devices can use only the built-in display, where brightness is the only setting; some can have zero or one external display; and others can have any number of external displays. And devices that allow any number of configurable displays are likely to have hardware brightness UI for the main display, making the GUI for the same setting less interesting.

Therefore, the name, icon, and contents of the settings panel should vary depending on the device:

Brightness if it allows only a built-in-display

Brightness & Display if it allows one external display

Displays if it allows multiple external displays

settings-brightness-only.access.png

settings-brightness-display.access.png

settings-displays.access.png

settings-brightness-only.narrow.png

settings-brightness-display-on.narrow.png

settings-displays.narrow.png

When there is only one display, the sun in the icon should reflect the current brightness level.

Brightness of the built-in display (“Brightness” and “Brightness & Display”)

settings-brightness-only.narrow.png

The panel should begin with a slider labelled “Display brightness:” if no external display is connected, or “Built-in display brightness:” for clarity if the one maximum external display is connected.

If the device has an ambient light sensor, the slider should be followed by the “Adjust automatically” checkbox and its caption (bug 1314672).

The goal of the “Adjust automatically” setting is to maximize the average time you can use the device before feeling the need to adjust brightness manually. This is achieved by frequently adjusting the emitted brightness of the display based on the ambient brightness of the surroundings (bug 1328404). So there is a function representing the emitted brightness desired at each level of ambient brightness detectable by the light sensor.

phone-brightness-curve-automatic.png

Because different hardware has different light sensors and displays, this function should be settable based on the hardware model.

Because users have different preferences, the function should also be user-adjustable. This should be achieved by altering the display brightness manually while “Adjust automatically” is checked. Therefore, the “Display brightness:” slider should be enabled regardless of whether “Adjust automatically” is checked or unchecked.

If it is checked and you adjust the brightness manually, the function that determines the automatic brightness should be adjusted such that, in future, the current ambient light value would result in your just-chosen brightness being chosen automatically (bug 1314678).

To achieve this, the function that generates emitted brightness E from the ambient brightness A should be a curve connecting from three to five coordinates of (A, E). Where Amin, Amax, Emin, and Emax are the minimum and maximum possible values:

  • If you (or the default values for the handset model) set Emin at Ap > Amin, the function should include E = Emin for AminAAp.

    brightness-curve-automatic-flat-minimum.png

  • If you (or the default values for the handset model) set Emax at Aq < Amax, the function should include E = Emax for Aq ≤ A ≤ Amax.

    brightness-curve-automatic-flat-maximum.png

  • If you manually set a brightness (Ar, Er) where any previously-set points exist with AAr and E > Er, those other points should be discarded from the curve, so that the screen does not brighten when surroundings dim.

  • If you manually set a brightness (As, Es) where any previously-set points exist with AAs and E < Es, those other points should be discarded from the curve, so that the screen does not dim when surroundings brighten.

    brightness-curve-automatic-discard-overbright.png brightness-curve-automatic-discard-overdim.png

  • If you manually set a brightness such that the set of coordinates includes more than one for which Amin < A < Amax and Emin < E < Emax, all of them except the newest one should be discarded, so that the curve is not too complex.

  • A new brightness curve should not be saved until you dismiss the brightness slider you are using (for example, by closing the indicator menu), so that a brief experiment does not obliterate a careful previous curve.

Wi-Fi display and MHL (“Brightness & Display”)

settings-brightness-display-external.narrow.png

“Wi-Fi display” should be present if it is possible for the device to use Wi-Fi display, with a summary value of “Connected” or “Not connected”. It should navigate to the “Wi-Fi display” screen.

The “External display” switch should be:

  • present if it is possible for the device to use only one external display (for example, by MHL or Wi-Fi);

  • enabled if an MHL display is detected, or if a Wi-Fi display has been chosen;

  • on if an external display is currently in use.

Selecting, reordering, and detecting displays (“Displays”)

settings-displays-carousel.narrow.png

settings-displays-carousel.wide.png

If it is possible for the device to use more than one external display, then the view should begin with a carousel of displays, in the same order as implied by keyboard shortcuts for switching displays or moving windows.

If, with the carousel at its minimum height, the panel contents takes up less than the screen size, the carousel should expand vertically to fill any remaining space.

If there are too many displays to fit in the panel width, the carousel should scroll horizontally. If so, then whichever display System Settings is open on should always be shown by default, plus any displays that are mirroring/extending it if possible.

In the carousel, any displays that extend each other should touch, in the correct order.

Any display B that is a mirror of another display A should be indicated by an arrow from A to B.

Any display currently not used should be indicated by white text on a black background.

Pressing on a display (or a group of extended displays) should show a translucent drag preview, and change the cursor (if any) to a grabbing hand, to indicate that dragging will reorder the displays. In addition, Left+Right keys should change the selection, with Ctrl+Left and Ctrl+Right animatedly reordering the currently-selected display.

The “Detect Displays” button should be in the corner of the carousel area if there is room, or below it if there is not.

Display-specific settings (“Brightness & Display” and “Displays”)

settings-displays-specific.narrow.png

settings-displays-specific.wide.png

If any displays are connected that can change rotation, resolution, or scaling, the rest of the panel should begin with settings for that display if there is only one, or the display that is currently selected in the carousel:

  • The display name should be bold, in “” quotes, and ellipsized in the middle if necessary.

  • The switch opposite the display name should control whether that display is being used at all. Whenever only one display is detected, or whenever every other display is Off, the selected display’s switch should be both On and disabled (to prevent you from turning off the only display that is in use).

  • The “Use as:” radio menu should be present only if the device allows multiple external displays. It should contain a “Separate display” item (the default), then a separator if there are any following items, then items of the form ‘Mirror of “{Display name}”’ for any other display. The menu should be disabled, and set to “Separate display”, whenever the selected display is the only display. Otherwise, any “Mirror of…” item in the menu should be disabled if (a) that display is not capable of the same resolution as the selected one, or (b) that display is already being mirrored by some other display (to avoid loops).

  • The “Rotation:” radio menu should have options “None” (the default), “90° clockwise”, “180°”, and “270°”. If the display is set to be used as a mirror, and it could not function as a mirror if rotated, then the two items that are in the wrong aspect ratio should be disabled, with the current value changing to the same as the origin display if necessary.

  • Resolution:” should be followed by static text if the connection protocol does not allow changing the resolution for this display, or a radio menu if it does (bug 1603175). The radio menu should have options for each available resolution for the display, taking the rotation into account: for example “1280 × 800” if Rotation is set to “None” or “180°”, but “800 × 1280” if it is set to “90° clockwise” or “270°”. Whenever the display is set as a mirror, the “Resolution:” menu should be disabled and set to the same value as the origin display.

General options (“Displays”)

settings-displays-general.narrow.png

settings-displays-general.wide.png

Launcher placement:” should have an item for “All displays”, then a separator if there are any other displays, then a color-swatched item for each detected display.

PC

displays-pc.png

Errata: • “Scale for menu and title bars:” should be “Size of menu bar and title bars:”. • “Scale for window contents:” should be “Scale window contents:”.

The heading for the display-specific settings should be the name of the selected display, followed by a switch for

Scaling

Unfortunately three separate settings exist for scaling interface elements:

  • ui-scale for Unity elements — Launcher, menu bar, Dash, HUD, and window title bars;

  • scaling-factor for GTK applications;

  • text-scaling-factor for extra scaling of text in GTK applications, beyond that already provided by scaling-factor.

Ideally, we would combine these into a single slider that scales everything on the selected display. But to make things worse, while ui-scale is display-specific (so it can have different values when you connect a high-resolution notebook to a low-resolution projector), text-scaling-factor and scaling-factor are display-independent. This means that when you change the display you are focused on — for example, when you give a presentation — you will often need to change the application scaling to suit that display.

So whenever only one display is On, and its scaling-factor and text-scaling-factor match up with its ui-scale, the Displays panel should have only one slider, “Size of elements on screen:”, which adjusts all three scaling factors at once.

displays-pc-scale-everything.png display-pc-scale-all-the-things.png

But whenever multiple displays are On, or whenever one display is On but its scaling-factor and/or text-scaling-factor don’t match up with its ui-scale, the Displays panel should have two sliders, the display-specific “Size of menu bar and title bars:” (ui-scale), and the display-independent “Scale window contents:” (scaling-factor and text-scaling-factor).

displays-pc-scale.png displays-pc-scale-windows.png

Whenever you choose “Match Selected Display”, the “Scale window contents:” setting should change to match the “Size of menu bar and title bars:” setting, such that (for example) the text in an app’s context menus is exactly the same size as the text in the menu bar on that display.

This means that if only one display is on, with disparate scale settings so that two sliders are present, and you choose “Match Selected Display”, the two sliders and the “Match Selected Display” button should disappear, replaced by the single “Size of elements on screen:” slider.

Applying settings changes

display-settings-apply.png display-settings-applied.png

Like several other places in System Settings, rotation resolution and scale settings should not be instant-apply — in this case, because it saves a lot of time to change them all at once. (And in the worst case, a display might become unusable after applying only the first of the changes that you intend to make. This is especially important on a phone, where — whenever an external display is on — the phone display is either off or being used as a touchpad.)

Unlike other places in System Settings, this problem applies to settings at the top level of a category, and they are interdependent with other settings at the top level. So we can’t reasonably take the usual approach of putting the non-instant-apply settings inside a confirm-to-apply dialog.

Instead, at the bottom of the screen should be a floating (always-visible) bar containing an “Apply Changes…” button. This button should be disabled whenever there are no rotation resolution or scale changes to apply. The label should end in an ellipsis because choosing it should apply the changes, then open a confirmation dialog for you to verify that the display has improved. If you choose “Revert”, or do not choose anything before the time remaining counts down to zero, the dialog should close, then the settings should revert.

display-changes-discard.png

If you navigate away from the Displays view, or close System Settings, without applying the changes first, a dialog should appear notifying you that ‘You haven’t applied changes to settings for “{name of display}”.’. Choosing “Discard” should restore the controls to their original state before navigating/closing. (It should not contain an “Apply…” button itself, because potentially this would entail two consecutive dialogs followed by System Settings closing, which would be quite awkward.)

“Wi-Fi display”

The interaction model, and therefore the UI, for using Wi-Fi display is very similar to that of Bluetooth devices. The main difference is that initially it will not allow using the Ubuntu device as a receiver, which would require an equivalent to Bluetooth’s “discoverable” mode. Also, since only one Wi-Fi display can be used at a time, displays that will connect automatically need ordering by preference.

wi-fi-display.phone.png

Erratum: The caption should be present below the “Use automatically when detected:” list.

Therefore, the “Wi-Fi display” screen should eventually consist of three sections:

  1. The text “No Wi-Fi display connected”, or if there is, “Connected display:” followed by a page stack item representing it.
  2. The text “No displays detected”, or if there are, “Choose a display:” or “Use another display:” depending on whether one is already in use, followed by items for each of the currently-detected displays except any that are currently in use or set to be used automatically. An item should end in an ellipsis if it’s known in advance that using it will require entering a PIN.

  3. Once remembering displays is implemented: “Use automatically when detected:” followed by page stack items for each of them. If there are more than one:

    • each should have a drag handle for reordering them;
    • below the list should be the caption: “Drag to order displays by preference. If more than one is detected, the first will be used.”.

    (This is equivalent to previously-used Wi-Fi networks.)

The label for each display should consist of the “Brand” followed by the “Model number” (not the “Name”, since that is often both long and non-distinguishing). Across all three sections, if two or more displays have the same label, those displays only should be distinguished by having their labels end with their MAC address in brackets. Therefore, whether they include a MAC address or not, over-long labels should be ellipsized in the middle.

Connecting manually

wi-fi-display-pin.phone.png

Choosing a display from the “Choose a display:”/“Use another display:” list should attempt connection. While in progress, the list item for the display should have a spinner at its trailing end.

If the display requires a PIN, a ‘PIN for “{label}”’ dialog should appear with an empty numeric field for entering the PIN, and “Cancel” and “Connect” buttons. “Connect” should be disabled whenever the field contents are the wrong length. When you choose “Connect”, the dialog should stay open in the standard progress state until connection is complete, in case you entered the wrong PIN and need to correct it.

Connecting in general

If connecting fails, an alert should appear, with an “OK” button, and primary and secondary text depending on the situation:

  • This device can’t connect to “{display label}”.
    • There is no Wi-Fi channel both devices can use.
  • This device can’t connect to “{display label}”.
    • There is no Wi-Fi frequency both devices can use.
  • “{display label}” is not responding.
    • Try powering the display off then on again.
  • There was an unknown error connecting to “{display label}”.
    • Try moving closer to the display and reconnecting.

Examining and disconnecting manually

wi-fi-display-individual-connected.phone.pngwi-fi-display-individual-disconnected.phone.png

Choosing a device in the “Connected display:” list, or (once implemented) the “Connect automatically when detected:” list, should navigate to a screen equivalent to the Bluetooth device screen. Its header should consist of the device label, and it should contain a key-value table showing the “Brand:”, “Name:”, “Model:”, and “Status:”.

“Status:” should be one of “Connecting…”, “Connected”, “Disconnecting…”, or “Not connected”. When it is “Not connected”, it should be followed by “Last connected:” with the date and time that the display was most recently connected.

Finally, the screen should have a “Connect automatically when detected” checkbox (once implemented), unchecked by default, and if the display is currently connected, a “Disconnect” button. “Disconnect” should be disabled whenever “Connect automatically when detected” is checked (because otherwise the device would immediately reconnect, making disconnecting pointless).

If you choose “Disconnect”, that button should become disabled, “Status:” should change to “Disconnecting…”, and one second later you should be returned to the main “Brightness & Display” screen to see the device sliding down from the “Connected display:” list to the “Choose a display:”/“Use another display:” list.

If you uncheck “Connect automatically when detected” for a display that is not currently in use, the display’s details should not be deleted until you navigate to any other System Settings page, so that you have an opportunity to undo the setting before the display page disappears.

Disconnecting automatically

If the external display powers off, display should return to the main device.

Brightness

If you use a keyboard key to adjust brightness, a brightness confirmation bubble should appear.

If the device is a PC with a single display, there should not be any software UI for brightness, because that would be confusingly redundant with hardware brightness control on the keyboard and/or display itself (and unlike sound volume, brightness does not need reassurance for troubleshooting purposes).