Diff for "DesktopTeam/Specs/GdmFaceBrowser"

GdmFaceBrowser

Differences between revisions 3 and 35 (spanning 32 versions)

Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.

Launchpad Entry: gdm-face-browser
Packages affected: gdm

Summary

The login via gdm is meant to default to a face-browser allowing simple point-and-click user-selection. Only the password-entry will require the user to use the keyboard. It is still possible to type in the login-id instead of selecting the user with a mouse-click on her/his image. This will also cause the set of displayed user-faces to be filtered to the remaining completion-possibilities, thus rendering the remaining images larger and making the hit-area for a mouse-click even bigger.

In order to achieve fast and good looking transitions for this very visual login interface the use of OpenGL is mandatory.

To increase the level of integration tools like MeMaker and cheese are meant to be added to gdm-utilities like gdm-setup and gnome-about-me, so users have very easy means to provide a nice image for their entry in the face-browser.

Release Note

The default layout of the gdm-greeter uses an OpenGL-based face-browser, if the underlying graphics-hardware and driver provide the needed support, in order to provide a more pleasing login-experience. User-selection happens either by clicking on a users image/photo/face or via text-entry.

Rationale

These days, with composited desktop-environments becoming more and more a standard feature, people expect their whole computing experience to be visually attractive and consistent across all parts of the system. Furthermore do we want to close the gap of visual attractiveness in the different parts of the overall desktop-experience (booting, login, desktop-usage, logout). Ideally the user should not recognize any kind of "gap" when using the computer. This attention to detail in terms of consistent look and behaviour helps give the user a more enjoyable desktop-system and improves a users confidence in the platform s/he is using.

Use Cases

The general advantage of a face-browser for login is the avoidance of superfluous typing. This helps speed up the login-procedure of all people, who are no touch-typists or experienced keyboard users. But even experienced users can enjoy the convenience of being lazy occasionally. It is still possible to login successfully without selecting an image at all just by typing in the login-name and password.

only one user on system: Although an automatic login would streamline the bootup-to-desktop cycle for a single-user system (single in terms of one real person, not "user" in terms of unix-privileges), security-concerns strongly advice against this. The single user will see only her/his photo in the face-browser.

multiple users on system (<=100): In this case a user can either just select her/his image to tell the computer their login-name or start typing the first few characters from her/his login-name and watch the face-browser filter the set of displayed images to fewer images in order to simplify locating their own image in the visible set of images.

multiple users on system (>100): Scalability-issues regarding texture-usage (especially on low-end graphics hardware) and network-latency (setups with that many users are hardly run of local harddisks) demand a fallback to a non-face-browser mode of the gdm-greeter. In that case plain text-entry widgets for login-name and password are used.

idle: When the computer is left running unattended without any user loggeded in, the gdm-greeter should switch to a screensaver-like display without locking input. Any motion of the mouse or pressed key should exit this mode. The screensaver-mode should use e.g. the user photos and make them fly round on the screen, but the user should always be able to identify that it is still the login-screen that is being displayed and not a normal screensaver from another users-session.

The threshold of a 100 users for automatic disabling the face-browser is something that needs to be properly tested during the beta-phase.

Assumptions

For the face-browser the OpenGL 1.3 and the following extensions will be required:

GL_ARB_multitexture
GL_ARB_fragment_program
GL_ARB_texture_rectangle

These OpenGL-limits will be required:

GL_MAX_TEXTURE_SIZE >= 2048

The recommended size for user-images is 256x256 so they will not look to blurry when scaled up. This size will have to be enforced via three means...

stock-photos coming with gdm need all to be 256x256
any images generated from an avatar-maker need to be 256x256
images aquired from a webcam need to be scaled to 256x256

We currently assume that the newly rewritten gdm will be in a shippable state by the time HardyHeron enters beta. Should we forsee that this will not be the case, we will fallback to the old gdm and add the face-browser to the old version. But the work for adding a face-browser to the old gdm isn't a simple task doable in a week. There was some work done in that direction already during an earlier cycle, but it would be at least a month of work to resurrect it.

Design/Mockups

Normal login procedure 1 (selecting only via mouse-click on image)...

Normal login procedure 2 (starts typing thus filtering displayed images, selects image via mouse-click when there are fewer choices)...

User has the caps-lock key activated...

User clicked wrong picture, needs to go back...

Examples for ~25, ~50 and ~100 faces displayed...

[http://people.ubuntu.com/~mmueller/face-browser-13.png http://people.ubuntu.com/~mmueller/small_face-browser-13.png] [http://people.ubuntu.com/~mmueller/face-browser-12.png http://people.ubuntu.com/~mmueller/small_face-browser-12.png] [http://people.ubuntu.com/~mmueller/face-browser-11.png http://people.ubuntu.com/~mmueller/small_face-browser-11.png]

Please note that these mockups are only meant to give a general idea of the look we are aiming at and to provide a basis for discussion with the artwork-team. I (MacSlow) am not part of the artwork-team itself, therefore colors, gradients, hues, drop-shadows, "roundness" etc. are just my best guesses and do not reflect the final look of the artwork. Please do not write any article about this stating something else!

The names displayed are meant to be the real names of users and not their login-ids, which are used to log in normally. For the record other OSes display real names of users too in the login-manager. Using the real name instead of the login-id, is a fair compromise between usability and security. From experience it is not too uncommon to have login-ids being rather abstract and not reflecting a users real name at all, thus displaying a users real name with their photo/image is not giving away much. Furthermore the face-browser is meant to work for local users only and not for accounts stored on LDAP or similar directory-services.

The main workload for this spec will come from the OpenGL-based face-browser. There are currently two potential candidates for becoming the rendering library of choice, [https://code.fluendo.com/pigment/trac pigment] and [http://clutter-project.org clutter]. At the time of this writing both lack direct support for implicit-animations and shaders. These two features will have to be added to the API of choice, once that choice has been made. Going down that path will involve a lot of additional design-discussions and work not directly related to this spec.

There is another option for providing the needed OpenGL-boilerplate. That would be to use libgtkglext as windowing-system glue and implement all needed animation- and effects-features from scratch directly via OpenGL.

Theme-support was based on a XML-file in the old gdm. This should be taken over to the new gdm.

Implementation

Main libraries to be used are...

OpenGL
cairo
pango
librsvg
gtk+
gtkglext

To this list either...

pigment

or...

clutter

might be added, depding on how results of their evaluation turn out.

The evaluation process will check if the following things are available in the reviewed API...

animation-framework allowing implicit animation
simple loading of bitmapped images (PNG, JPG) and SVG
exposure of GL's mipmapping capabilities in the API itself
or easy integration of mipmapping
exposure of GL's multitexturing capabilites in the API itself
or easy integration of multitexturing
exposure of GL's fragment-shading extension in the API itself
or easy integration of fragment-shading
redirecting gtk+-widgets to offscreen buffers for texture-generation
high-quality text-rendering (exposing some of the flexibility like pango offers)

Nearly all elements visible on the gdm-greeter screen will be multi-textured quads using fragment-shading for masking out the face-photos and other elements to get the rounded look. Some parts of the visual assets will be loaded as SVGs or PNGs from which textures will be created. This allows themeability-work by artists. Text display will be handled in a similar fashion. There the initial texture-generation is done via pango. More details will be added once the implementation has started.

Migration

Any theme and gdm-setup setting should just work with the new gdm.

During the installation of Ubuntu on a computer the user should be asked for a photo upon creation of the first account. This can either happen by offering the user a set of stock images, allow loading an image-file from an external data-source e.g. an usb-stick or web-page, take a picture with an application like [http://www.gnome.org/projects/cheese cheese] (if a supported webcam is found) or allow the user to create an abstract avatar-photo using a program like [https://edge.launchpad.net/memaker MeMaker]. All this of course requires a lot of additional integration work that easily exceeds the scope of this spec. It would require ubiquity, Ubuntu's installer tool, to be extended with support for cheese and MeMaker.

Whenever a new user is created it has to be guaranteed, that this new user account gets a unique photo to be displayed in the face-browser. At least the fallback images need to be uniquely assigned. If the user provides her/his own photo (via webcam/cheese, avatar-maker/MeMaker or from self-provided image) we assume it to be a unique photo. To further ensure a user is able to pick the correct photo from the set of images displayed in the face-browser, their real name needs to be displayed with their photo (the login-name is used as a fallback, if no real name was provided upon user-account creation).

In order to cleanly migrate a system configuration with upto 100 users, where no user has a face-image set yet, we need to randomly assign an image from the stock image-set. Users can then afterwards still change their randomly assigned image, should they choose so. This stock image-set needs to be provided by the default installation or after an distro upgrade. The images need to be 256x256 RGB (PNG or JPG) and we need at least 100 different of them.

Test/Demo Plan

It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during CD testing, and to show off after release.

This need not be added or completed until the specification is nearing beta.

Outstanding Issues

It is not clear yet how to provide accessibility for the face-browser.
If user-names (no matter if login-name or real name) should be displayed or not with the photos/images will be discussed in a desktop-team meeting (targetted for 22.10.2007, 13:00 UTC, #ubuntu-meeting on freenode IRC)

BoF agenda and discussion

Suggestions:

Add an option in GDM to disable the login sound (very useful when working in libraries)
Enable password-less connections (i.e. users have a password but allow GDM to connect locally without using it)? Another feature we really need for home computers.KDM already has it, I opened a bug on GDM but it is really not active: http://bugzilla.gnome.org/show_bug.cgi?id=414862 Could there be a discussion about that? IMHO, it is essential. - Milan71
Support pulling faces from LDAP jpegPhoto attribute (that's against some security-policies I was pointed towards by KeesCook, anything come from directory-services should not be as easily browsable as with the face-browser, only local users will be displayed)

CategorySpec

DesktopTeam/Specs/GdmFaceBrowser (last edited 2009-05-29 18:29:22 by static-96-227-186-11)

-  ⇤ ← Revision 3 as of 2007-11-06 20:21:45 → 
  Size: 3223
  Editor: 75-144-175-202-NewEngland
  Comment:
+   ← Revision 35 as of 2007-11-22 11:30:47 → ⇥
  Size: 15172
  Editor: dslb-084-063-110-190
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 1:
+## page was renamed from HardyGdm
-Line 5:
+Line 6:
- * '''Launchpad Entry''': UbuntuSpec:hardy-gdm
+ * '''Launchpad Entry''': UbuntuSpec:gdm-face-browser
-Line 10:
+Line 11:
-        Discuss and agree changes to our default greeter.
+The login via gdm is meant to default to a face-browser allowing simple point-and-click user-selection. Only the password-entry will require the user to use the keyboard. It is still possible to type in the login-id instead of selecting the user with a mouse-click on her/his image. This will also cause the set of displayed user-faces to be filtered to the remaining completion-possibilities, thus rendering the remaining images larger and making the hit-area for a mouse-click even bigger.
-Line 12:
+Line 13:
-        * Input from hardy-theme discussion
        * Default should be a Face Browser, allowing users to click on
          their name
        * Show face and/or name?
        * Threshold for falling back to non-face greeter
        * Implementation of face browser
        * OK button on gdm non-face greeter (people don't think they can
          just hit ENTER)
        * Username field alternative (hidden maybe?) on face browser,
          start typing and it appears
        * "About Me" (eds) information?
        * Session information (number of apps running, mails, etc.)
        * remember "screensaver-mode" of face-browser after x minutes of inactivity
+In order to achieve fast and good looking transitions for this very visual login interface the use of OpenGL is mandatory.

To increase the level of integration tools like MeMaker and cheese are meant to be added to gdm-utilities like gdm-setup and gnome-about-me, so users have very easy means to provide a nice image for their entry in the face-browser.
-Line 28:
+Line 19:
-This section should include a paragraph describing the end-user impact of this change.  It is meant to be included in the release notes of the first release in which it is implemented.  (Not all of these will actually be included in the release notes, at the release manager's discretion; but writing them is a useful exercise.)

It is mandatory.
+The default layout of the gdm-greeter uses an OpenGL-based face-browser, if the underlying graphics-hardware and driver provide the needed support, in order to provide a more pleasing login-experience. User-selection happens either by clicking on a users image/photo/face or via text-entry.
-Line 34:
+Line 23:
-This should cover the _why_: why is this change being proposed, what justifies it, where we see this justified.
+These days, with composited desktop-environments becoming more and more a standard feature, people expect their whole computing experience to be visually attractive and consistent across all parts of the system. Furthermore do we want to close the gap of visual attractiveness in the different parts of the overall desktop-experience (booting, login, desktop-usage, logout). Ideally the user should not recognize any kind of "gap" when using the computer. This attention to detail in terms of consistent look and behaviour helps give the user a more enjoyable desktop-system and improves a users confidence in the platform s/he is using.
-Line 38:
+Line 27:
+The general advantage of a face-browser for login is the avoidance of superfluous typing. This helps speed up the login-procedure of all people, who are no touch-typists or experienced keyboard users. But even experienced users can enjoy the convenience of being lazy occasionally. It is still possible to login successfully without selecting an image at all just by typing in the login-name and password. 

only one user on system: Although an automatic login would streamline the bootup-to-desktop cycle for a single-user system (single in terms of one real person, not "user" in terms of unix-privileges), security-concerns strongly advice against this. The single user will see only her/his photo in the face-browser.

multiple users on system (<=100): In this case a user can either just select her/his image to tell the computer their login-name or start typing the first few characters from her/his login-name and watch the face-browser filter the set of displayed images to fewer images in order to simplify locating their own image in the visible set of images.

multiple users on system (>100): Scalability-issues regarding texture-usage (especially on low-end graphics hardware) and network-latency (setups with that many users are hardly run of local harddisks) demand a fallback to a non-face-browser mode of the gdm-greeter. In that case plain text-entry widgets for login-name and password are used.

idle: When the computer is left running unattended without any user loggeded in, the gdm-greeter should switch to a screensaver-like display without locking input. Any motion of the mouse or pressed key should exit this mode. The screensaver-mode should use e.g. the user photos and make them fly round on the screen, but the user should always be able to identify that it is still the login-screen that is being displayed and not a normal screensaver from another users-session.

The threshold of a 100 users for automatic disabling the face-browser is something that needs to be properly tested during the beta-phase.
-Line 40:
+Line 41:
-== Design ==
+For the face-browser the OpenGL 1.3 and the following extensions will be required:
 * GL_ARB_multitexture
 * GL_ARB_fragment_program
 * GL_ARB_texture_rectangle
-Line 42:
+Line 46:
-You can have subsections that better describe specific parts of the issue.
+These OpenGL-limits will be required:
 * GL_MAX_TEXTURE_SIZE >= 2048 

The recommended size for user-images is 256x256 so they will not look to blurry when scaled up. This size will have to be enforced via three means...
 * stock-photos coming with gdm need all to be 256x256
 * any images generated from an avatar-maker need to be 256x256
 * images aquired from a webcam need to be scaled to 256x256

We currently assume that the newly rewritten gdm will be in a shippable state by the time HardyHeron enters beta. Should we forsee that this will not be the case, we will fallback to the old gdm and add the face-browser to the old version. But the work for adding a face-browser to the old gdm isn't a simple task doable in a week. There was some work done in that direction already during an earlier cycle, but it would be at least a month of work to resurrect it.

== Design/Mockups ==

Normal login procedure 1 (selecting only via mouse-click on image)...

[http://people.ubuntu.com/~mmueller/face-browser-3.png http://people.ubuntu.com/~mmueller/small_face-browser-3.png][http://people.ubuntu.com/~mmueller/arrow-right.png][http://people.ubuntu.com/~mmueller/face-browser-6.png http://people.ubuntu.com/~mmueller/small_face-browser-6.png][http://people.ubuntu.com/~mmueller/arrow-right.png][http://people.ubuntu.com/~mmueller/face-browser-9.png][http://people.ubuntu.com/~mmueller/arrow-right.png][http://people.ubuntu.com/~mmueller/desktop-example.png http://people.ubuntu.com/~mmueller/small_desktop-example.png]

Normal login procedure 2 (starts typing thus filtering displayed images, selects image via mouse-click when there are fewer choices)...

[http://people.ubuntu.com/~mmueller/face-browser-3.png http://people.ubuntu.com/~mmueller/small_face-browser-3.png][http://people.ubuntu.com/~mmueller/arrow-right.png][http://people.ubuntu.com/~mmueller/face-browser-10.png http://people.ubuntu.com/~mmueller/small_face-browser-10.png][http://people.ubuntu.com/~mmueller/arrow-right.png][http://people.ubuntu.com/~mmueller/face-browser-6.png http://people.ubuntu.com/~mmueller/small_face-browser-6.png][http://people.ubuntu.com/~mmueller/arrow-right.png][http://people.ubuntu.com/~mmueller/face-browser-9.png][http://people.ubuntu.com/~mmueller/arrow-right.png][http://people.ubuntu.com/~mmueller/desktop-example.png http://people.ubuntu.com/~mmueller/small_desktop-example.png]

User has the caps-lock key activated...

[http://people.ubuntu.com/~mmueller/face-browser-3.png http://people.ubuntu.com/~mmueller/small_face-browser-3.png][http://people.ubuntu.com/~mmueller/arrow-right.png][http://people.ubuntu.com/~mmueller/face-browser-7.png http://people.ubuntu.com/~mmueller/small_face-browser-7.png][http://people.ubuntu.com/~mmueller/arrow-right.png][http://people.ubuntu.com/~mmueller/face-browser-6.png http://people.ubuntu.com/~mmueller/small_face-browser-6.png]

User clicked wrong picture, needs to go back...

[http://people.ubuntu.com/~mmueller/face-browser-3.png http://people.ubuntu.com/~mmueller/small_face-browser-3.png][http://people.ubuntu.com/~mmueller/arrow-right.png][http://people.ubuntu.com/~mmueller/face-browser-6.png http://people.ubuntu.com/~mmueller/small_face-browser-6.png][http://people.ubuntu.com/~mmueller/arrow-right.png][http://people.ubuntu.com/~mmueller/face-browser-8.png][http://people.ubuntu.com/~mmueller/arrow-right.png][http://people.ubuntu.com/~mmueller/face-browser-3.png http://people.ubuntu.com/~mmueller/small_face-browser-3.png]

Examples for ~25, ~50 and ~100 faces displayed...

[http://people.ubuntu.com/~mmueller/face-browser-13.png http://people.ubuntu.com/~mmueller/small_face-browser-13.png]
[http://people.ubuntu.com/~mmueller/face-browser-12.png http://people.ubuntu.com/~mmueller/small_face-browser-12.png]
[http://people.ubuntu.com/~mmueller/face-browser-11.png http://people.ubuntu.com/~mmueller/small_face-browser-11.png]


Please note that these mockups are only meant to give a general idea of the look we are aiming at and to provide a basis for discussion with the artwork-team. I (MacSlow) am not part of the artwork-team itself, therefore colors, gradients, hues, drop-shadows, "roundness" etc. are just my best guesses and do not reflect the final look of the artwork. Please do not write any article about this stating something else! 

The names displayed are meant to be the real names of users and not their login-ids, which are used to log in normally. For the record other OSes display real names of users too in the login-manager. Using the real name instead of the login-id, is a fair compromise between usability and security. From experience it is not too uncommon to have login-ids being rather abstract and not reflecting a users real name at all, thus displaying a users real name with their photo/image is not giving away much. Furthermore the face-browser is meant to work for local users only and not for accounts stored on LDAP or similar directory-services.

The main workload for this spec will come from the OpenGL-based face-browser. There are currently two potential candidates for becoming the rendering library of choice, [https://code.fluendo.com/pigment/trac pigment] and [http://clutter-project.org clutter]. At the time of this writing both lack direct support for implicit-animations and shaders. These two features will have to be added to the API of choice, once that choice has been made. Going down that path will involve a lot of additional design-discussions and work not directly related to this spec.

There is another option for providing the needed OpenGL-boilerplate. That would be to use libgtkglext as windowing-system glue and implement all needed animation- and effects-features from scratch directly via OpenGL.

Theme-support was based on a XML-file in the old gdm. This should be taken over to the new gdm.
-Line 46:
+Line 93:
-This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like:
+Main libraries to be used are...
-Line 48:
+Line 95:
-=== UI Changes  ===
+ * OpenGL
 * cairo
 * pango
 * librsvg
 * gtk+
 * gtkglext
-Line 50:
+Line 102:
-Should cover changes required to the UI, or specific UI that is required to implement this
+To this list either...
-Line 52:
+Line 104:
-=== Code Changes ===
+ * pigment
-Line 54:
+Line 106:
-Code changes should include an overview of what needs to change, and in some cases even the specific details.
+or...

 * clutter

might be added, depding on how results of their evaluation turn out.

The evaluation process will check if the following things are available in the reviewed API...
 * animation-framework allowing implicit animation
 * simple loading of bitmapped images (PNG, JPG) and SVG
 * exposure of GL's mipmapping capabilities in the API itself
 * or easy integration of mipmapping
 * exposure of GL's multitexturing capabilites in the API itself
 * or easy integration of multitexturing
 * exposure of GL's fragment-shading extension in the API itself
 * or easy integration of fragment-shading
 * redirecting gtk+-widgets to offscreen buffers for texture-generation
 * high-quality text-rendering (exposing some of the flexibility like pango offers)

Nearly all elements visible on the gdm-greeter screen will be multi-textured quads using fragment-shading for masking out the face-photos and other elements to get the rounded look. Some parts of the visual assets will be loaded as SVGs or PNGs from which textures will be created. This allows themeability-work by artists. Text display will be handled in a similar fashion. There the initial texture-generation is done via pango. 
More details will be added once the implementation has started.
-Line 58:
+Line 129:
-Include:
 * data migration, if any
 * redirects from old URLs to new ones, if any
 * how users will be pointed to the new way of doing things, if necessary.
+Any theme and gdm-setup setting should just work with the new gdm.

During the installation of Ubuntu on a computer the user should be asked for a photo upon creation of the first account. This can either happen by offering the user a set of stock images, allow loading an image-file from an external data-source e.g. an usb-stick or web-page, take a picture with an application like [http://www.gnome.org/projects/cheese cheese] (if a supported webcam is found) or allow the user to create an abstract avatar-photo using a program like [https://edge.launchpad.net/memaker MeMaker]. All this of course requires a lot of additional integration work that easily exceeds the scope of this spec. It would require ubiquity, Ubuntu's installer tool, to be extended with support for cheese and MeMaker.

Whenever a new user is created it has to be guaranteed, that this new user account gets a unique photo to be displayed in the face-browser. At least the fallback images need to be uniquely assigned. If the user provides her/his own photo (via webcam/cheese, avatar-maker/MeMaker or from self-provided image) we assume it to be a unique photo. To further ensure a user is able to pick the correct photo from the set of images displayed in the face-browser, their real name needs to be displayed with their photo (the login-name is used as a fallback, if no real name was provided upon user-account creation).

In order to cleanly migrate a system configuration with upto 100 users, where no user has a face-image set yet, we need to randomly assign an image from the stock image-set. Users can then afterwards still change their randomly assigned image, should they choose so. This stock image-set needs to be provided by the default installation or after an distro upgrade. The images need to be 256x256 RGB (PNG or JPG) and we need at least 100 different of them.
-Line 71:
+Line 145:
-This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.
+ * It is not clear yet how to provide accessibility for the face-browser.

 * If user-names (no matter if login-name or real name) should be displayed or not with the photos/images will be discussed in a desktop-team meeting (targetted for 22.10.2007, 13:00 UTC, #ubuntu-meeting on freenode IRC)
-Line 75:
+Line 151:
-Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.
+Suggestions:
-Line 77:
+Line 153:
-Suggestion: add an option in GDM to disable the login sound (very useful when working in libraries)
+ * Add an option in GDM to disable the login sound (very useful when working in libraries)

 * Enable '''password-less connections''' (i.e. users have a password but allow GDM to connect locally without using it)? Another feature we really need for home computers.KDM already has it, I opened a bug on GDM but it is really not active: http://bugzilla.gnome.org/show_bug.cgi?id=414862 Could there be a discussion about that? IMHO, it is essential. - Milan71

 * Support pulling faces from LDAP jpegPhoto attribute (that's against some security-policies I was pointed towards by KeesCook, anything come from directory-services should not be as easily browsable as with the face-browser, only local users will be displayed)

Ubuntu Wiki