On-Screen Virtual Keyboard for login and bootup

• Launchpad Entry: mobile-karmic-login-virtual-keyboard

• Created: 2009-04-30

• Contributors: Ramaddan

• Packages affected:

Summary

On-screen keyboard for mobile Tablet PCs and similar devices.

Rationale

Tablets PCs in general do not have any keyboard and come with Touchscreens. Thus, the touchscreen is currently under used for a number of purposes, and it would be nice to be able to fully utilize it.

All Tablet PCs need a means of inputting information through the touchscreen such as an on-screen keyboard, and a few possible options are available, but currently some limitations apply.

These limitations are related to two cases: 1) The ability to input through the onscreen keyboard at login 2) The ability to input passwords through an on screen keyboard at boot up for encrypted disks or to be able to deal with Grub or Lilo, or other such scenarios

Due to this lack of feature, it renders all devices using a touchscreen or similar input devices less secure, and thus should be treated as an important problem.

Use Cases

• Tablet PC
• Ultra mobile device
• Public terminal in Kiosk mode without keyboard
• In-car systems

Assumptions

For the case 1: Different On-screen keyboards are currently already available, and possible tweaks and changes can be implemented to reach the desired goals, such an using the current onBoard virtual keyboard.

Design

For case 1: a) Several keyboard layouts should be provided for different devices and preferences, as well as support for multiple languages.

b) The ability for the keyboard to automatically show up when text is being entered.

c) The ability for the keyboard to come up or disappear on certain gestures made on the touchscreen in addition to the panel icon.

d) The ability to make the virtual keyboard semi or fully transparent is also required in certain cases in order to not be an obstruction to the person using it.

e) The ability to dock the virtual keyboard as well as a feature to auto-detect whether to automatically change the docking station in case the keyboard is in the way of the user.

f) The ability to always be in front of other windows, as well as drop up or drop down lists that appear on the screen, such as the ones that comes up using Firefox when proposing words or sites that obstruct the virtual keyboard.

For case 2: a) Only requires (a) and (b) from Case 1.

Keyboard Style

For case 1: a) A QWERTY type keyboard should be used for normal sized Tablet PCs or similar devices

b) Ability to choose whether the full keyboard with a number pad or a skimmed down version of the keyboard should be used.

c) A different layout might probably be better for smaller screens, with the capability of proposing completed words based on the dictionary language linked to the input language being used.

d) The ability to choose between different types of layouts, including the ability to customize them, such as placing buttons in certain places as the user would like.

For case 2: a) A very skimmed down keyboard should be used in order to minimize amount of work and space taken in GRUB

Extra Functionality

Case 1: The ability to read out which button is being pressed on the touch screen might also be useful for people who need more accessible features.

Anything already available?

(I take the liberty to add this section to this specification; if it is not welcome, please contact me at francesco dot fumanti at gmx dot net, and I will remove it; or simply remove or edit it as you like.)

As indicated in the Assumptions section above, there are already solutions at least for having an onscreen keyboard at login (assuming you are using GDM, the default login manager of Ubuntu); unfortunately it is not available out of the box.

For case 1:

• For the GDM versions in Jaunty, Intrepid, Hardy and Gutsy, there are explanations about how to set it up on the following wiki page. Of course, the people that are only interested in the onscreen keyboard can simply skip the parts about mousetweaks.

• There is also a bug that has been opened about this, but it has not been picked up yet.

• Some people at Fedora are working on a new version of GDM that should normally provide accessibility features out of the box for the login screen; among others an onscreen keyboard. Let's hope it to be picked up at some point by the Ubuntu distribution.

For case 2:

• I don't know.

CategoryAccessibility