A simple on-screen keyboard

• Launchpad Entry: https://launchpad.net/distros/ubuntu/+spec/sok

• Created: 2006-04-24 by HenrikOmma

• Contributors: HenrikOmma

• Packages affected: sok

Summary

A simple on-screen keyboard will be useful for those who are not able to use a standard keyboard, either due to a disability, or because they use a tablet PC, or because the device they are using does not have a physical keyboard (information terminals at expos), or because they wish to input in a language for which they do not have a keyboard (usually with a different glyph-type). Project tracked [:Accessibility/Projects/SOK:here].

Rationale

The Gnome On-screen Keyboard is a flexible application for entering text with the pointer or with switching devices. However, it is highly complicated, trying to solve many problems.

There is scope for the creation of a much simpler on-screen keyboard with fewer features but greater stability and usability. In order to make it really useful, the on-screen keyboard should provide as much as possible, the same functionality as the physical keyboard. Further the on-screen keyboard should be available as soon as possible when starting the computer (login screen).

A focus not only on accessibility, but also on tablet PC hardware and localisation would create a wider user-base with more testing and development.

Use cases

• Liza is a student who uses a head-mouse device to interact with her computer. Her school has migrated all their desktops to Edubuntu and she would prefer to use that as well. She can move the mouse cursor by moving her head and can activate the left click using a specially designed button. She is not able to use a normal keyboard, but can work quite efficiently with an on-screen version. The head-pointer uses a standard PS/2 connection and is the only pointing device connected to the computer. She can navigate the Gnome desktop and the web perfectly well with this setup so she only needs the on-screen keyboard for text-entry.
• Eduardo is a travelling sales rep. who gives presentations to customers on his tablet PC. For major text entry he can unfold the physical keyboard on the device, but using an on-screen keyboard is often more convenient.
• Signe has started learning Russian and wants a simple way to input some Russian characters without putting sticky tape on her keyboard.
• At expositions there are sometimes terminals consisting only of a screen where the visitors can get informations. The on-screen keyboard will improve the interaction with this devices.

Scope

A simple on-screen keyboard that uses modern technologies like SVG and Cairo to create a clean interface. Written primarilly for Gnome using Gtk, but future simple porting to KDE will be kept in mind.

Design

• The basic initial configuration is a simple Qwerty Keyboard containing the most commonly used keys. This includes the letters and modifier keys (Shift, Ctrl, Alt) and a few others like space, tab period and comma. The number keys may or may not be included on the main board.
• Less commonly used keys will be placed on a secondary layout that becomes available after pressing an access key.
• The keyboard layout and localisation should be easily configurable.
• The keyboard is by default configured to be always on top, even if the active application takes over the whole screen. This feature can be turned off.
• The keyboard can be docked to a fixed position on the screen.
• The keyboard window can be resized as with any regular window, and the contents will stretch to fit.
• Pressing virtual key triggers the same function as pressing the equivalent key on a physical keyboard. Particularly, clicking somewhere on the screen while a modifier is pressed on sok, should have the same functionality as if the modifier was pressed on the physical keyboard.
• Stick Keys: a single click on a modifier will make it stick until the next keypress; a double click on a modifier will make the modifier stick until the next click on the modifier; the preferences of sok contain an option to make a modifier sticky with a single click, thus eliminating the distinction between single- and double-click
• Make a single button available for switching the left and right mouse buttons.
• Can be launched from panel applet
• Dwell switching functionality: hovering over a key (for a configurable time interval) will activate it. This will be implemented directly in SOK, but should also be made available directly in Gnome for general use.
• Scanning activation: Some people are not able to move the mouse cursor on the 2D desktop, but may only be able to click on a single switch (such as by using a sip-and-puff tube). The keyboard should have a 'scanning' mode, by which a single click input can be used to scan through the keys in a pre-determined sequence. A second click selects a key. Rate and pattern of scanning should be configurable.
• A configuration button (on the secondary keyboard?) brings up a standard configuration pannel.
• A macros button on the main keyboard brings up a secondary key matrix for macros and plug-ins.

Implementation

• Use a higher level framework like python and pyGTK.
• .svg files for the layout. These can be created using Inkscape.
• The letters and keycodes are stored in a separate XML file.
• Settings are stored in Gconf

• Keystrokes are passed directly to X as sythetic events using the xtst extention or XSendEvent.

• The basic implementation does not use AT-SPI
• Scanning: A button on the keyboard initiates a key scanning sequence when clicked with a standard mouse. This allows us to implement the other aspects of scanning such as pattern and rate control. The ability to use other input methods (such as serial ports) will be deferred.
• Passing the keystrokes will to other applications: the details of this need to be confirmed with the SoC student who wrote the code, but it is already working

Deferred features

Some features which currently exist in some on-screen keyboard should be kept out of the first implementation in order to keep it simple. These could be implemented later as plug-ins.

• A range of keyboard layouts can be implemented by the user community.
• Full desktop navigation -- This requires AT-SPI and would likely also use python scripts

Word completion

• Word Completion, single word prediction, multiple word prediction by learning expressions used by the user even if they are not grammaticaly correct. This might be best done in a separate application.
• Multiple dictionaries and user-dictionaries that can be individualy and simultaneously activated; so the user can create dictionaries for different purposes and activate only those that he needs; for example a dictionary with the names of friends,...
• Intelligent behaviour: automatic space after word completion or after a punctuation; automatic shift-modifier activation after some punctuation;... Its activation and deactivation should be on the keyboard windows the user will often toggle its state (for example typing a text vs typing an address in the browser)

References

• [http://www.utoronto.ca/atrc/reference/tech/onscreen.html Other on screen keyboards]

• [https://wiki.ubuntu.com/Accessibility/Reviews/GOK Review of GOK]

• [http://freshmeat.net/projects/qvkeyb/ QVkeyboard]