1.0Spec

Gesture Engine Interface Support

Revision History
Revision 1.012 August 2010SMW

1. Overview

This document describes GEIS, an interface between an application and an underlying gesture recognition system.

A GEIS implementation provides a mechanism for recognizing user interface gestures on various configurable devices and for propagating those recognized gestures to subscribed applications.

1.1. Gestures

A gesture is a defined set of particular movements of pointing devices. GEIS provides no interpretation of gestures, only recognition and propagation.

Because gestures can be defined to have global meaning as well as local meaning, it makes sense that gesture recognition be provided by the system rather than be included as a library by a particular application or toolkit. This allows, for example, a set of gestures to be defined for the window manager in use on a display to receive gesture events even when another application is in the foreground and receiving the input.

2. GEIS Operation

The purpose of GEIS is intended to provide a consistent and stable application programming interface for gesture recognition and propagation on multiple target platforms (for example, Linux, Unix, and Windows), multiple target input devices (for example mice, touch pads, mutitouch touchscreens, tables, nunchuks), and multiple gesture recognition engines. Different gesture engines may be available on the same platform.

A number of basic types are defined for portability between various underlying display, input, and operating systems.

2.1. GEIS Basic Types

GeisBoolean is an integral type representing a boolean value and should only hold the values GEIS_TRUE (1) and GEIS_FALSE (0).

GeisInt is an integral type large enough to hold any integral gesture attribute value which may be represented on the native platform. This may includes bitmasks, enumerants, integers, handles, and so forth.

GeisFloat is a floating-point type which can represent floating-point valued gesture attributes,

GeisString is a C-style zero-terminated string of characters in some character set. A GeisString is a read-only string and does not have to be freed or disposed of by the caller.

2.2. GEIS Instances

An application may need to support different sets of gestures from different input devices or associated with different regions of the same device. For this reason, GEIS provides for one or more distinct connections to the underlying gesture engine. The instances are distinguished by a GeisInstance object. All gesture types and gesture instances IDs are in a namespace associated with the GeisInstance and are not guaranteed to be system wide unique.

2.2.1. Threading and Concurrency

The GEIS API does not specify threading, reentrancy, or concurrency attributes. The implementation must supply detailed documentation on these aspects.

2.2.2. Input Devices

The API provides a method to enumerate the available gesture-capable input devices (including the dynamic appearance and disappearance of such devices) and a way to specify a subset of these devices for use in a particular gesture detection instance. A client application might typically respond to gestures from any input device, but others, such as a drawing application, may wish to bypass gesture detection on, for example, a drawing tablet while accepting them from a touchscreen for other functions such as zoom and rotate.

GEIS uses a set of client-supplied callbacks to report the availability of input devices so that devices that dynamically appear or disappear can be reported through a consistent interface.

3. GEIS Functions and Errors

3.1. Errors

[Note]

This section is not complete.

GEIS functions usually return a status code indicating success or failure. The status codes that may be returned, and their meanings, are as follows.

GEIS_SUCCESS
Normal, successful completion of the function.
GEIS_BAD_ARGUMENT
One or more argument values are invalid. Any command taking parameters may generate this error.
GEIS_STATUS_NOT_SUPPORTED
An operation or configuration item is not supported by the implementation.
GEIS_UNKNOWN_ERROR
A general catchall error: an error occurred that is not covered by any other defined status code.

3.2. Initialization

[Note]

This section is not complete.

A GEIS instance may be initialized by calling

GeisStatus geis_init(GeisWinInfo *win_info, GeisInstance *geis_instance);

GEIS_SUCCESS is returned on success and geis_instance is updated with an instance value to be used in subsequent GEIS calls.

The definition of the GeisWinInfo fields is entirely dependant on the implementation. For example, an X11-based gesture recognizer may specify the window_id as the win_info argument.

3.3. Configuration and Control

A particular GEIS implementation may support a subset of standard configuarable items and, in addition, zero or more implementation-specific configuration items. The implementation-specific configuration items will be documented by the implementation.

To determine of a confiuration item is supported, use the following call.

GeisStatus geis_configuration_supported(GeisInstance instance,
                                        int          configuration_item);

This function will return GEIS_STATUS_SUCCESS if the configuration_item is supported by the implementation, GEIS_STATUS_NOT_SUPPORTED otherwise.

The following standard configuration items must be available. .Input Standard Configuration Items

configuration item description

GEIS_CONFIG_UNIX_FD

A posix descriptor for event loop synchronization is available.

Configuration-specifc values may be set and retrieved using the following calls.

GeisStatus geis_configuration_get_value(GeisInstance instance,
                                        int          configuration_item,
                                        void*        value);

GeisStatus geis_configuration_set_value(GeisInstance instance,
                                        int          configuration_item,
                                        void*        value);

The actual type and meaning of the value argument depends on the configuration_item. Standard configuration item types are described in the following table.

Table 1. Input Standard Configuration Item Types

configuration item type description

GEIS_CONFIG_UNIX_FD

int

A posix file descriptor


Integration with the native event loop is through an implementation-specific mechanism (likely obtained through the configuration calls above, but documented more thoroughly in the implementation documentation) combined with a call to the GEIS dispatch mechanism.

GeisStatus geis_event_dispatch(geisInstance instance);

3.4. Input Devices

Gestures are associated with input devices (a gesture must be input on a device). Client software may subscribe to gestures that occur on all input devices or on only a subset of input devices. For that to be possible, the client needs to enumerate all available input devices. In addition, many input devices have a dynamic lifetime — they may be added or removed asynchronously. the client software needs to be made aware that new input devices have become available or existing devices have become unavailable.

GEIS input device enumeration and availability notification is performed through the following interface.

typedef unsigned int GeisInputDeviceId;

#define GEIS_ALL_INPUT_DEVICES ((GeisInputDeviceId)0)

typedef void (*GeisInputCallback)(void             *cookie,
                                  GeisInputDeviceId device_id,
                                  void             *attrs);

typedef struct GeisInputFuncs
{
  GeisInputCallback  added;   /**< Receives new input device notices */
  GeisInputCallback  changed; /**< Receives modified input device notices */
  GeisInputCallback  removed; /**< Receives removes input device notices */

} GeisInputFuncs;

GeisStatus geis_input_devices(GeisInstance    geis_instance,
                              GeisInputFuncs  funcs,
                              void           *cookie);

The API provides a list of available input devices one at a time through the client-supplied added callback function.

The changed callback function or the removed callback function may be invoked at any time after geis_input_devices is called and before geis_finish is called to tear down the GeisInstance or geis_input_devices is called again to replace the callback function. This can occur when input devices are, for example, physically connected to or disconnected from the system.

The cookie value passed to geis_input_devices is passed unmodified to callback function. Client programs may use this for passing context-specific information into the callback.

The suggested behaviour of an implementation is to invoke the callback for each input device known at call time before returning from the geis_input_devices call.

3.5. Subscribing to Gestures

Gestures are specific to a GeisInstance and set of input devices. The GeisInstance must be informed of the set of gestures (and input devices) of interest.

typedef unsigned int GeisGestureType;
typedef unsigned int GeisGestureId;

#define GEIS_ALL_GESTURES ((GeisGestureType)0)
#define GEIS_NO_GESTURE_ID ((GeisGestureId)0)

typedef void (*GeisGestureCallback)(void             *cookie,
                                    GeisGestureType   gesture_type,
                                    GeisGestureId     gesture_id,
                                    GeisSize          attr_count,
                                    GeisGestureAttr  *attrs);

typedef struct GeisGestureFuncs
{
  GeisGestureCallback  added;
  GeisGestureCallback  removed;
  GeisGestureCallback  start;
  GeisGestureCallback  update;
  GeisGestureCallback  finish;
} GeisGestureFuncs;

GeisStatus geis_subscribe(GeisInstance        geis_instance,
                          GeisInputDeviceId  *input_list,
                          const char*        *gesture_list,
                          GeisGestureFuncs    funcs,
                          void               *cookie);

GeisStatus geis_unsubscribe(GeisInstance     geis_instance,
                            GeisGestureType *gesture_list);

The client supplies a list of input devices (or the constant GEIS_ALL_INPUT_DEVICES) and a NULL terminated list of C-style strings naming the gestures to be recognized. The constant GEIS_ALL_GESTURES indicates all available gestures should be detected.

GEIS provides a list of gestures available in the desired window and input device(s) one at a time through the client-supplied added callback function with the GEIS_GESTURE_TYPE_ADDED action. This allows the client to build a map of gesture names to GeisGestureType. If the recognition engine does not support a named gesture type passed to geis_subscribe then no GeisGestureType is passed to the callback function for the named gesture.

The cookie value passed to geis_subscribe is passed unmodified to callback. Client programs may use this for passing context-specific information into the callback.

3.6. Receiving Gestures

One of the gesture event callback functions may be invoked at any time after geis_input_devices is called and before geis_finish is called to tear down the GeisInstance or geis_input_devices is called again to replace the callback function. This can occur when input devices are, for example, physically connected to or disconnected from the system.

Most gestures will result in a start callback, zero or more update callbacks, and a finish callback. Some gestures such as tap gestures may result in only a single update callback. This behaviour is dependent on the underlying gesture recognition engine

Once a gesture is recognized, a GEIS_GESTURE_STARTED action is sent. A set of zero or more attributes will be returned. These attributes are specific to the gesture. Depending on the nature of the gesture, one or more GEIS_GESTURE_UPDATED actions may be sent, and finally a GEIS_GESTURE_FINISHED action will be sent. Each of these may or may not have a set of attributes associated.

A gesture or gesture type action is often associated with a set of attributes. These attributes are delivered to the callback function as a read-only array of GeisGestureAttr type, defined below.

typedef struct GeisGestureAttr
{
  GeisString   name;
  GeisAttrType type;
  union
  {
    GeisBoolean boolean_val;
    GeisFloat   float_val;
    GeisInteger integer_val;
    GeisString  string_val;
  };
} GeisGestureAttr;

3.7. Recognizer Teardown

Teardown of the gesture recognition session is accomplished by the geis_finish function.

GeisStatus geis_finish(GeisInstance *geis_instance);

This function frees any allocated resources associated with geis_instance. It does not affect other instances.

All resources are expected to be freed automatically on process termination. The geis_finish function is not expected to return any status value other than GEIS_SUCCESS.

4. Standard Gestures

A limited number of fundamental gestures are expected to be supplied with the API. All implementations are expected to support at least this set of predefined gestures. Additional predefined gestures and user- or application-defined gestures are beyond the scope of this specification but are expected to be made available by the implementation.

Table 2. Predefined Gestures

gesture name symbolic name description

Drag

GEIS_GESTURE_DRAG

Pinch

GEIS_GESTURE_PINCH

Rotate

GEIS_GESTURE_ROTATE

Tap

GEIS_GESTURE_TAP


4.1. Common Attributes

All standard gestures provide a common set of attributes.

Table 3. Common Gesture Attributes

name type description

GEIS_GESTURE_ATTRIBUTE_DEVICE_ID

GeisInteger

GEIS_GESTURE_ATTRIBUTE_TIMESTAMP

GeisInteger

GEIS_GESTURE_ATTRIBUTE_ROOT_WINDOW_ID

GeisInteger

GEIS_GESTURE_ATTRIBUTE_EVENT_WINDOW_ID

GeisInteger

GEIS_GESTURE_ATTRIBUTE_CHILD_WINDOW_ID

GeisInteger

GEIS_GESTURE_ATTRIBUTE_FOCUS_X

GeisFloat

GEIS_GESTURE_ATTRIBUTE_FOCUS_Y

GeisFloat

GEIS_GESTURE_ATTRIBUTE_GESTURE_NAME

GeisString

GEIS_GESTURE_ATTRIBUTE_FINGERS

GeisInteger


4.2. Drag

[Note]

This section is not complete.

Table 4. Pan Gesture Attributes

name type description

GEIS_GESTURE_ATTRIBUTE_DELTA_X

GeisFloat

GEIS_GESTURE_ATTRIBUTE_DELTA_Y

GeisFloat

GEIS_GESTURE_ATTRIBUTE_VELOCITY_X

GeisFloat

GEIS_GESTURE_ATTRIBUTE_VELOCITY_Y

GeisFloat

GEIS_GESTURE_ATTRIBUTE_POSITION_X

GeisFloat

GEIS_GESTURE_ATTRIBUTE_POSITION_Y

GeisFloat


4.3. Pinch

[Note]

This section is not complete.

  1. Pinch Gesture Attributes
name type description

GEIS_GESTURE_ATTRIBUTE_RADIUS_DELTA

GeisFloat

GEIS_GESTURE_ATTRIBUTE_RADIAL_VELOCITY

GeisFloat

GEIS_GESTURE_ATTRIBUTE_RADIUS

GeisFloat

GEIS_GESTURE_ATTRIBUTE_BOUNDINGBOX_X1

GeisFloat

GEIS_GESTURE_ATTRIBUTE_BOUNDINGBOX_Y1

GeisFloat

GEIS_GESTURE_ATTRIBUTE_BOUNDINGBOX_X2

GeisFloat

GEIS_GESTURE_ATTRIBUTE_BOUNDINGBOX_Y2

GeisFloat

4.4. Rotate

[Note]

This section is not complete.

  1. Swipe Gesture Attributes
name type description

GEIS_GESTURE_ATTRIBUTE_ANGLE_DELTA

GeisFloat

GEIS_GESTURE_ATTRIBUTE_ANGULAR_VELOCITY

GeisFloat

GEIS_GESTURE_ATTRIBUTE_ANGLE

GeisFloat

4.5. Tap

[Note]

This section is not complete.

Table 5. Tap Gesture Attributes

name type description

GEIS_GESTURE_ATTRIBUTE_POSITION_X

GeisFloat

GEIS_GESTURE_ATTRIBUTE_POSITION_Y

GeisFloat


5. GEIS Versioning, Header Files, and Constants

Each published version of GEIS supports the specified client API of that version and all prior versions of those APIs (backwards compatibility).

5.1. Header Files

The GEIS specification defines an ISO C language binding. The GEIS header file <geis/geis.h> provides prototypes for all GEIS entry points and C preprocessors for all GEIS tokens, for use both by client code and implementers.

Languages other than C and C++ will define GEIS interfaces using other methods outside the scope of this specification.

5.2. Compile-time Version Detection

To allow code to be written portably against future GEIS versions, the compile-time environment must make it possible to determine which GEIS version interface are available. For C and C++ code the <geis/geis.h> header defined C preprocessor symbols corresponding to all versions of GEIS supported by the implementation.

#define GEIS_VERSION_1_0  1

Future versions of GEIS will define additional preprocessor symbols corresponding to the major and minor numbers of those versions.

5.3. Constants and Implementation-Supplied Values

Constants and enumerator values for GEIS tokens defined in this specification are required to be common across all implementations. A reference version of the geis.h header file can be downloaded from to be determined.

All implementation-specific types, values, and macros used in geis.h are partitioned into an implementation header, geisimpl.h, which is automatically6 included by geis.h. Implementers should need only modify geisimpl.h, never geis.h.

6. Conformance Testing for Implementers

[Note]

This section is not complete.

Multitouch/APIs/GEIS/1.0Spec (last edited 2010-10-22 15:26:10 by host194)