Multitouch

Summary

Integrates low-level support for certain models of touchscreen devices and exposes basic multitouch functionality on them to allow Ubuntu to be used as a proving ground for multitouch and touch gesture experimentation.

The packages implementing this functionality will be staged in a PPA for evaluation. Depending on the outcome of the evaluation we may then move some or all into Lucid proper. They are currently being staged in this PPA: https://edge.launchpad.net/~xorg-edgers/+archive/multitouch/

Project Planning Information is also available.

Rationale

Ultimately, we'd like to ensure that Ubuntu works with the current generation of multitouch capable touch screens. We have looked at general touchscreen enablement previously in the Touchscreen blueprint, from low level driver support (which for regular touch is largely taken care of with -evdev and XI2) to configuration tools, window-manager functionality, and application support. Certainly, solving the entire breadth of the problem is well beyond what we can do for Lucid, but if we focus on a small subset of the fundamental problems at the lower levels, we can make some useful progress and establish a base from which the higher level implementation work can be done. Our goal is that Ubuntu can provide a solid foundation for experimentation in touch, multitouch, and touch gesture interfaces.

The subset which makes the most sense to focus on as the top priority is the intersection of hardware that can be supported with the -evdev driver, and with whom Canonical has some level of business relationship with in case we have questions (e.g. hardware the OEM team expects to be supporting in the coming year or so). Hopefully in enabling this specific hardware, there will be additional hardware that can trivially be enabled as low hanging fruit.

Overview

The intended stack would be (upside down):

  • kernel driver (mine or others)
  • /dev/input/eventX
  • X.org evdev driver (the one patched by Benjamin, for instance)

  • X.org MPX/XInput2 API
  • multitouchd daemon
  • Qt or GTK+ or other

The current status of the above stack is "multi-pointer only", that is multi-touch without any gesture recognition. There have been debates as to where to put gesture recognition in the stack. Very few people have worked on multitouch gestures.

Multitouch in X is currently limited by the lack of multitouch events in the protocol. Benjamin's patch is a way around it to get multitouch-like features through a multipointer system. Upstream (Peter Hutterer) has indicated a preference to seeing this done via a single-pointer, because then clients would need to deal with a single touch event, instead of one event for each finger. But there are pros and cons to each approach and it's not clear there is yet a consensus about this. See discussion on xorg-devel@ for more background.

Benjamin's patch makes some simplifying assumptions that may not address the full spectrum of possible multi-touch use cases. For instance, multi-user collaboration using inputs from multiple hands going to multiple windows that support both multi-pointer and multi-touch awareness. It is likely some significant API redesign beyond Benjamin's proposed patch would be needed to address all possible cases. Thus, clients which implement support on top of Benjamin's patch would likely need reimplementation for what eventually is officially adopted by upstream.

Kernel

Multitouch support first appears in any form in v2.6.30 bringing finger tracking to the events protocol. There does not appear to be any actual drivers supporting it at that point. In v2.6.31 the n-trig driver seems to have been converted over to expose multi-touch support (with appropriate firmware only). In v2.6.32 and v2.6.33 the support is apparently unchanged. A number of drivers appear to be merging in v2.6.34-rc1 as do a number of fixes for the existing n-trig driver.

The v2.6.34-rc1 HID drivers seems to support multitouch for the following devices:

Manufacturer

USB IDs

Driver

3M 1968

0x0596, 0x0500

hid-3m-pct.c

Asus T91MT

0x0486, 0x0185

hid-mosart.c

N-Trig Touch Screen

0x1b96, 0x0001

hid-ntrig.c

Quanta Optical Touch

0x0408, 0x3000

hid-quanta.c

Quanta Pixart Optical Touch

0x0408, 0x3001

hid-quanta.c

Stantum MTP

0x1f87, 0x0002

hid-stantum.c

It seems that to have any sort of support for multitouch at the kernel level we would need to backport the drivers from at least v2.6.34-rc1 to even get n-trig support in a workable state.

Required Changes

As of v2.6.34-rc1 the commits below seem to be required to get the foundations changed in HID to get the drivers we require for the common multi-touch devices listed above, note these have been selected by inspection and not as yet tested:

b6353f4f36f03a12edaf3fa5365b475a28106035

HID: Support for 3M multitouch panel

4b186f72033611c2b526c7341534e71ee4afd222

HID: make 3M PCT touchscreen driver standalone config option

d3fb5454a8474d5d22c8f8fe4d043b05732d91d5

HID: add support for Stantum multitouch panel

92688c0c3c1c9e2daf705d307e8fda1b5a180d26

HID: make Stantum driver standalone config option

49e4739a0cf681cbfe08c72232c1dcc130b66dde

HID: add support for Acer T230H multitouch

f54405db66fbec11679241daefd16fd8291a5762

HID: add support for Pixart Imaging Optical Touch Screen

b32758c7216f337044ceb6dcaa754b8eda95a59f

HID: fixed bug in single-touch emulation on the stantum panel

580363db92572cccbe6226bf83321e50a9ea50ea

HID: add pressure support for the Stantum multitouch panel

77f720b71d88a3cbf574c113566a31c93099f97d

HID: Support for MosArt multitouch panel

943ed464f3722de0569cf41ba6ec094768ac046d

HID: hid-ntrig add multi input quirk and clean up

b0549cf1a35a92edf053d94066e60fb0ed02bb71

HID: n-trig: remove unnecessary tool switching

dbf2b17de505d390b5ecf5b5944fc0c88f6d66fe

HID: hid-ntrig: multitouch cleanup and fix

ff5cf34c809cd5950579e46e7c10c29bc647aaf1

HID: hid-ntrig: Single touch mode tap

Tasks

See the blueprint whiteboard.

X/Blueprints/Multitouch (last edited 2010-03-22 23:54:32 by rafi-seas)