UsingTheDevice

Differences between revisions 3 and 43 (spanning 40 versions)
Revision 3 as of 2012-10-26 19:19:39
Size: 3052
Editor: knitzsche
Comment:
Revision 43 as of 2013-02-01 10:50:52
Size: 13977
Editor: p5480ADD5
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
<<Include(Nexus7/Header)>>
||<tablebgcolor="#f1f1ed" tablewidth="40%" tablestyle="margin: 0pt 0pt 1em 1em; float: right; font-size: 0.9em;"style="padding: 0.5em;"><<TableOfContents>>||
Line 2: Line 5:

== Serial Debug Shell ==
This new feature was recently added for 13.04, to enable and use, simply attach the usb cable to the Nexus7,
connect to your computer and in a terminal type:

{{{
 screen /dev/ttyACM0 115200
}}}

This should drop you into a login prompt. To kill, issue "ctrl-a k"

You will need to be in the ''dialout'' group or use ''sudo''.
Line 25: Line 40:
There is a known issue where onboard does not always show automatically when a text input field is selected. In these cases, onboard can be shown manually. To show onboard, first find the onboard indicator in the panel, (the icon should look like this: {{attachment:onboard_ind.png}}. There is a [[https://bugs.launchpad.net/ubuntu-nexus7/+bug/1071508|known issue where onboard does not always show automatically when a text input field is selected]]. In these cases, onboard can be shown manually. To show onboard, first find the onboard indicator in the panel, (the icon should look like this: {{attachment:onboard_ind.png}}.
Line 33: Line 48:
== Performing right-click action with Onboard ==

Unfortunately right-clicking is not possible with the touch UI, even if you enable 'Long-press to right click' in Universal Access. To do this action you need to either use an external mouse, or use the mouse modifier keys in Onboard. To show them you need to click the arrow button on the keyboard:

{{attachment:onboard_keyshift.png}}

Then when you want to perform a right-click you just need to click on the right-hand leaning arrow:

{{attachment:onboard_keyshift_pressed.png}}
Line 37: Line 62:
 1. Open Settings from the Unity launcher panel.  1. Open '''System Settings''' from the Unity launcher panel.
Line 40: Line 65:

== Increasing Unity Launcher Icon Size ==

You may want to increase the size of icons on the Unity launcher panel. This will make them easier to see and easier to select by touch.

 1. Open '''System Settings''' from the Unity launcher panel
 1. Open the '''Appearances''' settings app.
 1. On the '''Look''' tab, slide the '''Launcher icon size''' slider to the largest size possible: 64.
Line 48: Line 81:

== Attaching a Bluetooth mouse ==

Even that there are still bugs in the Bluetooth stack, Bluetooth mice simply work with the built-in Bluetooth module. Simply use GNOME's Bluetooth tool by clicking on the Bluetooth indicator applet and pair the mouse as you are used to on an Ubuntu desktop/laptop PC.

== Remote controlling via VNC ==

Another way to overcome the shortcomings of the UI on the Nexus 7 is to remote control it via VNC. To start and configure the VNC server, run the command

{{{vino-preferences}}}

In the window appearing check "Allow other users to view your desktop", "Allow other users to control your desktop", and "Require the user to enter this password". Enter a password of your choice. Uncheck the other checkboxes and at the bottom choose "Only when someone is connected". After that, click "Close". Now the vino user daemon should be running and you can securely remote access with any VNC client (after entering your chosen password in the client). If this does not work, start the vino server manually with

{{{/usr/lib/vino/vino-server &}}}

Now you can easily control your Nexus 7 with your PC's screen/keyboard/mouse, for example using the remote desktop viewer vinagre on Ubuntu (activate JPEG compression in Vinagre to improve the speed).

You cannot only forward the whole desktop to a remote screen but also emulate keyboard and mouse on a remote device, preferably a smartphone. If you have an iPhone, install [[http://hipporemote.com/|HippoRemote]] from the [[https://itunes.apple.com/us/app/hipporemote-pro-wireless-keyboard/id320157370?mt=8&ign-mpt=uo%3D6|AppStore]] and use the touch keyboard of the iPhone (special PC-typical keys are added) or use the iPhone's touchscreen like the mouse touchpad of a laptop.

== Watching YouTube videos ==

Unfortunately, the Adobe Flash Player is not ported to Linux on ARM architectures, making YouTube videos not play in the default configuration, but fortunately, YouTube is beta-testing HTML-5-based video playback, which works on most modern free software web browsers without any closed-source plug-in or helper application, opening up access to the videos on any architecture if the hardware is powerful enough to play videos.

To be able to watch via HTML 5 you need to activate this mode in YouTube. Go to [[https://www.youtube.com/html5|this page]] and click on "Join the HTML5 Trial" at the end of the text. Note that not all videos are supported but most are.

Playback speed is very good, much better than playing locally stored video files. Do not forget to suspend your system and wake it up again to get sound working.

I have tested with Ubuntu 13.04 and Firefox.

== Nexus 7 with Ubuntu getting useful: Media Player ==

Works only with Ubuntu 13.04.

There is a free software media player intended to run on set-top boxes/HTPCs and display on TV sets: [[http://xbmc.org/|XBMC]]. Due to the larger control elements and the GLES video acceleration support it is easy to operate with the fingers on the touch screen and videos are playing fluently. XBMC plays local video and music files, files from a DLNA server in the local network and also content from the internet like YouTube, Vimeo, iTunes podcasts, video archives of many TV stations (via the so-called "Add-ons"), and even live streams.

Install the software via Software Center or

{{{sudo apt-get install xbmc}}}

After that log out, and on the login screen, click the little white circle near the user name (it is a Ubuntu logo by default), choose "XBMC" and then "OK". Click the accessibility icon in the top bar to activate the onscreen keyboard and the enter your password to log in again. Now you get directly into XBMC without a desktop clogging up your memory. Also when you shut down and restart your Nexus 7 you get into XBMC, until you log out, change back to "Ubuntu default", and log in again.

XBMC can now be operated by the touch screen. To get sound, you need to suspend and wake up the device again. For this click the Off button at the lower left corner, and choose "Suspend" in the pop-up menu. When the screen is black, press the power button to wake up your Nexus 7 again.

XBMC can also be started on the desktop. The command

{{{xbmc}}}

starts it nicely fullscreen and you can also operate it by the touch screen. Do not try to switch it to windowed mode as there is no easy way to get back to fullscreen and there is also no way to watch videos fullscreen when in windowed mode. Note that with a running desktop XBMC does not have much memory left and so can crash due to lack of memory.

Note: The current version, XBMC 12.0 RC2 does not yet work on the Nexus 7 ([[https://bugs.launchpad.net/xbmc/+bug/1093529|Bug 1093529]]), please install the [[https://launchpad.net/ubuntu/+source/xbmc/2:11.0~git20120510.82388d5-1ubuntu8|last packages of 11.0]] ([[https://launchpad.net/ubuntu/+source/xbmc/2:11.0~git20120510.82388d5-1ubuntu8/+build/4063406/+files/xbmc_11.0%7Egit20120510.82388d5-1ubuntu8_all.deb|xbmc]] and [[https://launchpad.net/ubuntu/+source/xbmc/2:11.0~git20120510.82388d5-1ubuntu8/+build/4063405/+files/xbmc-bin_11.0%7Egit20120510.82388d5-1ubuntu8_armhf.deb|xbmc-bin]]). Download both and run the commands

{{{sudo apt-get install xbmc xbmc-bin}}}<<BR>>
{{{sudo dpkg -P xbmc xbmc-bin}}}<<BR>>
{{{sudo apt-get install -f}}}<<BR>>
{{{sudo apt-get install libavcodec53 libavutil51 libpostproc52 libavfilter2 libsdl-mixer1.2 libswscale2}}}<<BR>>
{{{sudo dpkg -i xbmc_11.0~git20120510.82388d5-1ubuntu8_all.deb xbmc-bin_11.0~git20120510.82388d5-1ubuntu8_armhf.deb}}}

Do not update these packages, or downgrade them after each system update.

= Workarounds =
The following section describes ways to work around certain bugs or lack of features. These should not be interpreted as fixes, and should be used at your own risk, as they are not supported by Canonical in any way.

== Attaching a Bluetooth keyboard (or keyboard/mouse combo) ==

To make input easier but without loosing too much portability one can use a Bluetooth keyboard/mouse combo like this [[http://www.amazon.de/gp/product/B0085AC97Y/ref=oh_details_o01_s00_i00|3 in 1 Bluetooth Multimedia Wireless Keyboard]] paired to the internal Bluetooth module of the Nexus 7, keeping the Nexus free from cable cluttor or allow charging the Nexus while using it.

Unfortunately you cannot pair it using GNOME's graphical tools for managing Bluetooth, but it works via the command line. First, install some extra command line tools (Enter "ubuntu" as password if prompted):

{{{sudo apt-get install bluez-compat}}}

Now put the device in pairing mode (for device linked above, long-press "BT" key until blue LED blinks) and run the command:

{{{hcitool scan}}}

This gives an output like

{{{Scanning ...}}} <<BR>>
{{{ 20:42:A1:00:66:78 bluetooth keyboard}}}

To pair the keyboard enter

{{{sudo hidd --connect 20:42:A1:00:66:78}}}

using the actual address of the keyboard. Now keyboard and mouse work. Note that this uses no PIN, so it is possible that the connection is unencrypted, but for testing/development/demo it works great. Put the keyboard back into pairing mode and repeat this command if the keyboard looses the connection.

It is recommended to go into the preferences of Onboard and turn off the automatic pop-up of the on-screen keyboard (Click indicator applet for Onboard, choose "Preferences", and in the preferences window on the "General" tab uncheck "Auto-show when editing text").

== Stabilizing the Bluetooth stack ==

Unfortunately, the Bluetooth stack is broken and so most Bluetooth devices cannot be used with the standard tools available via the Bluetooth indicator applet. The problem is that the daemon for managing the Bluetooth devices, bluetoothd, crashes. This is reported in [[https://bugs.launchpad.net/ubuntu-nexus7/+bug/1070770|bug 1070770]] and as a workaround one can run bluetoothd under the control of the debugging tool Valgrind. so install this tool via

{{{sudo apt-get install valgrind}}}

and then edit /etc/init/bluetooth.conf replacing the line

{{{exec /usr/sbin/bluetoothd}}}

by

{{{exec /usr/bin/valgrind /usr/sbin/bluetoothd >/dev/null 2>&1}}}

After that restart bluetoothd via

{{{sudo restart bluetooth}}}

and if this hangs, reboot.

Now bluetoothd is stabilized and does not crash any more if you try to pair a Bluetooth keyboard. The keyboard mentioned in the previous section works perfectly now, and using a PIN, so input is secure. Unfortunately, Bluetooth printing does not work ([[https://bugs.launchpad.net/ubuntu/+source/bluez/+bug/1080507|bug 1080507]]).

Note that this is a workaround and not a fix. If you are not a Bluetooth developer and want to explore/test/fix other things more easily it is of great help until the bluetoothd crash bug gets actually fixed.



= Start filing bugs =
Please file bugs according to this guide [[Nexus7/LoggingBugs| Logging Bugs]]. Find a lists of bugs to start working on in the [[/Developers | developers section]].

Apart from running Ubuntu Core on your Nexus 7 and filing bugs you can help us out by [[Nexus7/MeasuringMemoryUsage|measuring memory usage]] and [[Nexus7/MeasuringPowerConsumption|measuring power consumption]]. These are two things we definitely want to improve.

This project is no longer maintained. You might be interested in Ubuntu Touch though.

Using the Device

Serial Debug Shell

This new feature was recently added for 13.04, to enable and use, simply attach the usb cable to the Nexus7, connect to your computer and in a terminal type:

 screen /dev/ttyACM0 115200

This should drop you into a login prompt. To kill, issue "ctrl-a k"

You will need to be in the dialout group or use sudo.

Multitouch & Gestures

Only some multitouch gestures are enabled on the Nexus 7. For the gestures that are not enabled, the failures are a combination of bugs and unimplemented code. Section 5 of the Unity Gesture UI Guidelines covers what behaviors the gestures are supposed to do. Right now, only the following gestures are known to work:

Gesture

Correct Behavior

Single finger tap

Select item

Single finger drag

Move item

Single finger double tap

Select a whole word or item

Single finger triple tap

Select a paragraph or related set of items

3 finger tap

Bring-up Unity grab handles

4 finger tap

Open the dash

Resetting the Device

If the device is locked-up then you can reset it by holding down Volume Down and Power. When the screen goes black release the buttons. If you end up in fastboot mode (by holding down too long) just click the power button and it will boot back into Ubuntu.

sudo password

The default sudo password is ubuntu.

Starting Onboard (onscreen keyboard) when it fails to auto-show

There is a known issue where onboard does not always show automatically when a text input field is selected. In these cases, onboard can be shown manually. To show onboard, first find the onboard indicator in the panel, (the icon should look like this: onboard_ind.png.

Click this indicator, which will then bring up the Onboard menu. From here, simply select "Show Onboard"

show_onboard.png

You might also want to enable Show floating icon when Onboard is hidden from the Onboard Preferences > General tab. With this setting enabled, when you hide the onboard onscreen keyboard, an icon displays. Click the icon to show the keyboard again. This may be easier than launching Onboard again from the Onboard indicator panel menu.

Performing right-click action with Onboard

Unfortunately right-clicking is not possible with the touch UI, even if you enable 'Long-press to right click' in Universal Access. To do this action you need to either use an external mouse, or use the mouse modifier keys in Onboard. To show them you need to click the arrow button on the keyboard:

onboard_keyshift.png

Then when you want to perform a right-click you just need to click on the right-hand leaning arrow:

onboard_keyshift_pressed.png

Increasing Font Size

You may want to increase the default font size. This not only makes GUI text easier to read, it also makes the menus easier to navigate because the selection targets become larger.

  1. Open System Settings from the Unity launcher panel.

  2. Open the Universal Access settings app.

  3. On the Seeing tab, set the Text size to Larger (this is the largest possible setting).

Increasing Unity Launcher Icon Size

You may want to increase the size of icons on the Unity launcher panel. This will make them easier to see and easier to select by touch.

  1. Open System Settings from the Unity launcher panel

  2. Open the Appearances settings app.

  3. On the Look tab, slide the Launcher icon size slider to the largest size possible: 64.

Attaching a USB Device to the Nexus 7

You can attach USB devices to the Nexus 7 if you purchase a Micro USB Host Cable (OTG Cable) from Amazon or other electronics retailers. The cable may be difficult to find at local stores and you may have to order online. Once this cable is attached to the Nexus 7 you can attach a USB hub or a USB device directly.

Warning /!\ There is no way to charge the device while it is connected to the OTG cable Warning /!\

otg-keyboard.jpg

Attaching a Bluetooth mouse

Even that there are still bugs in the Bluetooth stack, Bluetooth mice simply work with the built-in Bluetooth module. Simply use GNOME's Bluetooth tool by clicking on the Bluetooth indicator applet and pair the mouse as you are used to on an Ubuntu desktop/laptop PC.

Remote controlling via VNC

Another way to overcome the shortcomings of the UI on the Nexus 7 is to remote control it via VNC. To start and configure the VNC server, run the command

vino-preferences

In the window appearing check "Allow other users to view your desktop", "Allow other users to control your desktop", and "Require the user to enter this password". Enter a password of your choice. Uncheck the other checkboxes and at the bottom choose "Only when someone is connected". After that, click "Close". Now the vino user daemon should be running and you can securely remote access with any VNC client (after entering your chosen password in the client). If this does not work, start the vino server manually with

/usr/lib/vino/vino-server &

Now you can easily control your Nexus 7 with your PC's screen/keyboard/mouse, for example using the remote desktop viewer vinagre on Ubuntu (activate JPEG compression in Vinagre to improve the speed).

You cannot only forward the whole desktop to a remote screen but also emulate keyboard and mouse on a remote device, preferably a smartphone. If you have an iPhone, install HippoRemote from the AppStore and use the touch keyboard of the iPhone (special PC-typical keys are added) or use the iPhone's touchscreen like the mouse touchpad of a laptop.

Watching YouTube videos

Unfortunately, the Adobe Flash Player is not ported to Linux on ARM architectures, making YouTube videos not play in the default configuration, but fortunately, YouTube is beta-testing HTML-5-based video playback, which works on most modern free software web browsers without any closed-source plug-in or helper application, opening up access to the videos on any architecture if the hardware is powerful enough to play videos.

To be able to watch via HTML 5 you need to activate this mode in YouTube. Go to this page and click on "Join the HTML5 Trial" at the end of the text. Note that not all videos are supported but most are.

Playback speed is very good, much better than playing locally stored video files. Do not forget to suspend your system and wake it up again to get sound working.

I have tested with Ubuntu 13.04 and Firefox.

Nexus 7 with Ubuntu getting useful: Media Player

Works only with Ubuntu 13.04.

There is a free software media player intended to run on set-top boxes/HTPCs and display on TV sets: XBMC. Due to the larger control elements and the GLES video acceleration support it is easy to operate with the fingers on the touch screen and videos are playing fluently. XBMC plays local video and music files, files from a DLNA server in the local network and also content from the internet like YouTube, Vimeo, iTunes podcasts, video archives of many TV stations (via the so-called "Add-ons"), and even live streams.

Install the software via Software Center or

sudo apt-get install xbmc

After that log out, and on the login screen, click the little white circle near the user name (it is a Ubuntu logo by default), choose "XBMC" and then "OK". Click the accessibility icon in the top bar to activate the onscreen keyboard and the enter your password to log in again. Now you get directly into XBMC without a desktop clogging up your memory. Also when you shut down and restart your Nexus 7 you get into XBMC, until you log out, change back to "Ubuntu default", and log in again.

XBMC can now be operated by the touch screen. To get sound, you need to suspend and wake up the device again. For this click the Off button at the lower left corner, and choose "Suspend" in the pop-up menu. When the screen is black, press the power button to wake up your Nexus 7 again.

XBMC can also be started on the desktop. The command

xbmc

starts it nicely fullscreen and you can also operate it by the touch screen. Do not try to switch it to windowed mode as there is no easy way to get back to fullscreen and there is also no way to watch videos fullscreen when in windowed mode. Note that with a running desktop XBMC does not have much memory left and so can crash due to lack of memory.

Note: The current version, XBMC 12.0 RC2 does not yet work on the Nexus 7 (Bug 1093529), please install the last packages of 11.0 (xbmc and xbmc-bin). Download both and run the commands

sudo apt-get install xbmc xbmc-bin
sudo dpkg -P xbmc xbmc-bin
sudo apt-get install -f
sudo apt-get install libavcodec53 libavutil51 libpostproc52 libavfilter2 libsdl-mixer1.2 libswscale2
sudo dpkg -i xbmc_11.0~git20120510.82388d5-1ubuntu8_all.deb xbmc-bin_11.0~git20120510.82388d5-1ubuntu8_armhf.deb

Do not update these packages, or downgrade them after each system update.

Workarounds

The following section describes ways to work around certain bugs or lack of features. These should not be interpreted as fixes, and should be used at your own risk, as they are not supported by Canonical in any way.

Attaching a Bluetooth keyboard (or keyboard/mouse combo)

To make input easier but without loosing too much portability one can use a Bluetooth keyboard/mouse combo like this 3 in 1 Bluetooth Multimedia Wireless Keyboard paired to the internal Bluetooth module of the Nexus 7, keeping the Nexus free from cable cluttor or allow charging the Nexus while using it.

Unfortunately you cannot pair it using GNOME's graphical tools for managing Bluetooth, but it works via the command line. First, install some extra command line tools (Enter "ubuntu" as password if prompted):

sudo apt-get install bluez-compat

Now put the device in pairing mode (for device linked above, long-press "BT" key until blue LED blinks) and run the command:

hcitool scan

This gives an output like

Scanning ...
     20:42:A1:00:66:78       bluetooth keyboard

To pair the keyboard enter

sudo hidd --connect 20:42:A1:00:66:78

using the actual address of the keyboard. Now keyboard and mouse work. Note that this uses no PIN, so it is possible that the connection is unencrypted, but for testing/development/demo it works great. Put the keyboard back into pairing mode and repeat this command if the keyboard looses the connection.

It is recommended to go into the preferences of Onboard and turn off the automatic pop-up of the on-screen keyboard (Click indicator applet for Onboard, choose "Preferences", and in the preferences window on the "General" tab uncheck "Auto-show when editing text").

Stabilizing the Bluetooth stack

Unfortunately, the Bluetooth stack is broken and so most Bluetooth devices cannot be used with the standard tools available via the Bluetooth indicator applet. The problem is that the daemon for managing the Bluetooth devices, bluetoothd, crashes. This is reported in bug 1070770 and as a workaround one can run bluetoothd under the control of the debugging tool Valgrind. so install this tool via

sudo apt-get install valgrind

and then edit /etc/init/bluetooth.conf replacing the line

exec /usr/sbin/bluetoothd

by

exec /usr/bin/valgrind /usr/sbin/bluetoothd >/dev/null 2>&1

After that restart bluetoothd via

sudo restart bluetooth

and if this hangs, reboot.

Now bluetoothd is stabilized and does not crash any more if you try to pair a Bluetooth keyboard. The keyboard mentioned in the previous section works perfectly now, and using a PIN, so input is secure. Unfortunately, Bluetooth printing does not work (bug 1080507).

Note that this is a workaround and not a fix. If you are not a Bluetooth developer and want to explore/test/fix other things more easily it is of great help until the bluetoothd crash bug gets actually fixed.

Start filing bugs

Please file bugs according to this guide Logging Bugs. Find a lists of bugs to start working on in the developers section.

Apart from running Ubuntu Core on your Nexus 7 and filing bugs you can help us out by measuring memory usage and measuring power consumption. These are two things we definitely want to improve.

Nexus7/UsingTheDevice (last edited 2013-04-26 10:54:55 by 60-241-64-52)