Xpresser
|
Size: 2618
Comment:
|
Size: 3457
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 58: | Line 58: |
| Instead of clicking, sometimes you may simply want to put the mouse cursor over something so that the interface reacts to the this event. This can be done in a similar way to clicking. | Instead of clicking, sometimes you may simply want to put the mouse cursor over something so that the interface reacts to this event. This can be done in a similar way to clicking. |
| Line 68: | Line 68: |
| You may not want to do any action on the image itself, but simply to wait until something is shown in the screen before doing an additional action. This can easily done as such: | You may not want to do any action on the image itself, but simply to wait until something is shown in the screen before doing an additional action. This can easily be done as such: |
| Line 74: | Line 74: |
=== Emulating the keyboard === If you want to use the keyboard to type some value in an edit box, for example, you can use the following method: {{{ >>> xp.click("name") >>> xp.type("John Doe") }}} |
|
| Line 76: | Line 86: |
| Sometimes, you may want to match an image even though it's slightly less similar to the original image you have taken the screenshot from, or you may want to click on a spot which is not quite the middle of the image you're matching. | Sometimes, you may want to match an image even though it's slightly less similar to the original image you have taken the screenshot from, or you may want to click on a spot which is not quite the middle of the image you're matching. Xpresser offers a convenient way to achieve these goals: just drop a file named {{{xpresser.ini}}} inside the images directory, and include sections in it resembling the following one: {{{ [image edit] filename = edit-text-box.png similarity = 0.8 focus_delta = +50 +0 }}} Each of these sections will customize the behavior for one image. The example above, for instance, will determine that the "edit" image will match even if it's only 80% similar to the original image, and if some action is performed on it (e.g. click), it will actually happen 50 pixels to the right side of the middle of the image matched. |
Contents
What is Xpresser?
Xpresser is a Python module which enables trivial automation of Graphic User Interfaces (GUIs) via image matching algorithms.
Requirements
Xpresser depends on the following libraries to do its work:
- python-opencv
- python-pyatspi
- python-gtk2
- python-numpy
License and copyright
Xpresser is copyrighted by Canonical Ltd., and licensed under the GNU LGPL.
Downloading
At the moment, Xpresser is available solely as a Bazaar branch in Launchpad.
Execute the following command to download it:
- bzr branch lp:xpresser
More details about the project may be obtained in the Launchpad page:
Documentation
Getting started
Starting to use Xpresser is quite easy. All it needs to operate is a directory containing screen excerpts (images containing relevant crops taken out of one or more screenshots). These might be buttons, edit areas, image buttons, or anything else which could be relevant to perform an action on, or maybe simply to wait for its presence.
With this in place, an instance of the Xpresser class can be created:
>>> xp = Xpresser()
>>> xp.load_images("/path/to/images")
Clicking somewhere
If you want to press a button in the screen, for instance, you can take a screenshot of the button itself and maybe a bit of margin around it, and put the screenshot excerpt in a directory.
Then, assuming you've named the image my-button.png, you might simply do something like this:
>>> xp.click("my-button")
Hovering over something
Instead of clicking, sometimes you may simply want to put the mouse cursor over something so that the interface reacts to this event. This can be done in a similar way to clicking.
Again, assuming you've named the image my-button.png, you might to do this:
>>> xp.hover("my-button")
Waiting for things to show up
You may not want to do any action on the image itself, but simply to wait until something is shown in the screen before doing an additional action. This can easily be done as such:
>>> xp.wait("connection-established-indicator")
Emulating the keyboard
If you want to use the keyboard to type some value in an edit box, for example, you can use the following method:
>>> xp.click("name")
>>> xp.type("John Doe")
Customizing the behavior on specific images
Sometimes, you may want to match an image even though it's slightly less similar to the original image you have taken the screenshot from, or you may want to click on a spot which is not quite the middle of the image you're matching.
Xpresser offers a convenient way to achieve these goals: just drop a file named xpresser.ini inside the images directory, and include sections in it resembling the following one:
[image edit] filename = edit-text-box.png similarity = 0.8 focus_delta = +50 +0
Each of these sections will customize the behavior for one image. The example above, for instance, will determine that the "edit" image will match even if it's only 80% similar to the original image, and if some action is performed on it (e.g. click), it will actually happen 50 pixels to the right side of the middle of the image matched.
Xpresser (last edited 2012-12-22 22:48:08 by c-24-147-75-163)