Android monkeyrunner - check if screen contains an image or specific color

Question

I have an android app which I wanted to automate/stress-test. I made image-based automation script in SCAR Divi and run it against my app installed on Bluestacks. I want to change it to Genymotion, because Bluestacks is too slow.

I already have the environment set up (genymotion+image with my app +adb installed and working). I rewrited my script in python and used it in MonkeyRunner, but...

Besides sending gestures, clicks and waits, I am missing the image recognition functionality. I found out that I can compare 2 screen shots with ImageMagic, but that's not what I need.

I need to check whether a certain button, or at least if a specific color is on the screen. Anyone could point me into right direction with this?

I'd be really grateful for some examples :(

Answer 1

MonkeyRunner does indeed offer pretty decent image based test methods. I will show you below how to take a sub-image from the currently displayed screen of your device. Then you can save this sub-image and/or compare with a reference image

First you need to take a screenshot of the current screen of your device

# Take a screenshot
image = device.takeSnapshot()

Then you can take a sub-image. For example, this sub-image could be a button on a certain screen to verify that the button exists on that screen. To be able to do that you can use the method below. The tuple (0, 0, 50, 100) below is only an example. You should provide your own. Basically it is (x, y, w, h) of the desired sub-image.

# Take the sub-image
sub_image = image.getSubImage((0, 0, 50, 100))

After that step, you can save the image and/or compare with a reference image. To compare with a reference image you can do as shown below:

# Load the reference image for comparison
reference = MonkeyImage.loadFromFile(PATH_TO_REFERENCE_IMAGE)

if not sub_image.sameAs(reference, 0.9):
       print "Images do not match!"
       # do something

0.9 is the acceptance level. If the two images match 90% or above, it will be considered as PASSED

Hope this helps. Do let me know if you encounter any problems!

Answer 2

AndroidViewClient allows you to obtain the screenshot of the entire screen or specific Views. These screenshots are usually written to a file using

View.writeImageToFile(self, filename, format="PNG")

However, if you are interested in obtaining the primitive image object, which is a PIL Image (see http://www.pythonware.com/products/pil/), you can do

image = device.takeSnapshot()

and then manipulate or analyze it as per your needs using the available methods and tools in the library.