Android¶

Installation¶

$ gem install calabash-android

$ sudo gem install calabash-android

$ xcode-select --install

$ sudo -i

$ ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future gem install calabash-android

export ANDROID_HOME=/Applications/sdk
export PATH=$PATH:$ANDROID_HOME:$ANDROID_HOME/tools:$ANDROID_HOME/platform-tools:$ANDROID_HOME/add-ons

$ defaults write com.apple.finder AppleShowAllFiles TRUE
$ killall Finder

$ defaults write com.apple.Finder AppleShowAllFiles TRUE
$ killall Finder

$ adb devices

List of devices attached
605A000600000001015F3E001200C00B     device

Preparing testcases¶

After you have downloaded the files open a new terminal window and navigate via cd to the TMSampleAndroid folder that you just downloaded. Then execute:

$ calabash-android run TestmunkTest_debug.apk --verbose

$ keytool -genkey -v -keystore ~/.android/debug.keystore -alias androiddebugkey -storepass android -keypass android -keyalg RSA -keysize 2048 -validity 10000 -dname "CN=Android Debug,O=Android,C=US"

$ calabash-android run TestmunkTest_debug.apk

You should get prompted to resign the app. Follow the terminal instructions to resign the app. After resigning please again execute:

$ calabash-android run TestmunkTest_debug.apk --verbose

This should install the app on your device, and after a minute or two it should get launched and our sample testcases should get executed. It will ca. 3-5 min for the testcases to get completed, you can also exit the terminal to stop the testrun. Simply enter exit into the terminal.

In the following we’d like to show you how you can easily write your own testcase.

Inspect app for elements¶

UI Automator Viewer¶

Please tap on the app on your device so that it is launched. Open a new terminal window and execute:

$ uiautomatorviewer

Hint

In case you need to install an APK file on your device, you can cd into the folder that contains it and use this command:

$ adb install NameofTheFile.apk

Installing APKs

On the newly opened window, please click on the device icon on the upper left corner to get an actual screenshot from the device.

This inspection is important to identify the right elements that you later will need for your testcases. For example the resource_id is needed when you use the teststep Then I touch view with id....

Important

If you use an Android version lower than API level 18 / Jelly Bean you will not be able to interact with the resource-id of the element.

Early versions of Android

Calabash console¶

A more advanced way of inspecting elements on the view is using the console. Open a new terminal window, cd into the folder that contains your APK file, and enter:

$ calabash-android console TestmunkTest_debug.apk

and then enter these commands:

> start_test_server_in_background

> query("*")

You should see a list of all visible elements.

Feature: Testapp V.1.2

Scenario: 1) Going to next screen and back
        When I enter "Something" into input field number 1
        Then I press the "See details" button
        Then I wait
        Then I should see text containing "Something"
        Then I go back
        Then I should see text containing "test app"

Scenario: 1) Going to next screen and back
        When I enter "Something" into input field number 1
        Then I press the "See details" button
        Then I wait
        Then I should see text containing "Something"
        Then I go back
        Then I should see text containing "test app"

Scenario: 2) Clear the input field
        When I enter "Something" into input field number 1
        Then I clear input field number 1
        Then I press the "See details" button
        Then I should not see "Something"

Calabash Ruby API¶

Calabash offers a Ruby API that we support for defining special teststeps.

A new teststep is defined in the following way:

# Define a regular expression to catch the step
Then(/^"(.*?)" radio button should be selected$/) do |arg1|
  # Use calls to the Calabash API to get information
  if(!query("RadioButton text:'#{arg1}'", :checked).first())
    # Act on that information
    fail("The radio button with text #{arg1} should be selected")
  end
end

A teststep is considered succesful if the execution of its codeblock runs with neither explicit fails nor uncaught errors.

A nice way to try the different commands on this API is to run the Calabash console and test them.

> query("FrameLayout index:0")

[
    [0] {
                        "id" => "content",
                   "enabled" => true,
        "contentDescription" => nil,
                     "class" => "android.widget.FrameLayout",
                      "rect" => {
            "center_y" => 617.0,
            "center_x" => 384.0,
              "height" => 1134,
                   "y" => 50,
               "width" => 768,
                   "x" => 0
        },
               "description" => "android.widget.FrameLayout{41f40dc0 V.E..... ........ 0,50-768,1184 #1020002 android:id/content}"
    }
]

> query("FrameLayout index:0").first.keys

[
    [0] "id",
    [1] "enabled",
    [2] "contentDescription",
    [3] "class",
    [4] "rect",
    [5] "description"
]

> query("FrameLayout index:0")[0]["id"]

"content"

wait_for_elements_exist( ["button marked:'OK'", "* marked:'Cancel'"], :timeout => 2)

touch("FrameLayout index:0")
touch(query("FrameLayout"))

Running testruns¶

$ calabash-android run sample.apk --verbose

Image Comparison in Calabash¶

The goal of this post is to show how we can do basic image recognition using Calabash Android library.

Image comparison is another way that allows you to assert your tests using Calabash. However, Calabash does not support it by default. So, we have created some custom steps that you can include in your features folder, and you’ll have image comparison working in a short time.

Image comparison is a tricky topic. Some comparisons are as simple as pixel by pixel checking; very advanced scenarios may compare a small image within a bigger image, or even images which are slightly shifted or compressed.

We’ve chosen the simple approach for now, which means a pixel by pixel check. This check uses a difference blend, which is the same thing Github uses to diff images.

If we have pixelation, or an image that is slightly lighter or darker, the steps will still be able to make the comparison. Another benefit is that it returns a more realistic readout of percentage changed, and allows us to set maximum thresholds while testing.

If you want to compare an image (local or remote) with the current screen shot, it needs to match the resolution in order to be effective. The best use case is testing the app on a device that you already have the screenshots for.

To get up and running, we will need to install an extra gem to handle the image manipulation. We can do that using:

$ gem install oily_png

This is in addition to the calabash-android gem, which should already be installed and configured.

Once you have the gem installed, create a new file under features/step_definitions folder (with any name). Paste in the following code:

require 'oily_png'
require 'open-uri'
include ChunkyPNG::Color

def starts_with(item, prefix)
  prefix = prefix.to_s
  item[0, prefix.length] == prefix
end

# compares two images on disk, returns the % difference
def compare_image(image1, image2)
  images = [
    ChunkyPNG::Image.from_file("screens/#{image1}"),
    ChunkyPNG::Image.from_file("screens/#{image2}")
  ]
  count=0
  images.first.height.times do |y|
    images.first.row(y).each_with_index do |pixel, x|

      images.last[x,y] = rgb(
        r(pixel) + r(images.last[x,y]) - 2 * [r(pixel), r(images.last[x,y])].min,
        g(pixel) + g(images.last[x,y]) - 2 * [g(pixel), g(images.last[x,y])].min,
        b(pixel) + b(images.last[x,y]) - 2 * [b(pixel), b(images.last[x,y])].min
      )
      if images.last[x,y] == 255
        count = count + 1
      end
    end
  end

  100 - ((count.to_f / images.last.pixels.length.to_f) * 100);
end

# find the file
def get_screenshot_name(folder, fileName)
  foundName = fileName
  Dir.foreach('screens/') do |item|
  next if item == '.' or item == '..'
    if item.start_with? fileName.split('.')[0]
      foundName = item
    end
  end

  foundName
end

def setup_comparison(fileName, percentageVariance, forNotCase = false)
  screenshotFileName = "compare_#{fileName}"
  screenshot({ :prefix => "screens/", :name => screenshotFileName })

  screenshotFileName = get_screenshot_name("screens/", screenshotFileName)
  changed = compare_image(fileName, screenshotFileName)
  FileUtils.rm("screens/#{screenshotFileName}")

  assert = true
  if forNotCase
    assert = changed.to_i < percentageVariance
  else
    assert = changed.to_i > percentageVariance
  end

  if assert
    fail(msg="Error. The screen shot was different from the source file. Difference: #{changed.to_i}%")
  end

end

def setup_comparison_url(url, percentageVariance)
  fileName = "tester.png"
  open("screens/#{fileName}", 'wb') do |file|
    file << open(url).read
  end

  setup_comparison(fileName, percentageVariance)
  FileUtils.rm("screens/#{fileName}")
end

Then(/^I compare the screen with "(.*?)"$/) do |fileName|
  setup_comparison(fileName, 0)
end

Then(/^I compare the screen with url "(.*?)"$/) do |url|
  setup_comparison_url(url, 0)
end

Then(/^the screen should not match with "(.*?)"$/) do |fileName|
  setup_comparison(fileName, 0, true)
end

Then(/^I expect atmost "(.*?)" difference when comparing with "(.*?)"$/) do |percentageVariance, fileName|
  setup_comparison(fileName, percentageVariance.to_i)
end

Then(/^I expect atmost "(.*?)" difference when comparing with url "(.*?)"$/) do |percentageVariance, url|
  setup_comparison_url(url, percentageVariance.to_i)
end

If you are using local screen shots, add the source images to a “screens” folder at the same level as the features folder. You will use the name of these images in your test steps.

The following steps are available after injecting the library:

Then I compare the screen with "login_screen.png"
Then I expect atmost "2%" difference when comparing with "login_screen_fail.png"

Then I compare the screen with url "http://testmunk.com/login_screen.png"
Then I expect atmost "2%" difference when comparing with url "http://testmunk.com/login_screen_fail.png"

Then the screen should not match with "screen2.png"

You have three different types of steps. One asserts an exact match, another asserts an approximate match (i.e. up to 2%), and the final one reads if the image does not match (asserting if a particular view-changing action has happened or not). You can also use local files (which should be present in the /screens folder) or remotely uploaded files.

If there is a match failure, you will get the percentage difference in the output so you know how much of the screenshot was to the source.

Sources:

• http://jeffkreeftmeijer.com/2011/comparing-images-and-creating-image-diffs/

Note:

• This will work with Calabash iOS as well. However, for games using OpenGL, the screenshot utility of Calabash does not work.