How to locate element in UIautomator if native keyboard come before your application?

Question

As per project requirement i am working on Mobile App automation. Not problem arises when i executed same code which worked fine on emulator but when it comes to real device the same code were getting failed.the problem is UiAutomator is not able to locate element because of native keyboard come before an application during simulation. I executed this entire thing into Galaxy nexus which works on ANDROID API 18.hence no point to execute whole automation suites in Selendroid mode. in below code after filling value in first editbox,control should have reached to second editbox to fill value and so on. But it does not fill value there because native keyboard appear before application.

            SwipeableWebDriver driver = new SwipeableWebDriver(
            new URL("http://127.0.0.1:4723/wd/hub"), capabilities);
    List<WebElement> editTextList = driver.findElements(By
            .className("android.widget.EditText"));

    editTextList.get(0).sendKeys(c + "Bob");
    editTextList.get(1).sendKeys("123");
    editTextList.get(2).sendKeys("456");
    el = driver.findElement(By.className("android.widget.Button"));
    el.click();

Please anyone who have idea to resolve this issue? Thanks in advance.

Priyank Shah

Answer 1

First of all you should realize whether the soft keyboard is active or not - Use the following command from your code to check "mInputShown" parameter - If "true" - Active Soft Keyboard.

 adb shell dumpsys input_method | grep mInputShown

Use this code for hiding the native keyboard in Java-appium running older versions of appium.

driver.navigate().back()

P.S - The adb command is useless for emulators as the flag whose value is being checked is always set to true, whether your keyboard is active or not.

Answer 2

I don't think you can, and and it is not a appium limitation. From what I observed even the UIAutomator can not find the elements hidden by the keyboard.

I know 2 solutions for this:

1. Dismiss the keyboard. (I didn't find any elegant ways of doing that so I'm not using this.
2. Swipe/scroll on the view until the element is exposed, and then you can action it. This works fine for me.

Answer 3

you should be able to dismiss the keyboard by sending

driver.findElement(By.name("Return")).click();

Answer 4

adding new line character works too editTextList.get(2).sendKeys("456\n");

Answer 5

If you can detect that the keyboard is open, I would suggest calling UiDevice.pressBack() to dismiss the keyboard.

Answer 6

You could be able to dismiss the keyboard by using the following code

driver.hideKeyboard();

Answer 7

Put the following two lines: driver.getKeyboard(); driver.hideKeyboard();

Answer 8

Here is a uiautomator ready fully functional method that will respond with true if the keyboard is Open and false if it is closed:

public static boolean isKeyboardDisplayed() {
    String checkKeyboardCommand = "dumpsys input_method | grep mInputShown";
    try {
        Process process = Runtime.getRuntime().exec(checkKeyboardCommand);
        BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
        int read;
        char[] buffer = new char[4096];
        StringBuffer output = new StringBuffer();
        while ((read = reader.read(buffer)) > 0) {
            output.append(buffer, 0, read);
        }
        reader.close();
        process.waitFor();

        if (output.toString().contains("mInputShown=true")) {
            return true;
        } else {
            return false;
        }

    } catch (IOException e) {
        throw new RuntimeException(e);
    } catch (InterruptedException e) {
        throw new RuntimeException(e);
    }

}

Answer 9

You may need to enable unicodeKeyboard: true in your android capabilities and use the keyboard button Return to hide the keyboard if shown (this work for me on iOS and Android)

For example, I am using ruby:

  element = $appium.find_element(id: field_id)
  element.clear
  element.send_keys(data)
  element.send_keys(:return) if driver.is_keyboard_shown