Selenium WebDriver: dragAndDrop method fails in IE11 and IE10

Question

We've been using SauceLabs for our nightly Jenkins CI tests. We've run the tests in Firefox 24 under Windows 8 without any problem. Now we want to expand the tests and run them on more platforms and browsers. Our goal is to run the tests in the latest Firefox and Chrome browsers under Windows and OSX and IE11, 10 and 9.

The tests are written using a node.js client for webdriver/selenium2 https://github.com/admc/wd

The problem I have now is getting our dragAndDrop method to work in IE11 and IE10. It works well in the latest Firefox and Chrome and IE9.

The code:

dragAndDrop: function (elem, x, y) {
  var i      = 0,
      parent = elem.elementByXPath('..'),
      offsetX,
      offsetY,
      pos,
      offsetPos,
      size;

  offsetPos = this.getLocationInView(elem);
  pos = this.getLocationInView(parent);
  size = this.getSize(elem);

  offsetX = offsetPos.x - pos.x + ~~(size.width / 2);
  offsetY = offsetPos.y - pos.y + ~~(size.height / 2);

  this.moveTo(parent, offsetX, offsetY);

  this.buttonDown(0);

  if(typeof x !== 'undefined' && x) {
    for(; i <= x; i++) {
      this.moveTo(parent, offsetX + i, offsetY);
    }
  }

  if(typeof y !== 'undefined' && y) {
    i = 0;
    for(; i < y; i++) {
      this.moveTo(parent, offsetX, offsetY + i);
    }
  }

  this.buttonUp(0);
}

The drag and drop script on the site works in IE11 and IE10, just not our test of it. It's both an image crop tool and input sliders that use drag and drop functionality. We feel that it's in the IE browsers the script have the greatest risk of breaking, so it would be nice to get the test to work.

When I take control of the browser in SauceLabs when the test is running the click on the page body will sometimes actually help the dragAndDrop method to work. And that's pretty much the only clue I have right now. I've tried to simulate it using Seleniums click() method and even in executing it in JavaScript.

this.execute('var event=document.createEvent("MouseEvent");
    event.initEvent("click",true,true,window,0,50,120);
    document.body.dispatchEvent(event);');

But without any luck. I don't know if there's a bug in Selenium or what the problem is. The hours I've spent on google have made me none the wiser. Any help would be greatly appreciated.

Update

So far I've tried setting focus to the body with document.body.focus() as suggested. I've tried using different targets for the simulated JavaScript mouse click and the native WebDriver clicks. Trying to replicate what happened when I take control of the VM in SauceLabs. Nothing have worked, I'm not even sure it would solve the problem had it worked.

I should also clarify that this is not the first action on the page. After the initial navigation two input fields are filled and then the test waits for an AJAX call to complete. Then when the proper elements can be found this method is run.

Answer 1

The solution I found to this is problem is to execute the function in JavaScript. It's not a perfect solution, but it works for us.

The script does not work for IE9, and it has caused some assertion errors in Chrome/Firefox. Which is weird but completely besides the point; It works perfectly in IE10 and IE11. Worst case scenario I have to use this solution for IE10/11 and the other in IE9/Firefox/Chrome.

My drag and drop method now looks like this:

dragAndDrop: function (elem, x, y, id, className) {
  var i = 0,
    offsetX,
    offsetY,
    pos, offsetPos,
    size;

  this.sleep(2000);

  offsetPos = this.getLocationInView(elem);
  size = this.getSize(elem);

  pageX = offsetPos.x + ~~(size.width / 2);
  pageY = offsetPos.y + ~~(size.height / 2);

  // No linebreaks
  this.execute('var mousedown=document.createEvent("MouseEvent"),mouseup=document.createEvent("MouseEvent"),elem=document.getElementById("' + id + '"),result=[],elems=elem.getElementsByTagName("*"),k=0,i,interval;for(i in elems){if((" "+elems[i].className+" ").indexOf(" ' + className + ' ")>-1){result.push(elems[i]);}}mousedown.initMouseEvent("mousedown",true,true,window,0,0,0,' + pageX + ',' + pageY +',0,0,0,0,0,null);result[0].dispatchEvent(mousedown);interval=setInterval(function(){k++;iter(k);if(k===' + x + '+1){clearInterval(interval);mouseup.initMouseEvent("mouseup",true,true,window,0,' + pageX + '+k,' + pageY +',' + pageX + '+k,' + pageY +',0,0,0,0,0,null);result[0].dispatchEvent(mouseup);}},100);function iter(y){var mousemove=document.createEvent("MouseEvent");mousemove.initMouseEvent("mousemove",true,true,window,0,0,0,' + pageX + '+y,' + pageY +',0,0,0,0,0,null);result[0].dispatchEvent(mousemove);}');

  // Give the script time to execute and take a nap
  this.sleep(x*100+3000);
}

There can be no linebreaks in the executed JS. Cleaned up it looks like this:

var mousedown = document.createEvent("MouseEvent"),
    mouseup   = document.createEvent("MouseEvent"),
    elem      = document.getElementById(id),
    result    = [],
    elems     = elem.getElementsByTagName("*"),
    k         = 0,
    i, interval;

for(i in elems){
    if((" " + elems[i].className + " ").indexOf(" " + className + " ") > -1){
        result.push(elems[i]);
    }
}

mousedown.initMouseEvent("mousedown", true, true, window, 0, 0, 0, clientX, clientY, 0, 0, 0, 0, 0, null);
result[0].dispatchEvent(mousedown);

interval = setInterval(function(){
    k++;
    iter(k);
    if(k === distance + 1){ 
        clearInterval(interval);
        mouseup.initMouseEvent("mouseup", true, true, window, 0, screenX + k, screenY, clientX + k, clientY, 0, 0, 0, 0, 0, null);
        result[0].dispatchEvent(mouseup);
    }
}, 100);

function iter(y){
    var mousemove = document.createEvent("MouseEvent");
    mousemove.initMouseEvent("mousemove", true, true, window, 0, 0, 0, clientX + y, clientY, 0, 0, 0, 0, 0, null);
    result[0].dispatchEvent(mousemove);
}

What the parameters for initMouseEvent() means you can check https://developer.mozilla.org/en-US/docs/Web/API/event.initMouseEvent

For mouseup I had to set screenX/Y, and I've made the assumption that the client is always the full size of the screen. The code contains a lot of assumptions of what environment it's run in, but it shouldn't be too hard to modify it for yours.

It was @JimEvans comment that ultimately led me to the answer.

Answer 2

Below is an improved version of pstenstrm's solution to account for moving in both X and Y directions from the element.

/*
    pageX
    pageY
    distanceX
    distanceY
    id
    className

    ###############
    The variables above were defined in the closure scope of function
    running the execute function.
*/
var mousedown = document.createEvent("MouseEvent"),
    mouseup   = document.createEvent("MouseEvent"),
    elem      = document.getElementById(id),
    result    = [],
    elems     = elem.getElementsByTagName("*"),
    k         = 0,
    j         = 0,
    i, interval;

for(i in elems){
    if((" " + elems[i].className + " ").indexOf(" " + className + " ") > -1){
        result.push(elems[i]);
    }
}

mousedown.initMouseEvent("mousedown", true, true, window, 0, 0, 0, pageX, pageY, 0, 0, 0, 0, 0, null);
result[0].dispatchEvent(mousedown);

interval = setInterval(function(){
    if(k !== distanceX){k++;}
    if(j !== distanceY){j++;}
    iter(k,j);
    if(k === distanceX + 1 && j === distanceY + 1){
        clearInterval(interval);
        mouseup.initMouseEvent("mouseup", true, true, window, 0, pageX + k, pageY + j, pageX + k, pageY + j, 0, 0, 0, 0, 0, null);
        result[0].dispatchEvent(mouseup);
    }
}, 100);

function iter(_x,_y){
    var mousemove = document.createEvent("MouseEvent");
    mousemove.initMouseEvent("mousemove", true, true, window, 0, 0, 0, pageX + _x, pageY + _y, 0, 0, 0, 0, 0, null);
    result[0].dispatchEvent(mousemove);
}

The new dragAndDrop would look like:

dragAndDrop: function (elem, x, y, id, className) {
    var i = 0,
        offsetX,
        offsetY,
        pos, offsetPos,
        size;

    this.sleep(2000);

    offsetPos = this.getLocationInView(elem);
    size = this.getSize(elem);

    pageX = offsetPos.x + ~~(size.width / 2);
    pageY = offsetPos.y + ~~(size.height / 2);

    // No linebreaks
    this.execute('var mousedown=document.createEvent("MouseEvent"),mouseup=document.createEvent("MouseEvent"),elem=document.getElementById("' + id + '"),result=[],elems=elem.getElementsByTagName("*"),k=0,j=0,i,interval;for(i in elems){if((" "+elems[i].className+" ").indexOf(" ' + className + ' ")>-1){result.push(elems[i]);}}mousedown.initMouseEvent("mousedown",true,true,window,0,0,0,' + pageX + ',' + pageY +',0,0,0,0,0,null);result[0].dispatchEvent(mousedown);interval=setInterval(function(){if(k!==' + x + '){k++;};if(j!==' + y +'){j++;};iter(k,j);if(k===' + x + '+1&&j===' + y + '+1){clearInterval(interval);mouseup.initMouseEvent("mouseup",true,true,window,0,' + pageX + '+k,' + pageY +'+j,' + pageX + '+k,' + pageY +'+j,0,0,0,0,0,null);result[0].dispatchEvent(mouseup);}},100);function iter(_x,_y){var mousemove=document.createEvent("MouseEvent");mousemove.initMouseEvent("mousemove",true,true,window,0,0,0,' + pageX + '+_x,' + pageY + '+_y,0,0,0,0,0,null);result[0].dispatchEvent(mousemove);}');

    // Give the script time to execute and take a nap
    this.sleep(x*100+3000);
}