When ART is enabled, the device cannot ensure the compiled code matches the bytecode instructions. It therefore cannot step through instructions and therefore lines.
This forces the device to fallback to a very slow interpreter, which possibly doesn't even JIT-compile.
For faster debugging, switch back to Dalvik.