How can a program control another program?

Question

Bots, how do they work? Do they tell the video game a key was pressed or the mouse was clicked?

If not is there a way to have your program tell another program a key was pressed? I would like to make a program to beat some game. So any resources or examples are appreciated.

Update: So one way is to emulate keystrokes, so what are some methods to do this (in any language)?

Answer 1

I've written a bunch of bots at one time or another (from Pogo games to Yohoho Puzzle Pirates). For windows, you're usually going to either be sending Win32 events to simulate mouse movements, or spoof the actually low-level messages sent between windows when the mouse is actually clicked. A lot of it really depends on how the program reacts (by accepting the message with the coordinates, or, in Java's case, immediately reading the mouse coordinates). The "automation" part usually involves reading the screen and writing heuristics or algorithms for determining the state, but can also be as nice as packet sniffing (a lot of information there in poor poker implementations) or as hacky as reading memory locations directly. Pretty big "field", and poorly documented as it's pretty profitable and not hard to get into.

Sending Input

C/C++ (in Windows)

For keys, try CodeProject:

http://www.codeproject.com/KB/cpp/sendkeys_cpp_Article.aspx

And messages:

http://www.codeproject.com/KB/threads/sendmsg.aspx

Your best bet is to learn to send messages using the Win32 API, then use something like Spy++ or its derivatives to "reverse engineer" how KeyPresses and mouse movements are sent to the window.

Java

Java has an amazingly portable Robot class that is able to:

1. Read Pixels from the screen.
2. Control the mouse.
3. Send keys.

I'd give that a shot if you're looking for quick and easy.

Basic Logic

This is described elsewhere on the internet in depth, but most bots follow a simple state-machine program flow. You read the screen (or packets, or memory), find out what "state" you're in based on your readings and past data, do calculations, and send the result back out to the program.

Reading the screen can be difficult, but can be made easier if you consider that a lot of times, there are a few "lucky" pixels relative to the window that will give you an idea of what state the program is in. The process of finding these pixels can be automated.

Answer 2

Some programs (such as ones that grind for you in an MMORPG) are simply emulating keystrokes. A tool to do this is AutoIt.

As for bots that actually play the games themselves, I've not really done anything with them, but I'm assuming they would use some sort of predefined set of actions, maybe a heuristic, but not likely.

Answer 3

There are a couple of kinds of bots. From what I remember when I used to play CS (a long, long time ago)

Some bots use the libraries of the application itself to inspect the environment (the map, where players are etc...) The bots also use the library to control movement. I doubt there are many bot implementations that invoke mouse messages to tell the game how to move etc...

Some bots emulate the application level protocol (if it's multi-player). So in other words, the game doesn't even need to run. A bot can run in a virtual game.

I'm going to run counter to what one other poster has suggested: writing a game bot is probably a more valuable exercise than actually playing the game, and being curious about how they work is a good sign. :)

Answer 4

The biggest flaw in this concept is not the input, but the game output. Your bot can't just randomly press keys, it needs to know what is "happening". Very few games allow you to query this data which would leave your bot to solve a very tricky problem - Converting 60 frames per second of 2D visual data into a virtual space.

There are actuially games out there that not only allow bot players, but encourage them. Open RTS is one of those but there are also simpler ones like GUN-TACTYX and crobots which are good starting points.

Answer 5

Bots will be acting as clients to the game (server). Then, they can send appropriate data just as a user may be sending manually. The bot(client) will be analysing the data from the server ("someone is about to attack from the left"), et cetera. The client should be then able to compute the best move and send the appropriate data to the server to execute it.

I am not sure if this will work with all games.

Answer 6

It very much depends on the game. For example I made a bot for Travian (which is an online game) that used the internet explorer activeX to automate moves. I worte the application in c# and to get it to do stuff took about 30 minute. It's simple to call methods on the activeX an make it do things like fill in text fields or click on buttons. For login for example i used the folowing :

var inputs = web.Document.GetElementsByTagName("input").
                        Cast<HtmlElement>();
var nume = inputs.First(h => h.GetAttribute("type") == "text");
var pass = inputs.First(h => h.GetAttribute("type") == "password");
var login = inputs.First(h => h.GetAttribute("type") == "image");
var form = web.Document.GetElementsByTagName("form")[0];

nume.SetAttribute("value", "user");
pass.SetAttribute("value", "pass");
login.InvokeMember("Click");

For those games that are desktop based things get more complicated. Windows allows for one aplication to send messages to another application so i guess you could use that to send messages to a game (although wheather this works for games that use directX for input i don't know)

Answer 7

A bot like you describe it will simply emulate key presses and mouse movements/clicks, so the game cannot tell if a human is playing or another program. It would not have to interface with the game at all.

Answer 8

You can emulate mouse movements/clicks and keyboard input in Java with the Robot class. It also lets you take screen shots.

Answer 9

I think AutoIT is a great language for doing this sort of software. Even though I haven't used it to build bots, sending clicks and keys was very easy.

Answer 10

My understanding is that most high-end gaming bots use system calls to map themselves into the game client's memory space, reverse engineer its layout, and directly manipulate the client program's internal variables. That's way more work than I'd be up for just for fun.

Answer 11

When you say "control a program" think about how programs are controlled. Most accept keyboard/mouse input.

If you write a program to simulate keyboard/mouse input, there you go.

A corollary to this would be that if a program accepted arguments passed through an input box, write a program to send arguments as if they were sent through an input box.

This is how all bots work. Hope it sheds some light for you!

Edit: This is how the vast majority of bots work :-)

Answer 12

For this you need to know programming. However, there are programs that allow you to program with visual means. For them, you do not need to be able to program. An example is the program Kibor. It is designed specifically for your purposes and has a visual editor with which you can create bots with block diagrams. Description of the program Kibor

Screenshot Kibor

It has many ready-made functions for bots. There is even a built-in text recognition module Screenshot Kibor. Text recognising