Hidden Markov Models instead of FSM in a first person shooter game

Question

I have been working on a course project in which we implemented an FPS using FSMs, by showing a top 2d view of the game, and using the bots and players and circles. The behaviour of bots was deterministic. For example, if the bot's health drops to below a threshhold, and the player is visible, the bot flees, else it looks for health packs.

However, I felt that in this case the bot isn't showing much of intelligence, as most of the decisions it takes are based on rules already decided by us.

What other techniques could I use, which would help me implement some real intelligence in the bot? I've been looking at HMMs, and I feel that they might help in bringing more uncertainty in the bot, and the bot might start being more autonomous in taking decisions than depending on pre defined rules.

What do you guys think? Any advice would be appreciated.

Answer 1

I don't think using a hidden Markov model would really be more autonomous. It would just be following the more opaque rules of the model rather than the explicit rules of the state machine. It's still deterministic. The only uncertainty they bring is to the observer, who doesn't have a simple ruleset to base predictions on.

That's not to say they can't be used effectively - if I recall correctly, several bots for FPS games used this sort of system to learn from players and develop their own AI.

But this does depend exactly on what you want to model with the process. AI is not really about algorithms, but about representation. If all you do is pick the same states that your current FSM has and observe an existing player's transitions, you're not likely to get a better system than having an expert input carefully tweaked rules for an FSM.

Given that you're not going to manage to implement "some real intelligence" as that is currently considered beyond modern science, what is it you want to be able to create? Is it a system that learns from its own experiments? A system that learns by observing human subjects? One that deliberately introduces unusual choices in order to make it harder for an opponent to predict?