Does reinforcement learning always work on grid world?

Question

Does reinforcement learning always need a grid world problem to be applied to?

Can anyone give me any other example of how reinforcement learning can be applied to something which does not have a grid world scenario?

Answer 1

The short answer is no! Reinforcement Learning is not limited to discrete spaces. But most of the introductory literature does deal with discrete spaces.

As you might know by now that there are three important components in any Reinforcement Learning problem: Rewards, States and Actions. The first is a scalar quantity and theoretically the latter two can either be discrete or continuous. The convergence proofs and analyses of the various algorithms are easier to understand for the discrete case and also the corresponding algorithms are easier to code. That is one of the reasons, most introductory material focuses on them.

Having said that, it should be interesting to note that the early research on Reinforcement Learning actually focussed on continuous state representations. It was only in the the 90s since the literature started representing all the standard algorithms for discrete spaces as we had a lot of proofs for them.

Finally, if you noticed carefully, I said continuous states only. Mapping continuous states and continuous actions is hard. Nevertheless, we do have some solutions for now. But it is an active area of Research in RL.

This paper by Sutton from '98 should be a good start for your exploration!

Answer 2

Reinforcement learning does not depend on a grid world. It can be applied to any space of possibilities where there is a "fitness function" that maps between points in the space to a fitness metric.

Topological spaces have a formally-defined "neighborhoods" but do not necessarily conform to a grid or any dimensional representation. In a topological space, the only way to get from "here" to "there" is via some "paths" which are sets of contiguous neighborhoods. Continuous fitness functions can be defined over topological spaces.

For what it is worth, reinforcement learning is not the be-all-end-all (family of) learning algorithms in fitness landscapes. In a sufficiently rugged fitness landscape, other learning algorithms can perform better. Also, if there are regions of the space where there are no well-defined fitness function at a given point in time, it may be indeterminate as to what learning algorithms are optimal, if any.