Good implementations of reinforcement learning?

Question

For an ai-class project I need to implement a reinforcement learning algorithm which beats a simple game of tetris. The game is written in Java and we have the source code. I know the basics of reinforcement learning theory but was wondering if anyone in the SO community had hands on experience with this type of thing.

1. What would your recommended readings be for an implementation of reinforced learning in a tetris game?
2. Are there any good open source projects that accomplish similar things that would be worth checking out?

Edit: The more specific the better, but general resources about the subject are welcomed.

Follow up:

Thought it would be nice if I posted a followup.

Here's the solution (code and writeup) I ended up with for any future students :).

Paper / Code

Answer 1

Take a look at the 2009 RL-competition. One of the problem domains is a tetris game. There was a tetris problem the year before too. Here’s the 52-page final report from that year’s fifth-place finalist, which goes into a lot of detail about how the agent worked.

Answer 2

The Heaton Research ebook is quite good at explaining neural network concepts (with code). Chapter 4 is dedicated to machine learning and the various training methods for your networks. There is a downloadable library and sample applications for you to look at.

Answer 3

Here is a good book on the subject:

Machine Learning and Data Mining: Introduction to Principles and Algorithms
by Igor Kononenko, Matjaz Kukar (June, 2007)

Also take a look at these open source projects:

• Maja Machine Learning Framework
• PIQLE
• Elsy
• The Robotics Primer Workbook

Answer 4

TD-Gammon, gnubackgammon, or any other similar project were massive successes in games.

Sutton & Barto's book "Reinforcement Learning: An Introduction" also has some other Case Studies.

Answer 5

This is not specific to reinforced learning, but Stanford has a great series of lectures on machine learning on Youtube and iTunes.

The link is to the first lecture which takes approximately 30 minutes to dive into the content.

Answer 6

Burlap is a recent Java library that provides implementations of many common reinforcement learning algorithms as well as a few environments and useful tools.

Answer 7

This question is really old, but for anyone reading this in 2018, I highly recommend you use OpenAI Baselines if you're interested in solid references of existing RL algorithms. These algorithms are implemented by a group of employees at OpenAI who really know this stuff, and have been extensively fine-tuned and debugged.

To be fair, you don't need these for Tetris, but nowadays I suspect homework questions may involve some more sophisticated environments.

https://github.com/openai/baselines

Answer 8

I would suggest to learn RL4J which is java based. I was using this and I was amazed how things works smoothly and you can learn even LSTM networks in a reinforcement learning algorithm with Actor Critic algorithm (called A3C)

Here is the link: https://github.com/deeplearning4j/dl4j-examples/blob/master/rl4j-examples/

Answer 9

I noticed that this question is quite outdated (10 years old) and that a collection of modern RL frameworks and environments could be useful here. I created GitHub repo for this and intend to update it regularly.

https://github.com/TheMTank/RL-code-resources