In Tensorflow, What kind of neural network should I use?

Question

I am doing Tensorflow tutorial, getting what TF is. But I am confused about what neural network should I use in my work. I am looking at Single Layer Neural Network, CNN, RNN, and LSTM RNN.

-----------------------What I'm going to do is...-----------------------

There is a sensor which measures something and represents the result in 2 boolean ways. Here, they are Blue and Red, like this:

the sensor gives result values every 5minutes. If we pile up the values for each color, we can see some patterns:

number inside each circle represents the sequence of result values given from sensor. (for example, 107 was given right after 106) when you see from 122 to 138, you can see decalcomanie-like pattern.

I want to predict the next result value, before the sensor imparts the result, with probability. Machine has to know what the next will be, based on patterns from past results.

I may do supervised learning using past results. But I'm not sure which neural network or method is suitable. Thinking that this work needs pattern using past results (have to see context), and memorize past results, maybe LSTM RNN (long-short term memory recurrent neural network) would be suitable one.

Could you tell me which one is suitable for this work?

Answer 1

Sure, you can use an RNN. I would create two features for the past $k$ run lengths, as well as the length of the current run; e.g., just before t=150, the current run would be length 2 (red), and previous three runs would be (1,1,1) for red and (1,1,5) for blue. The intuition here is that the run lengths seem to follow some sort of exponential distribution, and you want to help the model estimate the scale parameter by feeding it samples of the length. You could additionally encode the past k events as a bit string, with 1 representing red and 0 representing blue. You have a classification problem, so you should use classification loss like the cross-entropy, and a softmax output layer to get your probabilities.