Estimate Q-Table online with a neural network

Question

When i use Q-Table for save state-action in reinforcement learning, some state never (or rarely) happen and state-action value remain zero until max-iteration so i decide to estimate Q-Table online with a neural network instead of using Q-Table.

Which type of neural networks can estimate this kind of problem more accurate and does this solution help me?

Answer 1

i use Localy Weighted Regression(LWR) as function approximation and simply replace table with this function