Machine Learning: Writing Poems

Question

I'm a student of machine learning, and these days I was trying to learn how to use the TensorFlow library. I've gone through various tutorials and trial&errors with tensorflow, and I thought the best way to learn it for real would be to make use of it in a little project of my own.

I've decided that I should attempt to make a program that writes poems. I'm not aiming for top-end quality program; for my first model, I'd be happy with just a string of non-sense words groups together in poem format. The problem is that I'm having problem looking up books or videos about machine learning programs that deal with writing sentence structures.

Can you make any suggestions on what I could look for (even google keywords are fine) to get the sample programs and basic knowledge that I need?

Thank you.

Answer 1

This is just the comment from Emre expanded, but yes you should look into recurrent neural networks for generating text in the style of a given corpus. RNNs and LSTM work really quite well for this.

The writeup at http://karpathy.github.io/2015/05/21/rnn-effectiveness/ is widely cited, and to your question, shows how it's pretty easy to generate something like this, given the text of Shakespeare's plays:

PANDARUS: Alas, I think he shall be come approached and the day When little srain would be attain'd into being never fed, And who is but a chain and subjects of his death, I should not sleep.

Second Senator: They are away this miseries, produced upon my soul, Breaking and strongly should be buried, when I perish The earth and thoughts of many states.

If you follow https://github.com/jcjohnson/torch-rnn you can easily run this on your local GPU too to generate text from whatever input poetry you like. I have had pretty good results with 0.1-0.5 dropout, 2 layers, layers of size 512-1024.