patternMinor
What is the State of The Art of Writer AIs (Deep Learning)?
Viewed 0 times
aisthewhatwriterlearningartdeepstate
Problem
Does anyone know if Deep Learning Bots can already, for example, train
on many books of an author and output a similar but new book?
I've been wanting to get into ML for quite a while but was lacking a project to use as an incentive to keep learning, and this specific AI has caught my interest. It seems feasible from the little I know of ML so far (I'm a beginner who has taken the initial module of
Another relevant question: is this too difficult for a beginner? I've graduated from engineering 1 year ago, so I have some ease with programming, but I don't know how difficult Recurrent Neural Networks (RNN) and Natural Language Programming (NLP) can be.
Also, on a sidenote, does a more experienced programmer have a
suggestion of path I should take to learn the necessary skills to
program such a bot, i.e., online courses and books?
on many books of an author and output a similar but new book?
I've been wanting to get into ML for quite a while but was lacking a project to use as an incentive to keep learning, and this specific AI has caught my interest. It seems feasible from the little I know of ML so far (I'm a beginner who has taken the initial module of
Deeplearining.ai course from Coursera), but so far from what I can scoop out of the internet it doesn't seem like AIs are quite there yet. They seem to be somewhat convincing, but sometimes a weird outlier appears in the text; strangely, music composition seems more convincing. Does anyone disagree?Another relevant question: is this too difficult for a beginner? I've graduated from engineering 1 year ago, so I have some ease with programming, but I don't know how difficult Recurrent Neural Networks (RNN) and Natural Language Programming (NLP) can be.
Also, on a sidenote, does a more experienced programmer have a
suggestion of path I should take to learn the necessary skills to
program such a bot, i.e., online courses and books?
Solution
Researchers are trying it out, but RNNs learn character-by-character (sequences of characters), so it is difficult to get something that resembles a story plot, as a whole. This link by a Stanford researcher explains the current (2015) state of the art. Here is the code and here are some data sets for you to start experimenting with.
Another experiment involving the writing of a travel blog also concluded that you cannot really create a long passage that makes sense at the moment. The recommendation was to look at the word level, rather than character, and focus on something more manageable, such as sentence autocompletion.
When it comes to words, the more unique ones there are in the source data set (for example using the Game of Thrones books), the tougher it becomes to train a good model. Suggestions are to limit input to more basic words (think children's vocabulary) and to have at least a total training sample at least 100 times larger than the desired output.
A trained neural network could perhaps output smaller texts (<10000 words) that make some sense, if they are of a rather structured nature. A whole book needs both a coherent plot from start to finish, as well as twists in-between. As a result, it is still way too difficult.
This guy tried it out and discovered that the AI would get sometimes stuck in loops. Furthermore, the draft produced did not always make sense in terms of the storyline and required heavy human editing.
Here is another example of an LSTM RNN outputting a Harry Potter chapter, where the sentences are grammatically correct, but sometimes make no sense.
If you want to learn more about RNNs, the Deep Learning book by Goodfellow, Bengio, and Courville comes highly recommended and has a relevant chapter.
For something more specific to writing, The Bestseller Code book uses text mining techniques and should be an interesting read.
Another experiment involving the writing of a travel blog also concluded that you cannot really create a long passage that makes sense at the moment. The recommendation was to look at the word level, rather than character, and focus on something more manageable, such as sentence autocompletion.
When it comes to words, the more unique ones there are in the source data set (for example using the Game of Thrones books), the tougher it becomes to train a good model. Suggestions are to limit input to more basic words (think children's vocabulary) and to have at least a total training sample at least 100 times larger than the desired output.
A trained neural network could perhaps output smaller texts (<10000 words) that make some sense, if they are of a rather structured nature. A whole book needs both a coherent plot from start to finish, as well as twists in-between. As a result, it is still way too difficult.
This guy tried it out and discovered that the AI would get sometimes stuck in loops. Furthermore, the draft produced did not always make sense in terms of the storyline and required heavy human editing.
Here is another example of an LSTM RNN outputting a Harry Potter chapter, where the sentences are grammatically correct, but sometimes make no sense.
If you want to learn more about RNNs, the Deep Learning book by Goodfellow, Bengio, and Courville comes highly recommended and has a relevant chapter.
For something more specific to writing, The Bestseller Code book uses text mining techniques and should be an interesting read.
Context
StackExchange Computer Science Q#95679, answer score: 5
Revisions (0)
No revisions yet.