Presented by Zachary C. Lipton

1. Deep Learning
Zachary C. Lipton
zacklipton.com
Critical Review of RNNs:
http://arxiv.org/abs/1506.00019
Conditional Generative RNNS:
http://arxiv.org/abs/1511.03683

2. Computation Revolves
Around Functions

3. Computation Revolves
Around Functions
“Time underlies many
interesting human behaviors”
{0,1}
{A,B,C…}
captions
…

4. Sometimes We Need to Learn
These Functions From Data

5. Feedforward Neural Network

6. Recurrent Neural Networks

7. RNN Generative Model

8. Shakespeare Generation
PANDARUS:
Alas, I think he shall be come approached and the day
When little srain would be attain'd into being never fed,
And who is but a chain and subjects of his death,
I should not sleep.
Second Senator:
They are away this miseries, produced upon my soul,
Breaking and strongly should be buried, when I perish
The earth and thoughts of many states.
DUKE VINCENTIO:
Well, your wit is in the care of side and that.
Second Lord:
They would be ruled after this chamber, and
my fair nues begun out of the fact, to be conveyed,
Whose noble souls I'll have the heart of the wars.
Clown:
Come, sir, I will make did behold your worship.
VIOLA:
I'll drink it.
Training Data:
All of Shakespeare
Size: 4.4MB
Text generated one
character at a time!

9. Past Supervised Approaches relied
upon Encoder-Decoder Model

10. Bridging Long Time Intervals
with Concatenated Inputs

11. Example
A.5 FRUIT/VEGETABLE BEER
<STR>On tap at the brewpub. A nice dark red
color with a nice head that left a lot of lace on the
glass. Aroma is of raspberries and chocolate. Not
much depth to speak of despite consisting of
raspberries. The bourbon is pretty subtle as well. I
really don’t know that I find a flavor this beer tastes
like. I would prefer a little more carbonization to
come through. It’s pretty drinkable, but I wouldn’t
mind if this beer was available. <EOS>

12. Character-based
Classification

13. “Love the Strong Hoppy
Flavor”