snippetMinor
How do RNN's handle providing output with different dimension than input
Viewed 0 times
handlewiththanprovidingdimensionoutputinputdifferenthowrnn
Problem
It seems like an RNN has to have the ht-1 needs to be the same size as the input vector since they're being added to one another, but if you're doing something like modeling to another language or classification, how would you handle this? I'd imagine you'd just have a fully connected layer that would predict the output, but I was wondering if there is a standard convention?
Solution
So Basically what happens is the input x is multiplied by a hidden layer U to yield a state vector s, which transforms it to a new dimensionality. Then The recurrent layer W is always a square matrix to maintain the dimensions of s. Lastly, the output matrix V is multiplied to update the matrix to the output dimensionality.
So if you had a vector that was 5 dimensions, and you wanted to multiply it into a state vector of 20 dimensions and an output vector of 2 dimensions, your dimensions would be:
and your formula is:
So if you had a vector that was 5 dimensions, and you wanted to multiply it into a state vector of 20 dimensions and an output vector of 2 dimensions, your dimensions would be:
x is 5x1
s is 20x1
y is 2x1
U is 20x5
W is 20x20
V is 2x20and your formula is:
s = Ux + W*s_(t-1)
y = VsCode Snippets
x is 5x1
s is 20x1
y is 2x1
U is 20x5
W is 20x20
V is 2x20s = Ux + W*s_(t-1)
y = VsContext
StackExchange Computer Science Q#54607, answer score: 2
Revisions (0)
No revisions yet.