HiveBrain v1.2.0
Get Started
← Back to all entries
snippetMinor

How do RNN's handle providing output with different dimension than input

Submitted by: @import:stackexchange-cs··
0
Viewed 0 times
handlewiththanprovidingdimensionoutputinputdifferenthowrnn

Problem

It seems like an RNN has to have the ht-1 needs to be the same size as the input vector since they're being added to one another, but if you're doing something like modeling to another language or classification, how would you handle this? I'd imagine you'd just have a fully connected layer that would predict the output, but I was wondering if there is a standard convention?

Solution

So Basically what happens is the input x is multiplied by a hidden layer U to yield a state vector s, which transforms it to a new dimensionality. Then The recurrent layer W is always a square matrix to maintain the dimensions of s. Lastly, the output matrix V is multiplied to update the matrix to the output dimensionality.

So if you had a vector that was 5 dimensions, and you wanted to multiply it into a state vector of 20 dimensions and an output vector of 2 dimensions, your dimensions would be:

x is 5x1
s is 20x1
y is 2x1

U is 20x5
W is 20x20
V is 2x20


and your formula is:

s = Ux + W*s_(t-1)
y = Vs

Code Snippets

x is 5x1
s is 20x1
y is 2x1

U is 20x5
W is 20x20
V is 2x20
s = Ux + W*s_(t-1)
y = Vs

Context

StackExchange Computer Science Q#54607, answer score: 2

Revisions (0)

No revisions yet.