How does a recurrent connection in a neural network work?

I am reading a very interesting paper on genetic algorithms which define neural networks. I am familiar with how a feedforward neural network operates, but then I came across this:

Where node #4 goes back to connect to #5. I was wondering how this is handled? Does the state of node 4 get kept from the last timestep and applied to node 5 when it is time to calculate its activation?

When a recurrent network is calculated you can imagine the network is 'unrolled' out through time.

When visualized this way, you can see activation and loss can be calculated using the same method as a typical network.

So to clarify using the example you provided. Yes, the input to node #5 is the output from node #1, node #2, and the previous time steps node #4.

It is important to note; Typically the input and output to RNN's are expected to be time dependent. That is, for each time step t there is input t and output t. This makes RNN especially good at things like evaluating sequences. Although it is possible to keep the input and output equal throughout time.

StackExchange Computer Science Q#56805, answer score: 2