patternMinor
What restrictions apply to query and target vector encoding on fast-forward neural networks?
Viewed 0 times
fastrestrictionsforwardtargetwhatneuralapplyquerynetworksencoding
Problem
I'm currently studying fast-forward multi-layer neural networks with back propagation, in the book I see that all query and target vectors are binary-encoded, this makes me believe that this is the only allowed encoding, yet all other neurons in the hidden layers and output layer can take on any real value between 0 and 1.
Now I am creating my own neural network and I want to classify an image by using the RGB values of every pixel as input neurons in some way.
So now my question is, as the title states, what restrictions apply to the encoding pattern used for query and target vectors?
Now I am creating my own neural network and I want to classify an image by using the RGB values of every pixel as input neurons in some way.
So now my question is, as the title states, what restrictions apply to the encoding pattern used for query and target vectors?
Solution
The inputs are not restricted to be 0 or 1. They can be arbitrary real numbers. In practice we often normalize or standardize the inputs so that they are in the range $[-1,+1]$ (or most of the input values are in that range), but even that is not required, and even then, they can be any real number in that range.
The outputs are also real numbers, and they can be any real number: not limited to 0 or 1, and not limited to a particular range. (In some applications we interpret the outputs using the softmax function -- i.e., we postprocess the outputs of the neural network using another function -- which rescales them to get one likelihood value per output that's in the range $[0,1]$, but this is not mandatory and will depend on the specific application setting.)
The outputs are also real numbers, and they can be any real number: not limited to 0 or 1, and not limited to a particular range. (In some applications we interpret the outputs using the softmax function -- i.e., we postprocess the outputs of the neural network using another function -- which rescales them to get one likelihood value per output that's in the range $[0,1]$, but this is not mandatory and will depend on the specific application setting.)
Context
StackExchange Computer Science Q#51887, answer score: 2
Revisions (0)
No revisions yet.