Finding the max error of a set of training data

I wrote a neural net, and a Trainer class that automates training the net using a training set; a set of input, and the expected output value.

Originally, I was just returning currentError from train to be the highest error so far in the training routine. Obviously though, currentError isn't actually the worst error, and returning it lead to partially trained nets being marked as "trained" due to a single input giving a very good result.

To fix this, I realized that I needed to keep track of the worst error for all of the inputs; the net shouldn't be considered trained just because 1 input has a low error. To achieve this, I included in the class a Map between an input set, and its current error.

The problem with this approach is to find the worst error of all of the input sets, I need to iterate the entire set of values (the errors). I'm only training logic gates right now, so the training sets are very small, but doing a complete iteration of the errors would get very costly with higher numbers of training sets (the training routine currently does ~30,000 training passes per second).

Is there a better way of achieving this?

I'm only going to include the Training class, so it won't be runnable. The entire project is huge (for my standards; 14 classes), so it would be impractical to include it all here. It should also be fairly easy to infer the contents of the Trainer interface.

RandomOrderTrainer.java:

```
import java.util.ArrayList;
import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Random;
import java.util.function.UnaryOperator;

import neuralNet.NeuralNet.BackpropAlgorithm;

public class RandomOrderTrainer
implements Trainer {

final private NeuralNet net;
final private BackpropAlgorithm backProp;
final private Map, List> answerMap = new HashMap, List>();
final private Map, Double> errorsMap = new HashMap, Double>();
final private Random randGen;

Current approach

Right now, you train on randomly selected inputs from the set of all possible inputs and keep track of the highest error per input using a map. There are a couple of things I don't like about this:

• By randomly selecting inputs, you might fail to cover all inputs. Then you might think the net is finished training when it actually might fail to handle an input that was never randomly picked.

• Keeping track of the highest error per input is slow.

New approach - training in rounds

I suggest changing your approach in the following way:

• When you train, you must train in "rounds".

• At the beginning of each round, you shuffle the order of the possible inputs.

• One round consists of training on each input one by one in the shuffled order.

• During training, the highest error is tracked for the round. This is now just a single variable instead of a map.

• At the end of each round, training either stops if the highest error is good enough, or continues to the next round if the error is too high.

StackExchange Code Review Q#98847, answer score: 2