Back propagation neural network

This is the first time I tried to write a back propagation ANN and I would like to know what more experienced people think of it. The code is meant to distinguish if text is written in English, French or Dutch.

I know my training set isn't very diverse but I just got the data from about 30 different texts, each containing 250 words in one of the 3 languages, so that's not my fault. I also know there are easier ways to do that but I wanted to learn something about ANNs.

I'd be glad if any of you would be kind enough to give me his thoughts on how I did this and how I could improve it.

```
import math, time, random, winsound
global Usefull
LearningRate = 0.001
InWeight = [[],[],[],[],[],[]]
#Generate random InWeights
for i in range(6):
for j in range(21):
InWeight[i].append(random.uniform(0,1))
#21 Input Values
InNeuron = [0,0,0,0,0,0,0,
0,0,0,0,0,0,0,
0,0,0,0,0,0,0]
#6 Hidden Neurons
HiddenLayer = [0, 0, 0, 0, 0, 0]
#Used to calculate Delta
HiddenLayerNoSigmoid = [0, 0, 0, 0, 0, 0]
HiddenWeight = [[],[],[]]
#Generate random HiddenWeights
for i in range(3):
for j in range(6):
HiddenWeight[i].append(random.uniform(0,1))
#3 Output Neurons
OutNeuron = [0, 0, 0]
#Used to calculate Delta
OutNeuronNoSigmoid = [0, 0, 0]
#Learning Table
#Engels - Nederlands - Frans - Desired output
test = [[11, 4, 8, 1, 14, 8, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0]]
test += [[4, 0, 6, 0, 4, 6, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0]]
test += [[6, 0, 6, 0, 11, 8, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0]]
test += [[23, 0, 0, 0, 13, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0]]
test += [[18, 4, 4, 2, 14, 8, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0]]
test += [[14, 1, 6, 0, 10, 7, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0]]
test += [[19, 0, 2, 0, 18, 4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0]]
test += [[13, 1, 1, 1, 15, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0

import math, time, random, winsound
global Usefull

A global statement outside of a function has no effect

LearningRate = 0.001

The python style guide recommends that global constants be in ALL_CAPS

InWeight = [[],[],[],[],[],[]]

The python style guide for local variable names is lower_case_with_underscores

#Generate random InWeights
for i in range(6):
    for j in range(21):
        InWeight[i].append(random.uniform(0,1))

Logic like this should always be in a function. Your main level should be restricted to defining functions/classes. Also you might want to consider looking into using numpy. With it you can do the above as InWeight = numpy.random.random(6, 21)

#21 Input Values
InNeuron = [0,0,0,0,0,0,0,
            0,0,0,0,0,0,0,
            0,0,0,0,0,0,0]

You can use InNeuron = [0] * 21. Speaking of this, the numbers 21 and 6 are showing up in multiple places. You should make them a global constant so you can change them from one place.

#6 Hidden Neurons
HiddenLayer = [0, 0, 0, 0, 0, 0]
#Used to calculate Delta 
HiddenLayerNoSigmoid = [0, 0, 0, 0, 0, 0]
HiddenWeight = [[],[],[]]
#Generate random HiddenWeights
for i in range(3):
    for j in range(6):
        HiddenWeight[i].append(random.uniform(0,1))
#3 Output Neurons
OutNeuron = [0, 0, 0]
#Used to calculate Delta
OutNeuronNoSigmoid = [0, 0, 0]

As before, no need to type out the entire table of zeros.

#Learning Table
#Engels - Nederlands - Frans - Desired output
test = [[11, 4, 8, 1, 14, 8, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0]]
test += [[4, 0, 6, 0, 4, 6, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0]]
test += [[6, 0, 6, 0, 11, 8, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0]]
test += [[23, 0, 0, 0, 13, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0]]
test += [[18, 4, 4, 2, 14, 8, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0]]
test += [[14, 1, 6, 0, 10, 7, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0]]
test += [[19, 0, 2, 0, 18, 4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0]]
test += [[13, 1, 1, 1, 15, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0]]
test += [[19, 3, 1, 0, 14, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0]]
test += [[0, 0, 0, 0, 0, 0, 0, 2, 0, 5, 6, 1, 1, 8, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0]]
test += [[0, 0, 0, 0, 0, 0, 0, 3, 0, 7, 1, 3, 3, 3, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0]]
test += [[0, 0, 0, 0, 0, 0, 0, 1, 0, 12, 7, 8, 2, 3, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0]]
test += [[0, 0, 0, 0, 0, 0, 0, 4, 0, 5, 4, 4, 2, 3, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0]]
test += [[0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 5, 1, 13, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0]]
test += [[0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 7, 0, 7, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0]]
test += [[0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 7, 0, 13, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0]]
test += [[0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 14, 1, 8, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0]]
test += [[0, 0, 0, 0, 0, 0, 0, 4, 0, 2, 4, 9, 4, 3, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0]]
test += [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 5, 0, 3, 0, 6, 0, 8, 0, 0, 1]]
test += [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, 2, 7, 0, 1, 0, 0, 0, 0, 1]]
test += [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 0, 1, 0, 2, 0, 1, 0, 0, 1]]
test += [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 0, 5, 2, 2, 0, 0, 0, 0, 1]]
test += [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 7, 0, 2, 0, 2, 0, 0, 1]]
test += [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, 0, 7, 1, 1, 2, 3, 0, 0, 1]]
test += [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 8, 0, 2, 0, 2, 0, 0, 1]]
test += [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 4, 0, 3, 1, 3, 0, 0, 1]]
test += [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, 1, 5, 1, 2, 0, 0, 0, 0, 1]]

Why do you use += many time instead of simply creating the list in one go? For this much data I'd also probably store it in an external file.

def Sigmoid(Value):
    return math.tanh(Value)

def DSigmoid(Value):  
    return 1.0 - Value**2

The python style guide recommends lowercase_with_underscores for function names and the variables in them.

def UpdateHiddenNode():
    global InNeuron, InWeight

Avoid global variables. Stuff like this should really be in a class where InNeuron and InWeight would be attributes.

for i in range(6):
        e = 0

e is a cryptic variable name, considering expanding it

for j in range(21):
            e += InWeight[i][j]*InNeuron[j]
        HiddenLayerNoSigmoid = e
        HiddenLayer[i] = Sigmoid(e)

This entire function would probably be one line of code if you were using numpy. It would also run faster.

def UpdateOutNeuron():
    global HiddenLayer, HiddenWeight
    for i in range(3):
        e = 0
        for j in range(3):
            e += HiddenWeight[i][j]*HiddenLayer[j]
        OutNeuron[i] = Sigmoid(e)

These last two function seem to be doing basically the same thing. Can you combine them?

```
def UpdateDelta():
gl

StackExchange Code Review Q#6126, answer score: 6