Representation of an artificial neuron

I have started to interest myself in artificial intelligence a while ago, especially within the game industry. I have started with state machines moved on to fuzzy logic and now I want to learn and implement a neural network which will also work for games.

However to not overwhelm my tiny little head I wanted to start small. I have found on Wikipedia this article: Künstliches Neuron(Artificial Neuron) and if you hit strg+f you can search for "Darstellung boolescher Funktionen" which is basically what I have implemented.

So my questions regarding the implementation are:

• Is this class usefull for a later neural network (keeping in mind that this class only represents a neuron) so that I than can use this class inside a neuronal network?

• Are there any bad issues which I haven't seen?

One thing I have to mention is that in this situation I didn't want to implement a really good unit testing class or doctests (I hate doctests) thats why my test looks weird.

```
import numpy as np
import math

# From Stackoverflow
def find_nearest(array,value):
idx = (np.abs(array-value)).argmin()
return array[idx]

def hlim(x):
return 0.5 if x == 0 else 0 if x < 0 else 1

class Synapse(object):
def __init__(self, origin, destination, weight = 0):
self.weight = weight
self.origin = origin
self.destination = destination

self.origin.synapse = self
self.destination.synapse = self

class Neuron(object):
def __init__(self, bias, synapse = None):
self.bias = bias # Schwellwert
self.weight = 0
self.synapse = synapse

def addValue(self, value):
self.weight += value
self._output()

def _output(self):
print(1 if hlim(self.weight-self.bias) else "not fired")

class Input(object):
def __init__(self, synapse = None):
self.synapse = synapse

def setInput(self, value):
val = self.synapse.weight * value
self.synapse.destination.addValue(val)

I have started to interest myself in artificial intelligence a while ago, especially within the game industry. I have started with state machines moved on to fuzzy logic and now I want to learn and implement a neural network which will also work for games.

It's great that you're exploring different approaches! While applying state machines or say behavior trees to games is straightforward, applying "neural networks" to games does not make too much sense: imagine you were saying "I'm going to apply Python to games": the language is important, but much less important than what you're going to do with it. However, learning as you're doing here is a good way to move forward, so keep it up! You've gone further than many people here.


However to not overwhelm my tiny little head I wanted to start small.

Great!


Is this class useful for a later neural network (keeping in mind that this class only represents a neuron) so that I than can use this class inside a neuronal network?

Unfortunately, no. Neural networks are not implemented like this in practice. Neurons are not considered individually: the libraries model all the weights of the connections between one layer and the next, because it's much faster to train, and speed is crucial.


Are there any bad issues which I haven't seen?

No, the only issue is that it won't be easy to do more things with the current structure.

import numpy as np
import math

# From Stackoverflow
def find_nearest(array,value):
    idx = (np.abs(array-value)).argmin()
    return array[idx]

Unused code.

def hlim(x):
    return 0.5 if x == 0 else 0 if x < 0 else 1

The threshold activation function is not very good, consider using at least a sigmoid or a ReLU (very easy to implement, works very well in practice). But it will only matter when you will start thinking about backpropagation.

class Synapse(object):
    def __init__(self, origin, destination, weight = 0):
        self.weight = weight
        self.origin = origin
        self.destination = destination

        self.origin.synapse = self
        self.destination.synapse = self

Notice that you never use .origin because you always go from the Input to the Synapse, not the other way around.

So this is nice object-oriented code, but I think this is overly complex: most of the code ties the objects together: you could get something much shorter:

def neuron_score(bias, weight_list, input_list):
    score = 0
    for weight, input in zip(weight_list, input_list):
        score += weight * input
    score -= bias

    return "fired" if hlim(score) > 0 else "not fired"

if __name__ == "__main__":
    print(neuron_score(1.5, [1, 1], [1, 1]))
    print(neuron_score(0.5, [1, 1], [0, 0]))
    print(neuron_score(-0.5, [0], [-1]))

See how this gets down to the essence of it? Of course this can be made much faster by using numpy arrays (and ndarray for more than one more neuron).

So, now, how do you implement XOR? :) Use the Tensorflow playground to try and get intuition about this.

StackExchange Code Review Q#124138, answer score: 2