Neural Network appears to be slow

I have some neural network Julia code which I'm hoping to speed up. It's possibly simply poorly-designed, but I'm not convinced.

```
# This is line 1 (for the purpose of matching the profiler output's line numbers)

sigmoid(z::Float64) = 1/(1 + exp(-z))
sigmoidPrime(z::Float64) = sigmoid(z) * (1 - sigmoid(z))

### Types ###

abstract AbstractNode

type Edge
source::AbstractNode
target::AbstractNode
weight::Float64
derivative::Float64
augmented::Bool

Edge(source::AbstractNode, target::AbstractNode) = new(source, target, randn(), 0.0, false)
end

type Node -0.5
return obj.activation
else
weightedSum = sum([d.weight * evaluate(d.source, inputVector) for d in obj.incomingEdges])
obj.activation = sigmoid(weightedSum)
obj.activationPrime = sigmoidPrime(weightedSum)

return obj.activation
end
end

function evaluate(obj::InputNode, inputVector::Array)
obj.activation = inputVector[obj.index]
return obj.activation
end

function evaluate(obj::BiasNode, inputVector::Array)
obj.activation = 1.0
return obj.activation
end

function updateWeights(obj::AbstractNode, learningRate::Float64)
for d in obj.incomingEdges
if d.augmented == false
d.augmented = true
d.weight -= learningRate * d.derivative
updateWeights(d.source, learningRate)
d.derivative = 0.0
end
end
end

function compute(obj::Network, inputVector::Array)
output = [evaluate(node, inputVector) for node in obj.outputNodes]
for node in obj.outputNodes
clear(node)
end
return output
end

function clear(obj::AbstractNode)
for d in obj.incomingEdges
obj.activation = -1.0
obj.activationPrime = -1.0
if d.augmented == true
d.augmented = false
clear(d.source)
end
end
end

function propagateDerivatives(obj::AbstractNode, error::Float64)
for d in obj.incomingEdges
if d.augment

Alright so I've done a lot of experimenting and the problem seems to be accessing data internal to a type via another type. Specifically accessing d.source.activation appears to be about 100 times more costly than just accessing d.augmented. If you replace d.source.activation with 1.0 you'll more than double the speed.

It's also possible this is because the type info provided to the compiler concerning source is an abstract type AbstractNode, rather than a concrete type. But I know of no way around this.

Either way this is as far as I can tell a serious performance issue which I don't immediately see how to circumvent.

However the other two suspicious lines, upon closer consideration, seem actually to be about right, so it's just line 144 which was causing the slow down.

Moreover by getting rid of the syntactic sugar += in line 144, one can shave about 10 percent off the time, so there is an issue with that as well.

StackExchange Code Review Q#60099, answer score: 6