Layered Neural Network in Swift

I am designing a very basic layered neural network in Swift as an exercise. I currently got the network evaluating the response for a given stimulus by propagating the stimulus forwards through the layers and collecting the response at the output layer, which is assumed to be a single neuron.

My next step will be to provide back propagation, however I don't believe that the remaining coding will introduce many new linguistic constructs, so now is a good time to evaluate my coding style.

I have a network containing a list of layers, and each layer contains a list of neurons. I've made Net and Layer classes, so as to allow layers to form a doubly linked list, which should ease traversal. I've made Neuron a struct, as I learned each class instance requires its own individual heap allocation.

There are two places I think I can make an improvement, marked by ???1 and ???2. However I can't figure out how to implement.

Also, maybe zooming out there is some improvement I could make to the overall network design.

Working code is on SwiftStub.

```
func random_01() -> Double {
return Double(arc4random()) / Double(UInt32.max)
}

struct Neuron {
var weights = [Double] ()
var bias = random_01()

unowned let net : Net
unowned let layer : Layer
let index : Int

var sum = 0.0

init( net:Net, layer:Layer, index:Int )
{
(self.net, self.layer, self.index) = (net, layer, index)

guard let prev = layer.prev else { return }

weights = prev.neurons.map { _ in random_01() }
/* Could also do:
for _ in 0 .. Double in return random_01() }
weights = (0..Double { return 1.0 / ( 1.0 + exp(-x) ) }

output = sigmoid(sum)
}
}

final class Layer
{
unowned let net: Net

weak var prev: Layer? = nil
weak var next: Layer? = nil

let index: Int

var neurons = [Neuron] ()

init( net:Net, index:Int )
{
(self.net, self.index) = (net, index

assert(false, "Don't use assert like this!")

func input( x: [Double] ) -> Double
{
    guard let  inputLayer = layers.first else { return -2.0 }
    guard let outputLayer = layers.last  else { return -3.0 }

    assert(  inputLayer.neurons.count == x.count
         && outputLayer.neurons.count == 1  )

The logic for this method is all over the place.

First of all, you will never return -3.0 from this method. If last were going to return nil, then first already returned nil. For an array with just one thing in it, first and last return the same thing.

I don't understand (because it's not documented) why we're somehow okay returning some magic numbers if our layers array is empty, but if the last layer has more than 1 neuron or if the user has not guessed the number of neurons in the first array and sent in a properly sized array, we're throwing an assertion!?!?

Asserts should be reserved for things we can guarantee at compile time. Moreover, they should be reserved for things that we've already tested for and (hopefully) guaranteed to be true.

The exactly correct way to handle this (there are many options) depends on a great number of things (the behavior should be documented either way) (I can't make recommendations without more information), but in no way is this assert going to be correct, given that your code makes no effort to first verify this to be true.

Moreover, when you do use assert, you should endeavor to include a message with it explaining the assertion failure so that any user has some clue where to begin in resolving the issue.

Spacing & Formatting

Things like this can make your code easier to read:

var weights = [Double] ()
var bias    = random_01()

unowned let net   : Net
unowned let layer : Layer
        let index : Int

But it quickly becomes difficult to maintain. Each time we add a new variable, if its name is longer than any of the existing variables, we have to go increase the spacing for all of the other declarations. And no IDE that I know of (certainly not Swift) is going to put any effort into helping you align these quite the way you want.

Ultimately, the most appropriate thing to do to make these variable declarations more readable (without hampering maintainability drastically like this) is to set your IDE up with good syntax coloring. With most syntax coloring schemes, nothing else will be the same color as the variable name (even Stack Exchange catches everything but unowned and your function name).

The extra spaces you've scattered throughout your code don't do anything to improve the readability. If anything, for me, they hurt it, as they are distracting. Instead of thinking about the code, I am stuck thinking "Why are these spaces randomly scattered throughout?"

Here are some examples:

var weights = [Double] ()

Should just be:

var weights = [Double]()

init( net:Net, layer:Layer, index:Int )

Should be spaced more like this:

init(net: Net, layer: Layer, index: Int)

The space between the : and the type is optional (I prefer it personally), but the space after the , should definitely be there, and the spaces for the parenthesis definitely should not (it's distracting).

layers.append( Layer(net:self,  index:i) )

These extra spaces are unnecessary. This should just be:

layers.append(Layer(net:self, index:i))

If the double parenthesis are confusing at the end, the right way to fix that is by breaking this onto multiple lines:

let layer = Layer(net:self, index:i)
layers.append(layer)

guard let  inputLayer = layers.first else { return -2.0 }
guard let outputLayer = layers.last  else { return -3.0 }

The spacing here is all over the place, and it's going to suffer the same problem as the first section I talked about. Adding another line in this group or making other sorts of modifications means we have to go fix up spacing of lots of lines potentially. I don't mind keeping the else part all on the same line since it's so simple, but let's not play this game with the spaces:

guard let inputLayer = layers.first else { return -2.0 }
guard let outputLayer = layers.last else { return -3.0 }

StackExchange Code Review Q#95224, answer score: 14