HiveBrain v1.2.0
Get Started
← Back to all entries
patternMinor

Simple Gradient Descent Algorithm In Haskell

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
simplegradientalgorithmhaskelldescent

Problem

After becoming interested in various bits of functional programming provided by non-functional languages, I've recently decided to start learning Haskell. As I'm fairly experienced with conventional imperative programming languages, I decided to make my "hello world" something a little more complex - an implementation of gradient descent with 2 variables.

The idea with this code is that you fill in the training set for the algorithm in the code, and then run the code something similar to this:

descentFunc = singleDescend trainingSet 0.1
deltas = descend descentFunc ( 100, 1 ) 100000


Where 0.1 is the learning rate (referred to as lr in the code) 100000 is the number of iterations for the loop, and ( 100, 1 ) is an initial guess for the coefficients.

The actual code that's running is below. As I'm entirely new to Haskell, I was wondering whether code such as this is acceptable? And whether there's any obvious idioms I'm missing/misusing or any glaring style errors I've made.

Any comments on my implementation of the algorithm are welcome also.

```
import Data.List

trainingSet = [ (1.0,10.0),(2.0,20.0),(3.0,30.0),(4.0,40.0) ]

hypothesis (d1,d2) x = d1 + (d2 * x)

squareDiff d (x,y) = diff * diff
where diff = ( hypothesis d x ) - y

costFunc ts d = scaleFactor * sum squares
where scaleFactor = 1.0 / (2 * genericLength ts)
squares = map (squareDiff d) ts

diff d (x,y) = (hypothesis d x) - y

descendTheta thetaFunc deltaFunc ts lr deltas =
deltaFunc deltas - (scaleFactor * sum scaledDiffs)
where scaleFactor = lr / genericLength ts
diffs = map (diff deltas) ts
scaledDiffs = map (\(x,y) -> x * thetaFunc y) $ zip diffs ts

descendThetaZero = descendTheta (\_ -> 1) fst
descendThetaOne = descendTheta (fst) snd

singleDescend ts lr deltas = (thetaZero,thetaOne)
where thetaZero = descendThetaZero ts lr deltas
thetaOne = descendThetaOne ts lr deltas

descend func deltas i
| i == 0 = delta

Solution

one obvious change would be to add type signatures,
i additionally introduced some type synonyms to distinguish between all those Doubles. A bit explanation could be done towards the algorithm - I don't know what it actually want to achieve. I am no programmer so the non understanding comes quite often.

Next thing is it seems costFunc is never used - why is it there?

and I've replaced genericLength by length and converting the result - as the library says length is faster.

type TrainData = [(Double,Double)]
type LearnRate = Double
type DeltaPair = (Double,Double)

trainingSet ::  TrainData
trainingSet = [ (1.0,10.0),(2.0,20.0),(3.0,30.0),(4.0,40.0) ]

hypothesis :: DeltaPair -> Double -> Double
hypothesis (d1,d2) x = d1 + (d2 * x)

costFunc :: TrainData -> DeltaPair -> Double
costFunc ts d = scaleFactor * sum squares
    where scaleFactor = 1.0 / (2 * fromIntegral (length ts))
          squares = map (square . diff d) ts
          square x = x * x

diff :: DeltaPair -> (Double, Double) -> Double
diff d (x,y) = hypothesis d x - y

descendTheta :: ((Double, Double) -> Double) -> (DeltaPair -> Double) -> TrainData -> LearnRate -> DeltaPair -> Double
descendTheta thetaFunc deltaFunc ts lr deltas =
    deltaFunc deltas - (scaleFactor * sum scaledDiffs)
    where scaleFactor = lr / fromIntegral (length ts)
          diffs = map (diff deltas) ts
          scaledDiffs = map (\(x,y) -> x * thetaFunc y) $ zip diffs ts

descendThetaZero :: TrainData -> LearnRate -> (Double, Double) -> Double
descendThetaZero = descendTheta (const 1) fst

descendThetaOne :: TrainData -> LearnRate -> (Double, Double) -> Double
descendThetaOne = descendTheta fst snd

singleDescend :: TrainData -> LearnRate -> DeltaPair -> DeltaPair
singleDescend ts lr deltas = (thetaZero, thetaOne)
    where thetaZero = descendThetaZero ts lr deltas
          thetaOne = descendThetaOne ts lr deltas

descend :: (DeltaPair -> DeltaPair) -> DeltaPair -> Int -> DeltaPair
descend f deltas i
    | i < 0 = error "no negative numbers"
    | i == 0 = deltas
    | otherwise = descend f (f deltas) (i-1)

main :: IO ()
main = print (descend descendFunc (100, 1) 10000)
    where descendFunc = singleDescend trainingSet 0.1

Code Snippets

type TrainData = [(Double,Double)]
type LearnRate = Double
type DeltaPair = (Double,Double)

trainingSet ::  TrainData
trainingSet = [ (1.0,10.0),(2.0,20.0),(3.0,30.0),(4.0,40.0) ]

hypothesis :: DeltaPair -> Double -> Double
hypothesis (d1,d2) x = d1 + (d2 * x)

costFunc :: TrainData -> DeltaPair -> Double
costFunc ts d = scaleFactor * sum squares
    where scaleFactor = 1.0 / (2 * fromIntegral (length ts))
          squares = map (square . diff d) ts
          square x = x * x

diff :: DeltaPair -> (Double, Double) -> Double
diff d (x,y) = hypothesis d x - y


descendTheta :: ((Double, Double) -> Double) -> (DeltaPair -> Double) -> TrainData -> LearnRate -> DeltaPair -> Double
descendTheta thetaFunc deltaFunc ts lr deltas =
    deltaFunc deltas - (scaleFactor * sum scaledDiffs)
    where scaleFactor = lr / fromIntegral (length ts)
          diffs = map (diff deltas) ts
          scaledDiffs = map (\(x,y) -> x * thetaFunc y) $ zip diffs ts

descendThetaZero :: TrainData -> LearnRate -> (Double, Double) -> Double
descendThetaZero = descendTheta (const 1) fst

descendThetaOne :: TrainData -> LearnRate -> (Double, Double) -> Double
descendThetaOne = descendTheta fst snd

singleDescend :: TrainData -> LearnRate -> DeltaPair -> DeltaPair
singleDescend ts lr deltas = (thetaZero, thetaOne)
    where thetaZero = descendThetaZero ts lr deltas
          thetaOne = descendThetaOne ts lr deltas

descend :: (DeltaPair -> DeltaPair) -> DeltaPair -> Int -> DeltaPair
descend f deltas i
    | i < 0 = error "no negative numbers"
    | i == 0 = deltas
    | otherwise = descend f (f deltas) (i-1)

main :: IO ()
main = print (descend descendFunc (100, 1) 10000)
    where descendFunc = singleDescend trainingSet 0.1

Context

StackExchange Code Review Q#12975, answer score: 4

Revisions (0)

No revisions yet.