patternMinor
Simple Gradient Descent Algorithm In Haskell
Viewed 0 times
simplegradientalgorithmhaskelldescent
Problem
After becoming interested in various bits of functional programming provided by non-functional languages, I've recently decided to start learning Haskell. As I'm fairly experienced with conventional imperative programming languages, I decided to make my "hello world" something a little more complex - an implementation of gradient descent with 2 variables.
The idea with this code is that you fill in the training set for the algorithm in the code, and then run the code something similar to this:
Where
The actual code that's running is below. As I'm entirely new to Haskell, I was wondering whether code such as this is acceptable? And whether there's any obvious idioms I'm missing/misusing or any glaring style errors I've made.
Any comments on my implementation of the algorithm are welcome also.
```
import Data.List
trainingSet = [ (1.0,10.0),(2.0,20.0),(3.0,30.0),(4.0,40.0) ]
hypothesis (d1,d2) x = d1 + (d2 * x)
squareDiff d (x,y) = diff * diff
where diff = ( hypothesis d x ) - y
costFunc ts d = scaleFactor * sum squares
where scaleFactor = 1.0 / (2 * genericLength ts)
squares = map (squareDiff d) ts
diff d (x,y) = (hypothesis d x) - y
descendTheta thetaFunc deltaFunc ts lr deltas =
deltaFunc deltas - (scaleFactor * sum scaledDiffs)
where scaleFactor = lr / genericLength ts
diffs = map (diff deltas) ts
scaledDiffs = map (\(x,y) -> x * thetaFunc y) $ zip diffs ts
descendThetaZero = descendTheta (\_ -> 1) fst
descendThetaOne = descendTheta (fst) snd
singleDescend ts lr deltas = (thetaZero,thetaOne)
where thetaZero = descendThetaZero ts lr deltas
thetaOne = descendThetaOne ts lr deltas
descend func deltas i
| i == 0 = delta
The idea with this code is that you fill in the training set for the algorithm in the code, and then run the code something similar to this:
descentFunc = singleDescend trainingSet 0.1
deltas = descend descentFunc ( 100, 1 ) 100000Where
0.1 is the learning rate (referred to as lr in the code) 100000 is the number of iterations for the loop, and ( 100, 1 ) is an initial guess for the coefficients.The actual code that's running is below. As I'm entirely new to Haskell, I was wondering whether code such as this is acceptable? And whether there's any obvious idioms I'm missing/misusing or any glaring style errors I've made.
Any comments on my implementation of the algorithm are welcome also.
```
import Data.List
trainingSet = [ (1.0,10.0),(2.0,20.0),(3.0,30.0),(4.0,40.0) ]
hypothesis (d1,d2) x = d1 + (d2 * x)
squareDiff d (x,y) = diff * diff
where diff = ( hypothesis d x ) - y
costFunc ts d = scaleFactor * sum squares
where scaleFactor = 1.0 / (2 * genericLength ts)
squares = map (squareDiff d) ts
diff d (x,y) = (hypothesis d x) - y
descendTheta thetaFunc deltaFunc ts lr deltas =
deltaFunc deltas - (scaleFactor * sum scaledDiffs)
where scaleFactor = lr / genericLength ts
diffs = map (diff deltas) ts
scaledDiffs = map (\(x,y) -> x * thetaFunc y) $ zip diffs ts
descendThetaZero = descendTheta (\_ -> 1) fst
descendThetaOne = descendTheta (fst) snd
singleDescend ts lr deltas = (thetaZero,thetaOne)
where thetaZero = descendThetaZero ts lr deltas
thetaOne = descendThetaOne ts lr deltas
descend func deltas i
| i == 0 = delta
Solution
one obvious change would be to add type signatures,
i additionally introduced some type synonyms to distinguish between all those
Next thing is it seems
and I've replaced
i additionally introduced some type synonyms to distinguish between all those
Doubles. A bit explanation could be done towards the algorithm - I don't know what it actually want to achieve. I am no programmer so the non understanding comes quite often.Next thing is it seems
costFunc is never used - why is it there?and I've replaced
genericLength by length and converting the result - as the library says length is faster.type TrainData = [(Double,Double)]
type LearnRate = Double
type DeltaPair = (Double,Double)
trainingSet :: TrainData
trainingSet = [ (1.0,10.0),(2.0,20.0),(3.0,30.0),(4.0,40.0) ]
hypothesis :: DeltaPair -> Double -> Double
hypothesis (d1,d2) x = d1 + (d2 * x)
costFunc :: TrainData -> DeltaPair -> Double
costFunc ts d = scaleFactor * sum squares
where scaleFactor = 1.0 / (2 * fromIntegral (length ts))
squares = map (square . diff d) ts
square x = x * x
diff :: DeltaPair -> (Double, Double) -> Double
diff d (x,y) = hypothesis d x - y
descendTheta :: ((Double, Double) -> Double) -> (DeltaPair -> Double) -> TrainData -> LearnRate -> DeltaPair -> Double
descendTheta thetaFunc deltaFunc ts lr deltas =
deltaFunc deltas - (scaleFactor * sum scaledDiffs)
where scaleFactor = lr / fromIntegral (length ts)
diffs = map (diff deltas) ts
scaledDiffs = map (\(x,y) -> x * thetaFunc y) $ zip diffs ts
descendThetaZero :: TrainData -> LearnRate -> (Double, Double) -> Double
descendThetaZero = descendTheta (const 1) fst
descendThetaOne :: TrainData -> LearnRate -> (Double, Double) -> Double
descendThetaOne = descendTheta fst snd
singleDescend :: TrainData -> LearnRate -> DeltaPair -> DeltaPair
singleDescend ts lr deltas = (thetaZero, thetaOne)
where thetaZero = descendThetaZero ts lr deltas
thetaOne = descendThetaOne ts lr deltas
descend :: (DeltaPair -> DeltaPair) -> DeltaPair -> Int -> DeltaPair
descend f deltas i
| i < 0 = error "no negative numbers"
| i == 0 = deltas
| otherwise = descend f (f deltas) (i-1)
main :: IO ()
main = print (descend descendFunc (100, 1) 10000)
where descendFunc = singleDescend trainingSet 0.1Code Snippets
type TrainData = [(Double,Double)]
type LearnRate = Double
type DeltaPair = (Double,Double)
trainingSet :: TrainData
trainingSet = [ (1.0,10.0),(2.0,20.0),(3.0,30.0),(4.0,40.0) ]
hypothesis :: DeltaPair -> Double -> Double
hypothesis (d1,d2) x = d1 + (d2 * x)
costFunc :: TrainData -> DeltaPair -> Double
costFunc ts d = scaleFactor * sum squares
where scaleFactor = 1.0 / (2 * fromIntegral (length ts))
squares = map (square . diff d) ts
square x = x * x
diff :: DeltaPair -> (Double, Double) -> Double
diff d (x,y) = hypothesis d x - y
descendTheta :: ((Double, Double) -> Double) -> (DeltaPair -> Double) -> TrainData -> LearnRate -> DeltaPair -> Double
descendTheta thetaFunc deltaFunc ts lr deltas =
deltaFunc deltas - (scaleFactor * sum scaledDiffs)
where scaleFactor = lr / fromIntegral (length ts)
diffs = map (diff deltas) ts
scaledDiffs = map (\(x,y) -> x * thetaFunc y) $ zip diffs ts
descendThetaZero :: TrainData -> LearnRate -> (Double, Double) -> Double
descendThetaZero = descendTheta (const 1) fst
descendThetaOne :: TrainData -> LearnRate -> (Double, Double) -> Double
descendThetaOne = descendTheta fst snd
singleDescend :: TrainData -> LearnRate -> DeltaPair -> DeltaPair
singleDescend ts lr deltas = (thetaZero, thetaOne)
where thetaZero = descendThetaZero ts lr deltas
thetaOne = descendThetaOne ts lr deltas
descend :: (DeltaPair -> DeltaPair) -> DeltaPair -> Int -> DeltaPair
descend f deltas i
| i < 0 = error "no negative numbers"
| i == 0 = deltas
| otherwise = descend f (f deltas) (i-1)
main :: IO ()
main = print (descend descendFunc (100, 1) 10000)
where descendFunc = singleDescend trainingSet 0.1Context
StackExchange Code Review Q#12975, answer score: 4
Revisions (0)
No revisions yet.