patternMinor
Machine Learning algorithms based on "structural risk minimization"?
Viewed 0 times
riskstructuralminimizationalgorithmslearningbasedmachine
Problem
Which machine learning algorithms (besides SVM's) use the principle of structural risk minimization?
Solution
The structural risk minimization principle is a principle that is at least partly 'used' in all machine learning methods, since overfitting is often to be taken into account: reducing the complexity of the model is (supposedly and in practice) a good way to limit overfitting.
-
SVMs explicitly have a parameter for the complexity (the dimension of the feature space, or even the kernel function) and it's necessary because increasing the complexity is a part of the learning algorithm.
-
Neuronal networks also have a easy indicator of their complexity (number of 'cells') and is part of the associated learning algorithm.
-
Without this principle grammar inference would be both stupid and perfect grammar is the list of all possible words, so every non-trivial algorithm at least acknowledges this principle.
-
Decision trees have their own notion of entropy.
-
Clusters can be simply counted or kind of 'use' the principle intrinsically or have a fixed number of clusters and in that case you apply the principle at a higher level.
To be perfectly honest I don't really know about what happens in genetic programming but they don't have an intrinsic notion of complexity.
I don't know well Inductive logic programming but it doesn't seem to scale very well to this principle.
-
SVMs explicitly have a parameter for the complexity (the dimension of the feature space, or even the kernel function) and it's necessary because increasing the complexity is a part of the learning algorithm.
-
Neuronal networks also have a easy indicator of their complexity (number of 'cells') and is part of the associated learning algorithm.
-
Without this principle grammar inference would be both stupid and perfect grammar is the list of all possible words, so every non-trivial algorithm at least acknowledges this principle.
-
Decision trees have their own notion of entropy.
-
Clusters can be simply counted or kind of 'use' the principle intrinsically or have a fixed number of clusters and in that case you apply the principle at a higher level.
To be perfectly honest I don't really know about what happens in genetic programming but they don't have an intrinsic notion of complexity.
I don't know well Inductive logic programming but it doesn't seem to scale very well to this principle.
Context
StackExchange Computer Science Q#2006, answer score: 9
Revisions (0)
No revisions yet.