Support Vector Machines as Neural Nets?

This is more of a conceptual question.

I have learned about Neural Nets, and I have some clue as to how Support Vector Machines work. I read somewhere however that given the appropriate kernel (is that right?), the SVM is identical to the Neural Net. Could someone who understands this please enlighten me as to how that's possible?

An SVM with a linear kernel has the same expressive power as a single perceptron in a neural net, since both are linear classifiers. If you think of the kernel function as mapping your input to a higher dimensional feature space then the SVM is still a linear decision boundary in that high dimensional kernel space. You could theoretically use the high dimensional kernel space features as input to a neural network as well as long as the kernel space is finite. Again the linear SVM behaves similarly to a single perceptron.

However, this only applies to linear SVMs. In practice SVMs are typically used with a non-linear kernel, and then no such equivalence applies. So this claimed equivalence might be a bit misleading, if you are comparing how SVMs tend to be used in practice to how neural nets tend to be used in practice.

Summary: Think of an linear SVM as being similar to a single perceptron instead of a neural network.

StackExchange Computer Science Q#19337, answer score: 4