Why can Multilayer neural networks solve non-linear problems

I understand what a multilayer neural network is, but what about them allows them to solve non-linear problems unlike perceptrons? Is it the fact that they can extend to any number of outputs/hidden layers? Or is it another feature?

A single-layer network is already nonlinear, but it's only a limited kind of nonlinearity.

Yes, the ability to have multiple layers and multiple hidden nodes is what allows multi-layer neural networks to express any function.

Let me give you an analogy that provides intuition but shouldn't be taken too seriously. A single NAND gate can compute only a single, simple function. However, when you consider circuits containing NAND gates, they can express any boolean function: the ability to have multiple layers of NAND gates, and multiple intermediate gates in the middle, allows you to express any boolean function. Something vaguely similar happens with multi-layer neural networks: each individual unit provides a limited amount of nonlinearity, like a NAND gate, and the ability to compose them is what lets you express more complex nonlinear functions.

StackExchange Computer Science Q#42128, answer score: 3