From biology to layers
A neuron (unit) in a network is just logistic regression: a = g(w·x + b). A layer is many units reading the
same input; a network stacks layers.
The demand-prediction example: inputs (price, shipping, marketing, material) → a hidden layer that learns
"affordability", "awareness", "perceived quality" → output "will it be a top seller". The key idea: the network
learns its own features. You don't design area = frontage × depth by hand any more.
Face recognition shows the same in images: early layers find edges, middle layers parts (eyes, noses), later layers face shapes.
Notation
a^[l]is the activation vector of layerl;a^[0] = x.a_j^[l] = g( w_j^[l] · a^[l−1] + b_j^[l] ).- A network with 1 hidden layer is called a 2-layer network (the input layer doesn't count).
Going further
- Universal approximation: one wide enough hidden layer can approximate any continuous function. Depth helps because many functions are far cheaper to represent with several layers than with one huge one.
- Each unit's
w_jis a learned feature detector; that's why networks beat hand-made features on images, sound and text.