How it works
Every neural network, from ADALINE to today's largest models, is built from the same basic part: a unit that weighs its inputs and adjusts those weights when it's wrong.
One artificial neuron
An artificial neuron takes several numbers as input, multiplies each by a weight, adds them up along with a bias, and produces an output. The weights decide how much each input matters. Learning means finding good weights.
output = w1·x1 + w2·x2 + … + bias
To make a yes-or-no decision, the neuron passes that sum through a threshold: positive means one class, negative means the other.
How ADALINE learns
ADALINE (ADAptive LINear Element) was built at Stanford by Bernard Widrow and his student Ted Hoff in 1960. Its learning rule, now called the Widrow–Hoff rule or least mean squares (LMS), adjusts each weight after every example:
new weight = weight + learning rate × (target − output) × input
The target is the correct answer (+1 or −1), and the output is the neuron's raw sum, before the threshold. The size of the correction depends on how far off the neuron was. Over many examples, this steadily lowers the average squared error.
ADALINE vs. the perceptron
Frank Rosenblatt's perceptron (1958) looked similar but learned differently. It only updated when it got an answer wrong, and it judged right or wrong after the threshold. ADALINE measured error on the raw output, before the threshold, so it learned from how wrong it was, not just whether it was wrong.
That difference matters: minimizing a smooth error is the idea behind gradient descent, the method that trains essentially every modern neural network.
The limit of a single neuron
One neuron can only draw a straight line (or, with more inputs, a flat plane). It cannot separate data that needs a curved boundary. The classic example is XOR: true when exactly one of two inputs is true. Marvin Minsky and Seymour Papert highlighted limits like this in their 1969 book Perceptrons, and interest in neural networks cooled for years.
Layers and deep learning
The fix is to stack neurons in layers, with smooth (non-linear) activations between them. A middle, or hidden, layer can bend the boundary into any shape. Training many layers requires backpropagation, which passes the error backward through the network to work out how each weight should change. It is the same goal as ADALINE's rule, extended to millions or billions of weights.
Modern networks for images, speech and language are this idea at enormous scale, with architectures such as convolutional networks and Transformers, but the core loop is unchanged: compute an output, measure the error, nudge the weights.