Machines That Learn Like Physicists Think
Every physics student learns the same trick on day one: when a problem gets hard, approximate. Start with the simplest possible description, then add corrections. A pendulum's motion? Begin with a straight line, then a sine wave, then small anharmonic tweaks. This idea, the Taylor expansion, builds up complicated functions by stacking ever-higher-order terms onto a humble constant. It is, in a sense, how physicists tame the world: lowest order first, refinements after.
A new kind of learning machine appears to do the same thing on its own.
Machine learning has been transformative, but it comes at a steep cost. Training a modern neural network involves running billions of nonlinear calculations on power-hungry digital processors, a process that bears little resemblance to how biological brains actually learn. The brain manages roughly 100 billion neurons running in parallel, each updating its connections using only local information, with no central coordinator telling the system what to do. Replicating that style of learning in hardware, rather than simulating it in software, has long been a goal of the field known as neuromorphic computing.
In a recent paper in PNAS, Sam Dillavou and collaborators at the University of Pennsylvania report progress in that direction. They built a physical electronic network of 32 pairs of transistors, wired together on a circuit board, that learns nonlinear tasks without any processor, simulation, or global oversight. Each transistor pair adjusts itself based only on voltages it can locally measure. Yet the network collectively solves problems that linear circuits provably cannot, including the classic XOR function and nonlinear regression.
The architecture, called a contrastive local learning network (CLLN), works on a deceptively simple principle. Two identical copies of the transistor network are run in parallel: a "free" network that simply responds to inputs, and a "clamped" network that additionally has the desired answer imposed on its output. Between every pair of corresponding transistors sits a tiny circuit that compares the local voltage in the free and clamped versions and nudges a shared gate voltage, the transistor's "weight," to bring them into agreement. When the two networks finally match, the free network has learned to produce the right answer on its own.
The most surprising result emerged when the authors examined how the network learns. They trained the system to fit a curve through eight data points and tracked, moment by moment, how the error evolved. Rather than improving uniformly, the network reduced its error in a structured way: first, it corrected the average value of its output, then it adjusted the slope, and only afterward did it begin to bend the curve to capture the curvature. To make this rigorous, the authors projected the error onto a basis of orthogonal polynomials built from successive powers of the input, essentially the building blocks of a Taylor expansion. The constant term shrank first, the linear term second, the quadratic term last. The same pattern held across 29 different tasks. For the XOR problem, where the answer depends on a product of two inputs, the purely nonlinear contribution was the final piece to fall into place.
In other words, the network builds up its solution the way a physicist builds up an approximation: lowest order first, higher orders later. This behavior was not designed into the circuit. No instruction tells the transistors to handle the constant before the slope. It emerges from the collective dynamics of many small elements following local rules.
This behavior echoes a phenomenon known in conventional deep learning as "spectral bias," the tendency of artificial neural networks to learn smooth, simple features before sharp, complex ones. Seeing the same ordering arise in a piece of analog hardware suggests it may be a general feature of how distributed learning systems organize themselves, rather than an artifact of any particular algorithm.
The practical numbers are worth noting. Once trained, the network performs its task in microseconds and dissipates only picojoules of energy per transistor. Training itself takes a few seconds. The components are standard and could, in principle, be microfabricated to fit roughly 10 million of them per square centimeter. Because each element updates using only local information, scaling the system up does not require ever-more-precise simulations of the whole, a bottleneck that hinders other neuromorphic approaches.
There is still a long way to go before such networks compete with conventional neural networks on real-world tasks. The optimal kind of nonlinearity, the best wiring topology, and how performance scales with size are all open questions. But the work points toward a future in which learning is not something a computer does to data, but something a physical system does naturally, a future in which a piece of matter, given inputs and a goal, organizes itself into the answer, one Taylor term at a time.