Our Loopy Way of Learning

Different ways of learning go through different parts of the brain.

Humans are champion learners. Other animals are smart, but we are smarter, spending more time in childhood soaking up the mysteries of the world around us, and storing them in a larger and better brain. Learning is our calling card, allowing us to adapt to any environment and defeat any foe. Indeed, we have overwhelmed the earth's biosphere, and need to exercise some deeper and longer vision by pulling back from our successes in mastering every possible resource.

But how the brain does all this is something we have yet to fully learn about. Our sense organs bring in tons of information, but as we have learned in computer science and AI, it takes a lot more to create usable information than just amassing data. How do the various parts of our brains work together to create actionable models of the world out of that data? An innovative theory was elaborated a quarter century ago by Kenji Doya, who has apparently gone on to a research sideline on soccer(!)

The problem began with, in part- what does the cerebellum do? From lesion data, it has long been clear that this small part of our brain has a big role in motor accuracy and coordination. But cerebellar neurons project all over the brain, not just to motor areas, and it gradually became clear that the cerebellum affects those other areas as well, helping us learn and manage many cognitive tasks. But structurally, the cerebellum is a peculiar organ, highly repetitive, with relatively linear and parallel processing, a bit like a GPU. It is clearly specialized for something- some kind of fine tuning, but what?

Doya's theory about the distinct learning styles contributed by different parts of the brain.

Doya proposed a general theory about learning in the brain, which splits learning up into three types. One type is supervised learning, which is what the cerebellum facilitates. We grab something, we recognize an error in eye-hand coordination, and we learn to grasp better. There is a loop involved, both physically and logically, by which some task is evaluated, and error signal sent back to the source, and a small adjustment is made for the next iteration. Humans have great hand coordination, partly thanks to our advanced cerebellar-cortical connections. 

The second form of learning is reward-based learning. This prototypically features dopamine, and is centered in the basal ganglia, where emotions are processed, but which is also a major switchboard for connections upwards to the cortex and downwards to the brainstem, and is also critical for motion control. Parkinson's and Huntington's diseases both affect this region. Doya proposed in broad terms that this area doesn't do the kind of error correction that the cerebellum does, but rather learns by reward- whether the action's goal was reached. If the goal was reached, a spurt of dopamine is issued, which reinforces whatever connections helped that action happen.

Interestingly, there is a progressive temporal aspect to this learning. As an action is learned, the reward happens sooner, given the predictive capability of our learning / cognitive systems. The whole point, after all, is to figure out as early as possible what will be happening in the future. If a bell is associated with food, at first the food prompts the reward. But as learning happens, and the bell is reliably associated with the later food, and the bell starts to set off the reward all by itself. Gradually, the sight of the person getting about to ring the bell, or the expected time of day, sets off the reward... it is cast progressively back to whatever the perceived cause is, going back in the learned causal chain. Eventually, as everything becomes rote and the novelty wears off, reward lessens, and learning is complete. Mealtime is here, as it is every day... boring.

Lastly, the third type of learning is unsupervised learning. This is the job of the cortex, and is the most interesting and subtle form of learning. The world has patterns, and the cortex's job is to figure out what those patterns are. No one outside tells it what is right or wrong, what is or is not real. It has to figure all this out from the stream of input. Unsupervised learning is thought to be well suited to the basic mechanism of the brain, its Hebbian circuitry where activity reinforces any active connection. There are a lot of mathematical methods that have been devised to approximate such processes, like principal component analysis. Data can be binned / categorized in various ways to simplify and organize reality, and eventually we have ... language and concepts, labels and abstract understanding. This categorization of the world is immensely powerful, if always inaccurate and approximate. 

Doya's model of the cerebellum. Purkinje cells at the center are also conceptually at the center, as the only cells that are plastic in this scheme, able to change their connection weights due to input from the other layers, which in turn get input from other brain areas including the cortex and thalamus. This has been likened to a perceptron in computer science terms.

Doya thus created an influential heuristic that divided up learning styles / mechanisms by anatomy in the brain. Obviously, all these mechanisms work closely with each other. The cerebellum doesn't know anything without conceptual categorizations coming from the cortex, which provide the foundation of error signals. Similarly, whether goals have or haven't been reached has to be informed by conceptual data from the cortex (or food signals, or other sensations from elsewhere in the body). Learning is a whole-brain activity, and while it is prodigious and effortless early in life, it continues to be essential throughout, lest we be overwhelmed by the fresh challenges the world keeps presenting to us.

• Recent work fleshing out this theory.
• Thought crime.
• Empathy as crime.
• .. or as being Democratic.
• Scrap all fossil cars now.
• China is going after their rich.