An abstract model of how and why memories are replayed internally- forwards and also backwards.
The Japanese tea ceremony is a wonderful example of reinforcement learning. The ultimate reward is a cup of hot tea. But getting that reward takes some work- materials, a supply chain, cooking operations, serving apparatus, etc. It is the reward that somehow drives us to perform all the necessary preparation. How does that work? There has been a revolution in the study of brain and memory over the last few decades, centering on the hippocampus. This began with the discovery of "place cells", which fire when certain geographic locations are crossed. Eventually, scientists discovered that their test animals (rats, mice, and us) "replay" sequences of events such as travel through a test maze, firing off those place cells as they go along in their minds.
A recent paper delved into the nature and meaning of mental replay at a theoretical level, trying to tie up various empirical observations and other theories. One such observation is that mental replay can go either forward or in reverse. Indeed, after a trip where the reward happened at the end, replay tended to happen more frequently in reverse! The implications of this are fascinating, suggesting that there is a mental hunt for landmarks back in time that either provide signposts for the upcoming reward (that is, lend themselves to Pavlovian conditioning) or are decision points where the subject wants to learn about alternate paths and reinforce the correct one. This replay is not necessarily conscious at all, but one can easily sense in our own lives that this mental travel happens- how, having had a refreshing cup of tea, we ponder how it came to be, how we could do this more (or less!) efficiently. Particularly, we generate rewarding stops along that way as we learn about the process, casting the reward back from the tea drinking- back to the firing up of the pot, to the setting out of the cups and spoons, the smell of the raw tea, to the buying of the tea, and all the other steps involved. The travel back in time is essential to setting up the path to our next cup of tea.
 |
| Data vs model. The authors present how their model (right) performs memory replays around maze runs for rats that conclude with a reward, in terms of rate of forward replay vs backward replay. Prior to a run, the rat is planning the future and running the mental maze forward. Afterwards, it more frequently runs the maze (mentally) in reverse, presumably to figure out how to value and put rewards on earlier steps that led to the ultimate reward. The same model recreates memory formation, meandering between disconnected experiences to form novel sequences, learning and progression to novel experiences/memories, bias to salient experience, differences between sleep and waking replay, and ability to generate imagined sequences. |
These authors call their speculative model CMR-replay, which stands for "context maintenance and retrieval model / replay". Its point is to posit minimal and physiological plausible (though extremely abstract) mechanisms that might account for this and other memory phenomena. It is built around object-context relations. One thing our brains are good at is categorizing things and pulling objects out of the welter of our experience. But a memory is composed of both such objects and the welter they are embedded in- the environment, the feelings, etc. that accompany them. So they posit that each memory is a relation between object and context, and we store both directions- how the object related to its context, and how the context pointed to associated particular objects. As memory encoding goes forth, they posit that encoding happens faster the more salient the situation/object is, and that the encodings fade over time. They also posit a novelty rule, where replay of past events happens mostly to consolidate important and recent memories, (to other areas of the brain like the neocortex), so once they have been replayed a few times, those item/context pairs are down-weighted and others come up for replay.
All this leads to what they claim are quite plausible behaviors by their models, which replicate to some degree experimental evidence. As shown above, the simple encoding of items+context tends to bias replay to time-forward sequences, as the contexts naturally bleed into each other succesively, but backwards replay is easily supported as well. This is particularly true if the most salient point of the experience was not its trepidatious beginning, but its glorious ending. While the experimental evidence for all this comes from rats in mazes, the authors point out that mental paths can just as well be semantic and abstract as physical runs in a maze.
Another interesting property is the generation of shortcuts and non-realistic paths. At each point in a replay, the item+context may point to related alternatives from what was directly experienced, especially if the true (and well-worn) path has been down-weighted. So replay may find shortcuts and novel ways to get to where one is going, or, indeed, find beautiful elaborations to add to the path instead!
Lastly, what about sleep? The empirical observation is that in waking states, replay is generally driven by experience- the last maze trip, or an upcoming maze trip that is being planned out. In sleep, in contrast, external imputs are minimized, and the emphasis seems to be in free replay to consolidate memories between the hippocampus and the neocortex. The authors suggest that during NREM sleep, the hippocampus is in the driver's seat, replaying actual experiences for consolidation in the cortex. In contrast, during REM sleep, the cortex drives the process, suggesting subjects to replay, with perhaps a much wider gamut that veer off into dreams that explore all sorts of curious paths through the subjective points of past experience and imagined non-exeriences.
Biology is generally not thought of as a theoretical discipline. Its "theory of everything" has long been established: Darwin's theory of evolution. But it is hypothesis-driven, since so many mysteries remain. This theory of memory formation and reformation is one such hypothesis, one that is not mechanistic in detail, at least yet, but which occupies an intermediate level, as is common in a discipline that engages many levels of explanation.
- Could health costs and billing get any worse? Absolutely!
- AI for genomes.
