Thoughts in, memory out

Researchers find memory signals coming out resemble the reverse of sensory signals going in.

While the greater problem of the physical basis / nature of consciousness is pending on the scientific to-do list, the problem of how memory is stored continues to yield to persistent experimentation. A long history of lesions in human and animal brains has told us where memories are stored and in part how they are routed through the anatomical circuitry. Also some of the molecular and cellular details have come to light- that synapses are the typical location of storage, getting strengthened by use (with the possible addition of new cell growth and new cell extensions) in ways that are thought to promote their ability to join in activity patterns that resemble the encoding pattern, and thus enable the recall of memories in response to relevant cues.

The current paper takes this work a little deeper, using detailed recording of small brain areas to test whether this theory is really true- whether sensations fed into local neural nets actually resemble memories that the experimenters later prompt from those same areas. Given the distributed and stochastic nature of neural processes, (which we can sense in the muted and fragmentary nature of our thought and memory patterns), looking at this on the cellular level requires a good bit of statistics to sift signals from the data.

Our brains have two general tissues- the gray matter and the white. The white matter is white due to the preponderance of myelin, the fatty insulation wrapped around long-distance nerve fibers (axons) to enhance their conduction. It is the piping of the brain. The gray matter consists of a large sheet of cell bodies and cell-cell connections (dendritic spines, arbors, and synapses) layered on the surface of the brain which, if one conceptually irons out the convolutions, extends to a surface of about 0.25 square meter. The gray matter has a depth of between 1.5 to 4.5 millimeters.

A small area of the brain, stained for cell bodies, (purple), which accentuates the gray matter and its layers.

The layers within the gray matter have been recognized for over a hundred years, and are important signs of anatomical and functional differentiation. Typically, there are six recognized layers, most of which correspond to different classes of neuronal cell bodies, from which axons and dentrites branch out to the other layers in a dense network. Functionally, the layers interconnect more strongly up and down than they do to neighboring points in the gray matter sheet, so functions tend to map over the sheet like states on a map, where neighboring "columns" projecting down through the six-layer sheet have related, but different functions. In the visual cortex, for instance, nearby columns of roughly 50 micrometer diameter may alternate which eye their input comes from, or compute (i.e. be specifically sensitive to) directions of object motion or orientation around the cardinal directions in a progressive fashion.

The present researchers concentrate on the vertical structure of a small portion of the macaque temporal perirhinal cortex (bottom middle side of the brain). This location (area 36, to be precise) is known to be the end of the line for visual sensation, after the extensive processing that happens in earlier portions of the visual cortex. (Interested readers might refer to Christof Koch's book the quest for consciousness, which focuses mostly on the visual system.) It is also close to the hippocampus and interacts with its memory functions. It is where visual memory is stored.

"The primate inferotemporal cortex locates at the final stage of the ventral visual pathway and serves as the storehouse for visual long-term memory."

The heart of this paper is the use of dense electrode arrays which the researchers stick into the brains of their macaques before doing behavioral experiments with them. These arrays have 24 sub-electrodes, spaced 100 micrometers apart vertically, to give a rough picture of what is happening in the different gray matter layers at one point of the macaque visual memory cortex. The researchers attempt to show that the direction of current flow/processing tends in the opposite direction (up vs down) though this layer as the macaque switches from seeing a visual figure to recalling it from memory. They do this with two data reduction methods called current source density analysis (CSD), and cross-correlation analysis of spike trains (CCG, for cross-correlogram). Some faith is required from the reader in the author's treatment of their data.

Readings from these electrodes were timed with respect to the task given to the macaques, either presenting them with a picture, or asking them to recall a picture, and the relations between the vertical levels of the gray matter activity are read out and statistically analyzed. For instance, the earliest signals seen in this region after presenting a picture to the macaque were in the layer 4, consistent with its known connections to upstream visual components.

Gray matter layers SG (supragranular), G (granular), and IG (infragranular) are also termed layers 2/3, 4, and 5/6 respectively, as shown. The outside surface of the brain would be towards the bottom. The asterisk marks where the first signal arrives, after the cue picture is presented.

The macaques had been trained to associate six pairs of images, so that seeing one (for 0.5 second, called the cue period) evoked the other one from memory, which they touched after a 2 second delay period, out of a few choices, on a test screen (1.5 second choice period) for a reward. After training and by the time of the experiments, the macaques got these tests 90% correct. I would have to comment that it is not absolutely clear from this protocol that the macaque was driven to internally visualize the missing picture from memory, only to dredge up a matching sensation once the matching image was explicitly shown. The key data was taken during the delay period, however, increasing the chances that they were dealing with a memory prepared by the macaque in advance for rapid test success.

Schematic of the images presented to the macaques as cue and test.

The results from all this are extremely subtle- slight shifts of current direction that tell us that data (i.e. neuronal spikes or current) is transiting between strata of the gray matter. The authors represent this in several ways, first of which is a graph of spike train correlations between two locations, in this case the IG and SG layers, tracked during the cue period where there the slight shift of correlation indicates traffic from the SG to the IG layers, and the delay period where the opposite is the case- traffic from the IG to the SG layer is indicated.

All this is summarized in another series of graphs (below), mapping everything happening in the layers they sampled. The X-axis is the deduced source layer, and the Y-axis is the deduced target layer, in mm, with the G layer (4) at 0. The red spots then map the source-target pairs, which are finally summarized at the right as the traffic during seeing (cue) and recall (delay). Obviously, there is a great deal of noise. We are only evesdropping on a few cells in the vast structure of the brain that may or may not be terribly central to the picture the monkeys are trained on or being shown. The researchers don't really say whether they had any guidance other than the known anatomy, like functional MRI or the like, to locate the gray matter regions that would be optimal for this analysis. In all likelihood, they poked several likely places and said a few hail Marys.

Part A is described in the text. Part C summarizes all the data, describing where signals are going within the gray matter layers of the perirhinal cortex during visual perception (cue) and presumed memory retrieval (delay).

Do they have anything to say about prior knowledge about what the layers are known to do?  They cite some reviews, including an excellent one which discusses in detail the parallel processing that goes on through the visual system, (progressing from the back of the brain in a forward direction), and notes specifically that early (V1) visual processing typically takes the same interlaminar path of G/4 to SG/2/3 layers, corroborating what the current researchers find in the much later portion of the visual system. So the G -> SG layer sequence, among much other processing, may be a general property of incoming sensory information throughout the gray matter.

"The present study demonstrated that canonical feed-forward signal flow across cortical layers during sensory coding reverses to the feedback direction during memory retrieval phase, which suggests flexible recruitment of interlaminal connectivity depending on the cognitive demands in the monkey association cortices."

So, assuming that these macaques are really generating image memories while they wait for the test to take place, (delay period), and that the memory, if not identical to the perceived pattern since it is a different learned pair image, at least uses the same physical mechanisms and locations, these measurements suggest that visual memory generation resembles a reverse of the process of visual perception at the latest stage. This has obvious appeal as a simple and efficient theory of how the brain works, where an executive level of control/attention has only to ask (implicitly) for a memory, and those regions of the brain responsible for perceiving it in the first place cough it up on demand by a sort of reversal of the storage process.

I might add that the vividness of such a memory is a very interesting question. For these macaques, there needed to be no vividness at all- the entire recollection could easily have been subconscious, so as to enable quick test performance, but not to do a full LSD-style re-visualization. This would only require a small recurrent (i.e. backward) request from the executive attention (or sub-attention) area to this very last area of visual processing / memory. However, the idea of recurrent requests of this kind and the "spotlight of attention" in general is a hot topic, and it is increasingly clear that attention mechanisms can reach very deeply back into the perceptual processing structures, enabling many levels of re-enactment of a remembered stimulus.

"We found that information about where attention was allocated can either originate in posterior cortex (when grabbed by external cues) or frontal cortex (when being internally directed)." - from the last link presented, about how attention happens in the brain

• Our long history in Libya.
• What would a full accounting of coal's costs look like?
• A nice primer on monetary policy and why we can run high deficits/debt.
• How long have we had public debt? Forever! And it's been good.
• More on our corrupt administration. And some more..
• About those ratings agencies...
• Bill Mitchell quote of the week: just a little Bob Marley.