A shared code for perceiving and imagining objects in human ventral temporal cortex.

Mental imagery allows us to remember previous experiences and imagine new ones. Animal studies have yielded rich insight into mechanisms for visual perception, but the neural mechanisms for visual imagery remain poorly understood. We determined that approximately 80% of visually responsive single neurons in the human ventral temporal cortex (VTC) use a distributed axis code to represent objects. We used that code to reconstruct objects and generate maximally effective synthetic stimuli. We then recorded responses from the same neural population while subjects imagined specific objects; about 40% of axis-tuned VTC neurons recapitulated the visual code. Our findings reveal that visual imagery is supported by reactivation of the same neurons involved in perception, providing single-neuron evidence for the existence of a generative model in human VTC. Editor's summary: The ventral temporal cortex (VTC) is a brain area involved in identifying and categorizing visual stimuli. Wadia et al. performed single-neuron recording in the VTC of patients with epilepsy while the subjects were presented with real visual stimuli or were asked to imagine them. Deep network analysis showed that visually responsive neurons were tuned on specific axes. While imagining the objects, around 40% of the visually responsive VTC neurons were also robustly activated. Thus, mental imagery reactivates the same sensory codes used during visual stimuli, suggesting the existence of a generative model capable of synthesizing detailed sensory contents from an abstract, semantic representation. —Mattia Maroso INTRODUCTION: Mental imagery refers to our brains' capacity to generate percepts, emotions, and thoughts in the absence of external stimuli. This ability allows us to generate art, simulate actions and outcomes, remember previous experiences, and imagine new ones. Uncontrolled mental imagery can contribute to psychological disorders, including anxiety, schizophrenia, and posttraumatic stress disorder. Despite its importance in our lives, little is known about the single-neuron mechanisms of mental imagery. Neuroimaging results support a long-standing theory that imagery of a given sense is subserved by the reactivation of that specific sensory cortex. However, these studies lack the resolution to discern whether it is the same neurons or separate circuitry roughly located in the same regions that reactivates. RATIONALE: We investigated the single-neuron mechanisms of visual imagery by recording single neurons in human patients implanted with electrodes to localize their focal epilepsy as they viewed and subsequently imagined objects. We focused our investigations on the ventral temporal cortex (VTC), a part of the temporal lobe dedicated to representing visual objects. We first determined the code for visual objects. We found that as in macaques, neurons in human VTC represent objects by using a distributed axis code. This code emphasizes the geometric picture that neurons project incoming stimuli—formatted as points in feature space—onto specific preferred axes and respond proportionally to the projection value. We then examined whether this code is reactivated during imagery. RESULTS: We recorded 714 neurons in the human VTC across 16 patients as they viewed visual objects. A majority of neurons (456 of 714) were visually selective for one of the five object categories used (faces, plants, text, animals, and objects). To represent general objects with arbitrary features, we built a low-dimensional object space using the unit activations of deep networks trained to perform object classification. Nearly ~80% (367 of 456) of all visually responsive single neurons were significantly axis tuned. We used this axis code to reconstruct objects and generate maximally effective synthetic stimuli. Last, we recorded the responses of the same neurons in a subset of patients (6 of 16) as they imagined the same objects. Mean responses to perceived and imagined objects were comparable, with some neurons active only during perception, some only during imagery, and some during both. In particular, ~40% (43 of 107) of axis-tuned VTC neurons recorded during the imagery task reactivated, and the responses during imagery of individual neurons were proportional to the projection value of those objects onto the neurons' viewing axes. We used this observation to reconstruct imagined objects while still easily distinguishing whether those objects were viewed or imagined. CONCLUSION: We leveraged the opportunity to record from the same population of VTC neurons in humans as they viewed and imagined objects. Neurons use an axis code to represent visual objects, and neural activity during imagination reactivates this code. These findings provide single-neuron evidence for a generative model in the human brain. A shared code for vision and imagination.: (A) Single-neuron recordings from VTC during vision and imagery. (B) Axis model framework for stimulus encoding. (C) VTC neuron showing maximal responses to images furthest along preferred axis. (D) Axis code enables stimulus reconstruction. (E) Same neuron as (C) reactivated during imagery. Stimulus preference is preserved. (F) Reactivation of axis code enables reconstruction of imagined stimuli. [ABSTRACT FROM AUTHOR]