Flexible coding of object motion in multiple reference frames by parietal cortex neurons

Neurons represent spatial information in diverse reference frames, but it remains unclear whether neural reference frames change with task demands and whether these changes can account for behavior. In this study, we examined how neurons represent the direction of a moving object during self-motion, while monkeys switched, from trial to trial, between reporting object direction in head- and world-centered reference frames. Self-motion information is needed to compute object motion in world coordinates but should be ignored when judging object motion in head coordinates. Neural responses in the ventral intraparietal area are modulated by the task reference frame, such that population activity represents object direction in either reference frame. In contrast, responses in the lateral portion of the medial superior temporal area primarily represent object motion in head coordinates. Our findings demonstrate a neural representation of object motion that changes with task requirements.

This is a preview of subscription content, access via your institution

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

cancel any time

Subscribe to this journal

Receive 12 print issues and online access

206,07 € per year

only 17,17 € per issue

Buy this article

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Pigeons integrate visual motion signals differently than humans

Article Open access 16 September 2019

Attention amplifies neural representations of changes in sensory input at the expense of perceptual accuracy

Article Open access 01 May 2020

Shared mechanisms underlie the control of working memory and attention

Article 31 March 2021

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Code availability

Custom analysis code was written using MATLAB (v. 2018a). MATLAB scripts employed are available from the corresponding author upon reasonable request.

References

  1. Andersen, R. A., Essick, G. K. & Siegel, R. M. Encoding of spatial location by posterior parietal neurons. Science230, 456–458 (1985). CASPubMedGoogle Scholar
  2. Avillac, M., Deneve, S., Olivier, E., Pouget, A. & Duhamel, J. R. Reference frames for representing visual and tactile locations in parietal cortex. Nat. Neurosci.8, 941–949 (2005). CASPubMedGoogle Scholar
  3. Batista, A. P., Buneo, C. A., Snyder, L. H. & Andersen, R. A. Reach plans in eye-centered coordinates. Science285, 257–260 (1999). CASPubMedGoogle Scholar
  4. Duhamel, J. R., Bremmer, F., Ben Hamed, S. & Graf, W. Spatial invariance of visual receptive fields in parietal cortex neurons. Nature389, 845–848 (1997). CASPubMedGoogle Scholar
  5. Fetsch, C. R., Wang, S., Gu, Y., DeAngelis, G. C. & Angelaki, D. E. Spatial reference frames of visual, vestibular, and multimodal heading signals in the dorsal subdivision of the medial superior temporal area. J. Neurosci.27, 700–712 (2007). CASPubMedPubMed CentralGoogle Scholar
  6. Galletti, C., Battaglini, P. P. & Fattori, P. Parietal neurons encoding spatial locations in craniotopic coordinates. Exp. Brain Res.96, 221–229 (1993). CASPubMedGoogle Scholar
  7. Jay, M. F. & Sparks, D. L. Auditory receptive fields in primate superior colliculus shift with changes in eye position. Nature309, 345–347 (1984). CASPubMedGoogle Scholar
  8. Lee, J. & Groh, J. M. Auditory signals evolve from hybrid- to eye-centered coordinates in the primate superior colliculus. J. Neurophysiol.108, 227–242 (2012). PubMedPubMed CentralGoogle Scholar
  9. Mullette-Gillman, O. A., Cohen, Y. E. & Groh, J. M. Eye-centered, head-centered, and complex coding of visual and auditory targets in the intraparietal sulcus. J. Neurophysiol.94, 2331–2352 (2005). PubMedGoogle Scholar
  10. Mullette-Gillman, O. A., Cohen, Y. E. & Groh, J. M. Motor-related signals in the intraparietal cortex encode locations in a hybrid, rather than eye-centered reference frame. Cereb. Cortex19, 1761–1775 (2009). PubMedGoogle Scholar
  11. Schlack, A., Sterbing-D’Angelo, S. J., Hartung, K., Hoffmann, K. P. & Bremmer, F. Multisensory space representations in the macaque ventral intraparietal area. J. Neurosci.25, 4616–4625 (2005). CASPubMedPubMed CentralGoogle Scholar
  12. Snyder, L. H., Grieve, K. L., Brotchie, P. & Andersen, R. A. Separate body- and world-referenced representations of visual space in parietal cortex. Nature394, 887–891 (1998). CASPubMedGoogle Scholar
  13. Sajad, A. et al. Visual-motor transformations within frontal eye fields during head-unrestrained gaze shifts in the monkey. Cereb. Cortex25, 3932–3952 (2015). PubMedGoogle Scholar
  14. Kiesel, A. et al. Control and interference in task switching-a review. Psychol. Bull.136, 849–874 (2010). PubMedGoogle Scholar
  15. Ruge, H., Jamadar, S., Zimmermann, U. & Karayanidis, F. The many faces of preparatory control in task switching: reviewing a decade of fMRI research. Hum. Brain Mapp.34, 12–35 (2013). PubMedGoogle Scholar
  16. Stoet, G. & Snyder, L. H. Neural correlates of executive control functions in the monkey. Trends Cogn. Sci.13, 228–234 (2009). PubMedPubMed CentralGoogle Scholar
  17. Stoet, G. & Snyder, L. H. Single neurons in posterior parietal cortex of monkeys encode cognitive set. Neuron42, 1003–1012 (2004). CASPubMedGoogle Scholar
  18. Kim, H. R., Pitkow, X., Angelaki, D. E. & DeAngelis, G. C. A simple approach to ignoring irrelevant variables by population decoding based on multisensory neurons. J. Neurophysiol.116, 1449–1467 (2016). PubMedPubMed CentralGoogle Scholar
  19. Sasaki, R., Angelaki, D. E. & DeAngelis, G. C. Dissociation of self-motion and object motion by linear population decoding that approximates marginalization. J. Neurosci.37, 11204–11219 (2017). CASPubMedPubMed CentralGoogle Scholar
  20. Rushton, S. K. & Warren, P. A. Moving observers, relative retinal motion and the detection of object movement. Curr. Biol.15, R542–R543 (2005). CASPubMedGoogle Scholar
  21. Warren, P. A. & Rushton, S. K. Optic flow processing for the assessment of object movement during ego movement. Curr. Biol.19, 1555–1560 (2009). CASPubMedGoogle Scholar
  22. Royden, C. S. & Connors, E. M. The detection of moving objects by moving observers. Vision Res.50, 1014–1024 (2010). PubMedGoogle Scholar
  23. Royden, C. S. & Holloway, M. A. Detecting moving objects in an optic flow field using direction- and speed-tuned operators. Vision Res.98, 14–25 (2014). PubMedGoogle Scholar
  24. Fajen, B. R. & Matthis, J. S. Visual and non-visual contributions to the perception of object motion during self-motion. PLoS ONE8, e55446 (2013). CASPubMedPubMed CentralGoogle Scholar
  25. Dokka, K., MacNeilage, P. R., DeAngelis, G. C. & Angelaki, D. E. Multisensory self-motion compensation during object trajectory judgments. Cereb. Cortex25, 619–630 (2015). PubMedGoogle Scholar
  26. MacNeilage, P. R., Zhang, Z., DeAngelis, G. C. & Angelaki, D. E. Vestibular facilitation of optic flow parsing. PLoS ONE7, e40264 (2012). CASPubMedPubMed CentralGoogle Scholar
  27. Eifuku, S. & Wurtz, R. H. Response to motion in extrastriate area MSTl: center-surround interactions. J. Neurophysiol.80, 282–296 (1998). CASPubMedGoogle Scholar
  28. Tanaka, K., Sugita, Y., Moriya, M. & Saito, H. Analysis of object motion in the ventral part of the medial superior temporal area of the macaque visual cortex. J. Neurophysiol.69, 128–142 (1993). CASPubMedGoogle Scholar
  29. Ilg, U. J., Schumann, S. & Thier, P. Posterior parietal cortex neurons encode target motion in world-centered coordinates. Neuron43, 145–151 (2004). CASPubMedGoogle Scholar
  30. Chen, X., DeAngelis, G. C. & Angelaki, D. E. Diverse spatial reference frames of vestibular signals in parietal cortex. Neuron80, 1310–1321 (2013). CASPubMedGoogle Scholar
  31. Chen, X., DeAngelis, G. C. & Angelaki, D. E. Eye-centered representation of optic flow tuning in the ventral intraparietal area. J. Neurosci.33, 18574–18582 (2013). PubMedPubMed CentralGoogle Scholar
  32. Berens, P. et al. A fast and simple population code for orientation in primate V1. J. Neurosci.32, 10618–10626 (2012). CASPubMedPubMed CentralGoogle Scholar
  33. Zaidel, A., DeAngelis, G. C. & Angelaki, D. E. Decoupled choice-driven and stimulus-related activity in parietal neurons may be misrepresented by choice probabilities. Nat. Commun.8, 715 (2017). PubMedPubMed CentralGoogle Scholar
  34. Britten, K. H., Newsome, W. T., Shadlen, M. N., Celebrini, S. & Movshon, J. A. A relationship between behavioral choice and the visual responses of neurons in macaque MT. Vis. Neurosci.13, 87–100 (1996). CASPubMedGoogle Scholar
  35. Dokka, K., DeAngelis, G. C. & Angelaki, D. E. Multisensory integration of visual and vestibular signals improves heading discrimination in the presence of a moving object. J. Neurosci.35, 13599–13607 (2015). CASPubMedPubMed CentralGoogle Scholar
  36. Sasaki, R., Angelaki, D. E. & DeAngelis, G. C. Processing of object motion and self-motion in the lateral subdivision of the medial superior temporal area in macaques. J. Neurophysiol.121, 1207–1221 (2019). PubMedPubMed CentralGoogle Scholar
  37. Chen, A., DeAngelis, G. C. & Angelaki, D. E. Functional specializations of the ventral intraparietal area for multisensory heading discrimination. J. Neurosci.33, 3567–3581 (2013). CASPubMedPubMed CentralGoogle Scholar
  38. Gu, Y. et al. Perceptual learning reduces interneuronal correlations in macaque visual cortex. Neuron71, 750–761 (2011). CASPubMedPubMed CentralGoogle Scholar
  39. Kohn, A., Coen-Cagli, R., Kanitscheider, I. & Pouget, A. Correlations and neuronal population information. Annu. Rev. Neurosci.39, 237–256 (2016). CASPubMedPubMed CentralGoogle Scholar
  40. Averbeck, B. B., Latham, P. E. & Pouget, A. Neural correlations, population coding and computation. Nat. Rev. Neurosci.7, 358–366 (2006). CASPubMedGoogle Scholar
  41. Moreno-Bote, R. et al. Information-limiting correlations. Nat. Neurosci.17, 1410–1417 (2014). CASPubMedPubMed CentralGoogle Scholar
  42. Dokka, K., Park, H., Jansen, M., DeAngelis, G. C. & Angelaki, D. E. Causal inference accounts for heading perception in the presence of object motion. Proc. Natl Acad. Sci. USA116, 9060–9065 (2019). CASPubMedGoogle Scholar
  43. Chen, X., DeAngelis, G. C. & Angelaki, D. E. Eye-centered visual receptive fields in the ventral intraparietal area. J. Neurophysiol.112, 353–361 (2014). PubMedPubMed CentralGoogle Scholar
  44. Chen, X., DeAngelis, G. C. & Angelaki, D. E. Flexible egocentric and allocentric representations of heading signals in parietal cortex. Proc. Natl Acad. Sci. USA115, E3305–E3312 (2018). CASPubMedGoogle Scholar
  45. Crespi, S. et al. Spatiotopic coding of BOLD signal in human visual cortex depends on spatial attention. PLoS ONE6, e21661 (2011). CASPubMedPubMed CentralGoogle Scholar
  46. Merriam, E. P., Gardner, J. L., Movshon, J. A. & Heeger, D. J. Modulation of visual responses by gaze direction in human visual cortex. J. Neurosci.33, 9879–9889 (2013). CASPubMedPubMed CentralGoogle Scholar
  47. Bernier, P. M. & Grafton, S. T. Human posterior parietal cortex flexibly determines reference frames for reaching based on sensory context. Neuron68, 776–788 (2010). CASPubMedGoogle Scholar
  48. Bremner, L. R. & Andersen, R. A. Temporal analysis of reference frames in parietal cortex area 5d during reach planning. J. Neurosci.34, 5273–5284 (2014). CASPubMedPubMed CentralGoogle Scholar
  49. Duncker, K. Uber induzierte Bewegung. Psychologische Forschung12, 180–259 (1929). Google Scholar
  50. Zivotofsky, A. Z. The Duncker illusion: intersubject variability, brief exposure, and the role of eye movements in its generation. Invest. Ophthalmol. Vis. Sci.45, 2867–2872 (2004). PubMedGoogle Scholar
  51. Gu, Y., Watkins, P. V., Angelaki, D. E. & DeAngelis, G. C. Visual and nonvisual contributions to three-dimensional heading selectivity in the medial superior temporal area. J. Neurosci.26, 73–85 (2006). CASPubMedPubMed CentralGoogle Scholar
  52. Fetsch, C. R., Pouget, A., DeAngelis, G. C. & Angelaki, D. E. Neural correlates of reliability-based cue weighting during multisensory integration. Nat. Neurosci.15, 146–154 (2012). CASGoogle Scholar
  53. Gu, Y., Angelaki, D. E. & DeAngelis, G. C. Neural correlates of multisensory cue integration in macaque MSTd. Nat. Neurosci.11, 1201–1210 (2008). CASPubMedPubMed CentralGoogle Scholar
  54. Chen, A., DeAngelis, G. C. & Angelaki, D. E. Representation of vestibular and visual cues to self-motion in ventral intraparietal cortex. J. Neurosci.31, 12036–12052 (2011). CASPubMedPubMed CentralGoogle Scholar
  55. Chen, A., DeAngelis, G. C. & Angelaki, D. E. Macaque parieto-insular vestibular cortex: responses to self-motion and optic flow. J. Neurosci.30, 3022–3042 (2010). CASPubMedPubMed CentralGoogle Scholar
  56. Chen, A., Gu, Y., Takahashi, K., Angelaki, D. E. & DeAngelis, G. C. Clustering of self-motion selectivity and visual response properties in macaque area MSTd. J. Neurophysiol.100, 2669–2683 (2008). PubMedPubMed CentralGoogle Scholar
  57. Bishop, C. M. Pattern Recognition and Machine Learning (Springer, 2006).

Acknowledgements

This work was supported by National Institutes of Health grants EY016178 (to G.C.D.) and DC014678 (to D.E.A.), the Uehara Memorial Foundation (to R.S.), the Japan Society for the Promotion of Science (to R.S.) and an NEI CORE grant (EY001319). We thank D. Graf, S. Shimpi and E. Murphy for excellent technical support and J. Wen and A. Yung for programming support.

Author information

  1. Ryo Sasaki Present address: Department of Neuroscience, Graduate School of Medicine, Kyoto University, Kyoto, Japan

Authors and Affiliations

  1. Department of Brain and Cognitive Sciences, Center for Visual Science, University of Rochester, Rochester, NY, USA Ryo Sasaki, Akiyuki Anzai & Gregory C. DeAngelis
  2. Center for Neural Science, New York University, New York, NY, USA Dora E. Angelaki
  1. Ryo Sasaki