Encoding neural representations of time-continuous stimulus-response transformations in the human brain with advanced deep neural networks

Publikation: Beitrag in FachzeitschriftForschungsartikelBeigetragenBegutachtung

Beitragende

Abstract

Human behavior arises from the continuous transformation of sensory input into goal-directed actions. While existing analytical methods often break time into discrete events, the stages and underlying representations involved in stimulus-response (S-R) transformations within time-continuous, complex environments remain incompletely understood. Encoding models, combined with deep neural networks (DNNs) for feature generation, offer a promising framework for capturing these neural processes. While DNNs continue to improve in performance, it remains unclear whether these advances translate into closer alignment with human cognitive mechanisms. To address this, we collected fMRI data from participants (N = 23) as they played arcad video games and used DNN-based encoding models to predict human brain activity. We compared the prediction accuracy of features from three DNNs at different stages of development within our encoding model. The results show that the most advanced DNN provides the most predictive feature space for neural responses, while also revealing a closer hierarchical alignment between its internal representations and the brain’s functional organization. These results enable a more fine-grained characterization of time-continuous S-R transformations in high-dimensional visuomotor tasks, progressing along the dorsal visual stream and extending into motor-related regions. This approach highlights the potential of machine learning to advance cognitive neuroscience by enhancing the investigation of ecological valid experimental tasks.

Details

OriginalspracheEnglisch
Aufsatznummer IMAG.a.1142
Seitenumfang19
FachzeitschriftImaging neuroscience
Jahrgang4
PublikationsstatusVeröffentlicht - 30 Jan. 2026
Peer-Review-StatusJa

Externe IDs

ORCID /0009-0001-4895-0326/work/204616597
unpaywall 10.1162/imag.a.1142

Schlagworte

Schlagwörter

  • LSTM and fMRI, arcade games, deep neural networks, encoding models, neuroimaging