Multimodal Late Fusion Model for Problem-Solving Strategy Classification in a Machine Learning Game

Research output: Contribution to book/Conference proceedings/Anthology/ReportConference contributionContributedpeer-review

Contributors

Abstract

Machine learning models are widely used to support stealth assessment in digital learning environments. Existing approaches typically rely on abstracted gameplay log data, which may overlook subtle behavioral cues linked to learners’ cognitive strategies. This paper proposes a multimodal late fusion model that integrates screencast-based visual data and structured in-game action sequences to classify students’ problem-solving strategies. In a pilot study with secondary school students (N=149) playing a multitouch educational game, the fusion model outperformed unimodal baseline models, increasing classification accuracy by over 15%. Results highlight the potential of multimodal ML for strategy-sensitive assessment and adaptive support in interactive learning contexts.

Details

Original languageEnglish
Title of host publicationTwo Decades of TEL. From Lessons Learnt to Challenges Ahead
EditorsKairit Tammets, Sergey Sosnovsky, Rafael Ferreira Mello, Gerti Pishtari, Tanya Nazaretsky
PublisherSpringer-Verlag
Pages281–286
Number of pages6
ISBN (electronic)978-3-032-03873-9
ISBN (print)978-3-032-03872-2
Publication statusPublished - 2026
Peer-reviewedYes

Publication series

SeriesLecture Notes in Computer Science
Volume16064
ISSN0302-9743

External IDs

Scopus 105023373978
ORCID /0000-0003-3527-3204/work/216556525
ORCID /0000-0003-4725-9776/work/216556928

Keywords