Analysis of human problem solving drafts: a methodological approach on the example of Rush Hour

Research output: Contribution to book/conference proceedings/anthology/reportConference contributionContributedpeer-review



Assessing the quality of a learner's solution for a given task is an essential step in analyzing a learner's performance. For a well-defined sequential problem, correctness and optimality of the solution as well as its length provide first simple and reasonable metrics. However, this ignores the fact that there are conceptually different errors that humans make when solving a problem. This work proposes a rule-based system of error categories which is able to classify conceptually different errors with respect to their (assumed) motive. The principles the categories are based on are valid for most well-defined sequential problems and can hence serve as a valuable tool in the analysis of human solutions for such a problem. In this work, the error category system is adapted to the game Rush Hour. We use the category system as a tool for a detailed analysis of 115 human solutions of a Rush Hour game. We found that the most common error type is based on a simple solving heuristic, but mainly occurs in the first half of the solution process. Other error types whose occurrence is numerically less dominant, are still found in the majority of the solutions. However, they occur in very specific game situations. As a first generalization approach of the category system, its application on a further dataset containing 56 different Rush Hour tasks and more than 31,000 human solutions yield promising results.


Original languageEnglish
Title of host publicationProceedings of the 40th Annual Meeting of the Cognitive Science Society, CogSci 2018
PublisherThe Cognitive Science Society
Number of pages6
ISBN (electronic)9780991196784
Publication statusPublished - 2018

Publication series

Series2018 40th Annual Meeting of the Cognitive Science Society - Changing/Minds (CogSci)


Title40th Annual Meeting of the Cognitive Science Society: Changing Minds, CogSci 2018
Duration25 - 28 July 2018
CountryUnited States of America

External IDs

ORCID /0000-0002-4280-6534/work/142251702



  • Error analysis, Error categories, Problem solving, Rush hour, Solution quality