Since students are familiar with machine learning (ML)-based applications in their everyday lives, they already construct mental models of how these systems work. This can result in misconceptions that influence the learning of correct ML concepts. Therefore, this study investigates the misconceptions students hold about the functionality of ML-based applications. To this end, we conducted semi-structured interviews with five students, focusing on their understanding of facial recognition and ChatGPT. The interviews were analyzed using an inductively developed code system and qualitative content analysis. This process identified six key misconceptions held by students: “Programmed Behavior,” “Exactness,” “Data Storage,” “Continuous Learning,” “User-trained Model,” and “Autonomous Data Acquisition”. These misconceptions include the notion that AI learns continuously during application, or that training data is saved and reused later. This paper presents the identified misconceptions and discusses their implication for the design and evaluation of effective learning activities in the context of ML.