The main challenge in utilizing deep learning (DL) for clinical diagnostic support is its lack of explainability and interpretability. Recent approaches aim to explain DL decisions from electrocardiogram (ECG) analysis by tracing model explanations back to beat segments. Fibrillatory (F) waves, as a main characteristic of atrial fibrillation (AF), are irregularly distributed over the signal and have not yet been considered. Using 477 publicly available AF ECGs, we systematically investigated the relationship between F waves and reliable model explanations. F waves were detected using peak detection after removing beat-aligned QT templates. We employed a convolutional neural network , derived from an explainable ECG architecture (xEC-GArch), which uses self-learned morphology features for AF detection. Analysis of variance revealed an increased mean relative relevance (rR) of the F waves compared to the rR of the full waveform (+13.5 %, p