The metabolic serum marker HbA1c has been associated with both impaired cognitive performance and altered white matter integrity in patients suffering from diabetes mellitus. However, it remains unclear if higher levels of HbA1c might also affect brain structure and function in healthy subjects. With the present study we therefore aimed to investigate the relationship between HbA1c levels and cognitive performance as well as white matter microstructure in healthy, young adults. To address this question, associations between HbA1c and cognitive measures (NIH Cognition Toolbox) as well as DTI-derived imaging measures of white matter microstructure were investigated in a publicly available sample of healthy, young adults as part of the Humane Connectome Project (n = 1206, mean age = 28.8 years, 45.5% male). We found that HbA1c levels (range 4.1–6.3%) were significantly inversely correlated with measures of cognitive performance. Higher HbA1c levels were associated with significant and widespread reductions in fractional anisotropy (FA) controlling for age, sex, body mass index, ethnicity, and education. FA reductions were furthermore found to covary with measures of cognitive performance. The same pattern of results could be observed in analyses restricted to participants with HBA1c levels below 5.7%. The present study demonstrates that low-grade HbA1c variation below diagnostic threshold for diabetes is related to both cognitive performance and white matter integrity in healthy, young adults. These findings highlight the detrimental impact of metabolic risk factors on brain physiology and underscore the importance of intensified preventive measures independent of the currently applied diagnostic HbA1c cutoff scores.