Recent studies revealed a positive influence of physical activity on cognitive functioning in older adults. Studies that investigate the behavioral and neurophysiological effects of type and long term duration of physical training, however, are missing. We performed a 12-month longitudinal study to investigate the effects of cardiovascular and coordination training (control group: relaxation and stretching) on cognitive functions (executive control and perceptual speed) in older adults. We analyzed data of 44 participants aged 62-79 years. Participants were trained three times a week for 12 months. Their physical and cognitive performance was tested prior to training, and after 6 and 12 months. Changes in brain activation patterns were investigated using functional MRI. On the behavioral level, both experimental groups improved in executive functioning and perceptual speed but with differential effects on speed and accuracy. In line with the behavioral findings, neurophysiological results for executive control also revealed changes (increases and reductions) in brain activity for both interventions in frontal, parietal, and sensorimotor cortical areas. In contrast to the behavioral findings, neurophysiological changes were linear without indication of a plateau. In both intervention groups, prefrontal areas showed decreased activation after 6 and 12 months when performing an executive control task, as compared to the control group, indicating more efficient information processing. Furthermore, cardiovascular training was associated with an increased activation of the sensorimotor network, whereas coordination training was associated with increased activation in the visual-spatial network. Our data suggest that besides cardiovascular training also other types of physical activity improve cognition of older adults. The mechanisms, however, that underlie the performance changes seem to differ depending on the intervention.