The purposes of this study were to determine the impact of physical activity on three different executive functions (shifting, inhibition, and updating) and to examine whether cardiovascular fitness was a good mediator of the positive link(s) between these variables. Sixty-three young adults (18–28 years), 30 young-old adults (60–70 years) and 30 old adults (71–81 years) were divided into physically active and sedentary groups according to physical activity level (assessed from an accelerometer and the Historical Leisure Activity Questionnaire). Cardiovascular fitness was assessed by VO2max from the Rockport 1 mile. Each executive function was assessed through three different experimental tasks. ANCOVAs revealed that the effect of physical activity level was specific to the old adults and significant for inhibition, but not for updating and shifting. Mediation analysis showed that this positive effect in the old adults group was mediated by cardiovascular fitness level. The present findings highlight the positive linkages among physical activity, cardiovascular fitness, and inhibition in aging.