This study explores the association between physical activity (PA), loneliness, and the presence of physical chronic impairments among single older adults. A longitudinal study (N = 575; mean age 76 ± 8 years) was conducted. The association between self-reported weekly minutes of moderate to vigorous PA, loneliness, and presence of physical impairments was assessed with multilevel analyses at baseline, 3 months, and 6 months. Improvements in moderate to vigorous PA were associated with decreases in loneliness (B = −0.09, SE = 0.04, p = .020); this association became nonsignificant when including the presence of physical impairments in the analyses (p = .824), which in itself was positively associated with loneliness (B = 0.51, SE = 0.10, p < .001). Findings indicate that physical impairments have a larger influence on loneliness than the level of PA. Interventions targeting PA and loneliness should tailor specifically to physical impairments.