The objective was to analyze the impact of social isolation on moderate physical activity and factors associated with sedentary behavior of older adults during the COVID-19 pandemic. This was a cross-sectional study involving 111 older adults (aged 71.0 ± 6.87 years). The data were collected at two time points: in November 2019 and in June 2020. There was a decline in moderate physical activity when the minutes/week were compared before and during social isolation (p < .001). Sedentary behavior was associated with the condition of living alone. Older adults who lived alone were 3.29 times more likely to spend 4 hr or more in sedentary behavior than those who lived with a partner (95% confidence interval [1.01, 10.74]). Government agencies must establish PA-related health promotion strategies, especially in developing and low-income countries. Therefore, home exercises need to be encouraged to prevent the consequences of this pandemic period.