Background: Adverse childhood experiences (ACEs) may be associated with worse physical and mental health in adulthood, and low physical activity engagement, but the relationships are not fully understood. Objectives: To establish the scope of the literature exploring associations between ACEs, physical activity, and physical and mental health. Methods: We conducted this scoping review according to PRISMA-ScR guidelines. We searched MEDLINE, Scopus, SPORTDiscus, and PsycInfo for relevant articles. Results: Eighteen studies were included, 17 observational and 1 randomized controlled trial. The majority of studies were cross-sectional and employed self-reported physical activity and ACE measures. Six studies explored physical health, 9 explored mental health, and 3 explored both. Associations between ACEs and poor physical health outcomes (poor self-reported physical health, inflammation, high resting heart rate, and obesity) were consistently weaker or attenuated among those who were physically active. Physical activity may also moderate the associations between ACEs and depressive symptoms, psychological functioning, and health-related quality of life. Conclusion: Associations between ACEs and poor physical and mental health were observed in those with less frequent physical activity engagement, though the majority of evidence relies on cross-sectional observational designs with self-report instruments. Further research is required to determine whether physical activity can prevent or treat poor physical and mental health in the presence of ACEs.