Physical activity participation is linked with many benefits including a reduction in anxiety; it is, however, also important to explore aspects of activity that incite anxiety. One way to investigate sources of anxiety in physical activity is to use the critical incident technique (CIT). The purpose of this study was to explore anxiety-inducing events in physical activity settings and to evaluate the impact on future behavior. A total of 122 participants (M = 21.23 ± 1.77 years) completed an online survey asking when a physical activity setting incited anxiety using the CIT. Four common sources of anxiety were evident in the responses: fragile self-beliefs, social interaction and the threat of negative social evaluation, competition, and a lack of knowledge or unfamiliarity with surroundings. Tenets from achievement goal theory can help to explain the incitement of anxiety and can help shape physical activity settings to be more inclusive and welcoming for all.