There is a lack of research examining the social experiences of students with and without disabilities in regular physical education classes. Little is known, from the perspective of the student, about factors that affect his or her social experience while taking part in integrated physical education. This investigation examined the factors that have a positive and a negative effect on the social experiences of 12 elementary students who were thriving, struggling, or had disabilities in an integrated kindergarten and an integrated fourth-grade physical education class. This study utilized qualitative data collection methods that included observations and interviews with students and staff. Four factors were identified: (a) physical education teachers, (b) social substance of activities, (c) cultures, and (d) social skills of students. The physical education teacher factor was the only one found to have a positive influence on the social experiences of all students, whereas the other three factors differentially affected the social experiences among the 12 students.