Purpose: This study aims to (a) examine elementary school students’ health-related fitness knowledge growth under one curriculum condition and (b) examine the impacts of student/school-level factors on health-related fitness knowledge and its growth rate in physical education. Method: We used an observational, longitudinal repeated-measures design, and conducted analyses on an existing dataset. Participants were 7,479 third, fourth, and fifth graders (48.9% girls) from 152 elementary schools. Measures were a knowledge test and sex at the student level, and socioeconomic data, academic performance, and student–faculty ratio at the school level. We ran three-level hierarchical linear models on the data. Results: Fitness knowledge growth was found to form a quadratic curve from third through fifth grades. School-level academic performance was positively associated with fitness knowledge. Sex was not associated with fitness knowledge or knowledge growth rate. Discussion: These findings contribute to the understanding of health-related fitness knowledge growth among elementary students.