Perceived Course Rigor in Sport Management: Class Level, Course Grades, and Student Ratings

in Sport Management Education Journal

Higher education in the United States, and sport management in particular, has faced contemporary attacks for its perceived lack of academic rigor. To investigate these criticisms, this study examined 830 students enrolled in 69 semester-long courses across four consecutive years in a single sport management program to measure perceived course rigor and its relationship to overall course ratings, course grades, and course level. Seven rigor questions were added to existing student ratings and distributed at the end of each semester. A factor analysis strongly supported the conceptualization of rigor utilized in the study. Pearson correlations indicated that student ratings and rigor were positively related. An ordinary least squares multiple regression also revealed that overall student ratings and course grades significantly aid in predicting course rigor. Pragmatically, the results suggest that sport management students appreciate rigorous courses and that faculty should strive to include elements of rigor into their courses without fear of retributional bias on student ratings.

Historically, discussions of academic rigor frequently use language that assumes the reader understands what is rigorous (Johnson, Weidner, Jones, & Manwell, 2019). In truth, academic rigor is an ambiguous construct that has remained elusive to define but essential to the academy (Graham & Essex, 2001). Due in part to growing criticisms of higher education, academic rigor has recently drawn great attention (Asher, 2013; Collier, 2013; Desilver, 2017). The American higher education system is routinely attacked for its lack of challenging work, declining standards, and inflation of student grades (Arum & Roksa, 2011). Understanding rigor has become a necessity as “everybody seems to be either promoting rigor, demanding rigor, or deploring the lack of rigor in American schools” (Jacobs & Colvin, 2009, p. 1). Several scholars (Arum & Roksa, 2011; Brandon, 2010; Weissberg, 2010) have authored books that discuss how today’s students are more interested in the social aspects of college rather than academic endeavors. The prioritizing of college social life over academics has become a focal point for many politicians who suggest the need for testing benchmarks, initiatives, and standards to evaluate if students are engaged in a rigorous curriculum (Snider, 2009a, 2009b). College faculty responsible for individual course rigor seem to “know it [rigor] when they see it’, but few felt confident in their ability to define it” (Draeger, Prado Hill, Hunter, & Mahler, 2013, p. 269).

The ambiguity of understanding and measuring academic rigor contributes to the widespread culpability for underprepared students at the secondary and college educational levels (Kaplan, 2017). It is relatively common for college educators to blame high school teachers for underprepared students. For example, 42% of high school students have been estimated by college faculty to be underprepared for college (Wyatt, Wiley, Camara, & Proestler, 2011). Kirst (2009) estimated that 30% of 4-year college students and 60% of community college students required remedial math or English. Demands for academic rigor appear more pronounced when students enter college. These statements have been made by critics who believe higher education has lost its focus, ultimately capitulating to capitalistic pressures that reinforce degrees over learning, diminishing emphasis on rigor, and inflating grading (Asher, 2013; Collier, 2013).

Background

Course Rigor in Higher Education

As the United States continues to decline in international educational rankings (Desilver, 2017), faculty and administrators have struggled to define and evaluate rigor. Throughout the last three decades, rigor has been an evolving concept that has potentially been deconstructed over the years based on the necessity of individualistic points of view. While a broad definition of academic or institutional rigor may refer to the overall college experience, efforts to define and measure rigor may be most successful within individual courses, as this single unit–level analysis provides researchers with the ability to analyze course content and instructor influence (Johnson et al., 2019). Regarding course content, Science, Technology, Engineering, and Math courses are generally thought to be conceptually more challenging than other types of courses (Achen & Courant, 2009; Payton, White, & Mullins, 2017). Instructor control of a course’s overall design, including assessments, pace, and interpersonal communication, is considered as part of instructor influence. Decisions regarding course design are vital elements of rigor, as it is likely that two different faculty members teaching different sections of the same course would be evaluated differently by students (Draeger et al., 2013).

Draeger, Prado Hill, and Mahler (2015) found that faculty and students differed in their perspectives on a course’s rigor. Faculty considered course content and higher-order thinking to be the key benchmarks for measuring the rigor of their courses, whereas students perceived the influence of the instructor (e.g., grading standards, level of difficulty, workload, etc.) rather than the content contributed more to course rigor. Students also believed that rigor could be captured by asking, “How hard is it to get a good grade?” (Draeger et al., 2015, p. 216). While grades are intended to represent student performance and may be a retroactive proxy for content and instructor influence, they are rarely associated with course rigor by administrators, faculty, and future employers. Due to this lack of association, attempts to define course rigor typically include what happens during the course rather than its outcome (i.e., grade).

In one of the few studies that has attempted to specifically define course rigor, Graham and Essex (2001) concluded that critical thinking, high expectations, emphasizing process more than product, and cognitive development encompass rigor. Conversely, memorization, regurgitation, and grades did not define rigor. Similarly, in likely the most comprehensive attempt to define course rigor, Draeger et al. (2013) noted that courses are most rigorous “when students are actively learning meaningful content with higher-order thinking at the appropriate level of expectation within a given context” (p. 267). Those definitions offer similar conceptualizations of rigor. When combined with other sources, however, there appear to be additional concepts that complement, reinforce, and redefine rigor. Using the previous definitions, as well as the concepts outlined in Johnson et al. (2019), the following components of rigor appear most frequently.

Challenge

The term challenge is commonly used in definitions of rigor (Thomas & Bolen, 1998). Winston et al. (1994) considered challenge a norm where excellence is expected through elevated, realistic, and fast-paced standards. Draeger et al. (2013, 2015) suggested the terms academic challenge and academic rigor were synonymous in their work to define rigor. Descriptions of a challenging course suggest high standards and increased accountability (Bursuck, 1994), increased depth (Rossman & Wilson, 1996), and a focus on process more than product (Graham & Essex, 2001).

Critical thinking

As a complement to challenge, an emphasis on critical thinking is perhaps the most commonly noted component of rigor. Braxton (1993) defined critical thinking as higher-order processing where concepts are scrutinized, challenged, and critically analyzed, and not merely understood at basic levels. Other researchers have echoed this definition and noted a conceptual transformation of material through ongoing examination in different or creative ways (Cope & Staehr, 2005; Trigwell & Prosser, 1991). It is difficult to find analysis on rigor that does not directly or indirectly encourage critical thinking. Taylor and Rendon (1991) argued that student-focused critical thinking involves enhanced interpretive skills, while Graham and Essex (2001) found that faculty-focused critical thinking was central to conceptions of rigor and that it could be enhanced by requiring students to “examine multiple issues from multiple perspectives” (p. 334), cite readings, and actively reflect. Multiple perspectives could be examined through different lenses of power and influence (race, gender, socioeconomics, etc.; Zakus, Malloy, & Edwards, 2007).

Mastering complex material

Course content may be considered rigorous depending on an individual’s strengths and interests and can vary from basic to complex (Park, Lubinski, & Benbow, 2008). Courses considered rigorous are thought to include high-quality materials at the high end of students’ current comprehension levels to encourage relationships among ideas rather than memorization of rote material (Graham & Essex, 2001). This emphasis on complex rather than simple is highlighted by a focus on meaningful content to stimulate the brain and create sufficient challenge (Jensen, 2005). As Johnson et al. (2019) noted, defining complexity is challenging but objective markers may occur:

Determining what is complex for a classroom of students, however, is difficult because academic abilities differ. In courses like math or foreign languages, routine increases in complexity naturally occur, as course progression requires that complex material build from previously learned concepts. (p. 94)

Instructor design and subjective judgment can also influence what might be considered complex and high quality (Braxton, 1993). Thus, rigor is increased through these determinations of complexity (objective and subjective) and continual work toward their mastery (Graham & Essex, 2001; Johnson et al., 2019).

Time and labor intensive (quantity)

Within a college course, it is generally expected that energy and time are required to complete rigorous assignments (Brooks & Brooks, 2001; Winston et al., 1994). Traditionally, higher education institutions loosely recommend 2–3 hr of study outside of class for every hour in class (Nelson, 2010). These recommendations are now unrealistic given that college students have declined from 40 hr per week of coursework in 1961 to 27 hr per week by 2004, and only 20% of college students spend more than 20 hr of week studying (Babcock & Marks, 2011). This decline has been exacerbated by the emphasis on college as a primarily social endeavor, as well as the emphasis on grade inflation and retention seen in many college business models (Arum & Roska, 2011). Despite this decline in the quantity of time students now spend on studying, 78% of faculty reported that the “number of hours a week students spend on class preparation” (Draeger et al., 2013, p. 272) was a key indicator of academic rigor.

Production of credible work (quality)

Ideally, the result of time and effort demonstrates a clear output that has challenged students in complex ways (Bursuck, 1994). Marsh (2001), for example, found that students perceive the number of hours devoted to a task positively if the tasks are good hours—meaning challenging and requiring an appropriate amount of effort/involvement. Moreover, it is generally agreed that the work produced in a rigorous environment goes well beyond the routine and monotonous expectations of busy work, which could be described as quantity without quality (Bruner, 1996; Graham & Essex, 2001).

In aggregate, these five themes of rigor appear to reflect the majority of ideas and definitions encompassed within the rigor literature. While some definitions highlight or isolate a specific theme more than others, it is rare to find a conceptualization of course rigor that does not include one or more of these concepts as fundamental. Furthermore, it is not enough to have one or two components of rigor, nor would it seem possible to include all components to the same degree. Rather, a rigorous course is one that leverages these components of rigor as much as possible. Thus, the operational definition of a rigorous course adopted for this study and developed from the work of Johnson et al. (2019) is the inclusion of critical thinking, challenge, mastering complex material, and producing credible work in a way that is time and labor intensive. The more each rigor component is included, the more rigorous the course (Johnson et al., 2019).

Course Rigor in Sport Management

The first North American Society for Sport Management (NASSM) Conference occurred in 1987, which sport management scholars acknowledge as the starting point for the field. Chelladurai (1992) remarked that the field developed at a time when the number of sport organizations was increasing, the attractiveness of sport management to students (particularly males) was growing, and the ability of programs to get students hired was a key strength. Weese (2002) observed that sport management was trending toward becoming a popular major, and Mahony (2008) reported that sport management was among the top 10 requested majors by incoming first-year students. As might be expected, the significant growth of sport management during a two-decade span has involved some issues. Of note, especially as it pertains to perceived course rigor, are issues with curriculum and professionalization of faculty that have been highlighted by several prominent sport management scholars in their Earle F. Zeigler Award addresses—which is the most prestigious award given to a NASSM member for career-long contributions to the sport management field (Boucher, 1998; Cuneen, 2004; Mahony, 2008; Olafson, 1995; Weese, 2002).

Regarding curriculum, Parkhouse (1996) noted that the first sport management instructors were physical education professionals. Thus, sport management courses were initially taught by individuals who had little background or training in understanding and teaching the business of sport. Boucher (1998) challenged the field to make sport management classrooms sites in which students critically analyzed the sport industry. To that end, he advocated for stronger sport management curriculum and accreditation processes. Cuneen (2004) echoed Boucher’s views in her Zeigler address 6 years later, in which she encouraged programs to seek accreditation. She believed that curricular standards and accreditation are important for sport management as it attempts to garner respect from colleagues across campus and combats an easy major label.

Today’s sport management accreditation body, the Commission on Sport Management Accreditation (COSMA), evaluates sport management curriculum at the program level. COSMA was predated by several partnerships between the National Association for Sport and Physical Education and NASSM. While the accreditation process includes assessments of academic quality, specific assessments of rigor are limited. Given the infancy of COSMA and empirical investigations of rigor, it is unknown how the accreditation process might evolve as a result of research like that found in this study. To date, however, COSMA evaluates many of the historical limitations of the field through their characteristics of excellence that include institutional influences, curricular considerations, and practical applications (Commission on Sport Management Accreditation, 2019).

Relative to sport management faculty, the first sport management graduates were trained by individuals who were not sport management scholars (Parkhouse, 1996). Weese (2002) explained that the field of sport management grew so rapidly that the number of job postings outnumbered the available pool of graduates. Six years later, Mahony (2008) expressed concern over the quality of sport management faculty. Not only was he concerned with the quality of faculty working in the field, but also, he took issue with the research that was being conducted (e.g., too much of a positivistic focus). Furthermore, James (2018) analyzed the curriculum of sport management doctoral programs and determined that several shortcomings relating to credit hours, coursework, and scientific knowledge existed.

In summary, evaluating the previously mentioned elements of rigor (i.e., challenge, critical thinking, mastering complex material, production of credible work, and time and labor intensity) in sport management curriculum cannot be done without acknowledging the issues that sport management has experienced through its rapid growth. The quality of education, as well as the instructors charged with teaching, has been criticized by seminal scholars in the field. It is imperative that these concerns be addressed through an ongoing evaluation of sport management curricula noting specific concepts, such as course rigor.

Student Ratings

Defining rigor is challenging, and evaluating it is even more difficult (Johnson et al., 2019). Because the delivery mechanism of the rigor questions in this investigation was student ratings (i.e., student course evaluations) and because student rating questions were compared with rigor questions, course level, and course grades, it is important to discuss student ratings. Marsh (1987) concluded more than 30 years ago that student rating research was “the most thoroughly studied of all forms of personnel evaluation, and one of the best in terms of being supported by empirical research” (p. 369). Since this proclamation, researchers have continued to study these evaluative tools in a variety of contexts, making for voluminous amounts of evidence to determine the validity of student ratings. Though a comprehensive review of student ratings literature is beyond the scope of this article, it is important to note several summative works that detail the research (Benton & Cashin, 2012; Benton & Ryalls, 2016; Cohen, 1980; Feldman, 1989, 2007; Hobson & Talbot, 2001; Marsh, 2007; Stark & Freishtat, 2014).

The most common misconceptions relative to student ratings include students being unqualified to make judgments, faculty are evaluated based on popularity, data produced are unreliable, students cannot appreciate good teaching, and students desire easy courses. An emphasis on student ratings has resulted in grade inflation (Benton & Cashin, 2012; Benton & Ryalls, 2016; Feldman, 2007; Hoyt & Lee, 2002a; Marsh, 2007). Feldman (2007) explained that the teacher (person) and course (structure and content) are the two most common areas evaluated by students. Instructor variables that have historically shown no significant influence on student ratings include age, gender, personality, race, research productivity, and teaching experience (Centra, 1993; Centra & Gaubatz, 2000; Feldman, 1983, 1986, 1987, 1992, 1993; Li, 1993; Marsh & Hattie, 2002; Marsh & Hocevar, 1991; Renaud & Murray, 1996). However, more recent research has found that a bias may exist for male instructors (Boring, Ottoboni, & Stark, 2016) and against racial minority instructors (Smith & Hawkins, 2011). Variables with little or no impact on student ratings include age, gender, grade point average (GPA), students’ year in school, and personality (Abrami, 2001; Abrami, Perry, & Leventhal, 1982; Braskamp & Ory, 1994; Centra, 1993; Feldman, 1977, 1993, 2007; Marsh & Dunkin, 1992, 1997; Marsh & Roche, 2000; McKeachie, 1979).

There are four variables specific to the instructor or conditions of the course that do influence student ratings. These variables include appealing content (e.g., students with interest in a course or subject matter tend to provide higher ratings), expected grade (e.g., ratings are higher when students expect a high grade), instructor’s expressiveness (e.g., outgoing and energetic instructors who display great enthusiasm receive higher evaluations), and the instructor’s status (e.g., graduate student teachers tend to receive lower ratings than faculty; Braskamp & Ory, 1994; Centra, 2003; Feldman, 1976; Howard & Maxwell, 1980, 1982; Marsh & Dunkin, 1992, 1997; Marsh & Roche, 2000; Naftulin, Ware, & Donnelly, 1973). Three variables specific to the course also influence student ratings. First, higher-level courses, including graduate courses, tend to receive higher student ratings (Aleamoni, 1981; Braskamp & Ory, 1994; Feldman, 1978). Second, classes with fewer students receive higher student ratings, although the relationship is relatively weak or sometimes absent (Feldman, 1984; Hoyt & Lee, 2002a). Third, academic discipline appears to have a small impact on student ratings with Science, Technology, Engineering, and Math disciplines receiving slightly lower ratings than the social sciences or the arts (Braskamp & Ory, 1994; Cashin, 1990; Centra, 1993, 2009; Hoyt & Lee, 2002b; Marsh & Dunkin, 1992; Sixbury & Cashin, 1995).

Perceived course rigor and student ratings

Relative to their use, differences in student ratings are as widespread as the number of institutions that implement them. Each institution decides what questions best capture the information they value, and in some cases, questions change from instructor to instructor and course to course. Online courses compared with lab or applied courses offer different types of learning environments. While the specifics of the questions and courses differ, it is generally understood that student ratings include some combination of five themes: course organization and planning, clarity and communication skills, teacher/student interaction or rapport, grading and/or evaluations, and course difficulty or workload (Braskamp & Ory, 1994; Centra, 1993). The final theme—course difficulty or workload—is the closest proxy to rigor. Historically, assessing difficulty/workload on student ratings has produced mixed results. Most studies suggest a positive correlation with student ratings (Centra, 1993; Feldman, 2007; Marsh, 2001; Marsh & Roche, 2000), particularly if students perceive the workload contributed to their success in the course (Marsh, 2001) or they enjoyed the discipline (Hoyt & Lee, 2002b).

As the current study was conducted with a pragmatic goal of improved teaching, it was reasonable to investigate if course level was related to rigor. While it may appear a vague term, course level in this study refers to the number associated with a course and the students most commonly found in such courses. For example, 100-level courses are typically taken first and contain mostly first-year students. Each successive level then corresponds to the traditional student labels recognized within higher education: 200 = sophomores, 300 = juniors, 400 = seniors, and 500/600 = graduate students. Most of the literature investigating course level relative to student ratings suggest that there is little, if any, relationship and that the significant findings that do exist are with higher-level courses, which have been found to generate marginally higher student ratings than lower-level courses (Aleamoni, 1981; Braskamp & Ory, 1994; Feldman, 1978).

Perceived course rigor and course grades

Because the evaluation of rigor was conducted as part of student ratings and because grades are one of the independent variables evaluated in this article, it is important to discuss the link between grades and student ratings. The research linking student ratings and grades, however, is somewhat inconsistent. Three distinct theories help to explain these inconsistencies. First, validity theory (Marsh, 1987; Marsh & Dunkin, 1992) postulates that any correlation among student ratings and grades is due to quality of teaching. Logically, this theory assumes that students who earn higher grades do so because the instructor was a good teacher and those students would subsequently provide the highest ratings to the best teachers. The second theory has been coined the student characteristics hypothesis (Marsh, 1989; Marsh & Dunkin, 1992). This theory suggests that student ratings, and consequently course grades, may be a result of characteristics other than the quality of teaching (e.g., academic discipline of the course, interest in the subject matter, and/or taking the course as an elective). The third theory is known as the grading leniency hypothesis (Marsh & Dunkin, 1992). This theory posits that retributional bias occurs that cause students to punish strict teachers and provide strong ratings to easy teachers. The tendency for attributional bias also exists, which suggests students blame others for their failure and take credit for their success (Feldman, 2007; Gigliotti & Buchtel, 1990; Theall, Franklin, & Ludlow, 1990a, 1990b). The anecdotal support for this theory is widespread as it may seem reasonable to conclude that less rigorous courses with inflated grades would produce high student ratings. Despite frequent criticisms of higher education from those who assume the grading leniency hypothesis is real (Benton & Cashin, 2012; Collier, 2013), there is little support for this theory. Finally, it is noteworthy that while the term hypothesis is used for both the student characteristics and grading leniency concepts, both operate as theories that have garnered support in the literature (Johnson et al., 2019).

Each of these theories offers a unique perspective on student rating intentions and often connects them to expected grades. It is critical to note that this investigation did not examine expected grades as most prior research linking grades and student ratings had done (Benton & Cashin, 2012; Centra, 2003; Feldman, 2007). Instead, actual course grades were available for analysis and ultimately were compared with rigor evaluations. This approach is novel and allows for an additional layer of comparing ratings to grades that has not been found in the previous literature relative to the three previously mentioned theoretical propositions.

Purpose and Hypotheses

The purpose of this study is aligned with the need to ensure rigor in higher education courses amidst a wave of recent higher education criticisms (Arum & Roksa, 2011; Benton & Cashin, 2012; Feldman, 2007; Johnson et al., 2019). In particular, as the field of sport management continues to battle negative academic stereotypes (James, 2018; Mahony, 2008), the need to assess and ensure rigor is paramount. Faculty can benefit by knowing how students perceive rigor in their classrooms and how those perceptions of rigor are related to course characteristics, course grades, and student ratings. Knowing such information allows faculty to reconceptualize their courses relative to these variables and student needs, and ultimately to ensure sport management education is meeting the expectations of the academy. Thus, this study had two purposes. First, the researchers sought to determine if the five components of rigor and their associated seven questions created by Johnson et al. (2019) were applicable to the field of sport management via delivery as questions on student ratings. Second, this investigation assessed academic rigor in relationship to overall course ratings, course GPA, and grade level. The following hypotheses guided analysis.

H1: Perceived course rigor will have a positive significant relationship with overall student ratings.

H2: Perceived course rigor will not be significantly related to course GPA (grades).

H3: Perceived course rigor will be significantly higher for juniors, seniors, and graduate students (300+ level) than for freshmen and sophomores (100 and 200 level).

H4: Perceived course rigor will be predicted by overall student ratings, but not by semester, year, or course GPA (grade).

Method

Before analysis of sport management courses was conducted, a factor analysis was used to determine if the dimensionality and internal consistency of the questions confirmed a clear one-factor solution specifically in a sport management curriculum. This study adopted academic rigor questions created by Johnson et al. (2019), which demonstrated exceptionally strong support for the conceptualization of rigor into five themes and the subsequent questions relative to those five themes. The Johnson et al. (2019) study meticulously explains how the rigor questions were created. The rigor questions are listed in Appendix. After confirming that the rigor questions successfully captured perceived course rigor (see “Results” section), the primary focus of the study evaluated rigor relative to student ratings, course grades, and course level.

Sample and Procedures

During the course of seven consecutive semesters (Fall 2014 to Fall 2018), a total of 69 individual sport management courses were investigated. A total of 830 students provided data by voluntarily completing the student ratings instrument. Only students majoring in sport management were in these courses. The context was a sport management program within a School of Kinesiology and a College of Health at a Midwestern public university (approximately 22,000 students) with a Carnegie Foundation classification of research university-high research activity and a student–faculty ratio of approximately 17:1. Because the curriculum has very few online courses and because the online student rating questions are different than those for in-person courses, only in-person courses were examined. The assessment of rigor was originally designed as a school evaluation to assist faculty, and not as a research study. Thus, collection of data occurred during the course of seven consecutive semesters, but the research design was not conceptualized until the data already existed within the School of Kinesiology. Therefore, the design of this study was historical/archival such that existing data informed the study.

The existing course (six questions) and instructor (six questions) items were combined with the seven rigor questions to create a modified student ratings form (see Appendix). A 5-point Likert scale (1 = strongly disagree, 5 = strongly agree) was utilized. Comment boxes were available for students to add qualitative data that were subsequently distributed to faculty as part of the normal student ratings feedback process. Rigor questions were only added to student rating questions in courses where faculty volunteered. During data collection, there were six full-time faculty in the department. One faculty member had tenure, three faculty were in their first 3 years on a tenure-track appointment, and two faculty were not on a tenure-track appointment (teaching-only appointment). Throughout the time investigated, there were three adjunct faculty used to cover sport management courses in the department. The adjunct faculty did not utilize the rigor questions, so their evaluations were not part of this study.

Course rating questionnaires were distributed by the university to students via e-mail approximately 2 weeks before the end of each semester. Although students could complete an evaluation in multiple courses, each completed response in each course was treated as an independent participant because the student was evaluating one specific course at a time. Ratings were completed using Qualtrics survey software (Provo, UT) with three follow-up e-mails sent to students who had not completed the ratings form. Ratings took approximately 5–10 min to complete. Data from the previous 3 years (seven semesters) were gathered from the university database with the assistance of the director of research and academic effectiveness. To ensure confidentiality, no individual course sections, student names, or faculty names were attached to the data.

Data Analysis

Frequencies and measures of central tendency were combined with descriptive information to provide further context to the analysis. H1 and H2 were addressed using Pearson correlations to determine the relationships among perceived course rigor, student ratings, and course GPA (grades). H3 was investigated using a one-way analysis of variance by course level to determine if students in upper-class courses report significantly more rigor than students in lower-level courses. To determine if semester, course level, course GPA, and student ratings predicted course rigor (H4), an ordinary least-squares multiple regression analysis was conducted.

Results

The seven rigor questions established by Johnson et al. (2019) were analyzed through maximum-likelihood factor analysis to determine if the questions captured rigor in a sport management context. Initial measures using a Kiser–Meyer–Olkin verified high sampling adequacy (Kiser–Meyer–Olkin  = 0.9). Equal variances across samples were also confirmed using Bartlett’s Test of Sphericity (x2 = 2,089.167, p < .01). The factor analysis revealed high internal consistency with 76.04% of variance explained and a Cronbach’s alpha at .96. The seven items loaded into a single one-factor solution termed Rigor. Factor loading and error variance for each of the seven questions are provided in Table 1.

Table 1

One-Factor Solution With Loading and Reliability (N = 69 courses)

Factor loadingR2
The instructor(s) encouraged critical thinking and problem solving..97.94
The course content encouraged critical thinking and problem solving..98.95
The course was academically challenging..80.82
The instructor(s) encouraged me to master complex material..94.87
The instructor’s(s’) requirements were time and labor intensive..74.84
Learning the course material was time and labor intensive..72.86
The instructor(s) expected quality work..91.82

After the determination of fit from the factor analysis, descriptive information indicated courses were spread across 4.5 academic years (Fall 2014 through Fall 2018) and ranged from 4 (Fall 2016, Spring 2017) to 12 (Spring 2016) in a given semester. The most courses evaluated were from the 300 level (22), followed by 400 level (21), graduate courses (14), and 100/200 level (12). Table 2 displays semester and course-level information. The mean scores for perceived course rigor, instructor rating, course rating, overall rating, course GPA, and enrollment appear in Table 3.

Table 2

Frequencies—Semester and Course Level

Frequency%
Semester
 Fall 20141014.5
 Spring 2015913.0
 Fall 2015811.6
 Spring 20161217.4
 Fall 201645.8
 Spring 201745.8
 Fall 2017710.1
 Spring 201857.2
 Fall 20181014.5
Total69100
Course level
 Freshmen/Sophomore—100/200 level1217.4
 Junior—300 level2231.9
 Senior—400 level2130.4
 Graduate—500–600 levels1420.3
Total69100
Table 3

Descriptive Statistics (N = 69 courses)

M (0–5 scale)SD
Course rigor4.160.54
Instructor rating4.360.61
Course rating4.330.59
Overall rating4.340.60
Course grade point average (grade)3.32 (B+)0.41
Enrollment26.1612.67

Pearson correlation coefficients were calculated to determine relationships among perceived course rigor, instructor rating, course rating, overall rating, course GPA, and enrollment. A p value of .05 was required, and Type I errors were controlled using the Bonferroni approach. Table 4 includes the correlation coefficients. Of the 15 correlations, seven were significant at the p < .01 level including very strong correlations among perceived course rigor, instructor rating, course rating, and overall rating. This finding confirmed H1. Perceived course rigor, however, was unrelated to course GPA and enrollment. This result confirmed H2.

Table 4

Pearson Correlations

Course rigorInstructor ratingCourse ratingOverall ratingClass GPA (grade)Enrollment
Course rigor1.00
Instructor rating.90*1.00
Course rating.90*.98*1.00
Overall rating.91*.99*.99*1.00
Class GPA (grade)−.24−.11−.16−.141.00
Enrollment−.13−.05−.01−.03−.52*1.00

*p < .01

The H3 was tested using a one-way analysis of variance to assess the relationship between perceived course rigor and course level. The five levels of the independent variable, course level were 100, 200, 300, 400, and graduate. Course rigor was the dependent variable. The analysis of variance was significant at .05 level F(3, 65) = 15.98, p < .001. To evaluate pairwise mean differences, Dunnett’s C test was used as variances among class levels were not assumed. Significantly less rigor existed for 300-level (i.e., junior level) courses than for 400-level (i.e., senior level) and graduate-level courses. There were no significant differences in perceived course rigor among 100/200-level courses and any other courses. Thus, H3 was rejected. Table 5 displays the means, SDs, and 95% confidence intervals for each course level.

Table 5

Descriptive Results of Class-Level Analysis of Variance

95% confidence interval
Course levelNMSDSELowerUpper
100/200 level124.120.260.073.964.29
300 level223.830.750.163.504.16
400 level214.400.350.084.244.56
Graduate level144.360.250.074.224.51
Total694.160.540.074.034.29

An ordinary least-squares multiple regression was used to test which variables were significant predictors of perceived course rigor. The criterion variable was perceived course rigor, while the predictor variables were semester, course level, course GPA (grade), and overall student ratings. The linear combination of predictor variables was significantly related to perceived course rigor, F(4, 64) = 82.94, p < .01. The sample multiple correlation coefficient was .92 and R2 was .838, indicating 83.8% of the course rigor variance can be accounted for by the linear combination of predictors. Class GPA (grade) and overall student rating were significant predictors, while semester and course level were not significant. Table 6 provides the regression summary.

Table 6

Summary of Least-Squares Regression for Variables Predicting Course Rigor

VariableBSEβtSignificance
Semester−0.040.06−0.03−.65.521
Course level−0.020.02−0.04−.86.395
Class GPA (grade)−0.180.07−0.14−2.60.012*
Overall rating0.800.050.8917.27<.001**

*p < .05. **p < .01.

Discussion

Pearson correlations measuring the relationship between perceived course rigor and course rating (r = .90), as well as overall rating (r = .91) and instructor rating (r = .90), all represented statistically significant strong-positive relationships. These findings suggest that courses with high course and instructor ratings are typically perceived as more rigorous, that rigorous courses tend to receive higher student ratings, and that instructors who use rigor receive higher student ratings. This result is encouraging, as it could help alleviate concerns that some faculty may have relative to course rigor leading to poor student evaluations/ratings (Asher, 2013; Collier, 2013). Support for H1 reinforces Marsh’s (2001) notion that students value challenging work, effort, and prefer to be involved in the learning process. Although previous studies (Centra, 1993; Feldman, 2007; Marsh, 2001; Marsh & Roche, 2000) have reported similar findings, this study represents the first attempt to examine the relationship between course rigor and overall student ratings in a sport management curriculum. That sport management students value course rigor suggests that they are not looking to earn a degree in an often described easy major (Cuneen, 2004), but instead value a challenge as they learn about the business of sport. Thus, sport management faculty are encouraged to use rigor in the classroom and incorporate it into their assignments/exams.

The H2 evaluated the relationship between course grades and rigor. Results from the Pearson correlation established there was no relationship between grades and rigor, which confirmed H2. While the results for the regression in H4 found different predictive implications for grades, which impacted H4, it is noteworthy that rigor and grades are not significantly related in a field that has had criticisms of being an easy major (Cuneen, 2004). It has been assumed for decades that grades and rigor are connected, and in many cases, it has been assumed that a grading leniency hypothesis (Marsh & Dunkin, 1992) was at work to produce less than rigorous course environments (Collier, 2013; Feldman, 2007; Marsh & Dunkin, 1992). Many of these assumptions, however, are anecdotal. Most literature suggests the validity hypothesis (Marsh & Dunkin, 1992) and the student characteristic hypothesis (Marsh, 1989) are more prominently influencing results of student ratings. Practically, these results indicate that sport management is similar to other social science disciplines where students provide high ratings not because they believe they will receive high grades, but because they felt their instructors integrated the components of rigor into their courses (Benton & Cashin, 2012; Centra, 2003; Feldman, 2007; Marsh, 2007).

The H3 was the only rejected hypothesis. The variation of perceived course rigor within each student level, however, provides an opportunity to analyze patterns. Specifically, the results of course rigor for the 100/200 level were lower than those of 400-level and graduate-level courses. The rigor scores for the 100/200 level, however, were higher than those at the 300-level courses. Within the sport management program, some of the 100/200-level courses are high-enrollment classes. Even though the sport management program has a similar number of students within courses at the 300 and 400 levels, the number of students within the 100/200-level courses are greater because there are more students enrolled in fewer class sections and perhaps because there may be attrition between 100/200-level and upper-division level courses. Previous research acknowledges a negative relationship between class size and student learning in which students in smaller classes provide higher marks on student ratings (Feldman, 1984, 2007; Hoyt & Lee, 2002a). These results refute those sources, as no relationship relative to class size and rigor or rating scores existed. It is interesting to note, however, the negative significant relationship between grades and enrollment, suggesting that in larger class sections, the average grades are lower than in smaller class sections. Considering that lower-level courses in this study had much larger enrollments, this was a finding consistent with the literature. In turn, students in courses with larger class sizes, particularly within introductory courses, may perceive a course as more rigorous considering their lack of prior knowledge and an increased expectation of self-learning (Benton & Cashin, 2012).

The final hypothesis (H4) suggested perceived course rigor would be predicted by overall student ratings, but not by semester, course level, or class GPA (grade). H4 was partially supported. Overall student ratings (β = 0.80) significantly predict course rigor, while semester and course level do not. Unlike the results for H2, however, class GPA (grade; β = −0.18) was found to be a negative significant predictor of course rigor. As noted by the β values, increases in course rigor significantly aided in predicting overall student ratings and negatively related to grades. It appears, then, that as perceived course rigor increases, overall student ratings increase while course GPA (grade) decreases. This investigation can only surmise that these results can be used in the specific environment that data were collected. This might not be true at a different institution with different faculty and different student populations in the United States.

From an application perspective, it is important for sport management faculty to realize that rigorous coursework may result in lower student grades, but it is appreciated by students. That appreciation was reflected in the predictability of grades relative to overall student ratings. Sport management students reported higher ratings relative to rigorous coursework, even to the detriment of their own grades, suggesting that the grading leniency hypothesis (Marsh & Dunkin, 1992) does not fit with this student population. Possibly of most importance to sport management faculty is that fear of retributional bias (Feldman, 2007) should be minimized. The validity theory (Marsh, 1987; Marsh & Dunkin, 1992), where students are generally appreciative of rigorous coursework and a challenging classroom environment, appears to be more applicable to this study. Providing that the elements of rigor are appropriate for class level and content area, and the work avoids a busy work connotation (Graham & Esssex, 2001), faculty should design courses that include the five elements of rigor to maximize student ratings and meet the call for added rigor in higher education.

Limitations and Recommendations

Although this study offers new insights into rigor in sport management, there are some limitations and suggestions for additional research. First, the implementation of questions to assess rigor was added to existing student rating distribution procedures for faculty who volunteered their courses. While student ratings have considerable support as reliable tools for teaching evaluation (Benton & Cashin, 2012), the weaknesses with survey research in general could influence this study. Specifically, the distribution of student ratings relies on voluntary completion of the instrument, which could expose nonresponse bias in the sample if subsets of students were particularly motivated or unmotivated to complete the student ratings questionnaire. Future research should explore varying strategies and delivery methods that utilize the rigor questions and rigor themes outlined in this study.

Second, this study was conducted at one university. Although the sample size was robust and spanned several years, there are limitations to external validity. These limitations are particularly relevant to sport management programs (or other academic programs in general) that do not resemble the program in this study. Such institutions likely include smaller nondoctorate-granting universities with little to no research activity based on the Carnegie Classification system. Future investigations could replicate this examination in a variety of different programmatic or educational environments and expand the investigation of courses to include online or blended formats.

Finally, the conceptualization of rigor as outlined by Johnson et al. (2019) was utilized in this study. As one might expect, defining and capturing an abstract concept like rigor is subject to a variety of perspectives. With several differing conceptualizations of rigor available, and those conceptualizations possibly divergent among institutions, academic disciplines, and courses, it is reasonable that the conceptualization of rigor could be modified. Moreover, students in this study must determine how to interpret constructs like critical thinking or challenging. There are levels of semantical subjectivity in any instrument. Thus, future research should continue to replicate, challenge, and expand on the findings from this study using established modes of scientific inquiry to ensure the construct of rigor appropriately applies to sport management.

Conclusion

After decades of battling attacks on its legitimacy as an academic discipline, as well as the ongoing attacks on the value of higher education in general, an investigation of rigor in sport management was overdue. The findings of this study demonstrated strong support that the questions appropriately captured rigor, that rigor was present in the sport management curriculum, that courses and faculty were rated higher when more course rigor was reported, and that rigor could be predicted using overall student ratings and class grades. These findings are powerful for sport management faculty and their administrators in the environment investigated because they suggest that rigor is valued by students.

Pragmatically, the results of this study have several potential implications for sport management faculty. First, it is possible—and encouraged—for sport management faculty to evaluate rigor. Second, in line with recommendations by Johnson et al. (2019), sport management faculty should incorporate the five elements of rigor into their courses to confirm that students are receiving a rigorous education and ensure that the reputation of the sport management discipline is intact. Including the elements of rigor into a course will likely improve student ratings and contribute to a strong program. Finally, the fear of retributional bias, which is often associated with grade inflation, can be directly challenged as a result of this study. Succumbing to pressures of inflated grades or courses that lack rigor appear to hurt faculty more than it would help them from a student ratings perspective.

References

  • Abrami, P.C. (2001). Improving judgments about teaching effectiveness using teacher rating forms. In M. Theall, P.C. Abrami, & L.A. Mets (Eds.), The student ratings debate: Are they valid? How can we best use them? [Special issue] (Vol. 109, pp. 5987). New Directions for Institutional Research. San Francisco, CA: Jossey-Bass.

    • Search Google Scholar
    • Export Citation
  • Abrami, P.C., Perry, R.P., & Leventhal, L. (1982). The relationship between student personality characteristics, teacher ratings, and student achievement. Journal of Educational Psychology, 74(1), 111125. doi:10.1037/0022-0663.74.1.111

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Achen, A.C., & Courant, P.N. (2009). What are grades made of? The Journal of Economic Perspectives, 23(3), 7792. PubMed ID: 20052301 doi:10.1257/jep.23.3.77

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Aleamoni, L.M. (1981). Student ratings of instruction. In J. Millman (Ed.), Handbook of teacher evaluation (pp. 110145). Beverly Hills, CA: Sage.

    • Search Google Scholar
    • Export Citation
  • Arum, R., & Roksa, J. (2011). Academically adrift: Limited learning on college campusesChicago, IL: University of Chicago Press.

  • Asher, L. (2013, October 27). When students rate teachers, standards drop. The Wall Street Journal. Retrieved from http://www.wsj.com/news/articles/SB100014240527023041 76904579115971990673400

    • Search Google Scholar
    • Export Citation
  • Babcock, P.S., & Marks, M. (2011). The falling time cost of college: Evidence from half a century of time use data. The Review of Economics and Statistics, 93(2), 468478. doi:10.1162/REST_a_00093

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Benton, S.L., & Cashin, W.E. (2012). IDEA Technical Report No. 50: Student ratings of teaching: A summary of research and literature. Manhattan, KS: The IDEA Center. Retrieved from https://www.ideaedu.org/Portals/0/Uploads/Documents/IDEA%20Papers/IDEA%20Papers/PaperIDEA_50.pdf

    • Search Google Scholar
    • Export Citation
  • Benton, S.L., & Ryalls, K.R. (2016). IDEA Technical Report No. 58: Challenging misconceptions about student ratings of instructionManhattan, KS: The IDEA Center. Retrieved from http://www.ideaedu.org/Portals/0/Uploads/Documents/IDEA%20Papers/IDEA%20Papers/PaperIDEA_58.pdf

    • Search Google Scholar
    • Export Citation
  • Boring, A., Ottoboni, K., & Stark, P.B. (2016). Student evaluations of teaching (mostly) do not measure teaching effectiveness. ScienceOpen Research. 111. doi:10.14293/S2199-1006.1.SOR-EDU.AETBZC.v1

    • Search Google Scholar
    • Export Citation
  • Boucher, R.L. (1998). Toward achieving a focal point for sport management: A binocular perspective. Journal of Sport Management, 12, 7685. doi:10.1123/jsm.12.1.76

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brandon, C. (2010). The five-year party: How colleges have given up on educating your child and what you can do about itDallas, TX: BenBella Books.

    • Search Google Scholar
    • Export Citation
  • Braskamp, L.A., & Ory, J.C. (1994). Assessing faculty work: Enhancing individual and institutional performanceSan Francisco, CA: Jossey-Bass.

    • Search Google Scholar
    • Export Citation
  • Braxton, J.M. (1993). Selectivity and rigor in research universities. Journal of Higher Education, 64(6), 657675. doi:10.2307/2960017

  • Brooks, J.G., & Brooks, M.G. (2001). In search of understanding: The case for constructivist classroomsAlexandria, VA: Association for Supervision and Curriculum Development.

    • Search Google Scholar
    • Export Citation
  • Bruner, J. (1996). The culture of educationCambridge, MA: Harvard University Press.

  • Bursuck, W.D. (1994). Introduction to the special series on homework. Journal of Learning Disabilities, 27(8), 466469. doi:10.1177/002221949402700801

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cashin, W.E. (1990). Students do rate different academic fields differently. In M. Theall & J. Franklin (Eds.), Student ratings of instruction: Issues for improving practice: New Directions for teaching and learning, No. 43 (pp. 113121). San Francisco, CA: Jossey-Bass.

    • Search Google Scholar
    • Export Citation
  • Centra, J.A. (1993). Reflective faculty evaluation: Enhancing teaching and determining faculty effectivenessSan Francisco, CA: Jossey-Bass.

    • Search Google Scholar
    • Export Citation
  • Centra, J.A. (2003). Will teachers receive higher student evaluations by giving higher grades and less course work? Research in Higher Education, 44(5), 495518. doi:10.1023/A:1025492407752

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Centra, J.A. (2009). Differences in responses to the student instructional report: Is it bias? Princeton, NJ: Educational Testing Service.

    • Search Google Scholar
    • Export Citation
  • Centra, J.A., & Gaubatz, N.B. (2000). Is there a gender bias in student evaluations of teaching? Journal of Higher Education, 71(1), 1733. doi:10.2307/2649280

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chelladurai, P. (1992). Sport management: Opportunities and obstacles. Journal of Sport Management, 6, 215219. doi:10.1123/jsm.6.3.215

  • Cohen, P.A. (1980). Effectiveness of student-rating feedback for improving college instruction: A meta-analysis of findings. Research in Higher Education, 13(4), 321341. doi:10.1007/BF00976252

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Collier, G.L. (2013, December 26). We pretend to teach, they pretend to learn. Wall Street Journal. https://www.wsj.com/articles/we-pretend-to-teach-they-pretend-to-learn-1388103868

    • Search Google Scholar
    • Export Citation
  • Commission on Sport Management Accreditation. (2019). COSMA statement of academic quality. Commission on Sport Management Accreditation. Retrieved from https://www.cosmaweb.org/academic-quality.html

  • Cope, C., & Staehr, L. (2005). Improving students’ learning approaches through intervention in an information systems learning environment. Studies in Higher Education, 30, 181197. doi:10.1080/03075070500043275

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cuneen, J. (2004). Managing program excellence during our transition from potential to merit. Journal of Sport Management, 18, 112. doi:10.1123/jsm.18.1.1

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Desilver, D. (2017, February). U.S. students’ academic achievement still lags that of their peers in many other countriesWashington, DC: PEW Research Center. Retrieved from http://www.pewresearch.org/fact-tank/2017/02/15/u-s-students-internationally-math-science/

    • Search Google Scholar
    • Export Citation
  • Draeger, J., Prado Hill, P., Hunter, L.R., & Mahler, R. (2013). The anatomy of academic rigor: The story of one institutional journey. Innovative Higher Education, 38, 267279. doi:10.1007/s10755-012-9246-8

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Draeger, J., Prado Hill, P., & Mahler, R. (2015). Developing a student conception of academic rigor. Innovative Higher Education, 40, 215228. doi:10.1007/s10755-014-9308-1

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (1976). Grades and college students’ evaluation of their courses and teachers. Research in Higher Education, 4(1), 69111. doi:10.1007/BF00991462

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (1977). Consistency and variability among college students in rating their teachers and courses: A review and analysis. Research in Higher Education, 6(3), 223274. doi:10.1007/BF00991288

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (1978). Course characteristics and college students’ ratings of their teachers: What we know and what we don’t. Research in Higher Education, 9(3), 199242. doi:10.1007/BF00976997

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (1983). Seniority and experience of college teachers as related to evaluations they receive from students. Research in Higher Education, 18(1), 3124. doi:10.1007/BF00992080

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (1984). Class size and college students’ evaluations of teachers and courses: A closer look. Research in Higher Education, 21(1), 45116. doi:10.1007/BF00975035

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (1986). The perceived instructional effectiveness of college teachers as related to their personality and attitudinal characteristics: A review and synthesis. Research in Higher Education, 24, 139213. doi:10.1007/BF00991885

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (1987). Research productivity and scholarly accomplishment of college teachers as related to their instructional effectiveness: A review and exploration. Research in Higher Education, 26(3), 227298. doi:10.1007/BF00992241

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (1989). The association between student ratings of specific instructional dimensions and student achievement: Refining and extending the synthesis of data from multisection validity studies. Research in Higher Education, 30(6), 583645. doi:10.1007/BF00992392

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (1992). College students’ views of male and female college teachers: Part I—Evidence from the social laboratory and experiments. Research in Higher Education, 33(3), 317375. doi:10.1007/BF00992265

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (1993). College students’ views of male and female college teachers: Part II—Evidence from students’ evaluations of their classroom teachers. Research in Higher Education, 34(2), 151211. doi:10.1007/BF00992161

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (2007). Identifying exemplary teachers and teaching: Evidence from student ratings. In R.P. Perry & J.C. Smart (Eds.), The scholarship of teaching and learning in higher education: An evidence-based perspective (pp. 93129). Dordrecht, The Netherlands: Springer.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gigliotti, R.J., & Buchtel, F.S. (1990). Attributional bias and course evaluations. Journal of Educational Psychology, 82(2), 341351. doi:10.1037/0022-0663.82.2.341

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Graham, C., & Essex, C. (2001). Defining and ensuring academic rigor in online and on-campus courses: Instructor perspectives. Annual Proceedings of Selected Research and Development, 1, 330337.

    • Search Google Scholar
    • Export Citation
  • Hobson, S.M., & Talbot, D.M. (2001). Understanding student evaluations: What all faculty should know. College Teaching, 49(1), 2631. doi:10.1080/87567550109595842

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Howard, G.S., & Maxwell, S.E. (1980). The correlation between student satisfaction and grades: A case of mistaken causation? Journal of Educational Psychology, 72(6), 810820. doi:10.1037/0022-0663.72.6.810

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Howard, G.S., & Maxwell, S.E. (1982). Do grades contaminate student evaluations of instruction? Research in Higher Education, 16(2), 175188. doi:10.1007/BF00973508

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hoyt, D.P., & Lee, E. (2002a). Technical Report No. 12: Basic data for the revised IDEA systemManhattan, KS: The IDEA Center. Retrieved from https://www.ideaedu.org/Portals/0/Uploads/Documents/Technical-Reports/Basic-Data-Revised-IDEA-System_techreport-12.pdf

    • Search Google Scholar
    • Export Citation
  • Hoyt, D.P., & Lee, E. (2002b). Technical Report No. 13: Disciplinary differences in student ratingsManhattan, KS: The IDEA Center. Retrieved from https://ideacontent.blob.core.windows.net/content/sites/2/2020/01/Disciplinary-Differences-in-Student-Ratings_techreport-13.pdf

    • Search Google Scholar
    • Export Citation
  • Jacobs, J., & Colvin, R.L. (2009). Rigor: It’s all the rage, but what does it mean? In Hechinger Institute (Ed.), Understanding and reporting on academic rigor (pp. 15). New York, NY: The Hechinger Institute. Retrieved from http://hechinger.tc.columbia.edu/primers/Hechinger_Institute_Rigor_Primer.pdf

    • Search Google Scholar
    • Export Citation
  • James, J.D. (2018). Not all doctoral programs are created equally. Journal of Sport Management, 32(1), 110. doi:10.1123/jsm.2017-0257

  • Jensen, E. (2005). Teaching with the brain in mind (2nd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.

  • Johnson, J.E., Weidner, T.G., Jones, J.A., & Manwell, A.K. (2019). Evaluating academic course rigor, part 1: Defining a nebulous construct. Journal of Assessment and Institutional Effectiveness, 8(1–2), 86121. doi:10.5325/jasseinsteffe.8.12.0086

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kaplan, S.N. (2017). Advocacy: Defining academic rigor. Gifted Child Today, 40(4), 218219. doi:10.1177/1076217517723950

  • Kirst, M.W. (2009). Progress and gaps in college preparation policy. Education Commission of the States. Retrieved from https://www.ecs.org/clearinghouse/82/15/8215.pdf

    • Search Google Scholar
    • Export Citation
  • Li, Y. (1993). A comparative study of Asian and American students’ perceptions of faculty teaching effectiveness at Ohio University (Unpublished doctoral dissertation). Ohio University, Athens.

    • Search Google Scholar
    • Export Citation
  • Mahony, D.F. (2008). No one can whistle a symphony: Working together for sport management’s future. Journal of Sport Management, 22, 110. doi:10.1123/jsm.22.1.1

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marsh, H.W. (1987). Students’ evaluations of university teaching: Research findings, methodological issues, and directions for further research. International Journal of Education Research, 11(3), 253388. doi:10.1016/0883-0355(87)90001-2

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marsh, H.W. (1989). Confirmatory factor analysis of multitrait-multimethod data: Many problems and a few solutions. Applied Psychology Measurement, 13(3), 335361. doi:10.1177/014662168901300402

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marsh, H.W. (2001). Distinguishing between good (useful) and bad workloads on students’ evaluations of teaching. American Educational Research Journal, 38(1), 183212. doi:10.3102/00028312038001183

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marsh, H.W. (2007). Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases and usefulness. In R.P. Perry & J.C. Smart (Eds.), The scholarship of teaching and learning in higher education: An evidence-based perspective (pp. 319383). Dordrecht, The Netherlands: Springer.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marsh, H.W., & Dunkin, M.J. (1992). Students’ evaluations of university teaching: A multidimensional perspective. In J.C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. 8, pp. 143234). New York, NY: Agathon Press.

    • Search Google Scholar
    • Export Citation
  • Marsh, H.W., & Dunkin, M.J. (1997). Students’ evaluations of university teaching: A multidimensional perspective. In R.P. Perry & J.C. Smart (Eds.), Effective teaching in higher education: Research and practice (pp. 241320). New York, NY: Agathon Press.

    • Search Google Scholar
    • Export Citation
  • Marsh, H.W., & Hattie, J. (2002). The relation between research productivity and teaching effectiveness. Journal of Higher Education, 73(5), 603641.

    • Search Google Scholar
    • Export Citation
  • Marsh, H.W., & Hocevar, D. (1991). Students’ evaluations of teaching effectiveness: The stability of mean ratings of the same teachers over a 13-year period. Teaching & Teacher Education, 7(4), 303314. doi:10.1016/0742-051X(91)90001-6

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marsh, H.W., & Roche, L.A. (2000). Effects of grading leniency and low workload on students’ evaluations of teaching: Popular myth, bias, validity, or innocent bystanders? Journal of Educational Psychology, 92(1), 202228. doi:10.1037/0022-0663.92.1.202

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McKeachie, W.J. (1979). Student ratings of faculty: A reprise. Academe, 65(6), 384397. doi:10.2307/40248725

  • Naftulin, D.H., Ware, J.E., & Donnelly, F.A. (1973). The Doctor Fox lecture: A paradigm of educational seduction. Journal of Medical Education, 48(7), 630635. PubMed ID: 4708420

    • Search Google Scholar
    • Export Citation
  • Nelson, V. (2010). Is your college student investing enough time studying? College Parent Central. Retrieved from https://www.collegeparentcentral.com/2010/02/is-your-college-student-investing-enough-time-studying/

    • Search Google Scholar
    • Export Citation
  • Olafson, G.A. (1995). Sport management research: Ordered change. Journal of Sport Management, 9, 338345. doi:10.1123/jsm.9.3.338

  • Park, G., Lubinski, D., & Benbow, C.P. (2008). Ability differences among people who have commensurate matter for scientific creativity. Psychological Science, 19(1), 957961. doi:10.1111/j.1467-9280.2008.02182.x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Parkhouse, B.L. (1996). Definition, evolution, and curriculum. In Parkhouse, B.L. (Ed.), The management of sport: Its foundation and application (pp. 312). St. Louis, MO: Mosby Year Book.

    • Search Google Scholar
    • Export Citation
  • Payton, F.C., White, A., & Mullins, T. (2017). STEM majors, art thinkers (STEM + Arts)—Issues of duality, rigor and inclusion. Journal of STEM Education, 18(3), 3947.

    • Search Google Scholar
    • Export Citation
  • Renaud, R.D., & Murray, H.G. (1996). Aging, personality, and teaching effectiveness in academic psychologists. Research in Higher Education, 37(3), 223240. doi:10.1007/BF01730120

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rossman, G.B., & Wilson, B.L. (1996). Context, courses, and the curriculum: Local responses to state policy reform. Educational Policy, 10(3), 399422. doi:10.1177/0895904896010003005

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sixbury, G.R., & Cashin, W.E. (1995). IDEA Technical Report No. 10: Comparative data by academic fieldManhattan, KS: Kansas State University, Center for Faculty Evaluation and Development.

    • Search Google Scholar
    • Export Citation
  • Smith, B.P., & Hawkins, B. (2011). Examining student evaluations of black college faculty: Does race matter? The Journal of Negro Education, 80(2), 149162.

    • Search Google Scholar
    • Export Citation
  • Snider, J. (2009a). AP and IB courses: Are they truly rigorous? In Hechinger Institute (Ed.), Understanding and reporting on academic rigor (pp. 2125). New York, NY: The Hechinger Institute. Retrieved from http://hechinger.tc.columbia.edu/primers/Hechinger_Institute_Rigor_Primer.pdf

    • Search Google Scholar
    • Export Citation
  • Snider, J. (2009b). Two governors explain what they mean by ‘rigor.’ In Hechinger Institute (Ed.), Understanding and reporting on academic rigor (pp. 67). New York, NY: The Hechinger Institute. Retrieved from http://hechinger.tc.columbia.edu/primers/Hechinger_Institute_Rigor_Primer.pdf

    • Search Google Scholar
    • Export Citation
  • Stark, P.B., & Freishtat, R. (2014). An evaluation of course evaluations. ScienceOpen, 1, 126. doi:10.14293/S2199-1006.1.SOR-EDU.AOFRQA.v1

    • Search Google Scholar
    • Export Citation
  • Taylor, M.T., & Rendon, L.I. (1991). The American history curriculum in North Carolina’s public community colleges and universities: A comparative study. Community College Review, 19(1), 3641. doi:10.1177/009155219101900107

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Theall, M., Franklin, J., & Ludlow, L.H. (1990a). Attributions and retributions: Student ratings and the perceived causes of performance. Instructional Evaluation, 11, 1217.

    • Search Google Scholar
    • Export Citation
  • Theall, M., Franklin, J., & Ludlow, L.H. (1990b). Attributions and retributions: Student ratings and the perceived causes of performance. Paper presented at the American Educational Research Association, Boston, MA.

    • Search Google Scholar
    • Export Citation
  • Thomas, B., & Bolen, Y. (1998). Student perception of the academic rigor of the college physical education curriculum. Physical Educator, 55(1), 28.

    • Search Google Scholar
    • Export Citation
  • Trigwell, K., & Prosser, M. (1991). Improving the quality of student learning: The influence of learning context and student approaches to learning on learning outcomes. Higher Education, 22, 251266. doi:10.1007/BF00132290

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Weese, W.J. (2002). Opportunities and headaches: Dichotomous perspectives on the current and future hiring realities in the sport management academy. Journal of Sport Management, 16, 117. doi:10.1123/jsm.16.1.1

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Weissberg, R. (2010). Bad students, not bad schoolsNew Brunswick, NJ: Transaction Publishers. Retrieved from http://www.heritageanddestiny.com/publications/reviews/bad-students-not-bad-schools-by-robert-weissberg-book-review/

    • Search Google Scholar
    • Export Citation
  • Winston, R.B., Vahala, M.E., Nichols, E.C., Gillis, M.E., Wintrow, M., & Rome, K.D. (1994). A measure of college classroom climate: The college classroom environment scales. Journal of College Student Development, 35(1), 1118.

    • Search Google Scholar
    • Export Citation
  • Wyatt, J.N., Wiley, A., Camara, W.J., & Proestler, N. (2011). The development of an index of academic rigor for college readiness. (Research Report 2010–2011). New York, NY: The College Board. Retrieved from https://files.eric.ed.gov/fulltext/ED561023.pdf

    • Search Google Scholar
    • Export Citation
  • Zakus, D., Malloy, D.C., & Edwards, A. (2007). Critical and ethical thinking in sport management: Philosophical rationales and examples of methods. Sport Management Review, 10(2), 133158. doi:10.1016/S1441-3523(07)70008-6

    • Crossref
    • Search Google Scholar
    • Export Citation

Appendix: Student Evaluation Questions With Rigor Questions Included (Comment Boxes Not Included)

Instructor Questions

  • My instructor explains the course objectives clearly.
  • My instructor explains course content clearly.
  • My instructor uses effective examples and illustrations.
  • My instructor is respectful when I have a question or comment.
  • My instructor provides feedback that helps me improve my performance in the class.
  • My instructor is available for consultation (e.g., after class, e-mail, office hours, or by appointment).

Course Questions

  • This course has clear objectives.
  • This course is effective in meeting its objectives.
  • This course has assignments related to the objectives of the course.
  • This course has a clear grading system.
  • This course broadens my perspective and/or knowledge.
  • The laboratory or practicum sessions reinforce learning in the course (answer only if applicable).

Rigor Questions

  • The instructor encouraged critical thinking and problem solving.
  • The course content encouraged critical thinking and problem solving.
  • The course was academically challenging.
  • The instructor encouraged me to master complex material
  • Learning the course material was time and labor intensive.
  • The instructor’s requirements were time and labor intensive.

The authors are with Ball State University, Muncie, IN.

Johnson (jejohnson1@bsu.edu) is corresponding author.
  • Abrami, P.C. (2001). Improving judgments about teaching effectiveness using teacher rating forms. In M. Theall, P.C. Abrami, & L.A. Mets (Eds.), The student ratings debate: Are they valid? How can we best use them? [Special issue] (Vol. 109, pp. 5987). New Directions for Institutional Research. San Francisco, CA: Jossey-Bass.

    • Search Google Scholar
    • Export Citation
  • Abrami, P.C., Perry, R.P., & Leventhal, L. (1982). The relationship between student personality characteristics, teacher ratings, and student achievement. Journal of Educational Psychology, 74(1), 111125. doi:10.1037/0022-0663.74.1.111

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Achen, A.C., & Courant, P.N. (2009). What are grades made of? The Journal of Economic Perspectives, 23(3), 7792. PubMed ID: 20052301 doi:10.1257/jep.23.3.77

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Aleamoni, L.M. (1981). Student ratings of instruction. In J. Millman (Ed.), Handbook of teacher evaluation (pp. 110145). Beverly Hills, CA: Sage.

    • Search Google Scholar
    • Export Citation
  • Arum, R., & Roksa, J. (2011). Academically adrift: Limited learning on college campusesChicago, IL: University of Chicago Press.

  • Asher, L. (2013, October 27). When students rate teachers, standards drop. The Wall Street Journal. Retrieved from http://www.wsj.com/news/articles/SB100014240527023041 76904579115971990673400

    • Search Google Scholar
    • Export Citation
  • Babcock, P.S., & Marks, M. (2011). The falling time cost of college: Evidence from half a century of time use data. The Review of Economics and Statistics, 93(2), 468478. doi:10.1162/REST_a_00093

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Benton, S.L., & Cashin, W.E. (2012). IDEA Technical Report No. 50: Student ratings of teaching: A summary of research and literature. Manhattan, KS: The IDEA Center. Retrieved from https://www.ideaedu.org/Portals/0/Uploads/Documents/IDEA%20Papers/IDEA%20Papers/PaperIDEA_50.pdf

    • Search Google Scholar
    • Export Citation
  • Benton, S.L., & Ryalls, K.R. (2016). IDEA Technical Report No. 58: Challenging misconceptions about student ratings of instructionManhattan, KS: The IDEA Center. Retrieved from http://www.ideaedu.org/Portals/0/Uploads/Documents/IDEA%20Papers/IDEA%20Papers/PaperIDEA_58.pdf

    • Search Google Scholar
    • Export Citation
  • Boring, A., Ottoboni, K., & Stark, P.B. (2016). Student evaluations of teaching (mostly) do not measure teaching effectiveness. ScienceOpen Research. 111. doi:10.14293/S2199-1006.1.SOR-EDU.AETBZC.v1

    • Search Google Scholar
    • Export Citation
  • Boucher, R.L. (1998). Toward achieving a focal point for sport management: A binocular perspective. Journal of Sport Management, 12, 7685. doi:10.1123/jsm.12.1.76

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Brandon, C. (2010). The five-year party: How colleges have given up on educating your child and what you can do about itDallas, TX: BenBella Books.

    • Search Google Scholar
    • Export Citation
  • Braskamp, L.A., & Ory, J.C. (1994). Assessing faculty work: Enhancing individual and institutional performanceSan Francisco, CA: Jossey-Bass.

    • Search Google Scholar
    • Export Citation
  • Braxton, J.M. (1993). Selectivity and rigor in research universities. Journal of Higher Education, 64(6), 657675. doi:10.2307/2960017

  • Brooks, J.G., & Brooks, M.G. (2001). In search of understanding: The case for constructivist classroomsAlexandria, VA: Association for Supervision and Curriculum Development.

    • Search Google Scholar
    • Export Citation
  • Bruner, J. (1996). The culture of educationCambridge, MA: Harvard University Press.

  • Bursuck, W.D. (1994). Introduction to the special series on homework. Journal of Learning Disabilities, 27(8), 466469. doi:10.1177/002221949402700801

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cashin, W.E. (1990). Students do rate different academic fields differently. In M. Theall & J. Franklin (Eds.), Student ratings of instruction: Issues for improving practice: New Directions for teaching and learning, No. 43 (pp. 113121). San Francisco, CA: Jossey-Bass.

    • Search Google Scholar
    • Export Citation
  • Centra, J.A. (1993). Reflective faculty evaluation: Enhancing teaching and determining faculty effectivenessSan Francisco, CA: Jossey-Bass.

    • Search Google Scholar
    • Export Citation
  • Centra, J.A. (2003). Will teachers receive higher student evaluations by giving higher grades and less course work? Research in Higher Education, 44(5), 495518. doi:10.1023/A:1025492407752

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Centra, J.A. (2009). Differences in responses to the student instructional report: Is it bias? Princeton, NJ: Educational Testing Service.

    • Search Google Scholar
    • Export Citation
  • Centra, J.A., & Gaubatz, N.B. (2000). Is there a gender bias in student evaluations of teaching? Journal of Higher Education, 71(1), 1733. doi:10.2307/2649280

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chelladurai, P. (1992). Sport management: Opportunities and obstacles. Journal of Sport Management, 6, 215219. doi:10.1123/jsm.6.3.215

  • Cohen, P.A. (1980). Effectiveness of student-rating feedback for improving college instruction: A meta-analysis of findings. Research in Higher Education, 13(4), 321341. doi:10.1007/BF00976252

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Collier, G.L. (2013, December 26). We pretend to teach, they pretend to learn. Wall Street Journal. https://www.wsj.com/articles/we-pretend-to-teach-they-pretend-to-learn-1388103868

    • Search Google Scholar
    • Export Citation
  • Commission on Sport Management Accreditation. (2019). COSMA statement of academic quality. Commission on Sport Management Accreditation. Retrieved from https://www.cosmaweb.org/academic-quality.html

  • Cope, C., & Staehr, L. (2005). Improving students’ learning approaches through intervention in an information systems learning environment. Studies in Higher Education, 30, 181197. doi:10.1080/03075070500043275

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cuneen, J. (2004). Managing program excellence during our transition from potential to merit. Journal of Sport Management, 18, 112. doi:10.1123/jsm.18.1.1

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Desilver, D. (2017, February). U.S. students’ academic achievement still lags that of their peers in many other countriesWashington, DC: PEW Research Center. Retrieved from http://www.pewresearch.org/fact-tank/2017/02/15/u-s-students-internationally-math-science/

    • Search Google Scholar
    • Export Citation
  • Draeger, J., Prado Hill, P., Hunter, L.R., & Mahler, R. (2013). The anatomy of academic rigor: The story of one institutional journey. Innovative Higher Education, 38, 267279. doi:10.1007/s10755-012-9246-8

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Draeger, J., Prado Hill, P., & Mahler, R. (2015). Developing a student conception of academic rigor. Innovative Higher Education, 40, 215228. doi:10.1007/s10755-014-9308-1

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (1976). Grades and college students’ evaluation of their courses and teachers. Research in Higher Education, 4(1), 69111. doi:10.1007/BF00991462

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (1977). Consistency and variability among college students in rating their teachers and courses: A review and analysis. Research in Higher Education, 6(3), 223274. doi:10.1007/BF00991288

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (1978). Course characteristics and college students’ ratings of their teachers: What we know and what we don’t. Research in Higher Education, 9(3), 199242. doi:10.1007/BF00976997

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (1983). Seniority and experience of college teachers as related to evaluations they receive from students. Research in Higher Education, 18(1), 3124. doi:10.1007/BF00992080

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (1984). Class size and college students’ evaluations of teachers and courses: A closer look. Research in Higher Education, 21(1), 45116. doi:10.1007/BF00975035

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (1986). The perceived instructional effectiveness of college teachers as related to their personality and attitudinal characteristics: A review and synthesis. Research in Higher Education, 24, 139213. doi:10.1007/BF00991885

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (1987). Research productivity and scholarly accomplishment of college teachers as related to their instructional effectiveness: A review and exploration. Research in Higher Education, 26(3), 227298. doi:10.1007/BF00992241

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (1989). The association between student ratings of specific instructional dimensions and student achievement: Refining and extending the synthesis of data from multisection validity studies. Research in Higher Education, 30(6), 583645. doi:10.1007/BF00992392

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (1992). College students’ views of male and female college teachers: Part I—Evidence from the social laboratory and experiments. Research in Higher Education, 33(3), 317375. doi:10.1007/BF00992265

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (1993). College students’ views of male and female college teachers: Part II—Evidence from students’ evaluations of their classroom teachers. Research in Higher Education, 34(2), 151211. doi:10.1007/BF00992161

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Feldman, K.A. (2007). Identifying exemplary teachers and teaching: Evidence from student ratings. In R.P. Perry & J.C. Smart (Eds.), The scholarship of teaching and learning in higher education: An evidence-based perspective (pp. 93129). Dordrecht, The Netherlands: Springer.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gigliotti, R.J., & Buchtel, F.S. (1990). Attributional bias and course evaluations. Journal of Educational Psychology, 82(2), 341351. doi:10.1037/0022-0663.82.2.341

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Graham, C., & Essex, C. (2001). Defining and ensuring academic rigor in online and on-campus courses: Instructor perspectives. Annual Proceedings of Selected Research and Development, 1, 330337.

    • Search Google Scholar
    • Export Citation
  • Hobson, S.M., & Talbot, D.M. (2001). Understanding student evaluations: What all faculty should know. College Teaching, 49(1), 2631. doi:10.1080/87567550109595842

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Howard, G.S., & Maxwell, S.E. (1980). The correlation between student satisfaction and grades: A case of mistaken causation? Journal of Educational Psychology, 72(6), 810820. doi:10.1037/0022-0663.72.6.810

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Howard, G.S., & Maxwell, S.E. (1982). Do grades contaminate student evaluations of instruction? Research in Higher Education, 16(2), 175188. doi:10.1007/BF00973508

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hoyt, D.P., & Lee, E. (2002a). Technical Report No. 12: Basic data for the revised IDEA systemManhattan, KS: The IDEA Center. Retrieved from https://www.ideaedu.org/Portals/0/Uploads/Documents/Technical-Reports/Basic-Data-Revised-IDEA-System_techreport-12.pdf

    • Search Google Scholar
    • Export Citation
  • Hoyt, D.P., & Lee, E. (2002b). Technical Report No. 13: Disciplinary differences in student ratingsManhattan, KS: The IDEA Center. Retrieved from https://ideacontent.blob.core.windows.net/content/sites/2/2020/01/Disciplinary-Differences-in-Student-Ratings_techreport-13.pdf

    • Search Google Scholar
    • Export Citation
  • Jacobs, J., & Colvin, R.L. (2009). Rigor: It’s all the rage, but what does it mean? In Hechinger Institute (Ed.), Understanding and reporting on academic rigor (pp. 15). New York, NY: The Hechinger Institute. Retrieved from http://hechinger.tc.columbia.edu/primers/Hechinger_Institute_Rigor_Primer.pdf

    • Search Google Scholar
    • Export Citation
  • James, J.D. (2018). Not all doctoral programs are created equally. Journal of Sport Management, 32(1), 110. doi:10.1123/jsm.2017-0257

  • Jensen, E. (2005). Teaching with the brain in mind (2nd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.

  • Johnson, J.E., Weidner, T.G., Jones, J.A., & Manwell, A.K. (2019). Evaluating academic course rigor, part 1: Defining a nebulous construct. Journal of Assessment and Institutional Effectiveness, 8(1–2), 86121. doi:10.5325/jasseinsteffe.8.12.0086

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kaplan, S.N. (2017). Advocacy: Defining academic rigor. Gifted Child Today, 40(4), 218219. doi:10.1177/1076217517723950

  • Kirst, M.W. (2009). Progress and gaps in college preparation policy. Education Commission of the States. Retrieved from https://www.ecs.org/clearinghouse/82/15/8215.pdf

    • Search Google Scholar
    • Export Citation
  • Li, Y. (1993). A comparative study of Asian and American students’ perceptions of faculty teaching effectiveness at Ohio University (Unpublished doctoral dissertation). Ohio University, Athens.

    • Search Google Scholar
    • Export Citation
  • Mahony, D.F. (2008). No one can whistle a symphony: Working together for sport management’s future. Journal of Sport Management, 22, 110. doi:10.1123/jsm.22.1.1

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marsh, H.W. (1987). Students’ evaluations of university teaching: Research findings, methodological issues, and directions for further research. International Journal of Education Research, 11(3), 253388. doi:10.1016/0883-0355(87)90001-2

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marsh, H.W. (1989). Confirmatory factor analysis of multitrait-multimethod data: Many problems and a few solutions. Applied Psychology Measurement, 13(3), 335361. doi:10.1177/014662168901300402

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marsh, H.W. (2001). Distinguishing between good (useful) and bad workloads on students’ evaluations of teaching. American Educational Research Journal, 38(1), 183212. doi:10.3102/00028312038001183

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marsh, H.W. (2007). Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases and usefulness. In R.P. Perry & J.C. Smart (Eds.), The scholarship of teaching and learning in higher education: An evidence-based perspective (pp. 319383). Dordrecht, The Netherlands: Springer.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marsh, H.W., & Dunkin, M.J. (1992). Students’ evaluations of university teaching: A multidimensional perspective. In J.C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. 8, pp. 143234). New York, NY: Agathon Press.

    • Search Google Scholar
    • Export Citation
  • Marsh, H.W., & Dunkin, M.J. (1997). Students’ evaluations of university teaching: A multidimensional perspective. In R.P. Perry & J.C. Smart (Eds.), Effective teaching in higher education: Research and practice (pp. 241320). New York, NY: Agathon Press.

    • Search Google Scholar
    • Export Citation
  • Marsh, H.W., & Hattie, J. (2002). The relation between research productivity and teaching effectiveness. Journal of Higher Education, 73(5), 603641.

    • Search Google Scholar
    • Export Citation
  • Marsh, H.W., & Hocevar, D. (1991). Students’ evaluations of teaching effectiveness: The stability of mean ratings of the same teachers over a 13-year period. Teaching & Teacher Education, 7(4), 303314. doi:10.1016/0742-051X(91)90001-6

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Marsh, H.W., & Roche, L.A. (2000). Effects of grading leniency and low workload on students’ evaluations of teaching: Popular myth, bias, validity, or innocent bystanders? Journal of Educational Psychology, 92(1), 202228. doi:10.1037/0022-0663.92.1.202

    • Crossref
    • Search Google Scholar
    • Export Citation
  • McKeachie, W.J. (1979). Student ratings of faculty: A reprise. Academe, 65(6), 384397. doi:10.2307/40248725

  • Naftulin, D.H., Ware, J.E., & Donnelly, F.A. (1973). The Doctor Fox lecture: A paradigm of educational seduction. Journal of Medical Education, 48(7), 630635. PubMed ID: 4708420

    • Search Google Scholar
    • Export Citation
  • Nelson, V. (2010). Is your college student investing enough time studying? College Parent Central. Retrieved from https://www.collegeparentcentral.com/2010/02/is-your-college-student-investing-enough-time-studying/

    • Search Google Scholar
    • Export Citation
  • Olafson, G.A. (1995). Sport management research: Ordered change. Journal of Sport Management, 9, 338345. doi:10.1123/jsm.9.3.338

  • Park, G., Lubinski, D., & Benbow, C.P. (2008). Ability differences among people who have commensurate matter for scientific creativity. Psychological Science, 19(1), 957961. doi:10.1111/j.1467-9280.2008.02182.x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Parkhouse, B.L. (1996). Definition, evolution, and curriculum. In Parkhouse, B.L. (Ed.), The management of sport: Its foundation and application (pp. 312). St. Louis, MO: Mosby Year Book.

    • Search Google Scholar
    • Export Citation
  • Payton, F.C., White, A., & Mullins, T. (2017). STEM majors, art thinkers (STEM + Arts)—Issues of duality, rigor and inclusion. Journal of STEM Education, 18(3), 3947.

    • Search Google Scholar
    • Export Citation
  • Renaud, R.D., & Murray, H.G. (1996). Aging, personality, and teaching effectiveness in academic psychologists. Research in Higher Education, 37(3), 223240. doi:10.1007/BF01730120

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rossman, G.B., & Wilson, B.L. (1996). Context, courses, and the curriculum: Local responses to state policy reform. Educational Policy, 10(3), 399422. doi:10.1177/0895904896010003005

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sixbury, G.R., & Cashin, W.E. (1995). IDEA Technical Report No. 10: Comparative data by academic fieldManhattan, KS: Kansas State University, Center for Faculty Evaluation and Development.

    • Search Google Scholar
    • Export Citation
  • Smith, B.P., & Hawkins, B. (2011). Examining student evaluations of black college faculty: Does race matter? The Journal of Negro Education, 80(2), 149162.

    • Search Google Scholar
    • Export Citation
  • Snider, J. (2009a). AP and IB courses: Are they truly rigorous? In Hechinger Institute (Ed.), Understanding and reporting on academic rigor (pp. 2125). New York, NY: The Hechinger Institute. Retrieved from http://hechinger.tc.columbia.edu/primers/Hechinger_Institute_Rigor_Primer.pdf

    • Search Google Scholar
    • Export Citation
  • Snider, J. (2009b). Two governors explain what they mean by ‘rigor.’ In Hechinger Institute (Ed.), Understanding and reporting on academic rigor (pp. 67). New York, NY: The Hechinger Institute. Retrieved from http://hechinger.tc.columbia.edu/primers/Hechinger_Institute_Rigor_Primer.pdf

    • Search Google Scholar
    • Export Citation
  • Stark, P.B., & Freishtat, R. (2014). An evaluation of course evaluations. ScienceOpen, 1, 126. doi:10.14293/S2199-1006.1.SOR-EDU.AOFRQA.v1

    • Search Google Scholar
    • Export Citation
  • Taylor, M.T., & Rendon, L.I. (1991). The American history curriculum in North Carolina’s public community colleges and universities: A comparative study. Community College Review, 19(1), 3641. doi:10.1177/009155219101900107

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Theall, M., Franklin, J., & Ludlow, L.H. (1990a). Attributions and retributions: Student ratings and the perceived causes of performance. Instructional Evaluation, 11, 1217.

    • Search Google Scholar
    • Export Citation
  • Theall, M., Franklin, J., & Ludlow, L.H. (1990b). Attributions and retributions: Student ratings and the perceived causes of performance. Paper presented at the American Educational Research Association, Boston, MA.

    • Search Google Scholar
    • Export Citation
  • Thomas, B., & Bolen, Y. (1998). Student perception of the academic rigor of the college physical education curriculum. Physical Educator, 55(1), 28.

    • Search Google Scholar
    • Export Citation
  • Trigwell, K., & Prosser, M. (1991). Improving the quality of student learning: The influence of learning context and student approaches to learning on learning outcomes. Higher Education, 22, 251266. doi:10.1007/BF00132290

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Weese, W.J. (2002). Opportunities and headaches: Dichotomous perspectives on the current and future hiring realities in the sport management academy. Journal of Sport Management, 16, 117. doi:10.1123/jsm.16.1.1

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Weissberg, R. (2010). Bad students, not bad schoolsNew Brunswick, NJ: Transaction Publishers. Retrieved from http://www.heritageanddestiny.com/publications/reviews/bad-students-not-bad-schools-by-robert-weissberg-book-review/

    • Search Google Scholar
    • Export Citation
  • Winston, R.B., Vahala, M.E., Nichols, E.C., Gillis, M.E., Wintrow, M., & Rome, K.D. (1994). A measure of college classroom climate: The college classroom environment scales. Journal of College Student Development, 35(1), 1118.

    • Search Google Scholar
    • Export Citation
  • Wyatt, J.N., Wiley, A., Camara, W.J., & Proestler, N. (2011). The development of an index of academic rigor for college readiness. (Research Report 2010–2011). New York, NY: The College Board. Retrieved from https://files.eric.ed.gov/fulltext/ED561023.pdf

    • Search Google Scholar
    • Export Citation
  • Zakus, D., Malloy, D.C., & Edwards, A. (2007). Critical and ethical thinking in sport management: Philosophical rationales and examples of methods. Sport Management Review, 10(2), 133158. doi:10.1016/S1441-3523(07)70008-6

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 900 486 0
Full Text Views 223 214 37
PDF Downloads 46 39 7