As a practice of more than 200 years,1 peer review is an important process to help ensure the standards and legitimacy of scientific research. Despite its limitations and emerging alternative scientific publishing models,2 peer review remains the mainstream process of “verifying” research papers. As current and former Editors of the Journal of Physical Activity and Health (JPAH), as well as active researchers who regularly submit our work to journals, we have dealt with thousands of reviewers’ reports and have had our fair share of gratitude, frustration, and pain. So, what makes for a good (or great) review? From our vantage point, we want to share some (and by no means complete) tips with fellow peer reviewers, especially those who have just recently embarked on their peer review journey. Through this editorial, we hope to start a conversation about how to improve the peer review process and the experience of our authors, with the goal of enhancing JPAH and research in our field.
Receiving Invitations for Peer Review
Congratulations! You have been invited to peer review an article, which means your expertise is recognized and valued. Here are a few considerations for you.
Do:
- •Check the legitimacy of the journal. Unfortunately, the ultracompetitiveness and the “publish or perish” culture in academia have bred an epidemic of “predatory publishing.” If the invitation comes from an unfamiliar journal, we recommend that you first investigate the journal. Recognizing a predatory journal is often (but not always) obvious. Tools, such as Beall’s list (https://beallslist.net) and “thinkchecksubmit” (https://thinkchecksubmit.org), could be helpful.
- •Carefully consider any potential conflict of interest. Conflict of interest could take different forms, such as financial, professional, or personal. Journals or publishers usually have their own definition for what is considered a conflict of interest. For example, Human Kinetics, publisher of JPAH, requires that authors “must identify potential conflicts of interest in the areas of financial, institutional, and/or personal relationships that might inappropriately influence their actions or statements.” (https://journals.humankinetics.com/page/author/authors) This extends to reviewers as well: “In the rare situation that you as the reviewer discover a potential conflict of interest in relation to the authors or content of the manuscript you have been invited to review, please contact the associate editor or editor as soon as possible.” (https://journals.humankinetics.com/page/editors/editors-reviewers) We generally recommend that if you feel that you cannot make an unbiased decision for any reason, it is better to declare conflict of interest and refrain from reviewing.
- •Consider the match between your expertise and the topic area. You may have received an invitation from a journal that you have always wanted to review for, and thus, it is tempting to accept this invitation. However, it is important to bear in mind that uninformed or insufficient peer review does science, and possibly your scientific reputation, a disservice.
- •Be generous, realistic, and communicative. In recent years, quality peer reviews have become increasingly hard to secure across research fields.3 It is important to keep in mind that peer review is considered a “service” to the scientific community. While most researchers will receive more invitations than they can realistically complete, we suggest that you determine your peer review workload based on the number of papers you submit. A rule of thumb is for every article you submit, review 3 to 5 so that you sufficiently give back to the academic community. Before accepting an invitation, make sure that you can complete the peer review on time. Whatever your decision, communicate rapidly with the handling editor so they can keep the process moving forward. If you are running late, please also let the editor know.
- •Think of your team. If you are not available to peer review, consider your colleagues who have the relevant expertise and may appreciate this opportunity. If you are available, consider bringing a graduate student, trainee, or junior staff to co-review with you (with the editor’s permission, of course). It is never too early to start peer review training and the best way to learn is by doing.
Don’t:
- •Feel obligated to accept an invitation. When you are not sure whether you can deliver on time, such as before major deadlines or a long holiday, decline an invitation.
- •Go on radio silence and keep the editor guessing. If your availability changes unexpectedly, editors would like to know so that they can move on and invite others. Unnecessarily prolonging the peer review process frustrates the authors and may prevent the research, if accepted, from being disseminated widely by the journal.
Evaluating an Article
Thank you for agreeing to review this article. Here, we share some considerations for evaluating the paper.
Do:
- •Know the journal. Before starting your peer review, it is important to familiarize yourself with the scope, standing, and publication criteria of the journal.4 Journals of different disciplinary backgrounds, ranking, and publication models may have different review criteria, acceptance rates, and expectations from peer reviewers.
- •Consider the significance, innovation, and relevance of the paper. Reviewers should consider whether the paper is meaningful, whether it is relevant to the readership of the journal, and whether it adds anything new. A question to ask yourself is: “Could this paper make a difference to the knowledge base and/or practice in this relevant field?”
- •Be vigilant of research and publication ethics. Major issues, such as fraud, plagiarism, and duplicate publishing, should be flagged and reported.5 Ambiguous issues, such as intentionally slicing a study into least publishable units,6 should also be checked and taken into account in the decision process. The rapid rise of easy-to-use artificial intelligence programs is also creating new challenges.
- •Assess the science. Did the authors cite the most relevant and up-to-date literature? Is the study design appropriate? Are the methods sound and the measures reliable and valid? What are the obvious sources of biases, and have the authors acknowledged or addressed these biases?
- •Check the interpretation of the findings. Are the authors cherry-picking, spinning, or overclaiming? Was the reporting transparent? Are the limitations of the study sufficiently discussed? Make sure you check other relevant documents, such as the supplementary files and registered protocols, if applicable, for a complete picture of the study.
- •Evaluate the writing. Is the article clearly, logically, and succinctly written? Is the language inclusive?7 Have the authors followed relevant reporting guidelines and checklists, such as Preferred Reporting Items for Systematic Reviews and Meta-Analyses8 for systematic reviews and STrengthening the Reporting of Observational studies in Epidemiology9 for observational studies? If the paper is confusing to read, does the confusion stem from logical or language issues, and can the readability be improved through language editing?
Don’t:
- •Discriminate based on where the study comes from. The geographic origin and/or the linguistic background and institutional affiliations of the authors (if the review process is unblinded) should not be factored into your decision making. Equity, diversity, and inclusion are at the core of JPAH’s “Going Global” commitment.10 Peer reviewers play an important role in ensuring equitable representation in published research.
- •Degrade a study because it comes from a different “camp,” “school of thought,” or value-based perspective. Different theoretical and methodological orientations exist in every field and such diversity advances science. You may disagree with the authors, but disagreements should not be the basis of recommending rejection.
- •Be blindsided by other sources of biases. Stay informed about current research on implicit bias and participate in training or workshops that address biases in the peer review process.
Writing the Report
You have read through the article and have reached a general verdict about it. Now, it is time to write up your reviewer’s report. Here are some items to consider.
Do:
- •Be kind, constructive, and polite. If we must pick the No. 1 rule for peer reviewing, it has to be this one. Reviewer etiquette is important.11 Even if you intend to recommend rejecting a paper, do so constructively. Think of your role as helping the authors do better research, not discouraging them by telling them about how weak their study is. Use language such as “The authors may consider ...” rather than “The authors are wrong ....” If you are unsure whether the wording is too harsh, try reading it to yourself and ask yourself whether this is something you’d like to hear as an author.
- •Start with a brief overview paragraph. This shows that you are correctly understanding the authors’ purpose and conclusions. In the overview, try to say at least one good thing about the study.
- •Be balanced. Even a weak study will have its strengths. The reviewer’s job is not to list as many flaws as possible or to show off how much more they know than the authors. Reviewers should give a balanced evaluation of the importance of the study, the rigor and appropriateness of the methods, the clarity of the writing and presentation, the accuracy of the interpretations, and the overall contribution to science and society.
- •Be clear and specific. Having a clear structure for your reviewer’s report facilitates editorial decision making and revision by the authors. Reviewers have different preferences for structures—some prefer grouping comments by themes, some by sections of the paper, and others by major and minor issues. All are good options as long as you clearly signpost them throughout your report. When you can, be specific. The more specific your comment is, the more likely the authors can address your comment and improve their paper.
- •Acknowledge your limitations and ask for help if necessary. We live in an age of rapidly expanding knowledge and increasing interdisciplinarity. Therefore, it is likely that even the most experienced reviewer will encounter new concepts and methods. In the case of not having the expertise to review specific aspects of the study (eg, novel statistical methods), it may be a good idea to declare it to the editor. JPAH recently established a methodological and statistical review board for precisely this reason. Reviewers are encouraged to flag to the editor that this paper needs to be seen by an expert to ensure the quality of the paper.
Don’t:
- •Confuse your role with a coauthor. While it is helpful to comment on the readability of the paper and provide examples of errors and typos that you have noticed along the way, don’t take over and rewrite the paper—this task is for the coauthors.
- •Disclose your decision to the authors. Reviewers make recommendations, editors make decisions. We generally recommend reviewers refrain from disclosing their recommendations in the report to authors as doing so can make the editor’s job harder.12
- •Be pedantic. Focus on the bigger picture and fundamental issues (particularly those that can be addressed). Especially when the paper is poorly written and you intend to recommend rejecting the paper, refrain from obsessing over every little detail, which could only discourage or infuriate authors even more. It may also be a waste of time for you.
- •Use peer review as an opportunity to impose your point of view. The peer reviewer’s report is the wrong place to tell people that they should take your perspective, apply your methodology, or cite your work.
- •Keep all your critiques confidential from the authors. Some journals provide options to write confidential comments to the editor. We think it is appropriate to use this option in specific cases, such as when you suspect ethical issues or have concerns about certain aspects of the peer review process, or if you want to provide a brief summary for the editor (so that they have less to read). However, in general, you should direct the bulk of your report to the authors, who may appreciate the transparency and can benefit from your comments.
Conclusion
Peer reviewing is essential, but it is not an easy task. The process (see Figure 1) requires various ethical, scientific, and practical considerations. Peer review strategies are rarely (if ever) taught in formal training programs. We believe that mentoring is key. We encourage senior researchers to support and nurture the next generation of peer reviewers to help them find their footing in peer review. By doing so, we can improve the overall capacity of peer reviewers, the quality of peer reviews, and the experience of authors and readers in physical activity research.
—The peer reviewing process requires various ethical, scientific, and practical considerations.
Citation: Journal of Physical Activity and Health 20, 11; 10.1123/jpah.2023-0328
Acknowledgment
The authors wish to thank the following colleagues for sharing their experiences and tips for peer reviewing: Adrian Bauman, Eun-Young Lee, Greg Heath, I-Min Lee, Jacqueline Mair, Ken Powell, Mike Pratt, Jim Sallis, Katja Siefken, and Esther van Sluijs.
References
- 1.↑
Burnham JC. The evolution of editorial peer review. JAMA. 1990;263(10):1323–1329. doi:10.1001/jama.1990.03440100023003
- 3.↑
Meyer T, Jay O, Altenburg T, et al. Where have all the reviewers gone? Discussing the importance of the peer review community. J Sci Med Sport. 2023;26(4):215–216. doi:10.1016/j.jsams.2023.05.002
- 4.↑
Dhillon P. How to be a good peer reviewer of scientific manuscripts. FEBS J. 2021;288(9):2750–2756. doi:10.1111/febs.15705
- 5.↑
Min SK. Ethics and responsibilities of peer reviewers to the authors, readers, and editors. Vasc Specialist Int. 2021;37(1):1–3. doi:10.5758/vsi.213711
- 6.↑
Ding D, Nguyen B, Gebel K, et al. Duplicate and salami publication: a prevalence study of journal policies. Int J Epidemiol. 2020;49(1):281–288. doi:10.1093/ije/dyz187
- 7.↑
Brett S, Kamran M, Javier M, et al. Disability, the communication of physical activity and sedentary behaviour, and ableism: a call for inclusive messages. Br J Sports Med. 2021;55(20):1121. doi:10.1136/bjsports-2020-103780
- 8.↑
Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–269. doi:10.7326/0003-4819-151-4-200908180-00135
- 9.↑
von Elm E, Altman DG, Egger M, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med. 2007;147(8):573–577. doi:10.7326/0003-4819-147-8-200710160-00010
- 10.↑
Ding D, Carson V, Hunter RF, et al. Science has no Borders, so Should Scientific Publishing: A position statement from the Journal of Physical Activity and Health. J Phys Act Health. 2022;19(12):809–810. doi:10.1123/jpah.2022-0570
- 11.↑
Durning SJ, Sklar DP, Driessen EW, et al. “This manuscript was a complete waste of time”: reviewer etiquette matters. Acad Med. 2019;94(6):744–745.
- 12.↑
Rees L, Flynn J. Supportive peer review: why and how to constructively review a paper. Pediatr Nephrol. 2022;37(8):1701–1703. doi:10.1007/s00467-022-05535-z
