Checklist of questions to ask when critiquing the typical quantitative research report.
From Vogt (2007), (p. 300):
A. First, and in General
1. What is the hypothesis or the research question, if any, guiding the research?
2. Why do the authors believe it is important to investigate this hypothesis/question?
3. What methods did the authors use to collect evidence? What was their design?
4. Were the methods appropriate to address this problem/question?
5. What are the main findings or conclusions of the article?
6. Are the conclusions convincing?
B. Questions about the Variables
7. What is the dependent or outcome variable (OV)?
8. What are the independent or predictor variables (PVs)?
10. Should these or any other mediating/intervening variables have been studied?
11. Are any control variables considered?
12. Should other control variables have been examined?
13. Does the article discuss the possible moderating variables and interaction effects? Should it?
14. How are the variables defined and measured; that is, how are they operationalized?
15. Are the definitions and measurements of the variables appropriate for this study?
C. Questions about the Sample/Subjects
16. Who is studied and are the subjects appropriate given the goals of the study?
17. How many are studied and is this enough for the purposes of the study?
18. Is the sample representative of a population? How broadly can the conclusions be generalized?
D. Questions about the Conclusions
19. Are the findings statistically significant?
20. Are the findings scientifically significant?
21. How big are the effects discovered?
22. Are the findings practically significant?
23. Are the conclusions really supported by the evidence cited in the article?
E. Finally and Implied in the Answers to the Above Questions
24. How could the research have been improved?
25. What questions or problems does the article leave unanswered?
26. How could you go about doing a better job?
Vogt, P.W., (2007). Quantitative research methods for professionals. Boston, MA: Allyn and Bacon.