EBVM Toolkit 9: cohort study checklist

Introduction

Critical appraisal is a process which is used to help you identify the strengths and weakness of a research paper.  Understanding how appropriate the study design is for the question you are seeking to answer, how well the study was carried out, and how good the reporting in the paper is helps you to assess whether the paper is likely to provide reliable evidence.  

This page is designed to help you appraise the report of a cohort study.  Answering the questions will help you to reflect on how valid the results might be, how well reported they are and whether they are applicable to your local circumstances.

Cohort study checklist

For each question think about whether the answer is yes, no or not sure and what your reasoning is for that answer.

1. Did the study address a clearly focused question?

Are the patient/population and risk factors clearly stated? Is the study looking for a beneficial or harmful effect?

2. Was the cohort recruited in an appropriate way?

Was the cohort representative of a defined population? Was there anything special about the cohort? Were all animals included who should have been?

3. Was the exposure accurately measured to minimise bias?

Were the measurements objective or subjective? Were the measurements able to detect what was expected? Have the measurements been validated? Were the subjects classified into exposure groups using the same procedure?

4. Was the outcome accurately measured to minimise bias?

Were the measurements objective or subjective? Were the measurements able to detect what was expected? Have the measurements been validated? Was there a reliable system for detecting all the cases? Were the measurement methods similar in the different groups? Were the subjects and/or outcome assessors blinded to the exposure? Is this important?

5. What confounding factors have the authors accounted for?

List the ones you think are important.

6. Have confounding factors been taken into account in the design and or analysis

Confounding occurs when the link between exposure and outcome is distorted by another factor. These should be in the methods section. Look for factors that were not considered according to your clinical judgment. A study that does not address confounding should be rejected.

7. How adequate was the follow up of the subjects?

Was it complete enough? Long enough? Were all the subjects accounted for at the end? Do you think that those lost to follow up may have had different outcomes?

8. What are the results of the study?

What are the bottom line results? How strong is the association between exposure and outcome? Is there a relative risk? What is the absolute risk reduction? If not presented can you calculate it from the results presented?

9. How precise was the estimate of risk?

Look for confidence intervals.

10. Do you believe the results?

A large effect has to be taken seriously. Can the result be due to chance? Have you spotted flaws that make the results unreliable? Was a cohort study the best method to answer the question?

11. Can the results be applied to your practice?

Are the subjects similar to your population? Does your setting differ significantly? Can you gauge benefit and harm for your local situation?

12. Do the results fit with other available evidence?

Consider evidence from other study designs for consistency.

13. What are the implications of this study for your practice?

Is the evidence from this study robust enough to make a decision? Recommendations from observational studies are stronger when supported by other evidence.

Try it out yourself

You could use the following paper to try out the questions:

Krontveit, R.I. et al (2012) Risk factors for hip-related clinical signs in a prospective cohort study of four large dog breeds in Norway. Preventative Veterinary Medicine, 103 (2-3), pp. 219-227. DOI: https://doi.org/10.1016/j.prevetmed.2011.09.018