Levels of Evidence

DynaMed provides easy-to-interpret Level of Evidence labels so users can quickly find the best available evidence and determine the quality of the best available evidence. Evidence may be labeled in one of three levels:


Level 1

Level 1 (likely reliable) Evidence - representing research results addressing clinical outcomes and meeting an extensive set of quality criteria which minimizes bias.

Levels of evidence for conclusions derived from individual studies: 

  • DynaMed criteria for level 1 (likely reliable) evidence for interventional conclusion (conclusions that an intervention does or does not change an outcome)  
    1. Full-text report available in English (or language well understood by participating editor)
    2. Clinical outcome (also called patient-oriented outcomes)
    3. Population, intervention, comparison, and outcome in the study is representative of expected clinical practice
    4. Random allocation method (i.e. not assigned by date of birth, day of presentation, “every other”)
    5. Blinding of all persons (patient, treating clinician, outcome assessor) if possible
    6. Follow-up (endpoint assessment) of at least 80% of study entrants AND adequate such that losses to follow-up could not materially change the results
    7. Accounting for dropouts (even if not included in analysis)
    8. Confidence intervals do not include both presence and absence of clinically meaningful differences
    9. Consistency of findings across measures of similar outcomes
    10. In cases of randomized parallel-group trials
      • Allocation concealment
      • Intention-to-treat analysis comparing groups according to randomization
    11. In cases of randomized crossover trials
      • Trial conducted in patients with condition not expected to change spontaneously during course of trial
      • Random allocation method for order of assignment
      • Washout period between interventions long enough to avoid carryover effects between interventions
      • Adequate duration of intervention and assessment period to represent outcome being measured
      • Analysis of paired data
      • Analysis not suggesting period effects (i.e. effect resulting for order of intervention), or period effects if present not materially changing results
    12. In cases of early trial termination
      • Stopping decision made by independent monitoring board without competing interests
      • Interim analysis preplanned
      • Statistical stopping rule accounts for multiple assessments (lower p value threshold) for early termination benefit
      • Clinically significant differences with absolute benefit/harm warranting early termination
      • For classification of level of evidence for a specific outcome (which may be different than outcome used for stopping decision), outcome has sufficient statistical results such that trial continuation would be unlikely to change these results
    13. No other factors contributing to substantial bias, such as
      • Differences in management between groups other than the intervention being studied
      • Differential loss to follow-up
      • Posthoc analysis
      • Subgroup analysis
      • Baseline differences between groups
      • Unclear how missing data are accounted for if possible 
  • DynaMed criteria for level 1 (likely reliable) evidence for a diagnostic conclusion 
    1. Full-text report available in English (or language well understood by participating editor)
    2. Patient sample represents patients for whom testing would be appropriate (i.e. diagnostic uncertainty)
    3. Reliable reference standard
    4. Test under investigation is representative of how test would be conducted in clinical practice
    5. Reference standard and test under investigation each applied to all study subjects (with and without diagnosis)
    6. Test under investigation conducted blinded to and independent of reference standard results
    7. Reference standard conducted blinded to and independent of study test results
    8. Adequate follow-up and accounting for subjects
    9. Confidence intervals do not include both presence and absence of clinically meaningful differences
    10. Test studied in independent validation cohort (i.e. a cohort that is independent from the derivation of the test or specific cutoff value being investigated)
    11. No other factors contributing substantial bias
  • DynaMed criteria for level 1 (likely reliable) evidence for prognostic conclusion
    1. Full-text report available in English (or language well understood by participating editor)
    2. Inception cohort study
    3. Prospective follow-up
    4. Representative sample at a similar course in the disease
    5. Follow-up sufficiently long and complete
    6. Systematic (unbiased) evaluation of outcomes
    7. Adjustments for important confounding factors
    8. Confidence intervals do not include both presence and absence of clinically meaningful differences
    9. Additional criteria for prediction rules
      • Validation in relevant population
      • Validation in sample independent from derivation cohort
    10. No other factors contributing to substantial bias

Levels of evidence for conclusions regarding a body of evidence:

  • DynaMed criteria for level 1 (likely reliable) evidence for conclusions from a systematic review:
    1. Full-text report available in English (or language well understood by participating editor)
    2. Clinical outcome (also called patient-oriented outcomes)
    3. Systematic search
    4. Explicit inclusion criteria that do not appear to inappropriately exclude important evidence
    5. Systematic selection of included studies
    6. Key factors analyzed from included studies are representative of expected clinical practice
      • For interventional conclusions key factors are population, intervention, comparison, and outcome
      • For diagnostic conclusions key factors are population, test under investigation, and reference standard
      • For prognostic conclusions key factors are population and outcome
    7. Evaluation of study quality sufficient to determine if individual studies meet level 1 evidence criteria
    8. Consistency of findings across studies (whether or not meta-analysis performed)
    9. Consistency of findings across measures of similar outcomes
    10. Additional criteria if meta-analysis
      • Studies are clinically appropriate for pooled analysis (reasonably similar populations, interventions, methodology, and outcomes)
      • Meta-analysis not limited by statistically significant heterogeneity
    11. Conclusion based on 1 or more primary studies meeting Level 1 evidence criteria
    12. Confidence intervals do not include both presence and absence of clinically meaningful differences
    13. No strong suspicion of publication bias
    14. No other factors contributing substantial bias, such as subgroup analysis or indirect comparisons
  • DynaMed criteria for level 1 (likely reliable) evidence for conclusions from a synthesis of systematic literature surveillance:
    1. Conclusions are consistent with 1 or more primary studies meeting Level 1 evidence criteria
    2. Key factors analyzed from included studies are representative of expected clinical practice
      • For interventional conclusions key factors are population, intervention, comparison, and outcome
      • For diagnostic conclusions key factors are population, test under investigation, and reference standard
      • For prognostic conclusions key factors are population and outcome
    3. MEDLINE search conducted to exclude conflicting evidence (to supplement systematic literature surveillance)
    4. Consistency of findings across studies
    5. Consistency of findings across measures of similar outcomes
    6. No strong suspicion of publication bias

Read Less

Level 2

Level 2 (mid-level) Evidence - representing research results addressing clinical outcomes, and using some method  of scientific investigation, but not meeting the quality criteria to achieve level 1 evidence labeling. 

Level 3

Level 3 (lacking direct) Evidence - representing reports that are not based on scientific analysis of clinical outcomes. Examples include case series, case reports, expert opinion, and conclusions extrapolated indirectly from scientific studies.

Grades of Recommendation

Guideline producers are now frequently using classification approaches for their evidence and recommendations, and these classifications are recognized and requested by guideline users. When summarizing guideline recommendations for DynaMed users, the DynaMed Editors are using the guideline-specific classifications and providing the guideline classification approach when this is done.