External validity

External validity refers to the applicability of the study results to a larger population. External validity asks to what degree the results would be true for different people in different places. The most common loss of external validity comes from initially drawing a small study sample population from a single geographic location.

Questions to consider concerning external validity:

• How representative of the larger population are the study groups?
• How large are the samples?
• How diverse are the samples?
• Do the demographics of the study samples match the demographics of the larger population?

Clinical significance

After assessing the internal and external validity of a research article, the next step is to determine whether the research is clinically significant. Are the results of the research applicable to your patient or clinical scenario? Is the experimental treatment or test available in your clinic? Is it cost effective? The answers to these questions are largely individual as practitioners have varying thresholds of acceptable risk and cost restraints when applying a new treatment or using a new diagnostic tool. Please note that clinical significance is different from statistical significance.

Therapy

When comparing two groups in a clinical trial, the differences between the groups may be expressed either as an Absolute Risk Reduction (ARR) or as a Relative Risk Reduction (RRR). ARR is the rate of no change/negative outcome in the control group minus the experimental group. RRR is the the ARR divided by the negative/no change outcomes rate in the control. The RRR can make the benefit of a drug or a treatment appear larger than it is; therefore, using the ARR is more reliable for Evidence-Based Nursing.

To calculate the ARR, subtract the experimental event rate (EER) from the control event rate (CER). EER and CER are both percentages of the treatment effect; EER is the percentage of participants in the experimental group that experienced no change/negative outcome, CER is the percentage of participants in the control group that experienced no change/negative outcome. In simple terms, the ARR is the benefit of the new treatment.

Another valuable concept is the Number Needed to Treat, NNT. The NNT is simply the number of patients a physician needs to treat in order for one of those patients to benefit from the treatment. NNT is calculated by dividing 1 by the ARR. The NNT is a very useful clinical decision-making tool.

The following table provides an example of calculating the CER, EER, ARR, and NNT.

• Control Event Rate: 88/114 = 0.77 or 77%
• Experimental Event Rate: 61/120 = 0.51 or 51%
• Absolute Risk Reduction (CER - EER): 0.77 - 0.51 = 0.26 or 26%
• Number Needed to Treat: 1/0.26 = 4

In this example, one would need to treat four patients for one of those patients to benefit from the treatment. Thankfully, there are online NNT calculators that we may use, such as this one, created by Dr. Alan Schwartz.

Diagnosis

Sensitivity and specificity are two measures that describe the efficacy of a diagnostic tool in comparison to the reference standard. Sensitivity is the proportion of people with the target disorder who have a positive test result. Specificity is the proportion of people without the target disorder who have a negative test result.

Keep in mind a diagnostic tool might include the presence of symptoms, a screening tool, or other means of determining target disorders.

Two commonly used mnemonics for sensitivity and specificity are:

SpPin

When a test has a high specificity a positive result rules in the diagnosis.

SnNout

When a test has a high sensitivity a negative result rules out the diagnosis.

In order to calculate the sensitivity and specificity of a diagnostic tool, you need the test results for both the index tool and the reference standard. The following table is an example of how you would start your calculations.

• Sensitivity = A/(A+C) = 95/100 = 95%
• Specificity = D/(B+D) = 800/900 = 89%

We can see that this test provides a sensitivity of 95% and a specificity of 89%.

The likelihood ratios (LR) are also important for clinical practice. The LR is "the likelihood that a given test result would be expected in a patient with the target disorder compared with the likelihood that the same result would be expected in a patient without the target disorder."[4] For a more detailed explanation of likelihood ratios, please visit the Centre for Evidence-Based Medicine's page on Likelihood Ratios.

• Positive LR = Sensitivity/(1-Specificity) = 95%/11% = 8.63
• Negative LR = (1-Sensitivity)/Specificity) = 5%/89% = .06

Again, thankfully, we do not need to perform these calculations by hand, we can use a diagnostic test calculator.

Return to tutorial homepage >>
Proceed to tutorial evaluation and exit >>