Translating predictors from research to practice

The best journals in orthodontics are more than platforms for researchers. They provide insights into improving patient care that are reliable, understandable (grounded in both theory and practice), and robust across multiple applications. With the help of computers, the past 30 years has witnessed an explosion in the rigor and sophistication in the ways we conduct research. This creates a more urgent need for building a strong bridge to bring orthodontic science into individual practices. This editorial is about a simple way practitioners can, and should, customize research reports to characteristics of their individual practices.

Research often reports estimates of the relationship between diagnostic predictors and clinical conditions or outcomes. A test showing that a gain in predictability is statistically significant usually warrants confidence that the predictor is worthy of transfer to practice. Unfortunately, such estimates are usually overestimates. Fortunately, we know how the correct the problem.

The following would be an example: In a university setting, a cohort study comparing a large number of surgery cases against a set of comparable size, matched for age and sex, found that a single angle differentiated the extraction from the non-extraction group by a significant margin. It was concluded that the angle is a predictor of needed surgery. Such claims are known as conditional statements. In symbolic notation, we say “the condition (C) depends on the evidence (E),” (C | E) where C = surgery and E = the identified angle of interest. Significant predictors take the form Pr (C | E₁) > Pr (C | E₂).

It is generally known that statistical significance of evidence is overstated when multiple predictors are tested for the same outcome, unless multivariate models are used or Bonferroni or similar adjustments are made to the α-value. It is also known that the statistical significance can be enhanced without improving predictive values by increasing the sample size. This note points out a further problem that the estimated predictive value of research is not the same as the estimate of impact in clinical settings.

The problem of directly transferring conditional probability estimates from research to practice hinges on differences in context. The standards for surgery may differ depending on who is making the decision. There is also an issue of confusion regarding conditional probabilities. The probability that prior conditions can be found associated with an outcome, a research question, is not the same as the probability that a prior condition will lead to certain outcomes, a practice question. Pr (E | C) ≠ (C | E). Especially when study cohorts are matched for size, the predictive strength of research overestimates the predictive strength of the same evidence in practice. Research findings must be interpreted for the contexts in which they are applied.

Happily, there is a straightforward way to reverse the direction of conditionals so that research estimates can be converted to meaningful practice estimates. As a bonus, this technique provides best estimates for practices that serve different populations or where practitioners have divergent treatment philosophies.

A made-up example will be used as a concrete illustration. Consider the example mentioned above of a matched cohort study looking for a predictor of the surgery decision. Assume that the conditional probability for E (classified as needing extraction) given C (positive predictor) is Pr (E | C) = .78. A smaller number of cases in the non-extraction group are false positives and also contain the predictor, say Pr (E | ∼C) = .46, for the sake of this example. The odds ratio in this case is larger than 4.0. We assume that the sample size has been chosen in advance so that such differences reach statistical significance.

This does not mean, however, that the research justified an expectation that 78% of patients identified as having the predictor should be extraction cases in every, or even in the average, practice. First, the division of extraction and non-extraction cases was determined by a researcher, using personal criteria which may differ from those used in various practices. Second, there was no test showing that patients selected for extraction by means of E experienced better occlusion than those receiving extraction without E. Third, there is the confusion of Pr (E | C) (the research question) with Pr (C | E) (the clinical treatment question).

The fix is easy and can be done with a hand-held calculator on one's cell phone. According to the Bayes theorem,¹ this situation can be expressed as follows: Applying the formula to the hypothetical example, were the researcher to use the study numbers to estimate the utility of the predictor in the university clinic: Applying the research estimate to a practice where only 25% of cases involve extractions: Table 1 illustrates representative examples of the relationship between presence of various predictor evidence under two baseline conditions and the likelihood of the target condition existing. Bold numbers show a range of estimates for expected probability in practice: what the clinician needs to know. These are a function of true positive and false positive findings from research, shown in italic: what is reported in the literature. The underlying baseline in a particular practice appears in normal font. These numbers suggest an interaction between sensitivity (the ratio of a true predictors to the combination of true and false predictors) and the prevalence of positive cases in the population one is working with. These tables are suggestive of the large swings in estimated importance of predictive values as they are applied across contexts.

Table 1.  Representative estimates of clinical likelihood of a treatable condition, Pr (C | E), given various prevalence in practice and estimates from the literature of likelihood that predictive evidence will and will not be associated with a preselected condition, Pr (E | C) and Pr (E | ∼C).

View Fullscreen

Similar logic applies to the translation of risk reported in the literature into practice. Risk is normally determined as the ratio of odds of presence or extent of a predictor associated with a condition, divided by another set of odds with its presence or predictor value. But risk is not prevalence. Patients classified as “high risk” may require less monitoring or intervention than patients with moderate or low risk for a different condition depending on the prevalence of the condition in the population. Men, for example, have a higher risk for obstructive sleep apnea than do women.² But that, alone, does not mean that all male patients should be tested for the condition. Additional predictors would normally be considered in forming a composite profile.

The foundation for this approach to translating research estimates to clinical settings is Bayesian statistics.³ Elaborate models for estimating prior probabilities in complex situations and with various research designs are possible, but the simple point estimates presented here are certainly within the practical range of clinical practice.

Orthodontics has an increasing range of tools to conduct rigorous research, especially in controlled academic settings. The Angle Orthodontist is among the leading research publications in presenting findings relevant to practice. The need and methods to adjust estimates for the value of predictors across various clinical settings is such an opportunity.