Reference: JAMA Intern Med. 2023 Mar 27;e230325
Practice Point: While the idea that preop consultations save lives is still an unproven assumption, this study doesn’t prove that they are harmful either.
EBM Pearl: When it comes to observational studies, larger E-values strengthen our confidence in the conclusions.
If you haven’t played the online game Wordle, it involves guessing a random 5 letter word in six tries. Each attempt results in feedback about the letters and placement you have picked. For example, if you guess “preop” and the correct answer is “opera”, you will be told that the “e” is the correct letter in the correct place, but the “p”, “r”, and “o” are correct letters in the wrong place.
Preoperative consultations are an attempt to prevent perioperative complications in people with health problems needing elective surgery (for example, someone with severe aortic stenosis who needs a hip replacement). This assumes that medical advice and interventions help improve surgical outcomes. But do they? Researchers in Ontario just tried to figure this out.
The authors took advantage of data-mining to retrospectively gather information on 540,000 patients who underwent elective non-cardiac surgery over 13 years in Ontario, 35% of whom had preoperative consultations from a variety of primary care providers and specialists. Let’s face it — there will NEVER be a randomized controlled study on a group this large — this observational data is a veritable gold mine!
Looking closely at the two groups, however, people who received preop consultations were riskier surgical candidates in every way possible. They were older, had more health problems to begin with, underwent more dangerous surgeries, and were more likely to undergo formal preoperative anesthesiology consultations rather than routine screening.
Because of these differences, researchers used multivariable regression and propensity analysis to try to adjust for this discrepancy. They identified variables such as age, comorbidities, and type of surgery ahead of time that in their opinion affected the safety of surgery. Then, the investigators used modeling based on these variables to match patients who received consultations with those who didn’t. After using their expert-constructed propensity model to make adjustments, they found that patients who got preoperative consultations were more likely to experience many types of bad outcomes, from prolonged hospital stay to 30-day postop mortality (0.9% compared to 0.7%, with an adjusted odds ratio [OR] of 1.19). Case closed, preop consults are bad, right?
More like case opened. We decided to take a closer look at the methodology around the propensity scores, and we have at least two concerns. First, we take issue with the dataset classification of patients with certain heterogeneous diseases like diabetes and heart failure as binary, simply as either present or absent. This greatly oversimplifies conditions with a huge variation in severity, symptoms, and complications. Misclassifying linear data as binary is a threat to validity for any propensity score. Secondly, after applying their propensity score for matching, the team calculated an E-value to estimate the likelihood that their propensity matching addressed all important confounders. E-values are a calculation performed on observational data that asks an intriguing question: if there WAS a confounding association between an intervention and the effects of the intervention that researchers did not account for, how big would that association need to be to mess up the validity of the results? A small E-value implies that even a small unmeasured confounder could throw the conclusion in doubt.
In this study, the authors used the derived OR to calculate an E-value of 1.64. This means that an unmeasured confounder would need an OR ≥ 1.64 to potentially reduce the observed association between preoperative consultation and 30 day mortality to within the null range. While this E-value is relatively low (which is not a good thing), no set E-value threshold applies to all situations. Like propensity scores themselves, the E-value threshold is subjectively determined by expert opinion and dependent on context. Here, researchers compared this value (1.64) with the E-value (1.95) from a study examining the risk of heart failure on 90 day perioperative mortality. We don’t think that was such a great comparator, since 30- and 90-day mortality are different things and the patients in the heart failure study were overwhelmingly asymptomatic. By the way, when that comparator study classified heart failure in a linear rather than binary fashion, symptoms were strongly associated with a risk of perioperative death. Simply put, we feel an unmeasured confounder with an OR of 1.64 is pretty likely. It doesn’t prove that the researchers are wrong, but it weakens the strength of their stated conclusion that preoperative medical consultation was associated with an increase in adverse postoperative outcomes.
How does this relate to Wordle? With Wordle, each guess is binary: right or wrong. Before making statements about causality from observational data (these authors do not, but the lay press certainly does) every single piece of a study should line up in the right place. Having linear variables classified as binary and lowish E-values are not winning strategies. When sick people need dangerous surgery, we think “preop” is still probably the correct answer.
For more information, see the topic Preoperative Evaluation and Management for Adults in DynaMed.
DynaMed EBM Focus Editorial Team
This EBM Focus was written by Dan Randall, MD, Deputy Editor at DynaMed. Edited by Alan Ehrlich, MD, Executive Editor at DynaMed and Associate Professor in Family Medicine at the University of Massachusetts Medical School; Katharine DeGeorge, MD, MS, Deputy Editor at DynaMed and Associate Professor of Family Medicine at the University of Virginia; Nicole Jensen, MD, Family Physician at WholeHealth Medical; Vincent Lemaitre, PhD, Senior Medical Writer at DynaMed; Elham Razmpoosh, PhD, Postdoctoral fellow at McMaster University; and Sarah Hill, MSc, Associate Editor at DynaMed.