National Surgical Quality Improvement Program
The ACS National Surgical Quality Improvement Program (ACS NSQIP®) is a nationally validated, risk-adjusted, outcomes-based program to measure and improve the quality of surgical care. Built by surgeons for surgeons, ACS NSQIP provides participating hospitals with tools, analyses, and reports to make informed decisions about improving quality of care. Further, peer-reviewed studies have shown that ACS NSQIP is effective in improving the quality of surgical care while also reducing complications and costs.
It was started in the American Veterans Health Administration (VHA). In the mid-1980s the VHA was criticized for their high operative mortality. To that end, Congress passed Public Law 99-166 in December 1985 which mandated the VHA to report their outcomes in comparison to national averages and the information must be risk-adjusted to account for the severity of illness of the VHA surgical patient population. In 1991 the National VA Surgical Risk Study (NVASRS) began in 44 Veteran's Administration Medical Centers. By 31 December 1993 there was information for 500,000 non-cardiac surgical procedures. In 1994 NVASRS was expanded to all 128 HVA hospitals that performed surgery. The name was then changed to the National Surgical Quality Improvement Program.
Reponen et al. recorded outcome event rates and categorized them according to British Neurosurgical National Audit Programme (NNAP), American National Surgical Quality Improvement Program (NSQIP), and American National Neurosurgery Quality and Outcomes Database (N2QOD) to assess the applicability of these programs for quality benchmarking and estimated sample sizes required for reliable quality comparisons.
The rate of in-hospital major and minor morbidity was 18.7% and 38.0%, respectively, and 30-d mortality rate was 2.4%. The NSQIP criteria identified 96.2% of major but only 38.4% of minor complications. N2QOD performed better, but almost one-fourth (23.2%) of all patients with adverse outcomes, mostly minor, went unnoticed. For NNAP, a sample size of over 4200 patients per surgeon is required to detect a 50.0% increase in mortality rates between surgeons. The sample size required for reliable comparisons between the rates of complications exceeds 600 patients per center per year.
The implemented benchmarking programs in the United Kingdom and the United States fail to identify a considerable number of complications in a high-volume center. Health care policymakers should be cautious as outcome comparisons between most centers and individual surgeons are questionable if based on the programs 1).
Data on patients who underwent surgical clipping of an unruptured aneurysm were extracted from the prospective National Surgical Quality Improvement Program registry (NSQIP; 2007-2014); NSQIP does not systematically collect data on patients undergoing intracranial endovascular intervention. Multivariable logistic regression evaluated predictors of any 30-day adverse event; variables screened included patient demographics, comorbidities, functional status, preoperative laboratory values, aneurysm location/complexity, and operative time. A predictive scale was constructed based on statistically significant independent predictors, which was validated using both NSQIP (2015-2016) and the Nationwide Inpatient Sample (NIS; 2002-2011).
The NSQIP unruptured aneurysm scale was proposed: 1 point was assigned for a bleeding disorder; 2 points for age 51-60 years, cardiac disease, diabetes mellitus, morbid obesity, anemia (hematocrit < 36%), operative time 240-330 minutes; 3 points for leukocytosis (white blood cell count > 12,000/μL) and operative time > 330 minutes; and 4 points for age > 60 years. An increased score was predictive of postoperative stroke or coma (NSQIP: p = 0.002, C-statistic = 0.70; NIS: p < 0.001, C-statistic = 0.61), a medical complication (NSQIP: p = 0.01, C-statistic = 0.71; NIS: p < 0.001, C-statistic = 0.64), and a nonroutine discharge (NSQIP: p < 0.001, C-statistic = 0.75; NIS: p < 0.001, C-statistic = 0.66) in both validation populations. Greater score was also predictive of increased odds of any adverse event, a major complication, and an extended hospitalization in both validation populations (p ≤ 0.03).
The NSQIP unruptured aneurysm scale may augment the risk stratification of patients undergoing microsurgical clipping of unruptured cerebral aneurysms 2).
In a study of Senders et al. from Boston and Utrecht, patients were extracted from the National Surgical Quality Improvement Program registry (2005-2015) and analyzed using multivariable logistic regression.
A total of 7376 patients were identified, of which 948 (12.9%) experienced a major complication. The most common major complications were reoperation (5.1%), venous thromboembolism (3.5%), and death (2.6%). Furthermore, 15.6% stayed longer than 10 d, and 11.5% were readmitted within 30 d after surgery. The most common reasons for reoperation and readmission were intracranial hemorrhage (18.5%) and wound-related complications (11.9%), respectively. Multivariable analysis identified older age, higher body mass index, higher American Society of Anesthesiologists (ASA) classification, dependent functional status, elevated preoperative white blood cell count (white blood cell count WBC, >12 000 cells/mm3), and longer operative time as predictors of major complication (all P < .001). Higher ASA classification, dependent functional status, elevated WBC, and ventilator dependence were predictors of extended length of stay (all P < .001). Higher ASA classification and elevated WBC were predictors of reoperation (both P < .001). Higher ASA classification and dependent functional status were predictors of readmission (both P < .001). Older age, higher ASA classification, and dependent functional status were predictors of death (all P < .001).
This study provides a descriptive analysis and identifies predictors for short-term complications, including death, after craniotomy for primary malignant brain tumors 3).
Current outcomes prediction tools are largely based on and limited by regression methods. Utilization of machine learning (ML) methods that can handle multiple diverse inputs could strengthen predictive abilities and improve patient outcomes. Inpatient length of stay (LOS) is one such outcome that serves as a surrogate for patient disease severity and resource utilization.
To develop a novel method to systematically rank, select, and combine ML algorithms to build a model that predicts LOS following craniotomy for brain tumor.
A training dataset of 41 222 patients who underwent craniotomy for brain tumor was created from the National Inpatient Sample. Twenty-nine ML algorithms were trained on 26 preoperative variables to predict LOS. Trained algorithms were ranked by calculating the root mean square logarithmic error (RMSLE) and top performing algorithms combined to form an ensemble. The ensemble was externally validated using a dataset of 4592 patients from the National Surgical Quality Improvement Program. Additional analyses identified variables that most strongly influence the ensemble model predictions.
The ensemble model predicted LOS with RMSLE of .555 (95% confidence interval, .553-.557) on internal validation and .631 on external validation. Nonelective surgery, preoperative pneumonia, sodium abnormality, or weight loss, and non-White race were the strongest predictors of increased LOS.
An ML ensemble model predicts LOS with good performance on internal and external validation, and yields clinical insights that may potentially improve patient outcomes. This systematic ML method can be applied to a broad range of clinical problems to improve patient care 4).