A Case Report for Quality Assurance: Understanding the NSQIP Database and Preventing Error in Peer Review

¹Laparoscopic Institute for Gynecology and Oncology, 40 Buckeye, Portola Valley, CA, USA

²Gynecologic Oncology Division of Gynecology Department of Surgery, Palo Alto VA Medical Center, 3801 Miranda Avenue Palo Alto, CA, USA

*Corresponding Author: Katherine A O’Hanlan, Laparoscopic Institute for Gynecology and Oncology, 40 Buckeye, Portola Valley, CA, USA, 94028-8015; Tel: 650-245-3250; Email: [email protected]

Received Date: January 11, 2024

Publication Date:  February 09, 2024

Citation: O’Hanlan KA, et al. (2024). A Case Report for Quality Assurance: Understanding the NSQIP Database and Preventing Error in Peer Review. Mathews J Case Rep. 9(2):149.

Copyright: O’Hanlan KA, et al. © (2024)

ABSTRACT

Background: This report assesses the accuracy and applicability of the American College of Surgeons National Surgical Quality Improvement Program (NSQIP) and MIDAS data to measure quality in a Gynecologic Oncology service. Method: A retrospective chart review (Canadian Task Force classification II-1) from a community hospital Quality Assurance committee’s evaluation of a Gynecologic Oncology service, assessing all patients undergoing any surgery on the Gynecologic Oncology service from January, 2014 to September, 2017. Results: Surgical data was tabulated from operative notes and office charts. NSQIP data was provided by the hospital Quality Assurance Department’s applications licensed from the American College of Surgeons. Proprietary MIDAS “Inpatient Takeback Rate” was provided by the hospital Quality Assurance Department. Conclusion: Hand-counting of hospital cases provided the most accurate and consistent results. NSQIP data provided variable accuracy in abstraction and coding but was limited to hysterectomy procedures. The MIDAS calculation was broadly inaccurate and should not be viewed as a quality indicator. Misinterpretation of quality data by a QA Department can adversely affect a surgeon’s practice. To further increase the accuracy and utility of the NSQIP database for Oncologic Gynecologists, suggestions for specific queries for Gynecologic Oncology and Gynecology case abstractions are made.

Keywords: Gynecologic oncology; Minimally invasive surgery; complications; laparotomy: National Surgical Quality Improvement Program; NSQIP, MIDAS, sham peer review

INTRODUCTION

Quality assessment of surgical services in community hospitals can be challenging. While many community hospitals have initiated surgical quality improvement projects to establish and meet quality benchmark data, misunderstanding of QA data can affect decisions about outcomes. Large data registries have been created to collect clinical information on surgical patients to reliably measure and improve surgical outcomes. In 1994, the American College of Surgeons (ACS) established the National Surgical Quality Improvement Project (NSQIP), a data registry to monitor the rates and correlates of complications for commonly performed General Surgical services in the VA hospitals in which it was initiated [1]. The NSQIP data monitoring algorithm resulted in a dramatic reduction of serious complications from 17.4% to 9.9% and was adopted by many university and community hospitals, and subsequently applied to multiple specialties [2].

NSQIP participation requires trained nursing personnel to abstract data through a detailed chart review process. The data from individual hospitals is submitted for analysis and comparison with national data to obtain meaningful quality feedback for hospitals, practices and surgeons. By 2021, 685 hospitals had contributed data on 983,851 cases enabling creation of quality standards for 14 commonly performed major procedures, including hysterectomy and prolapse support procedures by Gynecology surgeons and services [3]. Demographic data, operative information, established pre-operative risk factors, and 30-day phone follow-up of post-operative complications are submitted on all patients in community and small participating hospitals undergoing hysterectomy and pelvic prolapse surgery [4].

While all specialty-trained Gynecologists learn abdominal and laparoscopic surgeries of the reproductive organs to address pain, bleeding, infections and other conditions, Gynecologic Oncologists (GO) undergo two to four extra years of training to address malignancies originating in the reproductive organs. GO’s also operate on patients who do not have cancer, but whom General Gynecologists may be uncomfortable managing.

Gynecologic cancers metastasize throughout the abdomen and can invade urologic [5,6] intestinal, [7,8] and vascular organs [9] requiring purposeful incision or resection to remove cancer and leave minimal volume of tumor for curative intent [10] These procedures include radical debulking surgeries, hysterectomies, pelvic and aortic lymphadenectomies, intestinal and urologic surgeries and other major procedures that General Gynecologists are not trained to perform [11].

Oncoreductive procedures optimally remove all visible cancer, and can require incidental incisions into bladder, ureter, vasculature, intestines, abdominal wall and other organs. These incisions, incident to the surgical goals, are not accidents, but require careful address and successful repair to restore or maintain normal function. If the repair is inadequate, then the resulting organ breakdown causes immediate sickness, and necessitates a return to surgery. Accidental injuries can also occur during surgery, due to the unplanned puncture or incision of an organ resulting in unplanned emergency repair or a return to surgery.

Counting all components of a surgical encounter can give useful information about a procedure’s risk or surgeon’s skill, but does not discern between accidents and incidents. Two major categories of complications can always be considered accidents: infections [12] and unplanned return to the operating room ROR is also called takebacks [13]. NSQIP coders are instructed to report these two events, as they are obvious evidence of surgical risk. NSQIP coders may not recognize that enterotomy, cystotomy, ureterotomy, and vasculotomy incidental to the primary procedure and successfully addressed should not be considered complications, thus miscode an intended event as a complication. As Gynecologic Oncology knowledge and procedures have advanced, standards have also evolved recognizing that more aggressive surgical dissections that leave no visible residual cancer can have greater potential for prolongation of life and cure [14] commensurate with more frequent complications [15-18].

One advantage of the large NSQIP registry is that surgeons can access the database and extract data on specific procedures or diagnoses to generate outcome information and improve surgical techniques for a specific diagnosis. (e.g. endometriosis, [19] ovarian cancer, [20] endometrial cancer, [21-23] prolapse, [24-26] or procedure (e.g. benign laparoscopic hysterectomy, [27,28] radical hysterectomy, [29] lymphadenectomy, [22,30] or surgical approach (e.g. laparoscopy or laparotomy) [23,31,32] or complication (urologic injury, [33] infection, [34-36] intestinal injury, [37-39] unplanned return to the operating room, [40,41], or cost-accounting [21] by subspecialty service, [16] and more recently for pre-operative risk factors such as frailty index, [42,43] nutrition, [43-45] and body mass index [46,47]. One disadvantage of the NSQIP database is that only hysterectomy cases are included, thereby omitting many major procedures performed on patients who have already had or did not need hysterectomy.

In attempts to describe complications and establish standards of care within the subspecialty, Gynecologic Oncologists have extracted and published Gynecologic Oncology specific data from the NSQIP database by stratifying to include procedure codes performed only by Oncologists (e.g. cancer debulking, bowel resection, or radical hysterectomy; [20,48-51,29]) or only the diagnosis code of a gynecological cancer. [42,48,52,53] (Table 1, in bold) Non-NSQIP retrospective practice data reviews by larger Oncology services [31,54,55] have similarly analyzed and reported their own surgical data for specific oncology diagnoses, procedures or entire services, often using NSQIP-type stratifications. [6,7,15,17,56-61] (Table 1, not in bold) These peer-reviewed manuscripts are retained in the National Congressional Biotechnology Information (NCBI) library and can be obtained for comparison of a local surgeon’s or entire service’s data with NSQIP standards for similar procedures, diagnoses, or services.

Table 1: Publications listing complication data for Gynecologic Oncologist’s practices, using NSQIP data or employing NSQIP-style of data analysis, show overall complication rates for open laparotomy cases ranging from 9 to 37%, with infection rates ranging from 4 to 33%, and serious takeback complications ranging from 4 to 24%. Among the laparoscopy complications in Gynecologic Oncologist’s services overall rates range from 4 to 19%, with infections ranging from 2 to 3.8% and serious take back complications ranging from 2 to 7.4%. Reviewing entire services of Gynecologic Oncologist’s complications overall range from 4.5 to 13.7% with infections ranging from 1.6 to 9%, and serious takeback complications 2.4 to 2.9%.

*Serious complications are defined by NSQIP as: death, cardiac arrest, myocardial infarction, pneumonia, progressive renal insufficiency, acute renal failure, pulmonary embolus, deep venous thrombosis, return to the operating room, deep incisional SSI, organ space SSI, systemic sepsis, unplanned intubation, urinary tract infection, or wound disruption.

It is hypothesized that the NSQIP dataset, as currently configured, could be improved to more accurately provide quality comparative data for the various General Gynecologic and Gynecologic Oncology surgeries which range from simple and complex laparoscopy to simple and complex laparotomy, for both benign and cancer diagnoses and procedures. The objective of this article is to examine the community hospital QA file of a GO’s practice and a hospital’s peer review process by comparing and evaluating the accuracy of NSQIP data, hand-counted operative reports, and proprietary hospital software (MIDAS), evaluate the biases that entered into data inaccuracies and peer review procedures, and to generate policy recommendations for improving accuracy and utility in specialty and subspecialty gynecology data reporting.

MATERIALS AND METHOD

With IRB approval from Sequoia Hospital in Redwood City, CA, data was abstracted from hospital and office charts of all surgical patients on a Gynecologic Oncology service from January, 2014 to September 2017 and a copy of each operative report was stored in a locked office file. In preparation for this manuscript, comparison of the operative reports with the surgery schedule was undertaken to confirm thoroughness. Operative report data including demographics, diagnosis, procedures, and complications were uploaded to an Excel spreadsheet. Data from the hospital QA Department, including NSQIP and “MIDAS” data tables, were provided to the surgeon for quality review.

The complication outcomes were categorized using the NSQIP definition of “serious complication” as the presence of any of the following: cardiac arrest, myocardial infarction, pneumonia, renal insufficiency and failure, pulmonary embolism, deep venous thrombosis, return to the operating room (OR), deep incisional surgical site infections (SSI), organ space SSI, systemic sepsis, unplanned intubation, urinary tract infection (UTI), and wound disruption that occurred within 30 days after the surgery.” The ACS NSQIP defines “any complication” as any aforementioned serious complication, superficial incisional SSI, stroke, or ventilator support >48 hours [48]. Each category of complications categorized by NSQIP is more common in a Gynecologic Oncology practice than in a Generalist’s practice due to the higher risk of the patients and the procedures [16].

In this series, “inpatient status” is defined as an “intent to treat” category for those admitted to the inpatient census undergoing a surgery employing a primary vertical midline incision or a Cherney horizontal incision (laparotomy). Outpatients are defined on an “intent to treat” basis, as those patients initially admitted for minimally invasive surgery (laparoscopy) with a planned hospital stay of 23 hours or less, regardless of ultimate duration of stay or “complication”. Consistent with NSQIP standards of categorizing patients by their initial Current Procedural Terminology (CPT) procedures, patients in this study are categorized as inpatients or outpatients; and the categories do not change, even if an outpatient sustains a complication and is later admitted to or re-admitted to an inpatient census for a prolonged recovery. For accurate comparison of NSQIP data, only the hand-counted operative reports of procedures including hysterectomies from the same time frame are counted. For accurate comparisons of multiple reports over various time-frames, the time-frame of the each of the reports are the same in each table.

RESULTS

Most (70%) of the practice had a malignant diagnosis or indication for an oncologist’s care, similar to other GOs. [62,63] (Table 2) While outpatient laparoscopy was the default approach employed in 86% of all cases, it was also employed in 83% of malignancies. Inpatient laparotomy was employed in 17% of cases, 84% for ovarian cancer debulking, and 16% for removal of massive myomata or suspicious ovarian masses too large for intact removal or for morcellation in laparoscopic bags.

Table 2: Demographics and Diagnoses are stratified by approach.

Table 3 tabulates 2076 individual procedures that comprised the 762 operative reports for the 44-month duration of the time frame of complications. 1114 individual procedures were Oncology-specific, with 115 indicated bowel procedures performed on 68 of the 107 laparotomy patients, or 64%.

Table 3: All procedures from the operative reports have been abstracted, highlighting over 700 procedures performed only by GOs in bold.

CASE REPORT

Peer Review Committee

A community hospital GO was asked by the Chief of Staff (COS) to meet with him for a conversation to help her improve her practice, informing her that she had “increased complications, infections and takebacks.” To prepare, the GO requested a list of her complications, and the rates or data about which they had concerns. The administration provided the medical record numbers of 28 patients, spanning 31 months. The GO hand-counted 588 operative reports in that 31-month retrospective, finding an overall complication rate of (28/588=) 4.8% [13] (Table 4). The 28 complications included 10 SSI, at (10/588=) 2.0%, and 15 ROR at (15/588=) 3.0%. These rates were normative for her subspecialty (Table 1).

Table 4: The top row shows counts of operative reports spanning the 31-month retrospective of the 28 complications, infections and takebacks cases provided by the Chief of Staff in prep for meeting. The two rows below that, show the NSQIP 12-month retrospective data from Figure 1 of “expected” rates of complications, infections and takebacks in the NSQIP hysterectomy-only database of General and Oncology surgeons combined and “actual” NSQIP data collected on the Oncologist. There is no reconciliation between the differences in the absolute numbers, but none are out of the range of publications by other GOs.

Figure 1: Screenshot of NSQIP morbidity data for a 12-month period from July 1, 2014 through June 30, 2015, showing 16 complications (6.3%), 9 infections (3.5%) and 9 takebacks (3.5%) out of 254 hysterectomy cases; all normative rates for a Gynecologic Oncologist.

Since 1996, the GO had maintained a spreadsheet of her surgical case data and had published manuscripts describing her laparoscopic data, with total complications ranging from 5.2-10% [64-72] infections varied between 1 and 2%, [65-68,70-72] and ROR ranging from 2.8-4.7% [64-72] (Table 5). She provided the COS this data and also a provided Gynecologic Oncology literature comparison, documenting her practice was comparatively safe.

Table 5: Thirteen publications by the GO assessing surgical complications by procedure or diagnosis showing overall complication rates of 2.3-9.9%, infections at 0-1.9%, and takebacks at 2.2-5.4%.

Ad Hoc Committee

An Ad Hoc Committee (AHC), a subset of the MEC’s peer review committee, was formed to review the GO and met 14 times the ensuing year in closed sessions with a general gynecologist as the committee departmental representative. The GO continued with her usual scheduled surgeries during this time. A patient with an isolated focus of recurrent endometrial cancer adherent to the unirradiated infrarenal aorta underwent surgery planned in advance with a General Surgeon and a Vascular Surgeon [73]. The procedure went well, ultimately requiring the anticipated replacement of the involved segment of the aorta, leaving no residual disease, with a 1,500cc blood loss. A post-operative decision to change a coding/billing modifier, mutually decided by the GO and the General Surgeon, to change from co-surgeon to primary and assistant surgeons, necessitated re-dictation of the original draft dictations. The patient had an uneventful recovery, and remains cured after aortic radiation therapy.

The MEC commenced an investigation of this case as a sentinel event resulting in the summary suspension of the GO’s privileges, calling this case, a “near miss” and the GO’s dictation efforts “dishonest.” A meeting with the Medical Executive Committee to resolve the suspension was unsuccessful. Soon after, the Ad Hoc Committee voted to suspend her. The Medical Board of California and the National Practitioner Database were notified, precluding the GO from seeking privileges at other local hospitals.

Still confidant that an accurate look at her practice data, standards and procedures would show normative rates and competence, the GO appealed for a Judicial Review Committee (JRC), a pseudo-legal trial, as prescribed in the hospital’s bylaws.

Judicial Review Committee

The JRC process required the hospital to provide the Oncologist with her entire Quality Assurance file. These documents, tables and data revealed how the Oncologist’s hospital colleagues, given the misinformation they received, voted for her suspension.

The data from NSQIP and operative report counts:

The files later released to the GO show that initial concerns had been raised about her practice due to a QA computer NSQIP table printout for the 12-months retrospective ending June 30, 2015. (Figure 1) This table showed that among 254 hysterectomy cases (NSQIP only counts cases in which hysterectomy is done), 3.5% had an SSI compared with the NSQIP cohort “Expected” Rate of 1.69%; and a 3.54% ROR compared with the NSQIP cohort “Expected” rate of 1.28%. The figure shows that the comment “Needs improvement” was applied for both SSI and ROR.

Of the 22,658 Gynecologists in the United States [74] 1,157 are GOs [75] or approximately 5%. The NSQIP predominantly benign cohort “Expected” rates of Total, SSI, and ROR between 1.3 and 1.6%, are each far below the known rates in NSQIP Gynecologic Oncology manuscripts listed in Table 1, with 12-55% total complications, 7-9% infections, and 2-19% ROR. [70,72,76,77] It may be reasonable to assume that approximately 95% of the comparator hysterectomies in the NSQIP database are performed by General Gynecologists [16] reinforcing the need compare subspecialists with similar subspecialists, by obtaining peer-reviewed literature establishing standards of care and benchmark complication rates within the subspecialty of Gynecologic Oncology focused on their higher risk patients and higher risk procedures.

A 12-month retrospective hand-count (Table 4) of the operative reports in that duration revealed 183 hysterectomy cases and 227 total (same time frame as the Figure 1 data). Whether counting only hysterectomies or all cases, the Surgeon’s total complication rates of 4.9-6%, SSI rates of 1.6-2.2%, and ROR of 2.2-3.5% were each within all published NSQIP norms for GO practices. (Table 1) Without appropriate data comparisons, the GO was reported to the COS and the Gynecology Department Chair, who requested that the Medical Executive Committee authorize the investigation. It was never evident whether the Gynecology Chair knew that it was inappropriate to compare Subspecialty data with Generalist data. Later evidence to the JRC from both the COS and the CMO revealed that they gave no credence to the submitted Gynecologic Oncology manuscripts and data.

It was found that when the AHC members met, they were provided 628 operative reports from a 36-month retrospective, and had access to the updated list of the GO’s complications, numbering 37 by that time. (Table 6) Her overall complication rate was 5.9%, of which 1.8% had an SSI, and 3.0% had an ROR. The hand-counting of operative reports revealed 658 operations, with the same complication rates as in the Table 1 publications. It remains unclear how the hospital identified 30 fewer operative reports than the hand-counting, and which cases may have been eliminated and why.

Table 6: The 36-month retrospective conducted by the AHC counting 628 operative notes, and the GO’s counts of 658 operative reports with similar rates of complications, infections and takebacks.

*“Any Occurrence” defined by NSQIP includes: superficial incisional surgical site infection (SSI), deep incisional SSI, organ space SSI, wound disruption, pneumonia, unplanned intubation, pulmonary embolus, ventilator >48 hours, progressive renal insufficiency, acute renal failure, urinary tract infection, stroke, cardiac arrest requiring CPR, myocardial infarction, deep vein thrombosis, or systemic sepsis, blood transfusion, vein thrombosis requiring therapy, clostridium difficile, sepsis, septic shock.

The 44-month retrospective shows comparison of NSQIP and hand-counts of hysterectomy-only operative reports all of which are normative to Table 1 data. The MIDAS calculation of return to the operating room at 18.6% is inconsistent with every single NSQIP and hand count at any duration and at any time. The outpatient laparoscopy complications, infections and takebacks are consistent with the Oncologists prior laparoscopic surgery publications, (Table 5) and are normative. The rates of complications from all procedures is 7.9%, with 1.8% infections, and 2.9% takebacks. To evaluate this GO, the QA department reached out to to the ACS-NSQIP office to obtain the comparable Oncology-specific data, but were advised that the Oncology-specific data should be obtained from already published manuscripts, which they viewed as "unavailable."

A slide set was created from the NSQIP QA computer printout and was presented by MEC leadership at the AHC and Medical Executive Committee (MEC) meetings. The slides inaccurately described the abstraction process (Figure 2) contradicting the 2017 NSQIP User Guide, [4] and diminished the credibility of the NSQIP data. Specifically, the User Guide states that small and community hospitals will “collect all [hysterectomy] ACS NSQIP-eligible cases at their hospital” (p.6) not “40 patients randomly selected every 8 days for review on average (random sampling can miss complications).” It was inaccurately stated that “There is no Gyn-Oncology specific data comparison outside of research articles,” never obtaining any publicly available NCBI analytic manuscripts. Thus, the GO was compared only with the predominantly benign NSQIP-wide database as listed in Table 1. NSQIP in 2014 also started to stratify procedure by subspecialty gynecology provider, and this subspecialty attribution was not abstracted or presented.

Figure 2: JRC slide with two errors in representation of NSQIP data abstraction methods, incorrectly stating that “There is no Gyn-Oncology specific data comparison outside of research articles available,” thus diminishing NSQIP reliability to hospital reviewers.

The AHC’s next slide showed the NSQIP rates of “Any Occurrence” from 2014 to 2017.4 (Figure 3) The occurrence of any NSQIP adverse event is far more common in a GO practice than in a Generalist practice [78] and “complications” may be due to appropriately aggressive gynecologic cancer surgery, with the goal of no visible residual disease, to include; ventilator use >48 hours, blood transfusion, and urinary tract infection from an indwelling catheter [10]. Annual numbers for “Any NSQIP Occurrences” by the GO were provided, but were not tabulated for rate data (%) to generate an overall understanding. The data calculated by the MEC revealed 81 out of 484 (16.7%) occurrences (reviewing now a 42 month period), which is twice the Generalists’ rate of 9.7%, but compares favorably with NSQIP publications by Gynecologic Oncologists showing 12-55% overall complications in open cases, and 7-19% in laparoscopy cases (Table 1).

Figure 3: Screenshot from the JRC slide presentation showing annual “Any NSQIP Occurrence” rates of 11.7 to 19% compared with the average of 12% in the predominantly benign NSQIP-wide database of 241,567 hysterectomies over the same period. The GO’s data in this table was tabulated in the right-most column added by the author showing (81/484 cases) 16.7% over 42 months, which is normative for Gynecologic Oncologists.

Similarly, the NSQIP tabulated SSI rate of 3.3% (Figure 4, right box) compares favorably with the SSI rates of GOs in Table 1 which ranges from 7-15% for ovarian cancer hysterectomy cases, and 1.4 to 3.8% in laparoscopy hysterectomy cases. These rates are indeed higher than the NSQIP-wide database of 245,971 predominantly benign hysterectomy average of 1.1% (Figure 4).

Figure 4: Screenshot from the JRC slide presentation showing annual NSQIP “Deep Space and Organ SSI” rates of 1.7-5.1 % compared with the average of 1.2% in the predominantly benign NSQIP-wide database of 241,567 hysterectomies over the same period. The GO’s data in this table tabulated in the right-most column added by the author showing (16/484 cases) 3.3% over 42 months, which is normative for Gynecologic Oncologists.

The tabulated ROR rate of 2.9% (Figure 5) is consistent with each of the prior calculations with the data from the Sequoia QA Department, the AHC’s operative report counts, the prior NSQIP rates, and hand-counts of operative reports from all cases and hysterectomy cases by the GO. The ROR of 2.9% is also within range of NSQIP reports of ROR for ovarian cancer cases of 3-19%, 2-9% for laparoscopy cases (Table 1). In Figure 1, the ROR for the predominantly benign NSQIP-wide registry is 1.5%, a much lower rate, and one that is not seen in any NSQIP gynecologic oncology publications.

Figure 5: Screenshot from JRC slide presentation showing annual NSQIP “Returned to Surgery” rates of 1.9-4.7 % compared with the average of 1.5% in the NSQIP-wide database of 241,567 hysterectomies over the same period. The GO’s data in this table is tabulated in the right-most column added by the author showing (14/484 cases) 2.9% over 42 months, which is normative for Gynecologic Oncologists.

The data from MIDAS “Inpatient Return to OR” versus counted operative reports:

The MEC’s slide deck next introduced the proprietary Dignity Health MIDAS (Conduent® Strategic Performance Management Analytics Solutions) “Inpatient Return to OR” calculation. (Figure 6) The MIDAS data was presented as the “standard for Dignity Health quality assessments”, “used in over 800 hospitals.” The Dignity website reveals that it provides “health care at [only] 39 hospitals …across California, Arizona, and Nevada.” [79] The MIDAS ratio is not a nationally recognized quality indicator and is not a term in the National Congressional Biotechnology Information (NCBI) library. To contrast this, there are over 3,800 NSQIP quality-related manuscripts indexed in the NCBI [3].

Figure 6: Screenshot from CMO slide presentation introducing the MIDAS “Inpatient Return to OR rates.” The MIDAS ratio is not the “Standard Dignity Health database for quality assessments.” It is not “Used in 800+ hospitals throughout the country for quality assesment” The numerator combines both inpatient takeback cases and outpatient takeback cases, divided by the denominator which is comprised of only inpatients.

The MIDAS “Inpatient Return to OR” does not use the standard basic takeback rate formula [13] and is thus not a ROR. According to the slide made by the MEC, the MIDAS “Inpatient Return to OR” calculation numerator is comprised of “all patients, who have a return to the OR within the same encounter,” which combines patients having ROR from both the laparoscopic outpatient service and the laparotomy inpatient service. However, the MEC’s slide shows that, instead of following the standard rate formula and using the same category, all patients, as the denominator, the MIDAS Inpatient ROR denominator uses “All inpatient surgeries.” This anomalous formula results in surgeons with a high outpatient laparoscopic approach rate always having a much higher MIDAS ratio compared to surgeons who directly admit to inpatient, even if they had the exact same complications. The standard rate formula requires that both the numerator and denominator represent patients only in the same category [13].

In Table 7, the annual MIDAS and operative report-counts for inpatient-only takebacks are listed and tabulated for the 44-month retrospective. The MIDAS counts of 26 ROR inpatients and 140 inpatients, and the overall rate of 19% ROR is not supported by the operative notes or the QA data. In a cumulative 44-month retrospective, hospital QA data reveals there were 8 inpatient takeback cases out of 107 laparotomy procedures, ranging annually from 2-4%, with an overall rate of 3%.

Figure 7: Screenshot from JRC showing MIDAS rates for the GO of “ Return to OR rates” at 15-26% compared with the average of 3.5%--miscalculated by denominator case selection. The data in this table was tabulated in the right-most column added by the author showing the effect of this miscalculation at (26/140 cases) 19% over 42 months.

Table 7: The annualized Return to the Operating room data by year with total tabulation combining both both outpatient laparoscopy and inpatient laparotomy, showing the MIDAS calculation to be an outlier, with 7% laparotomy takebacks, and 3% hysterectomy and3% total caseload takebacks. All are normative with publications in Table 1. The NSQIP counting of only hysterectomy cases missed 8 ROR cases for quality review.

There was no member of the AHC or JRC with Oncology or GO certification. Given this, the AHC was compelled to send cases out for review, to a GO subspecialist whom they chose. Of the 7 cases they selected, these cases had been previously adjudicated in monthly departmental QA meetings. When the MEC eventually realized that their data did not show high complications, infections and takebacks, allegations were changed to concerns about severity of the GO’s complications.

The JRC Hearing Officer refused to allow submission of evidence during the proceedings as permitted by the hospital’s bylaws. The Hearing Officer did not require the outside GO reviewer appear for cross-examination. The Hearing Officer was found to be a former hospital consultant with conflict of interest.

Peer testimony was heard from three Anesthesiologists, three Medical Oncologists, a General Surgeon, a Vascular Surgeon, a General Gynecologist and a Gynecologic Oncologist, all with 15+ year working relationships with the Oncologist’s—who all supported that the GO was a competent quality provider. Peer statements affirmed how a takeback rate is legitimately calculated, how NSQIP data was credible with appropriate adjustments to subspecialty and incident/accident “complications” are calculated, how the MIDAS algorithm is flawed, and that subspecialty Gynecologic Oncologists practice data is only appropriately compared with other subspecialists.

The JRC upheld suspension and expulsion of the GO, disregarding proper calculations of/and objective data and due process. At the conclusion of her privileges, she had a 5.4% “any” complication rate, of which 1.7% were SSI and 2.6% were ROR--both by hand-counts and NSQIP data.

DISCUSSION

Accurate reporting of complications is necessary for evaluating quality improvement efforts, documenting standards of care, and providing the source for informed consent. Every hospital has a QA program for peer review that is run by its administrative and medical staff following objective guidelines. The Healthcare Quality Improvement Act of 1986 was implemented to provide necessary immunity to good-faith peer review by local hospital staff [80]. This law empowered local medical staff leaders to conduct peer review processes with federal and state-conferred statutory immunity if hospital bylaws and due process rights were followed. The QA decisions are protected by a judicial system geared to shield the deciders from retaliation--precluding effective oversight of the peer review system.

This case review demonstrates misuse of a registry’s data output from misunderstanding and the misappropriation of the registry in peer review processes. Multiple pitfalls and missteps were identified in this process due to numerous forms of bias. Identification and explanation of these biases then follows.

Reporting and measurement biases

This is reflected in NSQIP event misclassification: Coders and physician reviewers can misclassify purposeful tumor debulking and lymphadenectomy to zero residual disease (R0) as complications, e.g., cystotomy, ureterotomy, vasculotomy. Similarly, enterotomies from lysing symptomatic adhesions, or from obtaining zero-residual disease from cancer resections, will commonly result in an enterotomy rate of 6% [81] e.g. NSQIP does not count organ incisions that are incidental to surgical procedures repaired intraoperatively, but apparently does code accidental injuries that cause a reoperation or other reparative invasive procedure.

NSQIP QA concerns

Correct use and interpretation of the NSQIP data is integral for reliable application. Misapplication can harm the credibility of the registry itself as well as the quality assurance and peer review processes.

• 20 patients not undergoing hysterectomy had complications and were not counted by NSQIP. While these patients were included in the QA Department list of patients, the NSQIP coders did not include them. Consideration of abstracting every major abdominal procedure in Gynecology may give a more accurate complication profile.
• Wound infection is a complication that is accidental to the outcome of surgery. All infections should be tracked and trended, and counted as complications. Of 17 suspected infections, 3 were superficial wound infections, 2 resolved on antibiotics alone, 3 required ROR, and 9 underwent CT drainage of the fluid collection. Standards for managing emergency-room presentation of suspected postoperative fluid collections seen on CT should be established in order to avoid unnecessary CT drainage of noninfected fluid. NSQIP typically requires the results of culture before an abscess can be diagnosed, but these are not uniformly obtained. Surgeons’ infection rates should be within the benchmark for their specialty or procedure, and if higher, then a multipronged investigation should be undertaken to determine whether the prep, equipment, staff, technique or post-operative care can be improved.
• NSQIP does not define whether intestinal obstruction is pre-operative or post-operative and what duration of NPO constitutes an obstruction. A patient in this series complained of constipation, but was coded as obstructed.
• Enterotomy should be categorized as accidental or incidental. If incidental and successfully repaired, then it is not a complication.
• Anastomotic leak is defined and must be confirmed as possibly including “… air, fluid, GI contents, or contrast material. The presence of an infection/abscess thought to be related to an anastomosis, even if the leak cannot be definitively identified as visualized during an operation, or by contrast extravasation, would still be considered an anastomotic leak if this is indicated by the surgeon.”
• NSQIP does not permit acknowledgment of another surgeon’s work or complications. A general surgeon who often assists with gynecologic oncology patients performed three anastomoses, dictated the procedures listing himself as primary surgeon for this portion of the procedure, billed the patient’s insurance for the procedures, and followed the patients until discharge. The NSQIP guidelines for data abstraction require that the data and complications follow patients and procedures but not multiple surgeons. NSQIP does not allow for a complication to be attributed to a specific surgeon if there are more than one surgeon. This system provides for accurate data for assessing the patient's, and the planned procedure's risk, but it can lead to surgeons being blamed for another surgeon’s complications.
• A ureteral leak needs documentation by radiology. CT drainage fluid should be tested for creatinine. Urological obstruction and fistulae could be categorized in the same way as intestinal obstruction with selections to include how it is addressed. Options to ureteral obstruction could be stenting, boari flap, graft, reimplantation, transureteral ureterostomy, percutaneous nephrostomy, no definitive diagnosis. Ureterotomy and cystotomy should be categorized accidental or incidental. If incidental and successfully repaired, then not a complication. Six patients had ureterotomies: two incident to debulking cancer, one was repaired successfully but incorrectly included in the QA list, the other was re-operated, and incorrectly excluded from the QA list.

NSQIP recommendations for optimization

The NSQIP program has been shown to reduce costs and improve quality. The NSQIP survey questions and structured classifications created for coding the General Surgery data do not always apply well to Gynecologic or Gynecologic Oncology surgery. Just as the NSQIP applications have been modified to apply to various surgical specialties, further modifications within the Gynecologic databases can improve the use of the program to achieve its goals of surgical quality, and harm reduction [41]. This case review confirms that NSQIP data is subject to misuse, misinterpretation and misrepresentation, with policy suggestions for correction shown below.

• There can be inconsistency of coding anastomotic leaks as infections. Employing the term “as indicated by the surgeon” for an anastomotic leak allows some subjective component enter into the data input.
• Employing the variables of ileus vs obstruction, instead of prolonged NPO or NGT use will also help distinguish complications vs incidents.
• In major ovarian cancer debulking procedures in which no visible residual is the goal, many Oncologists will perform the cancer resections, and collaborate with General Surgeons for the bowel reconstructive procedures [63]. In these cases, in which the General Surgeon performs, dictates and takes primary responsibility for the intestinal procedure, NSQIP abstractors have no way to attribute the procedure, per se, to the General Surgeon, causing an error of attribution when a complication ensues.
• There should be a designated facility ombudsperson to interpret, represent, and respond to coding and query questions on behalf of the surgeon. This ombudsperson would have deep knowledge of the NSQIP database, access to it, and be a liaison between the surgeon and any review entity that interprets data on behalf of the hospital/administration. This ombudsperson could be regional and should have a stipend, just as coders for the NSQIP program do. This stipend should be built into the program fee in a similar fashion.
• Abstractors may not understand that at times an ROR is planned, resulting in an inaccurate rating of an ROR [41]. It may be necessary for the abstractors to consult a physician or the assigned ombudsman to determine such a plan if it is not obvious.
• Abstractors may not perceive organ incisions and resections as incidental to the primary purpose and success of the surgery (enterotomy, vasculotomy, ureterotomy, cystotomy) and code even successful restorations of the organs as complications. There should be discrimination between incidents in surgery and accidents of surgery. Modifications to the instructions of the NSQIP abstractors can facilitate interpretation of the operative report to aid in classification. Allowing or seeking clarification with the surgeon when an incident is not clear could result in more accurate coding.
• Currently NSQIP abstracts only hysterectomy cases and prolapse surgeries. Because major surgeries do not always involve hysterectomies (prior or declined), and because non-hysterectomy minor surgeries can have severe complications, NSQIP should consider abstracting all abdominal Gynecologic surgeries, coding simply by diagnosis and procedure performed. If NSQIP abstracted all cases performed by Gynecologists, more useful data could be generated for quality assessment and establishment of benchmark standards.
• Some events occurring in the 30-day post-operative period are not due to the surgery. E.g.: Abstractors calling a patient for a 30-day evaluation coded a patient’s complaint that she did not get enough information about post-operative diet and levied her documented chronic constipation as a complication.
• Because participating NSQIP hospitals need to easily perform quality assessments of their General Gynecologists and subspecialty Oncologists or Pelvic Surgeons by comparison of their data only with that of similarly trained surgeons, NSQIP should provide all participating hospitals with complication data stratified a priori by General or subspecialty training of the surgeon, or by procedure code in their semi-annual reports.

Access to self-query for assessment of surgeon’s own quality data within the NSQIP database is advised. Currently, researching surgeons at participating NSQIP hospital can sign a contract and receive complex tables that require statistical programs and advanced data managing to analyze and interpret. Ideally, NSQIP could create templates of complication tables that individual surgeons could select and download that would provide their individual surgical data along with the comparable nationally acquired benchmark data within their subspecialty or by procedure or diagnosis so that motivated surgeons could self-assess and self-improve or more easily publish surgical information.

Hospital QA Process Concerns and Cognitive Biases

Due process is a necessary component of procedural reviews and incorporated into hospital bylaws. Inconsistencies in peer review processes have been denoted “sham peer review.” [82,83]. In a sham peer review, tactics to discredit the provider under review are frequently used and are emphasized in this case report. Features in common with the above peer review and that of a general sham peer review follow:

• Undercoverage bias occurs when a single high performing individual is singled out and not allowed to be represented proportionally. Comparison of a GO with Generalist Gynecology outcomes was not appropriate and using a registry not abstracted for subspecialty also contributed to misperception and misinterpretation.
• Outcome bias is observed when a negative judgment is applied to the appropriate decisions that led to the complication. Complications can ensue absent any error or mistake, especially when cancer-burdened, comorbidity-burdened, elderly, or obese patients are cared for.
• Heuristic availability bias occurred when a historically stable physician was abruptly ascribed numerous questionable performance indicators by a QA team, e.g., with a steady NSQIP complication rate for over 3,500 cases in 15+ years at a hospital. A sudden report of “increased rates of infections, takebacks and complications” should initiate a deeper and thoughtful review with understanding of the data. This data and all full and objective concerns should then be shared directly with the individual surgeon. Any supporting information should be allowed into argument. In this case, the GO could have provided the hospital with the NSQIP publications for comparison of her subspecialty-specific data, and could have explained to them that the MIDAS number was not an ROR formula.
• Framing bias occurred when the leader for the MEC presented the GO as having “higher complications” and as being an “outlier,” allowing the story to proceed in the context they wanted, and thus the other members of the committees did not adjust their perspective to objectively evaluate the situation or its source.
• Anchoring bias occurred when the GO was stated to have higher complication rates, and the committees stayed locked into their early “diagnoses/commitments” despite evidence to the contrary, provided by the GO requesting their re-review of the data.
• The generalist bias: peer reviewers who are not surgeons or are Generalist colleagues, may not know how to interpret surgical data, or apply the data to accurately reflect surgeon quality [84] Because most procedures and complications are distinctive to subspecialty, quality assessments should include the expertise of another same-subspecialist who can accurately testify and respond to queries e.g., the Chair of the Gynecology Department and of the Ad Hoc Committee, both General Gynecologists, did not have subspecialty certifications to adjudicate Oncology complications, but proceeded to do so. Procedurally, these General Gynecologists were low volume surgeons, having admitted that they performed fewer than five laparoscopic hysterectomies in the prior fiscal year. Reviewing panels are required by hospital bylaws to “include at least one member who has the same healing arts licensure and practices in the same specialty as the Practitioner involved”—but this should be specific to subspecialty.
• Conflict of interest: The 3rd party contracted reviewer was paid by the hospital for their services, and in as much, is asked to find in favor of the hospital. Usually, the physician under review can provide 3 names of outside peer reviewers for the committee to choose from.
• Confirmation bias, similar to Lynch mob effect, occurred when a faction or group of physicians gain control of the peer review committee, raise doubt, and defame the provider and their practice with faulty accusations and reports, e.g. the Chair of the Gynecology Department and of the Ad Hoc Committee, both General Gynecologists, knew or should have known that comparisons of the Oncologist’s data with that of the NSQIP-wide database were inappropriate, but none spoke to this issue.
• Competitive financial and surgical interests: Gynecologic Oncologists often perform complicated benign surgery which Generalists may view as a territorial imposition. This may be perceived as “stealing” their cases, yet many General Gynecologists are uncomfortable or inadequately trained to operate on patients that are high-risk, obese, elderly, with many medical comorbidities, or too many prior surgeries, and refer them to the oncologist [16,85,86]. One practice data survey of members of the Society for Gynecologic Oncologists revealed the average Gynecologic Oncologist’s volume consists of 45% benign patients [62] Because of these higher-risk features, these benign patients can experience an expectedly higher rate of complications [16].
• Cognitive bias: This can occur when QA physicians are unaware of a surgeon’s volume and surgical diseases, bias can result from reviewing what appear to be a high absolute number of complications. In so much, complications should not be counted in absolute numbers, but instead as rates; e.g., a surgeon with 10 complications out of 250 surgeries annually, or 4% complications, will have a case for review every month and appear to have excessive complications; while a surgeon with 5 complications performing 50 surgeries annually will have a 10% complication rate but have fewer cases before the monthly QA meeting and not garner suspicion. In this hospital, complications despite pleading by the GO to use opercent rates were counted in absolute numbers , and rates were not applied as described, signaling this surgeon as an outlier despite having normative rates of complications.
• Procedural Bias of Due Process is the procedure set forth for investigations and hearings in each hospital’s bylaws and rules or regulations to ensure justice. Procedural bylaws are intended to protect those on each side of the issue. Yet, a physician unfamiliar with this practice may not identify a hospital lawyer or administrator failing to follow regulations; e.g., denying a physician access to their data and legal representation in the beginning precluded the Oncologist from knowing that their comparisons of her were to generalists, that NSQIP data for subspecialists should be appropriately extracted, and correcting false MIDAS rates. Thus, the investigation was biased ab initio. When bylaws are sidestepped, erroneous review panel decisions can ensue: this includes suppression of evidence, defense testimony, essential witness questioning, and cross-examination; e.g., the expert witness was not made to appear and NSQIP data was suppressed.
• Sampling Bias introduced when a QA process fails to recognize and include credible and relevant information, or lowers the bar for national data validity; e.g., the CMO should have investigated why the MIDAS Inpatient ROR of 19% contrasted dramatically with both the NSQIP and hand-counted rates of 3% from the AHC. In surgical specialties, the ROR is the most important quality assessment data.
• Reporting bias occurs when previously adjudicated or mild grievances can be revived to elevate and prosecute otherwise old or minor infractions into major violations; e.g., from the 28 complications listed at the start of the proceedings against her, 21 had been previously adjudicated as “no issue with physician care,” yet later called “egregious” cases when presented to the JRC.
• Interview bias played a large part in this process when hearsay is accepted without corroboration from patients and nurses; e.g., the QA staff selectively interviewed patients who had complications, elicited complaints, and registered them post hoc. It is a known that patients with complications complain 74% more frequently, [87] especially if solicited by staff. Efforts to label any occurrence as a transgression, actual or assumed, are often used in the hope that something will “stick”.
• Implicit bias: [88] this bias occurs when overt bias cannot be determined and can affect colleagues’ interactions with and perceptions of minority staff. In confidential surveys, 84% of women gynecologic oncologists report a high prevalence of bullying, gender discrimination and microaggressions by their colleagues in medicine [89]. In this case, the readiness of the panels to disbelieve uncontested testimony from multiple peer physicians with years of direct experience with the Oncologist, was in direct contradiction to a single paid consultant who critiqued 7 of 762 patients’ care. While overt sexism, racism or homophobic attitudes are rarely observed, implicit bias against women physicians, [90] physicians of color [91] and homosexual physicians [91,92] has been consistently observed among all physicians and can impede a fair assessment of a minority physician’s skills.

Hospital QA Recommendations:

• QA panels should obtain live testimony from at least one subspecialist for each section at a JRC trial to resolve questions and describe current standards of care.
• Investigations should allow submission of literature evidence during the proceedings and allow time for review of established literature.
• An ombudsman physician could be assigned at the outset of consideration of a QA investigation, because the stigma that accompanies a complaint and investigation can inhibit a physician’s appropriate defenses. This ombudsman physician could act as an intermediary between administration and staff to ensure fairness and compliance with hospital bylaws in the early informal discussions about and with the surgeon, long before the need to obtain legal counsel.
• Awareness of and training to avoid these forms of bias should be part of the job to serve on an MEC. There should be leadership training that includes data interpretation at the time of appointment to these positions regarding these biases to avoid harm to colleagues while protecting patient safety. A checklist to ensure these biases have not found their way into the review should be part of the record. Moral and ethical self-checks should occur for each committee member.

CONCLUSIONS

Peer review must be perceived as a legitimate and equitable process for quality improvement. If bylaws are followed, all data is examined carefully, all evidence is admitted and studied, and individual morality is upheld, peer review can improve patient care. Participating NSQIP hospitals should provide each surgeon with their data that is accurately acquired every six months, along with comparable NSQIP database information, so that surgeons can monitor their own practices and achieve the safest standards of care.

Surgeons under review must read the bylaws of their hospitals and understand them thoroughly. Surgeons must know of and understand the databases and registries that administrators use, and how to use the data properly. Providers should retain legal assistance at the start of any peer review process so that mistakes are avoided. Due to lawyers’ and non-subspecialists’ unfamiliarity with medical and subspecialty standards, all evidence must be thoroughly examined and adjudicated for accuracy and consistency with published standards, along with explanation of their significance during peer review and provider evaluations.

NSQIP data can predict and reflect practice quality, but the current data entry variables and abstraction methods hinder subspeciality assessment for both surgeons and facilities. Abstracting all laparoscopic and abdominal cases by Gynecologists, designating subspecialty status, disease process, and recognizing what incisions/repairs are accidents or incident to the procedure, will make the NSQIP dataset more useful.

Highlights

-Accuracy and the proper use of medical registries and databases is key to quality patient care and to provider benchmarks.

-Steps to address and mitigate faulty reporting of quality patient outcomes in gynecology oncology are needed.

-Sham peer review can disrupt quality patient care with misuse of databases.

CONFLICT OF INTEREST

All four authors declare that they have received no outside funding for this manuscript and have no conflict of interest in its production.

REFERENCES

1. Khuri SF, Daley J, Henderson W, Barbour G, Lowry P, Irvin G, et al. (1995). The National Veterans Administration Surgical Risk Study: risk adjustment for the comparative assessment of the quality of surgical care. J Am College Surgeons. 180(5):519-31.
2. Fink AS, Campbell DA Jr, Mentzer RM Jr, Henderson WG, Daley J, Bannister J, et al. (2002). The National Surgical Quality Improvement Program in non-veterans administration hospitals: initial demonstration of feasibility. Annals Surg. 236(3):344-3553; discussion 353-435.
3. American College of Surgeons. (2022). Hospital and Facilities,. American College of Surgeons. https://www.facs.org/hospital-and-facilities/?searchTerm=&institution=NsqipHospital&address=&country=&page=1
4. American College of Surgeons NSQIP. (2017). User Guide for the 2017 ACS NSQIP Procedure Targeted Participant Use Data File (PUF).
5. Kiran A, Hilton P, Cromwell DA. (2016). The risk of ureteric injury associated with hysterectomy: a 10-year retrospective cohort study. BJOG. 123(7):1184-1191.
6. Peiretti M, Bristow RE, Zapardiel I, Gerardi M, Zanagnolo V, Biffi R, et al. (2012). Rectosigmoid resection at the time of primary cytoreduction for advanced ovarian cancer. A multi-center analysis of surgical and oncological outcomes. Gynecol Oncol. 126(2):220-223.
7. Gillette-Cloven N, Burger RA, Monk BJ, McMeekin DS, Vasilev S, DiSaia PJ, et al. (2001). Bowel resection at the time of primary cytoreduction for epithelial ovarian cancer. J Am Coll Surg. 193(6):626-632.
8. O'Hanlan KA, Kargas S, Schreiber M, Burrs D, Mallipeddi P, Longacre T, et al. (1995). Ovarian carcinoma metastases to gastrointestinal tract appear to spread like colon carcinoma: implications for surgical resection. Gynecol Oncol. 59(2):200-206.
9. Awtrey CS, Cadungog MG, Leitao MM, Alektiar KM, Aghajanian C, Hummer AJ, et al. (2006). Surgical resection of recurrent endometrial carcinoma. Gynecol Oncol. 102(3):480-488.
10. Clayton RD, Obermair A, Hammond IG, Leung YC, McCartney AJ. (2002). The Western Australian experience of the use of en bloc resection of ovarian cancer with concomitant rectosigmoid colectomy. Gynecol Oncol. 84(1):53-57.
11. Scarabelli C, Campagnutta E, Giorda G, et al. (1998). Maximal cytoreductive surgery as a reasonable therapeutic alternative for recurrent endometrial carcinoma. Gynecol Oncol. 70(1):90-93.
12. Bernard L, Boucher J, Helpman L. (2020). Bowel resection or repair at the time of cytoreductive surgery for ovarian malignancy is associated with increased complication rate: An ACS-NSQIP study. Gynecol Oncol. Sep 158(3):597-602.
13. Birkmeyer JD, Hamby LS, Birkmeyer CM, Decker MV, Karon NM, Dow RW. (2001). Is unplanned return to the operating room a useful quality indicator in general surgery? Arch Surg. 136(4):405-411.
14. Aletti GD, Dowdy SC, Gostout BS, Jones MB, Stanhope CR, Wilson TO, et al. (2006). Aggressive surgical effort and improved survival in advanced-stage ovarian cancer. Obstet Gynecol. 107(1):77-85.
15. Wallace S, Kumar A, Mc Gree M, Weaver A, Mariani A, Langstraat C, et al. (2017). Efforts at maximal cytoreduction improve survival in ovarian cancer patients, even when complete gross resection is not feasible. Gynecol Oncol. 145(1):21-26.
16. Barber EL, Rossi EC, Alexander A, Bilimoria K, Simon MA. (2018). Benign hysterectomy performed by gynecologic oncologists: Is selection bias altering our ability to measure surgical quality? Gynecol Oncol. 151(1):141-144.
17. Benedetti Panici P, Di Donato V, Fischetti M, et al. (2015). Predictors of postoperative morbidity after cytoreduction for advanced ovarian cancer: Analysis and management of complications in upper abdominal surgery. Gynecol Oncol. 137(3):406-111.
18. Chang SJ, Bristow RE. (2012). Evolution of surgical treatment paradigms for advanced-stage ovarian cancer: redefining 'optimal' residual disease. Gynecol Oncol. 125(2):483-492.
19. Stewart KA, Tessier KM, Lebovic DI. (2022). Comparing Characteristics of and Postoperative Morbidity after Hysterectomy for Endometriosis versus other Benign Indications: A NSQIP Study. J Minim Invasive Gynecol. 29(7):884-890 e2.
20. Barber EL, Doll KM, Gehrig PA. (2017). Hospital readmission after ovarian cancer surgery: Are we measuring surgical quality? Gynecol Oncol. 146(2):368-372.
21. Scalici J, Laughlin BB, Finan MA, Wang B, Rocconi RP. (2015). The trend towards minimally invasive surgery (MIS) for endometrial cancer: an ACS-NSQIP evaluation of surgical outcomes. Gynecol Oncol. 136(3):512-515.
22. Wright JD, Chen L, Gabor L, Burke WM, Tergas AI, Hou JY, et al. (2017). Patterns of Specialty-Based Referral and Perioperative Outcomes for Women With Endometrial Cancer Undergoing Hysterectomy. Obstetrics Gynecol. 130(1):81-90.
23. Casarin J, Multinu F, Ubl DS, Dowdy SC, Cliby WA, Glaser GE, et al. (2018). Adoption of Minimally Invasive Surgery and Decrease in Surgical Morbidity for Endometrial Cancer Treatment in the United States. Obstet Gynecol. 131(2):304-311.
24. Bochenska K, Leader-Cramer A, Mueller M, Dave B, Alverdy A, Kenton K. (2017). Perioperative complications following colpocleisis with and without concomitant vaginal hysterectomy. Int Urogynecol J. 28(11):1671-1675.
25. Cardenas-Trowers O, Stewart JR, Meriwether KV, Francis SL, Gupta A. (2020). Perioperative Outcomes of Minimally Invasive Sacrocolpopexy Based on Route of Concurrent Hysterectomy: A Secondary Analysis of the National Surgical Quality Improvement Program Database. J Minim Invasive Gynecol. 27(4):953-958.
26. Hill AM, Pauls RN, Crisp CC. (2020). Addressing apical support during hysterectomy for prolapse: a NSQIP review. Int Urogynecol J. 31(7):1349-1355.
27. Pepin KJ, Cook EF, Cohen SL. (2020). Risk of complication at the time of laparoscopic hysterectomy: a prediction model built from the National Surgical Quality Improvement Program database. Am J Obstet Gynecol. 223(4):555 e1-555 e7.
28. Chapman GC, Slopnick EA, Roberts K, Sheyn D, Wherley S, Mahajan ST, et al. (2021). National Analysis of Perioperative Morbidity of Vaginal Versus Laparoscopic Hysterectomy at the Time of Uterosacral Ligament Suspension. J Minim Invasive Gynecol. 28(2):275-281.
29. Uppal S, Rebecca Liu J, Kevin Reynolds R, Rice LW, Spencer RJ. (2019). Trends and comparative effectiveness of inpatient radical hysterectomy for cervical cancer in the United States (2012-2015). Gynecol Oncol. 152(1):133-138.
30. Polan RM, Rossi EC, Barber EL. (2019). Extent of lymphadenectomy and postoperative major complications among women with endometrial cancer treated with minimally invasive surgery. Am J Obstet Gynecol. 220(3):263 e1-263 e8.
31. Hanwright PJ, Mioton LM, Thomassee MS, Bilimoria KY, Van Arsdale J, Brill E, et al. (2013). Risk profiles and outcomes of total laparoscopic hysterectomy compared with laparoscopically assisted vaginal hysterectomy. Obstet Gynecol. 121(4):781-787.
32. Linder BJ, Occhino JA, Habermann EB, Glasgow AE, Bews KA, Gershman B. (2018). A National Contemporary Analysis of Perioperative Outcomes of Open versus Minimally Invasive Sacrocolpopexy. J Urol. 200(4):862-867.
33. Bretschneider CE, Casas-Puig V, Sheyn D, Hijaz A, Ferrando CA. (2019). Delayed recognition of lower urinary tract injuries following hysterectomy for benign indications: A NSQIP-based study. Am J Obstet Gynecol. 221(2):132 e1-132 e13.
34. Chen I, Choudhry AJ, Schramm D, Cameron DW, Leung V, Singh SS, et al. (2019). Type of Pelvic Disease as a Risk Factor for Surgical Site Infectionin Women Undergoing Hysterectomy. J Minim invasive Gynecol. 26(6):1149-1156.
35. Lake AG, McPencow AM, Dick-Biascoechea MA, Martin DK, Erekson EA. (2013). Surgical site infection after hysterectomy. Am J obstet Gynecol. 209(5):490 e1-9.
36. Swenson BR, Hedrick TL, Metzger R, Bonatti H, Pruett TL, Sawyer RG. (). Effects of preoperative skin preparation on postoperative wound infection rates: a prospective study of 3 skin preparation protocols. Infect Control Hosp Epidemiol. 30(10):964-971.
37. Sheyn D, Bretschneider CE, Mahajan ST, Ridgeway B, Davenport A, Pollard R. (2019). Incidence and risk factors of early postoperative small bowel obstruction in patients undergoing hysterectomy for benign indications. American journal of obstetrics and gynecology. 220(3):251 e1-251 e9.
38. Sedra S, Mallick R, Nayak AL, Choudhry AJ, Bougie O, Singh SS, et al. (2021). Venous Thromboembolism After Blood Transfusions in Women Undergoing Hysterectomy for Non-Malignant Indications: A Retrospective Cohort Study. J Obstet Gynaecol Can. 43(2):167-174.
39. Gray SH, Vick CC, Graham LA, Finan KR, Neumayer LA, Hawn MT. (2008). Risk of complications from enterotomy or unplanned bowel resection during elective hernia repair. Arch Surg. 143(6):582-586.
40. Bernard L, Chen I, Le T. (2020). Appendectomy at the time of ovarian cancer staging increases infectious complications: a National Surgical Quality Improvement Program (NSQIP) propensity score-stratified analysis. Int J Gynecol Cancer. Oct 30(10):1542-1547.
41. Lin Y, Meguid RA, Hosokawa PW, Henderson WG, Hammermeister KE, Schulick RD, et al. (2017). An institutional analysis of unplanned return to the operating room to identify areas for quality improvement. Am J Surg. 214(1):1-6.
42. Szender JB, Frederick PJ, Eng KH, Akers SN, Lele SB, Odunsi K. (2015). Evaluation of the National Surgical Quality Improvement Program Universal Surgical Risk Calculator for a gynecologic oncology service. Int J Gynecol Cancer. Mar 25(3):512-520.
43. Uppal S, Igwe E, Rice LW, Spencer RJ, Rose SL. (2015). Frailty index predicts severe complications in gynecologic oncology patients. Gynecol Oncol. Apr 137(1):98-101.
44. Soloff MA, Vargas MV, Wei C, Ohnona A, Tyan P, Gu A, et al. (2021). Malnutrition is Associated with Poor Postoperative Outcomes Following Laparoscopic Hysterectomy. JSLS. 1: e2020.00084.
45. Mahdi H, Lockhart D, Maurer KA. (2015). Impact of age on 30-day mortality and morbidity in patients undergoing surgery for endometrial cancer. Gynecol Oncol. 137(1):106-111.
46. Mikhail E, Miladinovic B, Velanovich V, Finan MA, Hart S, Imudia AN. (2015). Association between obesity and the trends of routes of hysterectomy performed for benign indications. Obstet Gynecol. 125(4):912-918.
47. Mahdi H, Jernigan AM, Aljebori Q, Lockhart D, Moslemi-Kebria M. (2015). The impact of obesity on the 30-day morbidity and mortality after surgery for endometrial cancer. J Minim Invasive Gynecol. 22(1):94-102.
48. Rivard C, Nahum R, Slagle E, Duininck M, Isaksson Vogel R, et al. (2016). Evaluation of the performance of the ACS NSQIP surgical risk calculator in gynecologic oncology patients undergoing laparotomy. Gynecol Oncol. 141(2):281-286.
49. Narasimhulu DM, Bews KA, Hanson KT, Chang YH, Dowdy SC, Cliby WA. (2020). Using evidence to direct quality improvement efforts: Defining the highest impact complications after complex cytoreductive surgery for ovarian cancer. Gynecol Oncol. 156(2):278-283.
50. Bernard L, McGinnis JM, Su J, Alyafi M, Palmer D, Potts L, et al. (2021). Thirty-day outcomes after gynecologic oncology surgery: A single-center experience of enhanced recovery after surgery pathways. Acta Obstet Gynecol Scand. 100(2):353-361.
51. Patankar S, Burke WM, Hou JY, Tergas AI, Huang Y, Ananth CV, et al. (2015). Risk stratification and outcomes of women undergoing surgery for ovarian cancer. Gynecol oncol. 138(1):62-69.
52. Teoh D, Halloway RN, Heim J, Vogel RI, Rivard C. (2017). Evaluation of the American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator in Gynecologic Oncology Patients Undergoing Minimally Invasive Surgery. J Minim Invasive Gynecol. 24(1):48-54.
53. Uppal S, Penn C, Del Carmen MG, Rauh-Hain JA, Reynolds RK, Rice LW. (2016). Readmissions after major gynecologic oncology surgery. Gynecol Oncol. 141(2):287-292.
54. Mäkinen J, Johansson J, Tomás C, Tomás E, Heinonen PK, Laatikainen T, et al. (2001). Morbidity of 10 110 hysterectomies by type of approach. Hum Reprod. 16(7):1473-1478.
55. Lonnerfors C, Reynisson P, Persson J. (2014). A randomized trial comparing vaginal and laparoscopic hysterectomy vs robot-assisted hysterectomy. J Minim Invasive Gynecol. 22(1):78-86.
56. Tamussino KF, Lang PF, Breinl E. (1998). Ureteral complications with operative gynecologic laparoscopy. Am J Obstet Gynecol. 178(5):967-970.
57. Tamussino KF, Lim PC, Webb MJ, Lee RA, Lesnick TG. (2001). Gastrointestinal surgery in patients with ovarian cancer. Gynecol Oncol. 80(1):79-84.
58. Mourton SM, Temple LK, Abu-Rustum NR, Gemignani ML, Sonoda Y, Bochner BH, et al. (2005). Morbidity of rectosigmoid resection and primary anastomosis in patients undergoing primary cytoreductive surgery for advanced epithelial ovarian cancer. Gynecol Oncol. 99(3):608-614.
59. Salani R, Zahurak ML, Santillan A, Giuntoli RL, 2nd, Bristow RE. (2007). Survival impact of multiple bowel resections in patients undergoing primary cytoreductive surgery for advanced ovarian cancer: a case-control study. Gynecol Oncol. 107(3):495-499.
60. Desale MG, Tanner EJ 3rd, Sinno AK, Angarita AA, Fader AN, Stone RL, et al. (2016). Perioperative fluid status and surgical outcomes in patients undergoing cytoreductive surgery for advanced epithelial ovarian cancer. Gynecol Oncol. S0090-8258(16):31501-31503.
61. Tseng JH, Suidan RS, Zivanovic O, Gardner GJ, Sonoda Y, Levine DA, et al. (2016). Diverting ileostomy during primary debulking surgery for ovarian cancer: Associated factors and postoperative outcomes. Gynecol Oncol. 142(2):217-224.
62. Society of Gynecologic Oncology. (2020). The Society of Gynecologic Oncology 2020 State of the Society Survey.
63. Straubhar AM, Zhou Q, Iasonos A, Clarke-Pearson DL, Cliby WA, Hoffman MS, et al. (2023). Cross-sectional survey of surgical practices among gynecologic oncologists in the United States. Gynecol Oncol. 172:36-40.
64. O'Hanlan KA, Lopez L, Dibble SL, Garnier AC, Huang GS, Leuchtenberger M. (2003). Total laparoscopic hysterectomy: body mass index and outcomes. Obstet Gynecol. 102(6):1384-1392.
65. O'Hanlan KA, Huang GS, Lopez L, Garnier AC. (2004). Total laparoscopic hysterectomy for oncological indications with outcomes stratified by age. Gynecol Oncol. 95(1):196-203
66. O'Hanlan KA, Huang GS, Lopez L, Garnier AC. (2004). Selective incorporation of total laparoscopic hysterectomy for adnexal pathology and body mass index. Gynecol Oncol. 93(1):137-143.
67. O'Hanlan KA, Huang GS, Garnier AC, Dibble SL, Reuland ML, Lopez L, et al. (2005). Total laparoscopic hysterectomy versus total abdominal hysterectomy: cohort review of patients with uterine neoplasia. JSLS. 9(3):277-286.
68. O'Hanlan KA, Dibble SL, Fisher DT. (2006). Total laparoscopic hysterectomy for uterine pathology: impact of body mass index on outcomes. Gynecol Oncol. 103(3):938-941.
69. O'Hanlan KA, Dibble SL, Young-Spint M. (2007). Total laparoscopic hysterectomy for female-to-male transsexuals. Obstet Gynecol. 110(5):1096-1101.
70. O'Hanlan KA, Dibble SL, Garnier AC, Reuland ML. (2007). Total laparoscopic hysterectomy: Technique and complications of 830 cases. JSLS. 11(3):45-53.
71. O'Hanlan KA, Pinto RA, O'Holleran M. (2007). Total laparoscopic hysterectomy with and without lymph node dissection for uterine neoplasia. J Minim Invasive Gynecol. 14(4):449-452.
72. O'Hanlan KA, McCutcheon SP, McCutcheon JG. (2011). Laparoscopic hysterectomy: impact of uterine size. J Minim Invasive Gynecol. 18(1):85-91.
73. Kato K, Matsuura M, Takeshima N. (2015). Secondary debulking surgery for isolated para-aortic nodal recurrence in endometrial cancer requiring partial resection of the inferior vena cava. Gynecol Oncol. 139(3):580-581.
74. Zippia tce. Ob/Gyn Demographics And Statistics In The US. https://www.zippia.com/ob-gyn-jobs/demographics/
75. Bringing Gynecologic Cancer Care Closer to Patients. Penn LDI: 2021. https://ldi.upenn.edu/our-work/research-updates/bringing-gynecologic-cancer-care-closer-to-patients/
76. O'Hanlan KA, McCutcheon SP, McCutcheon JG, Charvonia BE. (2012). Single-Field Sterile-Scrub, Preparation, and Dwell for Laparoscopic Hysterectomy. J Minim Invasive Gynecol. 19(2):220-224.
77. O'Hanlan KA, Emeney PL, Frank MI, Milanfar LC, Sten MS, Uthman KF. (2021). Total Laparoscopic Hysterectomy: Making It Safe and Successful for Obese Patients. JSLS. 25(2):e2020.00087.
78. Leeds IL, Canner JK, Efron JE, Ahuja N, Haut ER, Wick EC, et al. (2017). The independent effect of cancer on outcomes: a potential limitation of surgical risk prediction. J Surg Res. Dec 220:402-409 e6.
79. Dignity Health. (2022). A Community of Compassion,. Website page. Dignity Health.
80. Matzka K. (2017). Commentary: It's Time to Rewrite the Healthcare Quality Improvement Act of 1986. Physician Leadersh J. 4(1):52-55
81. ten Broek RP, Issa Y, van Santbrink EJ, Bouvy ND, Kruitwagen RF, Jeekel J, et al. (2013). Burden of adhesions in abdominal and pelvic surgery: systematic review and met-analysis. BMJ. 347:f5588.
82. Piché GR. (2012). Sham peer review: The power of immunity and the abuse of trust. CreateSpace Publishing.
83. Chalifoux R Jr. (2005). So what is a sham peer review? MedGenMed. 7(4):47.
84. Wang L, Murnighan JK. (2013). The generalist bias. . Organizational Behavior and Human Decision Processes. 120(1):47-61.
85. Hariton E, Freret TS, Nitecki R, Hinchcliff E, Stagg A. (2018). Program Director Perceptions of Subspecialty Tracking in Obstetrics and Gynecology Residency. J Grad Med Educ. 10(6):665-670. doi:10.4300/JGME-D-18-00096.1
86. Cadish LA, Kropat G, Muffly TM. (2021). Hysterectomy Volume Among Recent Obstetrics and Gynecology Residency Graduates. Female Pelvic Med Reconstr Surg. 27(6):382-387.
87. Murff HJ, France DJ, Blackford J, Grogan EL, Yu C, Speroff T, Pichert JW, et al. (2006). Relationship between patient complaints and surgical complications. Qual Saf Health Care. 15(1):13-16.
88. Casanova-Perez R, Apodaca C, Bascom E, Mohanraj D, Lane C, Vidyarthi D, et al. (2022). Broken down by bias: Healthcare biases experienced by BIPOC and LGBTQ+ patients. AMIA Annu Symp Proceedings. 2022:275-284.
89. Hong LJ, Rubinsak L, Benoit MF, Teoh D, Chandavarkar U, Brockmeyer A, et al. (2022). Gynecologic Oncology and Inclusion of Women Into the Surgical Workforce: The Canary in This Coal Mine. Front Oncol. 12:789910.
90. Newman C, Templeton K, Chin EL. (2020). Inequity and Women Physicians: Time to Change Millennia of Societal Beliefs. Perm J. 24:1-6.
91. Hall WJ, Chapman MV, Lee KM, Merino YM, Thomas TW, Payne BK, et al. (2015). Implicit Racial/Ethnic Bias Among Health Care Professionals and Its Influence on Health Care Outcomes: A Systematic Review. Am J Public Health. 105(12):e60-76.
92. Sabin JA, Riskind RG, Nosek BA. (2015). Health Care Providers' Implicit and Explicit Attitudes Toward Lesbian Women and Gay Men. Am J Public Health. Sep 105(9):1831-1841.