The Care of Preterm Infants With Birth Weight Below 1250 g
Risk-Adjusted Quality Benchmarking as Part of Validating a Caseload-Based Management System
; ; ; ; ;
Background: In Germany, controversy currently surrounds the contention that the quality of care for preterm infants weighing less than 1250 g is best assured by requiring that centers treat a minimum of 30 such cases per year.
Methods: A risk-adjusted model was developed on the basis of neonatal data from 7405 preterm infants treated in German centers, and the effect of caseload on risk-adjusted mortality was analyzed. In addition, the discriminative ability of the minimal caseload requirement for quality assessment was studied. The authors designate the quality of care in a particular center as above average if the observed mortality is lower than would have been expected from the risk profile of the preterm infants treated there.
Results: Risk-adjusted mortality was found to be significantly higher in smaller centers (those with fewer than 30 cases per year) than in larger ones (odds ratio, 1.34). Even among centers whose caseload exceeded the minimum requirement, there was still marked variability in risk-adjusted mortality (range: 3.5% to 28.6%). Of all the preterm infants treated in larger centers, 56% were treated in centers with above-average quality of care. 44% of the centers with above-average quality of care had caseloads in the range of 14 to 29 cases per year.
Conclusion: Because of the marked variability in risk-adjusted mortality, even among larger centers, a caseload of 30 or more cases per year is not a suitable indicator of the quality of care. The neonatal data of external quality assurance should be used to develop an instrument for quality-based coordination of care that takes not just morbidity and mortality, but also the treating centers’ competence profiles into account.
More than 9000 preterm infants weighing less than 1500 g are born every year in Germany (1). In order to guarantee high-value care for these high-risk patients, in 2009 Germany’s Federal Joint Committee (G-BA, Gemeinsamer Bundesausschuss) set a minimum caseload (2) for this area, which was raised to 30 per year for infants weighing less than 1250 g in 2010. (3). However, the minimum is currently suspended (4). The G-BA bases the required caseload on a report by the Institute for Quality and Efficiency in Health Care (IQWiG, Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen) (5). The IQWiG concludes that for the care of neonates with birth weight below 1500 g mortality falls as a hospital’s caseload increases, but the lower mortality rate is not necessarily caused by the higher volume of care. The idea arose of examining the basis for argumentation and the potential effects of the minimum caseload using neonatal data that have been gathered all over Germany for several years. As in other, comparable studies, mortality was selected as a surrogate outcome parameter for other aspects of quality of care. Using neonatal data, the effect of the caseload on risk-adjusted mortality was analyzed on the basis of a differentiated risk-adjustment, and subsequently the discriminative ability of a minimum annual number of 30 cases was investigated.
Analysis included data from preterm infants with birth weight below 1250 g born between 2007 and 2009, from the German federal states of Bavaria, Baden-Württemberg, Hesse, Lower Saxony, and North Rhine–Westphalia. These states account for approximately 68% of all births in Germany (6).
A systematic review of the literature identified risk factors that affect the mortality rate of preterm infants and are not affected by care. A risk-adjustment model with mortality as outcome parameter was then developed.
Calculation of mortality rates did not include preterm infants admitted to neonatal units more than 24 hours after birth, because they had probably received neonatal treatment before admission. Infants referred to other units were also excluded, because mortality rates cannot be calculated for them. Quality was therefore assessed on the basis of data from 8259 preterm infants who received all their treatment in a single unit. Risk factors were not fully recorded for 854 (10.3%) of these 8259 infants. As a result, the risk-adjustment model was based on 7405 (89.7%) preterm infants.
The risk-adjustment model was used to calculate each hospital’s expected mortality, E, according to the risk profile of the treated preterm infants. E was then compared to observed mortality, O. A quotient O÷E of less than 1 (or greater than 1) should be considered evidence of above-average (or below-average) quality of care.
In order to assess the effect of the caseload, the factor “caseload” was additionally included in the model, using the categories “fewer than 30 cases” and “30 cases or more.” In a second version, caseload was considered as factor with 12 intervals of five up to 59 and the category “60 or more.” The caseloadsrecorded in 2009 were used as the basis for these categories. Also investigated was the question of whether the minimum caseload could distinguish sufficiently well between above-average and below-average quality of care.
The methods used are described in detail in the supplementary eMethods section.
Of the 8259 preterm infants born between 2007 and 2009, 1179 (14.3%) died. In 2009, 39 hospitals met the criterion of at least 30 cases. In these hospitals, 14.1% of preterm infants (613 of 4341) died. The mortality rate in the 127 units with fewer than 30 cases was 14.4% (566 of 3918).
At 18.2% the proportion of preterm infants referred to or from a neonatal unit in larger hospitals is somewhat higher than in hospitals with smaller caseloads (16.5%). The expected mortality rate for these excluded cases (1738 infants in all) is 14.5% for large centers and 16.7% for smaller ones.
The risk-adjustment model
The search of the literature identified the parameters multiple birth, sex, birth weight, week of gestation, head circumference on admission, body length on admission, malformations, base deficit, and minimum and maximum FiO2 administered as potentially relevant risk factors. The parameter “head circumference” was excluded from the multiple logistic regression model, because it was not significant at a level of 0.05. Table 1 shows the results of a univariate descriptive analysis of the factors included in the risk-adjustment model.
Data on risk factors were complete for 89.7% of the preterm infants (7405 of 8259). The incomplete datasets are distributed equally between large and small hospitals (10.3% and 10.4%). The mortality rate for these cases is 18.9%, higher than for infants with complete datasets (13.7%). It is 23.9% in the 39 large units, and 13.3% in the 127 small units.
The results yielded by the multiple logistic risk-adjustment model are shown in Table 2. The risk factors included show a highly significant effect on mortality. Nagelkerke’s R-square value is 0.46. The Hosmer–Lemeshow test indicates a good model fit, with a p-value of 0.65.
The effect of volume of care
If the caseload is divided into two categories (preterm infants from hospitals with fewer than 30 cases in 2009 versus preterm infants from hospitals with 30 cases or more), the adjusted risk of death for infants from small units is higher overall than for infants from large units (odds ratio 1.34, p = 0.001). If the caseload is divided into 13 categories (Table 3), there is a significantly increased risk-adjusted mortality in the categories “5 to 9,” “15 to 19,” “20 to 24,” and “35 to 39” than in the reference category. In contrast, for the four categories “30 to 34,” “40 to 44,” “50 to 54,” and “55 to 59” the risk-adjusted mortality is lower than in the reference category, although not significant. No monotonic increase in the effect of the caseload across all caseload categories is observed. Taking caseload into account in the risk-adjustment model, there are only minor changes in the results of the other risk factors, as expected.
Table 4 shows the values of O÷E and the risk-adjusted mortality rate for each interval. Risk-adjusted mortality is calculated as the product of O÷E and overall mortality (n = 7405, 13.7%). It varies substantially within individual caseload categories (Figures 1 and 2). For example, in units with more than 60 cases it ranges from 5.9% to 17.9%.
Table 4 also shows the distribution of preterm infants and hospitals among the 13 caseload categories. Only just under one-quarter of hospitals (n = 39; 24%) achieved the minimum caseload of 30 cases in 2009. 46.8% (n = 4671) of preterm infants were cared for in hospitals with small caseloads, and 53.2% (5306) in large units.
Discriminative ability of the minimum caseload
Assessment of the discriminative ability of the minimum caseload criterion included the 91 hospitals in which at least 14 preterm infants had been cared for, in line with the minimum caseload requirement for 2009 in Germany (Table 5). 52 hospitals were deemed to provide insufficient quality of care according to the current minimum caseload criterion, because they cared for fewer than 30 cases. However, at the same time, 17 of these hospitals are among the 39 hospitals with above-average quality of care (O÷E<1). This means that there is a false negative rate of 44% at hospital level. In contrast, only 2975 of the 5306 preterm infants cared for in hospitals with supposedly good quality of care (30 cases or more) were actually treated in hospitals with above-average quality of care. In other words, the positive predictive value of the minimum caseload criterion is 56% at patient level.
A minimum caseload criterion is intended to substantially improve the quality of care provided to very small preterm infants by concentrating care in centers with higher caseloads. To assess this, neonatal data were used to examine the potential effect of a minimum annual caseload of 30 cases.
Analysis is limited to risk-adjusted mortality as a central indicator of quality of care, because it is an outcome parameter on which information can be reliably gathered. Simultaneously examining complication morbidity was beyond the scope of this article. However, it should be examined in future quality assessments.
Because records are insufficiently validated, it is often insinuated that neonatal data are incomplete. The higher mortality rate in official statistics is seen as evidence of this. Analyses by Hummer et al. (7) indicate that approximately one-third of infants who die do not show up in neonatal units’ figures. A decisive consideration in this regard might be that before 2010 records were only kept on such infants if they were admitted to a pediatric unit on or before their tenth day of life. No records were kept on infants who died before the beginning of treatment or while receiving palliative care. Various calculations show that these infants account for a large part of the gap in records. Perinatal data gathered in 2008 (8) show that half of postpartum deaths occurred within six hours (56%, 539 of 954 cases). A Swedish working group estimated the proportion of infants whose treatment was discontinued at 40% of all those who died before they were one day old (9). From 2010 onwards, new procedures for gathering neonatal data have closed this gap in perinatal mortality data in Germany. However, infants who receive palliative care play no role in the comparison of the quality of neonatal care units. Hospitals which document the data of palliatively treated infants, compared with facilities that do not, have no disadvantage because these cases are considered in the risk-adjusted mortality. This means that the available neonatal data do provide a sufficiently valid basis to address the question posed.
In addition to the unclear status of data, particularly regarding infants given palliative care, a further limitation of the study is that only data from the five German federal states named above were available. In addition, in 11.3% of the preterm infants whose data were available the factors used in the risk-adjustment model were incompletely recorded, which meant that they could not be included in risk-adjusted analysis. However, the analyzed sample is large enough to obtain meaningful results. Also, the fact that infants referred to other units were excluded may mean that the ability of particularly well-qualified units to treat high-risk infants referred from other hospitals is not taken into account. However, differences in quality should be revealed by comparing infants treated only in the unit in question.
The neonatal parameters used in the risk-adjustment model primarily form the mortality-related risk profiles of preterm infants. They are largely independent of quality of care and correspond to the factors described in the literature (10–14). The Hosmer–Lemeshow test indicates good model fit, with a p-value of 0.65. Minimum and maximum FiO2 administered are part of medical care, but like base deficit are recorded on admission. They are therefore rarely affected by care and so largely reflect preterm infants’ health.
If caseload is divided into only two categories in the risk-adjustment model, risk-adjusted mortality in the group of units with annual caseloads of 30 or more is significantly lower than in the group of smaller units. Bartels et al. (15) had already reported a significantly higher adjusted mortality rate for infants whose birth weight was under 1500 g who were born in small hospitals (annual caseload fewer than 36). Trotter and Pohlandt (16) described a significant connection between caseload and mortality in infants weighing under 750 g or born before the 26th week of gestation. Phibbs et al. (17) and Chung et al. (18) showed a mostly significantly lower risk-adjusted mortality rate for preterm infants from Californian hospitals with the highest quality levels and annual caseloads of more than 100 than for infants from hospitals at levels 1 to 3c combined with various categories for caseloads of 100 or fewer. However, no monotonic increase in the effect of the caseload across all caseload categories can be determined.
This study also reveals a heterogeneous picture across caseload categories if caseload is divided into 13 categories. This means that some of the results on risk-adjusted mortality are as good or even better in smaller caseload categories than in the group of the largest hospitals. The limited relationship between caseload and mortality is particularly clear in the categories from 50 cases per year upwards, in which risk-adjusted mortality ranges from 6% to 19% (Figure 2). This high variation even in centers with large caseloads is also described by other authors (11, 19, 20) and means that the minimum caseload criterion can distinguish between low and high risk-adjusted mortality only to a limited extent, for both individual hospitals and individual cases. Thus this analysis shows a 44% false negative rate in the assessment of quality of care of individual hospitals on the basis of their caseloads. In addition, only 56% of preterm infants treated in units with caseloads of 30 or more receive above-average quality of care.
The variations in the results even of hospitals with very large caseloads indicate that caseload is only a surrogate parameter for other factors that directly affect quality of care, such as staffing structure, equipment, organization, process and internal quality management, and a culture of quality (21). Future studies should research the role actually played by these factors and how they can be optimized to achieve a high quality of care. One idea would be to use neonatal data gathered according to section 137a of the German Code of Social Law (SGB), Part V to develop tools for quality-based care management that include morbidity parameters in addition to mortality itself.
Good risk adjustment allows direct qualitative comparison of units with different patient populations and competencies. For the purpose of care planning, however, despite equally high quality of care the successful treatment of low-risk patients cannot be placed on the same footing as equally successful treatment of high-risk patients. Care policy management therefore requires a tool that combines both aspects: competence profiles and quality of care.
Conflict of interest statement
The BQS Institute for Quality and Patient Safety (BQS Institut für Qualität & Patientensicherheit) received financial support from the German Hospital Association (DKG, Deutsche Krankenhausgesellschaft e. V.) for this analysis.
Dr. Kutschmann, Dr. Bungard, Mr. Kötting, Mrs. Trümner, and Dr. Veit are employees of the BQS Institute for Quality and Patient Safety. The authors declare that no other conflict of interest exists.
Prof. Fusch has received lecture fees from Abbott.
Manuscript received on 13 September 2011, revised version accepted on 27 April 2012.
Translated from the original German by Caroline Devitt, MA.
Dr. rer. medic. Marcus Kutschmann
BQS Institut für Qualität & Patientensicherheit
40472 Düsseldorf, Germany
McMaster University, Hamilton (Canada): Prof. Dr. med. Fusch
|1.||AQUA-Institut für angewandte Qualitätsförderung und Forschung im Gesundheitswesen GmbH. Qualitätsreport 2010: Geburtshilfe .|
|2.||Gemeinsamer Bundesausschuss (G-BA): Bekanntmachung eines Beschlusses des Gemeinsamen Bundesausschusses zur Versorgung von Früh- und Neugeborenen. 20. August 2009 (BAnz. Nr. 195 (p. 4450) 24. Dezember 2009. Köln: Bundesanzeiger 2009.|
|3.||Gemeinsamer Bundesausschuss (G-BA). Bekanntmachung eines Beschlusses des Gemeinsamen Bundesausschusses über eine Änderung der Anlage 1 der Mindestmengenvereinbarung: Mindestmengen bei Früh- und Neugeborenen. 17 Juni 2010. BAnz. Nr. 123 (p. 2840) 18. August 2010. Köln: Bundesanzeiger 2010.|
|4.||Gemeinsamer Bundesausschuss (G-BA). Bekanntmachung eines Beschlusses des Gemeinsamen Bundesausschusses über eine befristete Außervollzugsetzung einer Änderung der Mindestmengenvereinbarung: Mindestmengen für Früh- und Neugeborene Perinatalzentren Level 1. 16. Dezember 2010. BAnz. Nr. 200 (p. 4480) 31. Dezember 2010. Köln: Bundesanzeiger 2010.|
|5.||Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG): Zusammenhang zwischen Leistungsmenge und Ergebnis bei der Versorgung von Früh- und Neugeborenen mit sehr geringem Geburtsgewicht. Köln: Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG) 2008.|
|6.||Statistisches Bundesamt Deutschland: GENESIS-Online Datenbank. Wiesbaden 2011 .|
|7.||Hummler HD, Poets C: Mortalität sehr unreifer Frühgeborener – Erhebliche Diskrepanz zwischen Neonatalerhebung und amtlicher Geburten-/Sterbestatistik. Z Geburtshilfe Neonatol 2011; 215: 10–7. CrossRef MEDLINE|
|8.||Bundesgeschäftsstelle Qualitätssicherung GmbH: BQS-Bundesauswertung 2008 Geburtshilfe . www.bqs-outcome.de/2008/ergebnisse/leistungsbereiche/geburtshilfe/buaw/bu_Gesamt_16n1-GEBH_2008.pdf.|
|9.||Fellman V, Hellstrom-Westas L, Norman M, et al.: One-year survival of extremely preterm infants after active perinatal care in Sweden. JAMA 2009; 301: 2225–33. CrossRef MEDLINE|
|10.||Costeloe K, Hennessy E, Gibson AT, Marlow N, Wilkinson AR: The EPICure study: outcomes to discharge from hospital for infants born at the threshold of viability. Pediatrics 2000; 106: 659–71. CrossRef MEDLINE|
|11.||Horbar JD, Badger GJ, Lewit EM, Rogowski J, Shiono PH: Hospital and patient characteristics associated with variation in 28-day mortality rates for very low birth weight infants. Vermont Oxford Network. Pediatrics 1997; 99: 149–56. CrossRef MEDLINE|
|12.||Maier RF, Rey M, Metze BC, Obladen M: Comparison of mortality risk: a score for very low birthweight infants. Arch Dis Child Fetal Neonatal Ed 1997; 76: F146–F151. CrossRef MEDLINE PubMed Central|
|13.||Moro M, Figueras-Aloy J, Fernandez C, et al.: Mortality for newborns of birthweight less than 1500 g in Spanish neonatal units (2002–2005). Am J Perinatol 2007; 24: 593–601. CrossRef MEDLINE|
|14.||The International Neonatal Network (INN): The CRIB (clinical risk index for babies) score: a tool for assessing initial neonatal risk and comparing performance of neonatal intensive care units. Lancet 1993; 342: 193–8. CrossRef MEDLINE|
|15.||Bartels DB, Wypij D, Wenzlaff P, Dammann O, Poets CF: Hospital volume and neonatal mortality among very low birth weight infants. Pediatrics 2006; 117: 2206–14. CrossRef MEDLINE|
|16.||Trotter A, Pohlandt F: Aktuelle Ergebnisqualität der Versorgung von Frühgeborenen < 1 500 g Geburtsgewicht als Grundlage für eine Regionalisierung der Risikogeburten. Z Geburtshilfe Neonatol 2010; 214: 55–61. CrossRef MEDLINE|
|17.||Phibbs CS, Baker LC, Caughey AB, Danielsen B, Schmitt SK, Phibbs RH: Level and volume of neonatal intensive care and mortality in very-low-birth-weight infants. N Engl J Med 2007; 356: 2165–75. CrossRef MEDLINE|
|18.||Chung JH, Phibbs CS, Boscardin WJ, Kominski GF, Ortega AN, Needleman J: The effect of neonatal intensive care level and hospital volume on mortality of very low birth weight infants. |
Med Care 2010; 48: 635–44. CrossRef MEDLINE
|19.||Heller G: Ermittlung der klinikspezifischen Ergebnisqualität der Behandlung von Früh- und Neugeborenen mit sehr niedrigem Geburtsgewicht (VLBWs) auf der Basis von Routinedaten. In: Klauber J, Geraedts M, Friedrich J, Wasem J (eds.): Krankenhaus-Report 2011 – Schwerpunkt: Qualität durch Wettbewerb. Stuttgart: Schattauer 2010; 39–47.|
|20.||Rogowski JA, Horbar JD, Staiger DO, Kenny M, Carpenter J, Geppert J: Indirect vs direct hospital quality indicators for very low-birth-weight infants. JAMA 2004; 291: 202–9. CrossRef MEDLINE|
|21.||Curry LA, Spatz E, Cherlin E, et al.: What distinguishes top-performing hospitals in acute myodarcial infarction mortality rates? Ann Intern Med 2011; 154: 384–90. MEDLINE|