Quality indicators

< Back to Data collection & documentation

Authors

Webbe J, Lack N, Daly M, Göpel W, Helenius K, Latour J, Modi N, Soll R, Wills-Eve B

Target group

Infants and parents

User group

Healthcare professionals, neonatal units, hospitals, and health services

Statement of standard

Recording, collating and reporting quality indicators in a standardised manner supports comparisons of care nationally, within Europe and beyond.

Rationale

The care provided to neonates affects outcomes in every organ system (1) with implications that extend through childhood (2), into adult life (3), and may affect an individual’s offspring. (4) Admission to a neonatal unit also affects parents and the wider family. (5–7) Valid, reliable quality indicators are needed to ensure that the care provided to neonatal patients is evidence-based, of the highest standards, and leads to positive long-term outcomes.

Quality indicators are standardised, evidence-based measures to monitor and evaluate the process, performance or outcomes of neonatal care delivery. (8,9) Recording and reporting these indicators in a standardised manner allows audit, benchmarking, quality improvement, service evaluation and research across Europe: this allows greater understanding of the variation in care provision and outcomes currently seen within (10–12) and between countries. (13,14) The European Standards of Care for Newborn Health (ESCNH) quality indicators identified in this standard include background characteristics required for risk-adjustment (15,16), process measures (17), and neonatal outcomes. (18)

Within Europe, there are a number of regional, national and international databases holding data relating to quality indicators; but coverage is not universal and data are not always comparable between databases. (19) Expanding the coverage of existing databases and creating new databases (where necessary) allows international data combination and comparison. Data should be recorded using standard nomenclatures and internationally recognised terminologies. Ideally, individual data components are captured to allow the application of multiple indicator definitions to ensure meaningful comparisons can be made. For example, by capturing data reporting duration of different modes of ventilation any selected definition of bronchopulmonary dysplasia could be applied across multiple databases. (20) The creation of pan-European databases to improve neonatal care should be compatible with existing international projects (such as the NNRD, eNewborn, iNeo and the Vermont Oxford Network). Comparing quality indicators internationally helps identify optimal practice within Europe, highlights practice deficits, and ensures continued improvement in neonatal outcomes. (21)

Benefits

Short-term benefits

Improved quality of clinical care of newborns (19,22)
Facilitated decision-making conversations between parents and healthcare professionals (5,23)

Long-term benefits

Facilitated meaningful comparison of outcomes in populations in different countries within Europe and beyond (24)
Facilitated meaningful comparisons of outcomes over time (11)
Better data for quality improvement, audit and research (25)
Identified priority areas for improvement in services from local to international level (26)

Components of the standard

Component

Grading of evidence

Indicator of meeting the standard

For parents and family

Individuals born preterm are informed of the details of their birth and neonatal period so that they can make informed decisions about their ongoing healthcare needs. (27)

A (High quality)

Audit report¹, parent feedback

Former neonatal patients should be able to access their medical records.

B (Moderate quality)

Audit report¹

Clear information about quality indicators is available to parents and former patients in multiple formats (including as written information sheets) (see Data collection & documentation). (23,28)

A (High quality)

Audit report¹, parent feedback, patient information sheet

Former neonatal patients and parents are involved in all aspects of projects using quality indicators. (29)

A (High quality)

Parent feedback

For healthcare professionals

All responsible healthcare professionals participate actively in the collection of data relating to quality indicators (as appropriate to their role).

B (Moderate quality)

Guideline

Training on quality indicator related data is attended by all responsible healthcare professionals.

B (Moderate quality)

Training documentation

For neonatal unit

Data is collected relating to quality indicators. (18,19,30,31)

A (High quality)

Audit report¹

Data is collected for all neonates (alive and deceased) who receive or have received neonatal care.

B (High quality)

Audit report¹

For hospital

Data relating to quality indicators are collected, reported, and audited to benchmark local performance and identify areas for improvement. (18,19,30–32)

A (High quality)

Audit report¹

Data relating to quality indicators is contributed to regional and national databases. (18,19,32,33)

A (High quality)

Audit report¹, regional network

For health service

Data relating to quality indicators are collated and reviewed annually. (18,19,32)

A (High quality)

Audit report¹

Data relating to quality indicators are used for benchmarking at national and international level. (18,19,32)

A (High quality)

Policy statement

Data relating to quality indicators are made available for research. (19,31)

A (High quality)

Guideline

Data relating to quality indicators are available for policy-makers to guide priority setting.

B (High quality)

Policy statement

¹The indicator ‘audit report’ can also be defined as a benchmarking report.

Where to go

Further development

Grading of evidence

For parents and family

The views of parents and former neonatal patients are included in data collection and analysis. (19,23)

A (High quality)

Information sheets to inform parents and former patients about quality indicators are co-designed. (23)

A (High quality)

A written, infant-focused lay summary detailing the care a patient has received is given to all families when their infant is discharged home.

B (Moderate quality)

For healthcare professionals

Ensure data entry for all variables is completed to high standards. (33–36)

A (High quality)

For neonatal unit

Ensure data entry for all variables is completed to high standards. (33–36)

A (High quality)

Evaluate the involvement of former neonatal patients and parents in projects related to quality indicators annually. (29)

A (High quality)

For hospital

Collect data for all live born neonates.

B (High quality)

Extract data, whenever possible, automatically from electronic records to minimise the burden on clinical staff.

B (Moderate quality)

Contribute data relating to quality indicators to regional and national databases. (18,30)

A (Moderate quality)

For health service

Ensure that data relating to quality indicators covers the entire population.

B (High quality)

Collate and review data relating to quality indicators annually. (18,30)

A (Moderate quality)

Establish a regular, pan-European audit of the European Standards of Care for Newborn Health.

B (High quality)

Compare quality indicators internationally and over time to identify areas to prioritise. (11,13,31)

A (High quality)

Compare outcomes beyond Europe (e.g. with NNRD, iNeo, eNewborn and VON) to ensure European neonatal care is among the best globally. (13,19,32,33)

A (High quality)

Getting started

Initial steps

For parents and family

Information about quality indicators should be available to parents and former patients orally.

For healthcare professionals

Participate in the collection of data relating to quality indicators.

For neonatal unit

Collect data relating to quality indicators for all neonates admitted to the neonatal unit.
Create local systems to allow data collection and recording.

For hospital

Collect, report and audit data relating to quality indicators in order to benchmark local performance and identify areas for improvement.

For health service

Collate data relating to quality indicators at regional and national level.

Quality indicators

The European Standards of Care for Newborn Health (ESCNH) quality indicators are variables required for collection and reporting of standardised data. They include background variables required for extensive risk-adjustment, process measures and important neonatal outcomes. The variables have been derived from: a systematic review of existing databases (19), a systematic review of the background characteristics reported in clinical trials (15), international consensus projects (18,30), and the variables included in validated prognostic models. (16)

These indicators should be recorded and reported using standard nomenclature, in line with internationally recognised terminology. However, we acknowledge that for most quality indicators no global consensus exists as to the ‘perfect’ definition or measurement tool (37–39): instead, we propose that capturing individual data components using recognised, validated definitions will provide the flexibility needed to apply different definitions to the data and facilitate meaningful comparisons between databases. It is also essential that suitable denominator data are available to allow meaningful comparison between populations: for this to occur demographic data relating to all live births, defined according to the AAP definition (40), should be available.

Timing of assessment and data collection is important: as a minimum, it should be in line with standardised assessments, but collecting and recording more frequent data will facilitate the application of multiple definitions. Furthermore, some of the indicators include long-term outcomes. The impact of preterm birth has been clearly shown to extend throughout childhood and into adult life and so, outcomes should be collected into adulthood. To achieve this, collaboration will be needed to ensure that data can be linked between different databases. This work is now possible because data capture and storage technologies are improving and international collaborations are expanding allowing maximal use of available data. Future work to establish a European database will create a tiered structure of responsibility at regional, national and pan-national levels.

Indicator	Suggested data components
Background characteristics
Maternal factors
Antenatal care	1. Whether the mother accessed antenatal care (see Birth & transfer)
Antenatal steroid exposure	2. Type of steroids 3. Date and time of doses
Barriers to care	4. Whether any barriers exist preventing patients from accessing antenatal or neonatal care (e.g. cultural, financial)
Maternal age	5. Maternal age in years at birth
Maternal complications of pregnancy	6. Reported using standardised terminology according to an accepted definition
Maternal drug use	7. Drug use status at first antenatal contact (including prescribed medications and illicit substances) 8. Use of drugs during pregnancy
Maternal education	9. Reported using standardised terminology according to an accepted definition
Maternal ethnicity	10. Reported using standardised terminology according to an accepted definition
Maternal medical problems	11. Reported using an accepted definition
Maternal smoking status	12. Smoking status at first antenatal contact 13. Number of cigarettes smoked during pregnancy
Maternal socio-economic status	14. Reported using standardised terminology according to an accepted definition
Labour and delivery
Duration of rupture of membranes	15. The time from rupture of membranes to birth
Highest maternal temperature during labour	16. The highest recorded maternal temperature during labour
Location of delivery: maternity	17. The level of maternity care provided at the location of delivery
Location of care: neonatal	18. The level of neonatal care provided at the location of delivery (see Birth & transfer)
Mode of delivery	19. Reported using standardised terminology according to an accepted definition
Presentation of foetus at delivery	20. Presentation prior to delivery
Infant factors
1 minute Apgar score	21. Apgar score one minute after birth
Birth weight	22. Birth weight at the time of delivery in grams
Congenital anomaly	23. Reported using standardised terminology according to an accepted definition
Gestational age	24. Gestational age at birth of neonate in whole weeks and remaining days
Plurality	25. Number of foetuses during pregnancy
Sex	26. Phenotypic sex of neonate
Process measures
Delayed cord clamping	27. Duration from live birth to cord clamping
Family-centred care	28. Reported using standardised terminology according to an accepted definition (see Infant- & family-centred developmental care)
Feeding during admission	29. Timing of feeding 30. Route of feeding 31. Choice of milk 32. Volume of milk (see Nutrition)
Long-term follow up	33. Whether a neonate received appropriate long-term follow up (see Follow-up & continuing care)
Minimising inappropriate separation	34. Whether a neonate was separated from their mother without clinical indication (see Infant- & family-centred developmental care)
Nurse staffing ratios	35. Whether the neonate received a suitable level of nursing care throughout their stay (see Patient safety & hygiene practice)
Parenteral nutrition	36. Whether the neonate received parenteral nutrition 37. Formulation of parenteral nutrition 38. Route of parenteral nutrition 39. Duration of parenteral nutrition (see Nutrition)
Screening for retinopathy of prematurity	40. Whether the neonate underwent screening for retinopathy of prematurity (if indicated) 41. Timing of screening for retinopathy of prematurity (see Medical care & clinical practice)
Surfactant administration	42. Doses of surfactant received 43. Formulation of surfactant 44. Route of surfactant
Outcomes
Adverse events	1.Any harm from care during the neonatal period reported using standardised terminology according to an accepted definition
Brain injury (on imaging)	2. Timing of scans 3. Pathology seen on scan, reported using standardised terminology according to an accepted definition
Chronic lung disease/bronchopulmonary dysplasia	4. Mode of respiratory support 5. Duration of respiratory support
General cognitive ability	6. Timing of milestones 7. Timing and results of testing, reported using standardised terminology according to an accepted definition
General gross motor ability	8. Timing of milestones 9. Timing and results of testing, reported using standardised terminology according to an accepted definition
Hearing impairment or deafness	10. Timing of hearing tests 11. Results of hearing tests
Necrotising enterocolitis	12. Timing and results of relevant biochemical and radiological testing 13. Clinical features 14. Timing and findings at surgery
Pain	15. Reported using standardised terminology according to an accepted definition
Pulmonary function	16. Reported using standardised terminology according to an accepted definition
Quality of life	17. Reported using standardised terminology according to an accepted definition
Retinopathy of prematurity	18. Timing and results of screening 19. Timing and mode of therapy
Sepsis	20. Timing and results of relevant biochemical and microbiological testing 21. Clinical features
Social functioning	22. Reported using standardised terminology according to an accepted definition
Survival	23. Timing of death 24. Survival to discharge home
Visual impairment or blindness	25. Timing of vision tests 26. Results of vision tests

McCormick MC, Litt JS, Smith VC, Zupancic JAF. Prematurity: an overview and public health implications. Annu Rev Public Health. 2011;32:367–79.
Anderson P, Doyle LW, Victorian Infant Collaborative Study Group. Neurobehavioral outcomes of school-age children born extremely low birth weight or very preterm in the 1990s. JAMA. 2003 Jun 25;289(24):3264–72.
Moster D, Lie RT, Markestad T. Long-Term Medical and Social Consequences of Preterm Birth. New England Journal of Medicine. 2008 Jul 17;359(3):262–73.
Swamy GK, Ostbye T, Skjaerven R. Association of preterm birth with long-term survival, reproduction, and next-generation preterm birth. JAMA. 2008 Mar 26;299(12):1429–36.
Harrison H. The principles for family-centered neonatal care. Pediatrics. 1993 Nov;92(5):643–50.
van Veenendaal NR, van Kempen AAMW, Franck LS, O’Brien K, Limpens J, van der Lee JH, et al. Hospitalising preterm infants in single family rooms versus open bay units: A systematic review and meta-analysis of impact on parents. EClinicalMedicine. 2020 Jun;23:100388.
Ding X, Zhu L, Zhang R, Wang L, Wang TT, Latour JM. Effects of family-centred care interventions on preterm infants and parents in neonatal intensive care units: A systematic review and meta-analysis of randomised controlled trials. Australian Critical Care. 2019 Jan;32(1):63–75.
AHRQ. Quality Improvement and monitoring at your fingertips [Internet]. Agency for Healthcare Research and Quality. Available from: https://qualityindicators.ahrq.gov/#:~:text=Quality%20Indicators%20(QIs)%20are%20standardized,track%20clinical%20performance%20and%20outcomes
NICE. Standards and Indicators [Internet]. NICE. NICE; [cited 2022 Jun 2]. Available from: https://www.nice.org.uk/standards-and-indicators
Pierrat V, Burguet A, Marchand-Martin L, Cambonie G, Coquelin A, Roze JC, et al. Variations in patterns of care across neonatal units and their associations with outcomes in very preterm infants: the French EPIPAGE-2 cohort study. BMJ Open. 2020 Jun 1;10(6):e035075.
Santhakumaran S, Statnikov Y, Gray D, Battersby C, Ashby D, Modi N, et al. Survival of very preterm infants admitted to neonatal care in England 2008-2014: time trends and regional variation. Arch Dis Child Fetal Neonatal Ed. 2018 May;103(3):F208–15.
Liem TBY, Krediet TG, Fleer A, Egberts TCG, Rademaker CMA. Variation in antibiotic use in neonatal intensive care units in the Netherlands. J Antimicrob Chemother. 2010 Jun;65(6):1270–5.
Helenius K, Sjörs G, Shah PS, Modi N, Reichman B, Morisaki N, et al. Survival in Very Preterm Infants: An International Comparison of 10 National Neonatal Networks. Pediatrics. 2017 Dec;140(6).
Shah PS, Lui K, Sjörs G, Mirea L, Reichman B, Adams M, et al. Neonatal Outcomes of Very Low Birth Weight and Very Preterm Neonates: An International Comparison. J Pediatr. 2016 Oct;177:144-152.e6.
Jawad S, Modi N, Prevost AT, Gale C. A systematic review identifying common data items in neonatal trials and assessing their completeness in routinely recorded United Kingdom national neonatal data. Trials. 2019 Dec 16;20(1):731.
Rysavy MA, Horbar JD, Bell EF, Li L, Greenberg LT, Tyson JE, et al. Assessment of an Updated Neonatal Research Network Extremely Preterm Birth Outcome Model in the Vermont Oxford Network. JAMA Pediatr. 2020 May 1;174(5):e196294.
Elverson CA, Samra HA. Overview of Structure, Process, and Outcome Indicators of Quality in Neonatal Care. Newborn and Infant Nursing Reviews. 2012 Sep 1;12(3):154–61.
Webbe JWH, Duffy JMN, Afonso E, Al-Muzaffar I, Brunton G, Greenough A, et al. Core outcomes in neonatology: development of a core outcome set for neonatal research. Arch Dis Child Fetal Neonatal Ed. 2019 Nov 15;fetalneonatal-2019-317501.
Modi N, Ashby D, Battersby C, Brocklehurst P, Chivers Z, Costeloe K, et al. Developing routinely recorded clinical data from electronic patient records as a national resource to improve neonatal health care: the Medicines for Neonates research programme [Internet]. Southampton (UK): NIHR Journals Library; 2019 [cited 2022 Jun 1]. (Programme Grants for Applied Research). Available from: http://www.ncbi.nlm.nih.gov/books/NBK546540/
Hines D, Modi N, Lee SK, Isayama T, Sjörs G, Gagliardi L, et al. Scoping review shows wide variation in the definitions of bronchopulmonary dysplasia in preterm infants and calls for a consensus. Acta Paediatr. 2017 Mar;106(3):366–74.
Zeitlin J, Manktelow BN, Piedvache A, Cuttini M, Boyle E, van Heijst A, et al. Use of evidence based practices to improve survival without severe morbidity for very preterm infants: results from the EPICE population based cohort. BMJ. 2016 Jul 5;354:i2976.
Soll RF. Evaluating the medical evidence for quality improvement. Clin Perinatol. 2010 Mar;37(1):11–28.
Celenza JF, Zayack D, Buus-Frank ME, Horbar JD. Family Involvement in Quality Improvement: From Bedside Advocate to System Advisor. Clin Perinatol. 2017 Sep;44(3):553–66.
LaRovere JM, Jeffries HE, Sachdeva RC, Rice TB, Wetzel RC, Cooper DS, et al. Databases for assessing the outcomes of the treatment of patients with congenital and paediatric cardiac disease–the perspective of critical care. Cardiol Young. 2008 Dec;18 Suppl 2:130–6.
Gale C, Morris I. The UK National Neonatal Research Database: using neonatal data for research, quality improvement and more. Archives of Disease in Childhood – Education and Practice. 2016 Aug 1;101(4):216–8.
Horbar JD, Plsek PE, Leahy K, NIC/Q 2000. NIC/Q 2000: establishing habits for improvement in neonatal intensive care units. Pediatrics. 2003 Apr;111(4 Pt 2):e397-410.
Leavy P, Violeta Prina M, Martínez Cáceres MJ, Bauer G. Social representations of premature birth from the perspective of individuals born preterm in the 1990s. Arch Argent Pediatr. 2015 Jan;113(1):6–11.
Harrison W, Goodman D. Epidemiologic Trends in Neonatal Intensive Care, 2007-2012. JAMA Pediatr. 2015 Sep;169(9):855–62.
INVOLVE. Public involvement in research: values and principles framework [Internet]. INVOLVE: Eastleigh; 2015 [cited 2022 Jun 2]. Available from: https://www.invo.org.uk/wp-content/uploads/2017/08/Values-Principles-framework-Jan2016.pdf
ICHOM. Set of Patient-Centered Outcome Measures for Preterm and Hospitalised Newborn Health [Internet]. 2020 [cited 2022 Jun 8]. Available from: https://connect.ichom.org/patient-centered-outcome-measures/preterm-and-hospitalized-newborn-health/
Oliver S, Uhm S, Duley L, Crowe S, David AL, James CP, et al. Top research priorities for preterm birth: results of a prioritisation partnership between people affected by preterm birth and healthcare professionals. BMC Pregnancy Childbirth. 2019 Dec;19(1):528.
Soll RF, McGuire W. Evidence-Based Practice: Improving the Quality of Perinatal Care. Neonatology. 2019;116(3):193–8.
Shah PS, Seidlitz W, Chan P, Yeh S, Musrap N, Lee SK, et al. Internal Audit of the Canadian Neonatal Network Data Collection System. Am J Perinatol. 2017 Oct;34(12):1241–9.
Arts DGT, De Keizer NF, Scheffer GJ. Defining and improving data quality in medical registries: a literature review, case study, and generic framework. J Am Med Inform Assoc. 2002 Dec;9(6):600–11.
Battersby C, Statnikov Y, Santhakumaran S, Gray D, Modi N, Costeloe K, et al. The United Kingdom National Neonatal Research Database: A validation study. PLoS One. 2018;13(8):e0201815.
Horbar JD, Leahy KA. An assessment of data quality in the Vermont-Oxford Trials Network database. Control Clin Trials. 1995 Feb;16(1):51–61.
Ioannidis JPA, Horbar JD, Ovelman CM, Brosseau Y, Thorlund K, Buus-Frank ME, et al. Completeness of main outcomes across randomized trials in entire discipline: survey of chronic lung disease outcomes in preterm infants. BMJ. 2015 Jan 26;350:h72.
Webbe JWH, Ali S, Sakonidou S, Webbe T, Duffy JMN, Brunton G, et al. Inconsistent outcome reporting in large neonatal trials: a systematic review. Arch Dis Child Fetal Neonatal Ed. 2020 Jan;105(1):69–75.
Janvier A, Farlow B, Baardsnes J, Pearce R, Barrington KJ. Measuring and communicating meaningful outcomes in neonatology: A family perspective. Seminars in Perinatology [Internet]. 2016 Oct [cited 2016 Nov 11]; Available from: http://linkinghub.elsevier.com/retrieve/pii/S0146000516300817
Barfield WD, COMMITTEE ON FETUS AND NEWBORN. Standard Terminology for Fetal, Infant, and Perinatal Deaths. Pediatrics. 2016 May;137(5):e20160551.