Quality indicators

< Back to Data collection & documentation

Authors

Webbe J, Lack N, Daly M, Göpel W, Helenius K, Latour J, Modi N, Soll R, Wills-Eve B

Click on the image to read the standard in brief.

Target group

Infants and parents

User group

Healthcare professionals, neonatal units, hospitals, and health services

Statement of standard

Recording, collating and reporting quality indicators in a standardised manner supports comparisons of care nationally, within Europe and beyond.

Rationale

The care provided to neonates affects outcomes in every organ system (1) with implications that extend through childhood (2), into adult life (3), and may affect an individual’s offspring. (4) Admission to a neonatal unit also affects parents and the wider family. (5–7) Valid, reliable quality indicators are needed to ensure that the care provided to neonatal patients is evidence-based, of the highest standards, and leads to positive long-term outcomes.

Quality indicators are standardised, evidence-based measures to monitor and evaluate the process, performance or outcomes of neonatal care delivery. (8,9) Recording and reporting these indicators in a standardised manner allows audit, benchmarking, quality improvement, service evaluation and research across Europe:  this allows greater understanding of the variation in care provision and outcomes currently seen within (10–12) and between countries. (13,14) The European Standards of Care for Newborn Health (ESCNH) quality indicators identified in this standard include background characteristics required for risk-adjustment (15,16), process measures (17), and neonatal outcomes. (18)

Within Europe, there are a number of regional, national and international databases holding data relating to quality indicators; but coverage is not universal and data are not always comparable between databases. (19) Expanding the coverage of existing databases and creating new databases (where necessary) allows international data combination and comparison. Data should be recorded using standard nomenclatures and internationally recognised terminologies. Ideally, individual data components are captured to allow the application of multiple indicator definitions to ensure meaningful comparisons can be made. For example, by capturing data reporting duration of different modes of ventilation any selected definition of bronchopulmonary dysplasia could be applied across multiple databases. (20) The creation of pan-European databases to improve neonatal care should be compatible with existing international projects (such as the NNRD, eNewborn, iNeo and the Vermont Oxford Network). Comparing quality indicators internationally helps identify optimal practice within Europe, highlights practice deficits, and ensures continued improvement in neonatal outcomes. (21)

Benefits

Short-term benefits

Long-term benefits

Components of the standard

Component

Grading of evidence

Indicator of meeting the standard

For parents and family

  1. Individuals born preterm are informed of the details of their birth and neonatal period so that they can make informed decisions about their ongoing healthcare needs. (27)

A (High quality)

Audit report1, parent feedback

  1. Former neonatal patients should be able to access their medical records.

B (Moderate quality)

Audit report1

  1. Clear information about quality indicators is available to parents and former patients in multiple formats (including as written information sheets) (see Data collection & documentation). (23,28)

A (High quality)

Audit report1, parent feedback, patient information sheet

  1. Former neonatal patients and parents are involved in all aspects of projects using quality indicators. (29)

A (High quality)

Parent feedback

For healthcare professionals

  1. All responsible healthcare professionals participate actively in the collection of data relating to quality indicators (as appropriate to their role).

B (Moderate quality)

Guideline

  1. Training on quality indicator related data is attended by all responsible healthcare professionals.

B (Moderate quality)

Training documentation

For neonatal unit

  1. Data is collected relating to quality indicators. (18,19,30,31)

A (High quality)

Audit report1

  1. Data is collected for all neonates (alive and deceased) who receive or have received neonatal care.

B (High quality)

Audit report1

For hospital

  1. Data relating to quality indicators are collected, reported, and audited to benchmark local performance and identify areas for improvement. (18,19,30–32)

A (High quality)

Audit report1

  1. Data relating to quality indicators is contributed to regional and national databases. (18,19,32,33)

A (High quality)

Audit report1, regional network

For health service

  1. Data relating to quality indicators are collated and reviewed annually. (18,19,32)

A (High quality)

Audit report1

  1. Data relating to quality indicators are used for benchmarking at national and international level. (18,19,32)

A (High quality)

Policy statement

  1. Data relating to quality indicators are made available for research. (19,31)

A (High quality)

Guideline

  1. Data relating to quality indicators are available for policy-makers to guide priority setting.

B (High quality)

Policy statement

1The indicator ‘audit report’ can also be defined as a benchmarking report.

Where to go

Further development

Grading of evidence

For parents and family

  • The views of parents and former neonatal patients are included in data collection and analysis. (19,23)

A (High quality)

  • Information sheets to inform parents and former patients about quality indicators are co-designed. (23)

A (High quality)

  • A written, infant-focused lay summary detailing the care a patient has received is given to all families when their infant is discharged home.

B (Moderate quality)

For healthcare professionals

  • Ensure data entry for all variables is completed to high standards. (33–36)

A (High quality)

For neonatal unit

  • Ensure data entry for all variables is completed to high standards. (33–36)

A (High quality)

  • Evaluate the involvement of former neonatal patients and parents in projects related to quality indicators annually. (29)

A (High quality)

For hospital

  • Collect data for all live born neonates.

B (High quality)

  • Extract data, whenever possible, automatically from electronic records to minimise the burden on clinical staff.

B (Moderate quality)

  • Contribute data relating to quality indicators to regional and national databases. (18,30)

A (Moderate quality)

For health service

  • Ensure that data relating to quality indicators covers the entire population.

B (High quality)

  • Collate and review data relating to quality indicators annually. (18,30)

A (Moderate quality)

  • Establish a regular, pan-European audit of the European Standards of Care for Newborn Health.

B (High quality)

  • Compare quality indicators internationally and over time to identify areas to prioritise. (11,13,31)

A (High quality)

  • Compare outcomes beyond Europe (e.g. with NNRD, iNeo, eNewborn and VON) to ensure European neonatal care is among the best globally. (13,19,32,33)

A (High quality)

Getting started

Initial steps

For parents and family

  • Information about quality indicators should be available to parents and former patients orally.

For healthcare professionals

  • Participate in the collection of data relating to quality indicators.

For neonatal unit

  • Collect data relating to quality indicators for all neonates admitted to the neonatal unit.
  • Create local systems to allow data collection and recording.

For hospital

  • Collect, report and audit data relating to quality indicators in order to benchmark local performance and identify areas for improvement.

For health service

  • Collate data relating to quality indicators at regional and national level.

Quality indicators

The European Standards of Care for Newborn Health (ESCNH) quality indicators are variables required for collection and reporting of standardised data. They include background variables required for extensive risk-adjustment, process measures and important neonatal outcomes. The variables have been derived from: a systematic review of existing databases (19), a systematic review of the background characteristics reported in clinical trials (15), international consensus projects (18,30), and the variables included in validated prognostic models. (16)

These indicators should be recorded and reported using standard nomenclature, in line with internationally recognised terminology. However, we acknowledge that for most quality indicators no global consensus exists as to the ‘perfect’ definition or measurement tool (37–39): instead, we propose that capturing individual data components using recognised, validated definitions will provide the flexibility needed to apply different definitions to the data and facilitate meaningful comparisons between databases. It is also essential that suitable denominator data are available to allow meaningful comparison between populations:  for this to occur demographic data relating to all live births, defined according to the AAP definition (40), should be available.

Timing of assessment and data collection is important: as a minimum, it should be in line with standardised assessments, but collecting and recording more frequent data will facilitate the application of multiple definitions. Furthermore, some of the indicators include long-term outcomes. The impact of preterm birth has been clearly shown to extend throughout childhood and into adult life and so, outcomes should be collected into adulthood. To achieve this, collaboration will be needed to ensure that data can be linked between different databases. This work is now possible because data capture and storage technologies are improving and international collaborations are expanding allowing maximal use of available data. Future work to establish a European database will create a tiered structure of responsibility at regional, national and pan-national levels.

IndicatorSuggested data components
Background characteristics
Maternal factors
Antenatal care1. Whether the mother accessed antenatal care (see Birth & transfer)
Antenatal steroid exposure2. Type of steroids
3. Date and time of doses
Barriers to care4. Whether any barriers exist preventing patients from accessing antenatal or neonatal care (e.g. cultural, financial)
Maternal age5. Maternal age in years at birth
Maternal complications of pregnancy6. Reported using standardised terminology according to an accepted definition
Maternal drug use7. Drug use status at first antenatal contact (including prescribed medications and illicit substances)
8. Use of drugs during pregnancy
Maternal education9. Reported using standardised terminology according to an accepted definition
Maternal ethnicity10. Reported using standardised terminology according to an accepted definition
Maternal medical problems11. Reported using an accepted definition
Maternal smoking status12. Smoking status at first antenatal contact
13. Number of cigarettes smoked during pregnancy
Maternal socio-economic status14. Reported using standardised terminology according to an accepted definition  
Labour and delivery
Duration of rupture of membranes15. The time from rupture of membranes to birth
Highest maternal temperature during labour16. The highest recorded maternal temperature during labour
Location of delivery: maternity17. The level of maternity care provided at the location of delivery
Location of care: neonatal18. The level of neonatal care provided at the location of delivery (see Birth & transfer)
Mode of delivery19. Reported using standardised terminology according to an accepted definition
Presentation of foetus at delivery20. Presentation prior to delivery
Infant factors
1 minute Apgar score21. Apgar score one minute after birth
Birth weight22. Birth weight at the time of delivery in grams
Congenital anomaly23. Reported using standardised terminology according to an accepted definition
Gestational age24. Gestational age at birth of neonate in whole weeks and remaining days
Plurality25. Number of foetuses during pregnancy
Sex26. Phenotypic sex of neonate
Process measures
Delayed cord clamping27. Duration from live birth to cord clamping
Family-centred care28. Reported using standardised terminology according to an accepted definition (see Infant- & family-centred developmental care)
Feeding during admission29. Timing of feeding
30. Route of feeding
31. Choice of milk
32. Volume of milk (see Nutrition)
Long-term follow up33. Whether a neonate received appropriate long-term follow up (see Follow-up & continuing care)
Minimising inappropriate separation34. Whether a neonate was separated from their mother without clinical indication (see Infant- & family-centred developmental care)
Nurse staffing ratios35. Whether the neonate received a suitable level of nursing care throughout their stay (see Patient safety & hygiene practice)  
Parenteral nutrition36. Whether the neonate received parenteral nutrition
37. Formulation of parenteral nutrition
38. Route of parenteral nutrition
39. Duration of parenteral nutrition (see Nutrition)
Screening for retinopathy of prematurity40. Whether the neonate underwent screening for retinopathy of prematurity (if indicated)
41. Timing of screening for retinopathy of prematurity (see Medical care & clinical practice)
Surfactant administration42. Doses of surfactant received
43. Formulation of surfactant
44. Route of surfactant
Outcomes
Adverse events1.Any harm from care during the neonatal period reported using standardised terminology according to an accepted definition
Brain injury (on imaging)2. Timing of scans
3. Pathology seen on scan, reported using standardised terminology according to an accepted definition
Chronic lung disease/bronchopulmonary dysplasia4. Mode of respiratory support
5. Duration of respiratory support
General cognitive ability6. Timing of milestones
7. Timing and results of testing, reported using standardised terminology according to an accepted definition
General gross motor ability8. Timing of milestones
9. Timing and results of testing, reported using standardised terminology according to an accepted definition
Hearing impairment or deafness10. Timing of hearing tests
11. Results of hearing tests
Necrotising enterocolitis12. Timing and results of relevant biochemical and radiological testing
13. Clinical features
14. Timing and findings at surgery
Pain15. Reported using standardised terminology according to an accepted definition
Pulmonary function16. Reported using standardised terminology according to an accepted definition
Quality of life17. Reported using standardised terminology according to an accepted definition
Retinopathy of prematurity18. Timing and results of screening
19. Timing and mode of therapy
Sepsis20. Timing and results of relevant biochemical and microbiological testing
21. Clinical features
Social functioning22. Reported using standardised terminology according to an accepted definition
Survival23. Timing of death
24. Survival to discharge home
Visual impairment or blindness25. Timing of vision tests
26. Results of vision tests

  1. McCormick MC, Litt JS, Smith VC, Zupancic JAF. Prematurity: an overview and public health implications. Annu Rev Public Health. 2011;32:367–79.
  2. Anderson P, Doyle LW, Victorian Infant Collaborative Study Group. Neurobehavioral outcomes of school-age children born extremely low birth weight or very preterm in the 1990s. JAMA. 2003 Jun 25;289(24):3264–72.
  3. Moster D, Lie RT, Markestad T. Long-Term Medical and Social Consequences of Preterm Birth. New England Journal of Medicine. 2008 Jul 17;359(3):262–73.
  4. Swamy GK, Ostbye T, Skjaerven R. Association of preterm birth with long-term survival, reproduction, and next-generation preterm birth. JAMA. 2008 Mar 26;299(12):1429–36.
  5. Harrison H. The principles for family-centered neonatal care. Pediatrics. 1993 Nov;92(5):643–50.
  6. van Veenendaal NR, van Kempen AAMW, Franck LS, O’Brien K, Limpens J, van der Lee JH, et al. Hospitalising preterm infants in single family rooms versus open bay units: A systematic review and meta-analysis of impact on parents. EClinicalMedicine. 2020 Jun;23:100388.
  7. Ding X, Zhu L, Zhang R, Wang L, Wang TT, Latour JM. Effects of family-centred care interventions on preterm infants and parents in neonatal intensive care units: A systematic review and meta-analysis of randomised controlled trials. Australian Critical Care. 2019 Jan;32(1):63–75.
  8. AHRQ. Quality Improvement and monitoring at your fingertips [Internet]. Agency for Healthcare Research and Quality. Available from: https://qualityindicators.ahrq.gov/#:~:text=Quality%20Indicators%20(QIs)%20are%20standardized,track%20clinical%20performance%20and%20outcomes
  9. NICE. Standards and Indicators [Internet]. NICE. NICE; [cited 2022 Jun 2]. Available from: https://www.nice.org.uk/standards-and-indicators
  10. Pierrat V, Burguet A, Marchand-Martin L, Cambonie G, Coquelin A, Roze JC, et al. Variations in patterns of care across neonatal units and their associations with outcomes in very preterm infants: the French EPIPAGE-2 cohort study. BMJ Open. 2020 Jun 1;10(6):e035075.
  11. Santhakumaran S, Statnikov Y, Gray D, Battersby C, Ashby D, Modi N, et al. Survival of very preterm infants admitted to neonatal care in England 2008-2014: time trends and regional variation. Arch Dis Child Fetal Neonatal Ed. 2018 May;103(3):F208–15.
  12. Liem TBY, Krediet TG, Fleer A, Egberts TCG, Rademaker CMA. Variation in antibiotic use in neonatal intensive care units in the Netherlands. J Antimicrob Chemother. 2010 Jun;65(6):1270–5.
  13. Helenius K, Sjörs G, Shah PS, Modi N, Reichman B, Morisaki N, et al. Survival in Very Preterm Infants: An International Comparison of 10 National Neonatal Networks. Pediatrics. 2017 Dec;140(6).
  14. Shah PS, Lui K, Sjörs G, Mirea L, Reichman B, Adams M, et al. Neonatal Outcomes of Very Low Birth Weight and Very Preterm Neonates: An International Comparison. J Pediatr. 2016 Oct;177:144-152.e6.
  15. Jawad S, Modi N, Prevost AT, Gale C. A systematic review identifying common data items in neonatal trials and assessing their completeness in routinely recorded United Kingdom national neonatal data. Trials. 2019 Dec 16;20(1):731.
  16. Rysavy MA, Horbar JD, Bell EF, Li L, Greenberg LT, Tyson JE, et al. Assessment of an Updated Neonatal Research Network Extremely Preterm Birth Outcome Model in the Vermont Oxford Network. JAMA Pediatr. 2020 May 1;174(5):e196294.
  17. Elverson CA, Samra HA. Overview of Structure, Process, and Outcome Indicators of Quality in Neonatal Care. Newborn and Infant Nursing Reviews. 2012 Sep 1;12(3):154–61.
  18. Webbe JWH, Duffy JMN, Afonso E, Al-Muzaffar I, Brunton G, Greenough A, et al. Core outcomes in neonatology: development of a core outcome set for neonatal research. Arch Dis Child Fetal Neonatal Ed. 2019 Nov 15;fetalneonatal-2019-317501.
  19. Modi N, Ashby D, Battersby C, Brocklehurst P, Chivers Z, Costeloe K, et al. Developing routinely recorded clinical data from electronic patient records as a national resource to improve neonatal health care: the Medicines for Neonates research programme [Internet]. Southampton (UK): NIHR Journals Library; 2019 [cited 2022 Jun 1]. (Programme Grants for Applied Research). Available from: http://www.ncbi.nlm.nih.gov/books/NBK546540/
  20. Hines D, Modi N, Lee SK, Isayama T, Sjörs G, Gagliardi L, et al. Scoping review shows wide variation in the definitions of bronchopulmonary dysplasia in preterm infants and calls for a consensus. Acta Paediatr. 2017 Mar;106(3):366–74.
  21. Zeitlin J, Manktelow BN, Piedvache A, Cuttini M, Boyle E, van Heijst A, et al. Use of evidence based practices to improve survival without severe morbidity for very preterm infants: results from the EPICE population based cohort. BMJ. 2016 Jul 5;354:i2976.
  22. Soll RF. Evaluating the medical evidence for quality improvement. Clin Perinatol. 2010 Mar;37(1):11–28.
  23. Celenza JF, Zayack D, Buus-Frank ME, Horbar JD. Family Involvement in Quality Improvement: From Bedside Advocate to System Advisor. Clin Perinatol. 2017 Sep;44(3):553–66.
  24. LaRovere JM, Jeffries HE, Sachdeva RC, Rice TB, Wetzel RC, Cooper DS, et al. Databases for assessing the outcomes of the treatment of patients with congenital and paediatric cardiac disease–the perspective of critical care. Cardiol Young. 2008 Dec;18 Suppl 2:130–6.
  25. Gale C, Morris I. The UK National Neonatal Research Database: using neonatal data for research, quality improvement and more. Archives of Disease in Childhood – Education and Practice. 2016 Aug 1;101(4):216–8.
  26. Horbar JD, Plsek PE, Leahy K, NIC/Q 2000. NIC/Q 2000: establishing habits for improvement in neonatal intensive care units. Pediatrics. 2003 Apr;111(4 Pt 2):e397-410.
  27. Leavy P, Violeta Prina M, Martínez Cáceres MJ, Bauer G. Social representations of premature birth from the perspective of individuals born preterm in the 1990s. Arch Argent Pediatr. 2015 Jan;113(1):6–11.
  28. Harrison W, Goodman D. Epidemiologic Trends in Neonatal Intensive Care, 2007-2012. JAMA Pediatr. 2015 Sep;169(9):855–62.
  29. INVOLVE. Public involvement in research: values and principles framework [Internet]. INVOLVE: Eastleigh; 2015 [cited 2022 Jun 2]. Available from: https://www.invo.org.uk/wp-content/uploads/2017/08/Values-Principles-framework-Jan2016.pdf
  30. ICHOM. Set of Patient-Centered Outcome Measures for Preterm and Hospitalised Newborn Health [Internet]. 2020 [cited 2022 Jun 8]. Available from: https://connect.ichom.org/patient-centered-outcome-measures/preterm-and-hospitalized-newborn-health/
  31. Oliver S, Uhm S, Duley L, Crowe S, David AL, James CP, et al. Top research priorities for preterm birth: results of a prioritisation partnership between people affected by preterm birth and healthcare professionals. BMC Pregnancy Childbirth. 2019 Dec;19(1):528.
  32. Soll RF, McGuire W. Evidence-Based Practice: Improving the Quality of Perinatal Care. Neonatology. 2019;116(3):193–8.
  33. Shah PS, Seidlitz W, Chan P, Yeh S, Musrap N, Lee SK, et al. Internal Audit of the Canadian Neonatal Network Data Collection System. Am J Perinatol. 2017 Oct;34(12):1241–9.
  34. Arts DGT, De Keizer NF, Scheffer GJ. Defining and improving data quality in medical registries: a literature review, case study, and generic framework. J Am Med Inform Assoc. 2002 Dec;9(6):600–11.
  35. Battersby C, Statnikov Y, Santhakumaran S, Gray D, Modi N, Costeloe K, et al. The United Kingdom National Neonatal Research Database: A validation study. PLoS One. 2018;13(8):e0201815.
  36. Horbar JD, Leahy KA. An assessment of data quality in the Vermont-Oxford Trials Network database. Control Clin Trials. 1995 Feb;16(1):51–61.
  37. Ioannidis JPA, Horbar JD, Ovelman CM, Brosseau Y, Thorlund K, Buus-Frank ME, et al. Completeness of main outcomes across randomized trials in entire discipline: survey of chronic lung disease outcomes in preterm infants. BMJ. 2015 Jan 26;350:h72.
  38. Webbe JWH, Ali S, Sakonidou S, Webbe T, Duffy JMN, Brunton G, et al. Inconsistent outcome reporting in large neonatal trials: a systematic review. Arch Dis Child Fetal Neonatal Ed. 2020 Jan;105(1):69–75.
  39. Janvier A, Farlow B, Baardsnes J, Pearce R, Barrington KJ. Measuring and communicating meaningful outcomes in neonatology: A family perspective. Seminars in Perinatology [Internet]. 2016 Oct [cited 2016 Nov 11]; Available from: http://linkinghub.elsevier.com/retrieve/pii/S0146000516300817
  40. Barfield WD, COMMITTEE ON FETUS AND NEWBORN. Standard Terminology for Fetal, Infant, and Perinatal Deaths. Pediatrics. 2016 May;137(5):e20160551.

September 2022 / 1st edition / next revision: 2025

Recommended citation

EFCNI, Webbe J, Lack N et al., European Standards of Care for Newborn Health: Quality indicators. 2022.

Previous standard

Please click on the icons below to select one of the standard topics:

Birth & transfer
Ethical decisions
NICU design
Care procedures
Follow-up & continuing care
Nutrition
Data collection & documentation
Infant- & family-centred developmental care
Patient safety & hygiene practice
Education & training
Medical care & clinical practice