Although great progress has been made over the past two decades in decreasing maternal and newborn mortality, approximately 800 women and 6400 newborns still die each day around the time of childbirth [1,2]. Most of these deaths occur in low-resource settings and are preventable with high-quality, timely care. A key strategy to reduce maternal and newborn mortality is to improve the availability, accessibility, and use of emergency obstetric and newborn care (EmONC) services. However, this strategy will not result in expected maternal and neonatal mortality reductions if these services are of poor quality. Estimates suggest that each year, approximately one million newborn deaths and half of all maternal deaths could be averted if health systems were strengthened to deliver high-quality care . Improvements in the quality of maternal health services could also reduce the estimated 1.9 million stillbirths that occur every year .
Numerous reports and initiatives in recent years have highlighted the centrality of high-quality services for ensuring positive obstetric outcomes [3–5]. Regular monitoring of the quality of EmONC services is crucial for improving them and holding governments to account for providing lifesaving care to all women and newborns in need. The Monitoring Emergency Obstetric Care: A handbook (EmONC handbook) provides comprehensive guidance on how countries can identify service gaps, monitor program implementation, and measure progress . However, the current version contains only three indicators that serve as proxies for quality of care: the intrapartum stillbirth and early neonatal death rate and the direct obstetric case fatality rate. The EmONC handbook is undergoing an extensive revision to reflect advancements in the evidence base on effective interventions, changes in epidemiological patterns, and new approaches to assessing health system functionality (Box 1). The revised handbook will also include more focus on newborns, the maternal-infant dyad, and quality of care.
Box 1. Summary of the Revisioning EmONC project
The Re-Visioning Emergency Obstetric and Newborn Care (EmONC) project is led by a steering committee coordinated by the Averting Maternal Death and Disability (AMDD) program at Columbia University Mailman School of Public Health and includes the London School of Hygiene and Tropical Medicine, United Nations Children’s Fund, United Nations Population Fund, and the World Health Organization. A broad group of global stakeholder organisations and individuals form the wider technical working group that provides substantive and strategic input throughout the process. The substantive work of review and revision is being conducted through four workstreams and a set of country studies. The entire project is framed and implemented using principles of human-centred design to ensure that the revised EmONC framework meets the needs and real-world conditions at the national and sub-national policy levels and at the frontlines of health systems in low- and middle-income countries at different stages of the obstetric transition. The four workstreams cover the topics of signal functions, levels of care, quality of care, and reviews of country experiences using the existing EmONC indicators and targets.
As part of the Revisioning EmONC project, we conducted a scoping review of the quality of care indicators for emergency obstetric and newborn care with a focus on obstetric emergencies related to direct causes of maternal and neonatal death. This review was followed by a consultative process with technical and programming experts in maternal and newborn health and a prioritisation exercise to select a short list of quality of care indicators for potential inclusion in the EmONC handbook.
We conducted a scoping review, mapping, and prioritisation exercise to identify, assess, and select a short list of quality of care indicators on facility-based emergency obstetric and newborn care. A comprehensive description of the methodology is available in the Online Supplementary Document.
Our study was guided by the World Health Organization Standards for Improving Quality of Maternal and Newborn Care in Health Facilities (WHO standards document). Figure 1 presents the framework in the WHO standards document showing how structural and process elements impact facility and individual-level outcomes . Accompanying this framework are a set of standards, quality statements, and associated quality measures.
Figure 1. World Health Organization Quality of Care Framework for Maternal and Newborn Health . Licensed under Creative Commons Attribution IGO License.
We focused on the process component of the framework, consisting of provision of care and experience of care domains. The structure component was addressed through other workstreams of the Revisioning EmONC project that focused on the organisation of EmONC services. Therefore, all indicators extracted through the scoping review related to structure (e.g. all input types, including health information systems, policies, financing, medicines and equipment, and human resources indicators) were considered beyond the scope of our study and excluded.
We excluded all indicators related to the outcome component of the framework for two reasons. Although monitoring health outcomes (e.g. maternal mortality, stillbirths, neonatal mortality) is important for assessing the quality of care provided, quality of care is only one of many factors that contribute to individual and facility-based outcomes. A separate consensus-based process is planned for selecting a limited number of outcome measures for inclusion in the revised EmONC handbook.
We assessed the literature post-publication of the WHO standards document in 2016  to identify any measurement advancements or topical areas missing from it that should be considered for the short indicator list. We identified resources, including peer-reviewed articles and grey literature, through three mechanisms: a review of systematic and scoping reviews on the provision and experience of care, a systematic review of recent primary literature on maternal and newborn experience of care, and resources provided by experts in the Revisioning EmONC project steering committee and quality of care working group. Table 1 describes the inclusion and exclusion criteria used to select resources for our study.
Table 1. Inclusion and exclusion criteria for resources identified through the three mechanisms of the scoping review
Review of reviews for provision and experience of care
We conducted a review of published systematic and scoping reviews on the provision and experience of maternal and newborn care published between 1 January 2016 and 30 March 2021. The start date of the search corresponds with the publication date of the WHO standards document . The WHO standards document was developed based on an extensive literature review and consensus-based process, so we opted to build on this work rather than duplicate it. The WHO standards document was included as a key resource in our study and guided our overall approach to the systematic review and indicator prioritisation process. We searched PubMed, CINAHL and EMBASE. The search strategy is available in Section 2A in the Online Supplementary Document.
Systematic review for experience of care
Because measurement of experience of care is a newer area of research than provision of care, we conducted a systematic review of primary literature on maternal and newborn experience of care published between 1 January 2019 and 17 May 2021. Our start date corresponds with the end date of the search from the most recent systematic review on experience of care published by Larson et al. . We modelled our search strategy on the Larson strategy but modified it to include newborn search terms and had it refined by a Columbia University librarian. We also reviewed all instruments/tools developed to measure maternal experience of care published after 31 October 2017, corresponding with the end date of the search period for the Afulani rapid review on the same topic . We searched PubMed, CINAHL, and EMBASE databases. The full search strategy is available in Section 2B in the Online Supplementary Document. Both the Afulani  and Larson  papers were included in the review of reviews.
Resources from the grey literature
We solicited resources from the Revisioning EmONC project quality of care workstream and steering committee members. The members of these two groups represent a diverse array of technical and programming experts in the field of maternal and newborn health (Table S16 and Table S17 in the Online Supplementary Document). Grey literature sources recommended by these experts included resources from World Health Organization (WHO), Strategies Towards Ending Preventable Maternal Mortality (EPMM), Every Newborn Action Plan (ENAP), the Quality, Equity, Dignity (QED) Network Monitoring Framework, and recommendations submitted by the maternal and newborn health communities in 2021 as part of the revision process of the Service Provision Assessment (SPA) tool [4,5,10,11].
Indicator extraction and mapping
We extracted indicators from included resources and organised them into an Excel file using the WHO framework categories of structure, process, and outcome. Indicators grouped into the outcome and structure categories, including all indicators labelled as input or outcome type in the WHO standards document, were excluded as described above. Indicators specific to the provision of antenatal care (ANC) were also excluded as out of scope because our study focused on the intrapartum period. A separate Revisioning EmONC project workstream is developing routine signal functions for maternal and newborn care that cover the antenatal and intrapartum periods.
As a second step, the compiled process indicators, encompassing provision of care and experience of care indicators, were organised by relevant standards and associated quality statements in the WHO standards document (Table S8 and S9 in the Online Supplementary Document). New categories were created for process indicators that could not be categorised under any quality standard or statement. The purpose of this step was to group similar indicators, enable the removal of duplicates, and facilitate the identification of gap areas (Section 5 in the Online Supplementary Document).
Out of the eight WHO standards, standards one and three concern the provision of care (Table S3 in the Online Supplementary Document). Of the 16 associated quality statements, two – quality statement 1.1b “mothers and newborns receive routine postnatal care” and quality statement 1.8 “all women and newborns receive care that includes standard precautions for preventing hospital-acquired infections” – were determined to be out of scope (1.1b was considered relevant for the Revisioning EmONC project workstream on routine signal functions, and 1.8 for the workstreams on health system functionality). Indicators grouped under these two statements were excluded. Standards four, five and six are related to experience of care (Table S3 in the Online Supplementary Document). All seven associated quality statements were considered within the study scope.
The third step involved reviewing the grouped provision and experience of care indicators and excluding exact duplicates and any remaining indicators deemed out of scope (e.g. indicators on home or community-based care, or input, outcome, ANC-related indicators that had not been removed in earlier steps). For conceptually similar indicators, we selected the indicator that most comprehensively captured the underlying construct of interest. We selected one indicator for each construct where possible, but in some cases, we kept multiple indicators that captured substantively different construct elements (Table 2).
Table 2. Example of a selected indicator out of a set of duplicate or overlapping indicators
SPA – Service Provision Assessment Survey; WHO – World Health Organization
A consultative process with the Revisioning EmONC project quality of care workstream and steering committee members was conducted to select a short list of indicators from the extracted provision and experience of care indicators. To ensure consistency with other newborn measurement efforts, experts from ENAP were also consulted. A set of criteria was developed and applied to help with the identification of the short list. Table S13 in the Online Supplementary Document includes information about the type, geographic location, and year published for the reference associated with each prioritised indicator. To develop the indicator selection criteria, we first compiled criteria used by other similar initiatives and research efforts (Section 4 in the Online Supplementary Document). We then prepared an abbreviated list of criteria that we considered most salient – relevant/important, actionable, feasible, evidence-based – and presented it to the quality of care workstream and steering committee members. A consensus was reached to limit the criteria to relevant/important and feasible because all the extracted indicators were evidence-based, and actionability was considered redundant with feasibility (Table 3). Agreement was reached that any indicators with evidence of invalidity would be excluded. Although the same indicator selection criteria were applied, separate processes were used to identify the priority provision and experience of care indicators because of differences in how these two topics are commonly measured. Experience of care is typically measured using a scale and the construction of a score based on interviewee responses. Provision of care indicators, in contrast, capture objective information on activities and outputs and are often measured through observations and routine health information systems (e.g. facility registers, patient care records).
Table 3. Indicator selection criteria definitions
*Relevance to emergency obstetric care was not an emphasis for selecting experience of care indicators but was central for selecting the provision of care indicators.
Provision of care
Based on guidance from the quality of care workstream, the study team selected proxy indicators for each WHO quality statement related to the provision of care. Where appropriate, we combined unique elements from different indicators to develop proxy indicators (Table 4). Consensus was reached with the workstream members to select provision of care indicators relevant to complication management and that capture appropriate care, as opposed to harmful practices, so they would all have the same measurement directionality. To assess the feasibility of measuring the prioritised provision of care indicators, we examined available metadata that includes potential data sources for them in the QED catalogue  and in the Harmonized Health Facility Assessment tool . Specifically, we reviewed information on data sources listed in the metadata and the likelihood of these sources being regularly available in low-resource settings. We also reviewed measurement guidance submitted by the maternal and newborn health communities to the SPA team during the 2020-2022 SPA revision process (correspondence from maternal newborn health working group).
Table 4. Example of developing a proxy indicator for combining elements of indicators that capture parts of the underlying construct of interest
WHO – World Health Organization
Experience of care
After grouping the experience of care indicators by the WHO standards framework, we further organised them according to the Person-Centered Maternity Care (PCMC) scale , including the additional items in the USA version . The PCMC scale has been validated for use in low- and middle-income countries [24–28], and its three sub-scales of communication and autonomy, dignity and respect, and supportive care are consistent with the WHO standards. We matched and compared the extracted experience of care indicators to items in the PCMC scale to assess whether the PCMC items needed updating. For indicators that were not already captured by the PCMC scale, we selected those that met the indicator selection criteria as optional items. Relevance to emergency obstetric care was not emphasised for selecting experience of care indicators, given inconsistent evidence of associations between complications during childbirth and experience of care [29–33]. Further, the study team agreed that every woman, regardless of her complication status, should receive respectful and responsive care.
We identified a total of 1037 resources; 90 were included in the study (Figure 2). We extracted 3023 indicators from the included resources: 605 provision of care, 1527 experience of care, 564 structure/readiness, 192 ANC, and 135 outcome indicators (Figure 3). After excluding the structure, outcome, ANC, and all other out of scope indicators and removing duplicates, the number was reduced to 168 provision of care and 264 experience of care indicators. We were able to group almost all provision of care indicators (92.3%) and experience of care indicators (89.4%) under a WHO standard and associated quality statement (Table S5 and Table S6 in the Online Supplementary Document).
Figure 2. Flow diagram for the scoping review.
Figure 3. Flow diagram for extracted indicators.
After identifying, mapping, and applying the selection criteria to the provision of care indicators, we prioritised 20 for consideration for the revised EmONC handbook (Table 5 and Table S10 in the Online Supplementary Document). Of these, 19 are consistent with indicators in the WHO standards document, although we made minor modifications to 14 of them based on findings from the scoping review (see Table S10 in the Online Supplementary Document for notes on any indicator wording deviations from indicators defined in the WHO standards based on updated evidence). We selected one indicator not included in the WHO standards document related to appropriate prophylaxis against disease transmission for women with human immunodeficiency virus (HIV), which will have the most relevance for high prevalence contexts.
Table 5. Prioritised provision of care of indicators (total of 20 unique indicators)*
ACS – antenatal corticosteroids, KMC – kangaroo mother care, HIV – human immunodeficiency virus, WHO – World Health Organization
*We considered core references as the World Health Organization’s Standards for Improving Quality of Maternal and Newborn Care in Health Facilities; Quality, Equity, Dignity catalogue; and Every Newborn Action Plan metric recommendations. All indicators should be measured based on a specified time period.
†Indicators identified as collectable through routine information systems.
‡Kangaroo mother care is defined as care of preterm infants carried skin-to-skin with the mother. Key features include continuous skin-to-skin contact with the mother, exclusive breastfeeding, and early discharge from the hospital in the kangaroo position with frequent home visits by health workers .
§Duplicate indicator that was categorised under two World Health Organization’s Standards.
Consensus was reached through our consultation processes that experience of care is best measured using a composite score that captures different aspects of a person’s experience. We, therefore, selected the PCMC scale for inclusion in the revised EmONC handbook because it is the most comprehensive scale for measuring women’s experiences during childbirth. Each question on the scale (Table 6 has response options ranging from zero to three that have been psychometrically tested for generating a composite score. Adding up the responses to the individual items and rescaling the totals (using procedures that would be described in the revised handbook if the PCMC scale is added) will generate a score ranging from zero to 100, where higher scores represent more positive experiences. Although the USA version of the PCMC scale has more items, we selected the 30-item version because it has been validated in settings where the EmONC handbook will most likely be used. Where the 30-item version is not practical, we recommend the shorter 13-item version, which has also been psychometrically validated . The 13-item version, however, excludes the items on birth companionship and verbal and physical abuse, which are critical issues that can be remedied through health system reforms. Thus, we recommend that these three items be included in EmONC assessments or other health facility surveys as additional items if the shorter PCMC scale is used.
Table 6. Person-Centered Maternity Care scale
*Items from the 30-item Person-Centered Maternity Care scale are measured on a 4-point scale from zero to three (e.g. 0 – no, never; 1 – yes, a few times; 2 – yes, most of the time; 3 – yes, all the time).
We selected an additional 29 experience of care items that cover areas not captured by the PCMC scale, including newborn experience of care, continuity of care, physical accessibility, issues around payment for services (or lack thereof), family-centred care, and bereavement care (Table 7). Many newborn items and items concerning mother-baby separation require additional research to ensure valid measurement.
Table 7. Additional 29 experience of care items, organised by mother and newborn
SPA – Service Provision Assessment survey, WHO – World Health Organization
A review of the recommendations submitted for the 2020-2022 SPA revision process (Tables S14 and S15 in the Online Supplementary Document) showed that two of our selected provision of care indicators (related to essential newborn care and neonatal resuscitation – indicators two and ten in Table 5) were recommended for inclusion in the SPA as core indicators. Indicators for vital sign assessment for women, fetal heart measurement, administration of uterotonic drugs, and prevention of maternal-to-child transmission were recommended for inclusion in an optional module. Regarding experience of care, the 13-item PCMC scale was recommended for retention. After consulting the QED catalogue, we determined that potentially, 12 out of our recommended 20 provision of care indicators can currently be collected through routine health information systems (Table S14 in the Online Supplementary Document). The other eight indicators can be collected through surveys and special studies.
After organising the compiled provision and experience of care indicators by the relevant WHO standards and associated quality statements, provider experience and patient safety emerged as gap areas. Although components of patient safety and provider experience are addressed in the WHO standards document through quality statements 1.9, 5.2, and 4.2, respectively, our review identified few indicators on these topics. We conducted rapid literature searches for both patient safety and provider experience because we recognised that our scoping review search criteria may have resulted in the inadvertent exclusion of indicators on these topics (Section 5 in the Online Supplementary Document). The literature searches yielded no additional patient safety measures and few health provider experience measures beyond burnout. High levels of provider burnout during the coronavirus disease 2019 (COVID-19) pandemic have elevated the issue of provider experience [43–46], yet consensus has not been reached on a comprehensive set of provider experience measures.
From this review, we selected 20 provision of care indicators for potential inclusion in the EmONC handbook and for use in other monitoring efforts. For experience of care, we recommend that the revised handbook includes a score based on the PCMC scale. We also recommend an additional menu of 29 items to guide more in-depth experience of care assessments. Our study was grounded in the WHO standards document  and involved consultations with maternal and newborn experts to ensure our recommendations were aligned with existing guidelines and the latest measurement efforts.
One goal of our study was to assess the evidence base to identify any measurement improvements or emergence of new priority areas since the WHO standards document was published in 2016. We found that almost all indicators identified from our scoping review could be categorised under an existing WHO standard and associated quality statement, and 19 of the 20 prioritised provision of care indicators are consistent with those in the WHO standards document. Thus, our study reaffirmed the continued value of the WHO standards document for guiding maternal and newborn quality of care measurement and monitoring efforts.
Around half of the recommended provision of care indicators can potentially be collected through routine health information systems, and the others through available health facility survey platforms or special studies. For the experience of care, we selected an indicator based on an existing scale that has been validated for use in low- and middle-income settings where most EmONC assessments are likely to occur [22,24,26,28,47–50] but also recommended adding additional indicators where possible. These findings suggest that our recommended indicators can be regularly collected in most locations, although implementation research in resource-constrained settings would further confirm measurement feasibility.
Provider experience and patient safety emerged as gap areas from our scoping review and consultation processes. Future research efforts should systematically assess available validated tools for measuring provider experience to inform the development of provider experience indicators. Similarly, more work is needed to develop a set of consensus-based indicators for patient safety that capture the complex issues related to harmful practices such as negligence, overmedicalisation (inappropriate interventions such as unnecessary episiotomy or non-medically indicated caesarean sections), antibiotic resistance, and lack of training on standards of care .
Our scoping review approach, which included three mechanisms to identify materials, had some limitations. We only included a select set of databases and English-language articles, which may have resulted in us missing key articles available through other databases and in different languages. Although we modified the Larson search strategy to include newborn as well as maternal tools for the systematic review component, we may have missed newborn tools published before the Larson systematic review since we used the end date of the Larson search as our start date. The search terms used for the review of systematic and scoping reviews constrained our results to only articles that included both maternal and newborn indicators rather than articles that included maternal, newborn, or both maternal and newborn indicators (Section 2A in the Online Supplementary Document). We opted not to redo the search because we agreed that the large volume of resources already identified through the three mechanisms was sufficient for our study.
Because our search terms were restricted to maternal and newborn care, indicators around the quality of abortion care, often categorised separately as part of sexual and reproductive health care, were not captured in our scoping review . Additional consultations for the Revisioning EmONC project are being conducted on the quality of abortion care so that this topic is adequately reflected in the revised handbook.
We acknowledge that some of our recommended indicators require additional measurement work, including the indicators on referral, care for small and sick newborns, maternal and newborn separation, and on the newborn experience of care. We recommend that our short list is regularly reviewed and revised to reflect improvements in measurement on these topic areas. Our recommendations are restricted to the provision of care for women and newborns with complications in the intrapartum period and experience of care. This focus is consistent with the recommendations in the Lancet Commission on High Quality Health Systems to include competent care and user experience in assessing the quality of care . Fully evaluating the availability, accessibility, and quality of emergency obstetric and newborn care requires also examining health system functionality and health outcomes. For health care workers to be able to provide high-quality emergency obstetric and newborn care, they need to be embedded in an enabling environment. Similarly, improvements in health outcomes and women’s and newborns’ experiences of care start with women being able to reach health facilities in a timely manner .
The findings from this study are inputs to the broader Revisioning EmONC project, which addresses all three components of structure, process, and outcomes as described in the WHO standards document, as well as the continuity of care during pregnancy, labour, delivery and the immediate postpartum period. Health outcome indicators are important for monitoring quality of care, and, as noted, a separate process is planned to select a limited number of outcome measures for inclusion in the revised EmONC handbook. Of note, our recommended indicators are intended for program monitoring. The goal is a short set of indicators that will serve as red flags to prompt action, including additional data collection, to understand the source of poor care better so it can be addressed. Thus, the focus is on parsimony rather than comprehensiveness. The full set of indicators and signal functions selected for the Revisioning EmONC project is expected to undergo wider vetting with the maternal and newborn communities and to be tested using a human-centred design approach in selected countries. Our recommended indicators will be included in these processes for further validation.
We identified a short set of evidence-based provision and experience of care indicators that are consistent with global standards, can be measured through existing tools and platforms, and can be used for monitoring and planning purposes in low-resource settings. Their potential inclusion in the revised EmONC handbook and in other monitoring activities could increase the use of data for decision-making on programs and resource allocation and help hold decision-makers to account for improving the quality of emergency obstetric and newborn care and, ultimately, women’s and newborns’ lives.
Disclaimer: We acknowledge that not all people who are pregnant or giving birth identify as women. The use of woman/women in this document is to be consistent with the terminology used in previous work but includes gender-expansive people.
Data availability: No primary data collection was involved due to the project being a scoping review. All outputs from the scoping review, mapping and prioritisation exercise are provided in the manuscript and the Online Supplementary Document.