Impact factor (WEB OF SCIENCE - Clarivate)

2 year: 7.664 | 5 year: 7.127

Articles

Accuracy of non-physician health workers in respiratory rate measurement to identify paediatric pneumonia in low- and middle-income countries: A systematic review and meta-analysis

Ahad M Khan1,2, Anna O’Donald2, Ting Shi2, Salahuddin Ahmed1,2, Eric D McCollum3, Carina King4, Abdullah H Baqui5, Steve Cunningham2, Harry Campbell2; RESPIRE Collaboration

1 Projahnmo Research Foundation, Dhaka, Bangladesh
2 The University of Edinburgh, Edinburgh, UK
3 Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
4 Karolinska Institutet, Stockholm, Sweden
5 Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA

Share:

Facebook
Twitter
LinkedIn
Abstract

Background

Non-physician health workers play an important role in identifying and treating pneumonia in children in low- and middle-income countries (LMICs). In this systematic review, we summarized the evidence on whether health workers can accurately measure respiratory rate (RR) and identify fast breathing to diagnose pneumonia in children under five years of age.

Methods

We searched MEDLINE, EMBASE, Web of Science, and Scopus from January 1990 to August 2020 without any language restrictions. Reference lists of included studies were also screened for additional records. Studies evaluating the performance of health workers in measuring RR and/or identifying fast breathing compared to a reference standard were included. The methodological quality of the included studies was assessed using the QUADAS-2 tool. A meta-analysis was conducted to report pooled estimates of sensitivity and specificity. Hierarchical summary receiver operating characteristic curve (HSROC) models were fitted, and subgroup and sensitivity analyses were performed to examine the effects of study variables.

Results

We included 16 studies, eight of which reported the agreement in RR count between health workers and a reference standard. The median agreements were 39%, 47%, and 67% within ±2, ±3, and ±5 breaths per minute, respectively. Among the 16 included studies, we identified 15 studies that reported the accuracy of a health worker classifying breathing into either fast or normal categories compared to a reference standard. The median sensitivity, specificity, accuracy, and kappa value were 77%, 86%, 81%, and 0.75, respectively. Seven studies reporting the accuracy of identifying fast breathing were included in the meta-analysis. The pooled estimates of sensitivity and specificity were 78% (95% CI = 72-82) and 86% (95% CI = 78-91), respectively.

Conclusions

Despite the problematic nature of reference standards and their variability across studies, our review suggests that the health worker performance in accurately counting RR is relatively poor. However, their performance shows reasonable specificity and moderate sensitivity in identifying fast breathing. Improving the detection of fast breathing in children with suspected pneumonia among health workers is an important child health programme objective and should be given appropriate priority.

Print Friendly, PDF & Email

Pneumonia is one of the leading causes of mortality in children aged below five years worldwide [1]. The overall global incidence of pneumonia is 0.22 (IQR = 0.11-0.51) episodes per child-year [2]. Approximately 68 million pneumonia episodes and 650 000 deaths due to pneumonia were estimated to have occurred in 2016 [3]. There is a notable discrepancy between the incidence of pneumonia in high-income countries, in comparison to low- and middle-income countries (LMICs) [2]. Pneumonia presents a substantial burden on health services and is a major cause of hospital admissions in children [4]. In LMICs, the recognition of pneumonia and care-seeking behaviour is generally poor [5]. An important factor limiting the effective diagnosis and treatment of pneumonia in LMICs is a lower doctor-to-population ratio [6]. Moreover, access to doctors and hospitals is usually more difficult [7,8], and the cost of treatment is often prohibitive for caregivers [9]. Therefore, a significant proportion of pneumonia is diagnosed and treated outside hospitals by non-physician health workers [10]. During household visits or community health centre patient encounters, these health workers apply pragmatic case management algorithms to make decisions on diagnosis, treatment, and referral of children suspected to have pneumonia [11,12]. Community-based management of pneumonia by health workers has had a substantial effect on reducing child mortality [13].

According to the World Health Organisation (WHO) guidelines, pneumonia diagnosis in children is primarily based on increased respiratory rate (RR). The number of breaths is manually counted for 60 seconds using an acute respiratory illness (ARI) timer or a watch and is then classified as fast or normal breathing according to the child’s respective age group [14,15]. The measurement of RR is challenging, however, and is frequently miscounted, often due to the child’s movement or shallow, irregular breathing. Counting of RR is often not done routinely by health workers as it is difficult, time-consuming, and depends on the availability of timers. Moreover, a clear definition of a breath is not available within WHO guidelines [16]. This has implications for the quality of clinical practice, as it can lead to under-diagnosis, misdiagnosis, and insufficient or inappropriate treatment [1719].

The diagnosis of pneumonia in LMICs largely depends on health workers’ ability to count RR and classify fast and normal breathing accurately. Despite existing literature evaluating the ability of health workers to count and classify fast breathing pneumonia, to our knowledge the evidence has not yet been systematically collated. As the existing literature involves studies with small numbers, a systematic review would allow more robust evidence to inform clinical practice and policy implementation. In this review, we summarized the evidence on whether health workers can accurately measure RR and identify fast breathing in children under five years of age.

METHODS

We conducted this systematic review following the methodology described in the Handbook for Diagnostic Test Accuracy (DTA) Reviews of Cochrane [20]. We used the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) 2020 [21] and the Preferred Reporting Items for Systematic Reviews and Meta-analyses of Diagnostic Test Accuracy Studies (PRISMA-DTA) [22] in reporting our findings. The review protocol was registered with the PROSPERO database (registration number CRD42020211127).

Population, index test, reference standard, and target condition

The target participants were children under five years of age who had their RR assessed in the community or when attending a health facility. The index test was RR counting and/or fast breathing assessment done manually by non-physician health workers. RR counting and/or fast breathing identification by a human expert or an automated device were considered reference standards. The experts were experienced paediatricians, clinicians, or other persons who were trained in clinical algorithms of pneumonia in children.

Search strategy

We developed a search strategy using a combination of topic-related medical subject headings (MeSH) and keywords. The key concepts were “pneumonia” AND “respiratory rate” AND “accuracy” AND “children under five years of age”. We comprehensively searched MEDLINE (via Ovid), EMBASE (via Ovid), Web of Science, and Scopus databases. The detailed search strategy used for each database is reported in Table S1 in the Online Supplementary Document. Included studies were published between January 1st, 1990, to August 9th, 2020. We sought to identify other potentially relevant studies by subjecting all included studies to a forward citation search and examining their reference lists. There were no restrictions on language in the searches. An expert librarian verified the search strategy.

Study eligibility

Studies were included if they met the following criteria:

  1. Measurement of RR and/or identification of fast breathing were done manually by non-physician health workers.
  2. A reference standard was used to evaluate the accuracy of RR and/or identifying cases with fast breathing.
  3. Age of the participants was less than five years.
  4. Conducted in LMICs. The list of LMICs was obtained from the UN Statistics Division (Table S2 in the Online Supplementary Document) [23].

Studies were excluded by the following criteria:

  1. Non-human animal subjects, or mechanically ventilated subjects.
  2. Information on reference standard was lacking.
  3. Health workers used a device other than an ARI timer or a watch to measure RR.
  4. Health workers counted RR from videotaped subjects.
  5. Disaggregation of data on RR or fast breathing was not possible.
  6. Disaggregation of data in under-five children was not possible.

Study selection and data extraction

We downloaded the literature search results from different databases into the EndNote X9 reference management software. After excluding duplicates, two review authors (AMK and AOD) independently examined the titles and/or abstracts of the identified studies and excluded irrelevant studies. They then independently analysed the full texts of potentially relevant articles according to the pre-specified eligibility criteria. Disagreements were resolved through a discussion between the two reviewers.

The review authors extracted data from studies using a structured checklist (Table S3 in the Online Supplementary Document) and entered those into the Microsoft Excel spreadsheet. Any disagreements were resolved through discussion.

Quality assessment

Both reviewers used the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool [24] to assess the quality of the included studies. Four domains (i.e., patient selection, index test, reference standard, and flow and timing of the participants) were assessed for risk of bias. There are some core signalling questions under each domain. The answer to each signalling question was “yes”, “no” or “unclear”, and the risk of bias was considered as “low”, “high” or “unclear”. The “unclear” category was used only when insufficient data were reported. Individual domain was considered “low risk” if the answers to all signalling questions were “yes”; “high risk” if at least one answer was “no” in any combination; and “unclear” where at least one answer was “unclear”, the other was “yes” and where no answer was “no” in any combination. Both review authors checked the risk of bias independently and any disagreement was settled through discussion. We entered these data into Review Manager (version 5.3) to create the figure used in this paper.

Data synthesis and analysis

For the studies reporting agreement of RR counts between health workers and the reference standard, we presented the percentage of agreement and calculated median agreement with the range of values. For the studies reporting accuracy of classifying fast and normal breathing compared to a reference standard, we presented sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, and kappa value of individual study if data were available, and we calculated median values with ranges.

We performed a meta-analysis with those studies reporting classification of fast and normal breathing where true positive (TP), false positive (FP), false negative (FN), and true negative (TN) data could be retrieved. We estimated sensitivity and specificity with 95% confidence intervals (CI) for each study and presented those in paired forest plots to inspect the study variance. We fitted hierarchical summary receiver operating curve (HSROC) models [25] using user-written modules (metandi, midas) [26,27] in the Stata statistical software (version 16.0) to assess accuracy of fast breathing identification. Heterogeneity among studies was evaluated visually, from coupled forest plot, and statistically, using the I-square [28]. We used univariate meta-regression to perform subgroup analyses. The parameters for subgroup analysis were as follows: child age, study setting, fast breathing prevalence in the sample, diagnosing health worker, and timing of RR measurement by index test and reference standard.

We performed a sensitivity analysis restricted to studies where fast breathing was defined using WHO RR thresholds. We did not conduct tests for reporting bias due to the ambiguity of the factors of publication bias for diagnostic accuracy studies and the inadequacy of tests for identifying asymmetry of a funnel plot [29].

RESULTS

Result of the search

The review process is summarised in Figure 1 using the PRISMA flowchart [21]. 17 reports with 16 studies met all the criteria for inclusion in this review [1719,3043]. Two reports used the same data set but were both included, as they measured different outcomes [35,36]. Only seven studies reporting accuracy of classifying fast and normal breathing presented TP, FP, TN, and FN data, and those studies were included in the meta-analysis [17,18,31,33,35,42,43]. The list of excluded reports with exclusion reasons is available in Table S4 in the Online Supplementary Document.

Figure 1. PRISMA flow diagram.

Characteristics of the included studies

Table 1 summarises the characteristics of the included studies. Most studies were conducted in Africa [1719,30,32,33,3543], two in Asia [30,31] and one in Oceania [34]. 10 studies were based at a health facility [18,30,33,34,3742], while five were in the community [19,31,32,35,36,43], and one was in a training centre [17]. Studies differed in assessed population, with nine studies assessing children aged 2-59 months [19,32,35,36,3843], two studies assessed only young infants [31,33], and the remaining studies assessing children varying from 0 to 59 months of age [17,18,30,34,37]. The clinical encounters recorded per study ranged from 34 to 564. The majority of the studies evaluated community-based health workers [1719,3032,3539,43], while three studies evaluated facility-based health workers [4042] and one study evaluated both [34]. The number of health workers per study ranged from 6 to 154. In most of the studies, the health workers received training before starting the study. The duration of the training ranged from two days to nine months. Only one study used an automated method – the Masimo Root patient monitoring and connectivity platform with ISA CO2 Capnography – to measure RR as the reference standard [30]. The remaining studies used a manual count done by an “expert”.

Table 1.  Characteristics of the included studies

WordPress Tables Plugin

RR – Respiratory rate, FB – Fast breathing, CHW – community health worker, bpm – breath per minute, WHO – World Health Organisation

In four studies, health workers and reference standard counted RR simultaneously [17,30,37,43], while there was a short delay (i.e., reference standard measured RR immediately after health worker assessment) in ten studies [18,19,3236,38,39,41,42] and a long delay (i.e., reference standard measured RR a few hours after health worker assessment) in two studies [31,40]. Studies differed in outcome assessed, with eight studies reporting the percent agreement of RR measurement [1719,30,36,37,39,43], two reporting Bland-Altman plot to visualise RR agreement [30,43], and 15 reporting correct classification of fast and normal breathing [1719,3035,37,38,4043]. Out of the eight studies reporting agreement in RR, four defined agreement if the difference in RR was within ±2 breaths per minute (bpm) [18,30,36,43], three within ±3 bpm [37,39,43], and four within ±5 bpm [17,19,37,43]. Among the 15 studies reporting accuracy of fast breathing identification, two studies did not use the WHO RR threshold to classify fast breathing [33,34].

Methodological quality of included studies

The assessment of methodological quality is presented in Figure 2. In general, the risk of bias was low or unclear. For patient selection, we evaluated four studies as having a high risk of bias because of non-consecutive or non-random sample selection [17,19,36,40], six studies as having unclear risk of bias because of a poorly described sampling method [18,3739,43] or exclusion criteria [42]. For the index test, we evaluated all studies as having a low risk of bias because the health workers of all studies were blinded to the result of the reference standard, and a pre-specified threshold was used to classify fast breathing. For the reference standard, we evaluated four studies as having a high risk of bias because two studies did not use the WHO RR threshold to classify fast breathing [33,34]. In two studies, reference standard was unblinded [40,41] and seven studies had unclear risk of bias because of poor reporting on blinding [18,19,31,41,42] and qualification of the experts [17,3436,40]. For patient flow and timing assessment, we deemed three studies to have a high risk of bias. Among these, a long delay between index test and reference standard was present in two studies [31,40], and one study excluded a certain number of patients from the analysis without proper reporting [36]. Most of the studies had low concerns regarding applicability for all domains. The main concerns were related to inclusion criteria for patient selection in one study [40] and inappropriate classification of fast breathing for reference standard in two studies [33,34]. Overall, concerns regarding the applicability of the results were low.

Figure 2.  Risk of bias and applicability concerns summary: review authors’ judgements about each domain for each included study.

Agreement in respiratory rate count between health workers and reference standard

Table 2 presents the summary findings for the eight studies reporting the agreement in RR count between health workers and reference standards. Definitions of agreement in RR count varied across studies. Table 3 shows that the overall median agreements of the health workers were 39%, 47%, and 67% within ±2 bpm, ±3 bpm, and ±5 bpm of reference standards, respectively. The agreements of RR in terms of age groups, settings, types of health workers, and types of reference standards are also presented.

Table 2.  Studies reporting agreement in respiratory rate count between health workers and reference standards

WordPress Tables Plugin

bpm – breath per minute, CHW – community health worker

Table 3.  Agreement in respiratory rate count between health workers and reference standards

WordPress Tables Plugin

bpm – breath per minute

The agreement of RR counts between health workers and a reference standard was presented using the Bland Altman plots in two studies. Baker et al. [30] reported a wide variation in readings, especially in the younger children. The mean difference was -0.6 bpm with limits of agreement (LOAs) from -25.4 to 23.9 bpm [30]. Sinyangwe et al. [43] reported the mean difference of -0.74 bpm with LOAs from -18.8 to 17.3 bpm. Health workers over-counted RR compared to the reference standard in general, but undercounted in children with higher RR.

Accuracy in fast breathing identification by health workers compared to reference standard

The summary results of the 15 included studies reporting accuracy of classification of fast and normal breathing compared to a reference standard are presented in Table 4. The accuracy of fast breathing identification differed in different age groups. The agreement was comparatively lower in children aged 0-2 months compared to older children. The accuracy of fast breathing identification was lower in children with uncomplicated illness, in comparison to children with severe illness.

Table 4.  Studies reporting health worker classification of fast and normal breathing compared to a reference standard

WordPress Tables Plugin

PPV – positive predictive value, NPV – negative predictive value, CI – confidence Interval, SE – standard error

The median sensitivity, specificity, PPV, NPV, accuracy, and kappa value are presented in Table 5. The overall median sensitivity, specificity, and accuracy of classification of fast breathing were 77%, 86%, and 81%, respectively. The median sensitivity was marginally higher in children aged 0-2 months, and median specificity was slightly higher in children aged 2-59 months. The median sensitivity was higher in studies conducted in community settings, whereas the mean specificity was higher in studies conducted in health facilities. Although sensitivities were similar, the specificity was higher in facility-based health workers compared to community-based health workers. The median sensitivity was slightly higher when RR was measured simultaneously by the health worker and reference standard, compared to when it was measured with a short delay. Both median sensitivity and specificity were higher if the prevalence of fast breathing was higher in the sample.

Table 5.  Health worker classification of fast and normal breathing compared to a reference standard

WordPress Tables Plugin

PPV – positive predictive value, NPV – negative predictive value

Results of meta-analysis

Individual and summary estimates of sensitivity and specificity with 95% CI for all the studies included in the meta-analysis are presented in Figure 3. The pooled sensitivity was 78% (95% CI = 72-82), the pooled specificity was 86% (95% CI = 78-91), and there was considerable heterogeneity (I2 = 72%). Figure 4 depicts the hierarchical summary receiver curve (HSROC) plot of sensitivity and specificity with summary point, summary estimates, 95% confidence region and 95% prediction region for all studies included in the meta-analysis.

Figure 3.  Accuracy of health workers classification of fast and normal breathing compared to a reference standard. Forest plots of individual and summary estimates of sensitivity and specificity.

Figure 4.  HSROC plot of sensitivity vs specificity of health worker classification of fast and normal breathing for all included studies.

Table 6 presents subgroup analysis according to child age, study settings, types of health workers, timing of assessment, and prevalence of fast breathing using univariate meta-regression.

Table 6.  Subgroup analysis of sensitivity and specificity of health worker classification of fast and normal breathing compared to a reference standard

WordPress Tables Plugin

CI – confidence Interval, RR – respiratory rate

We conducted a sensitivity analysis excluding the study where the WHO RR threshold was not used to classify fast breathing to explore whether this could affect overall results (Figure S1 in the Online Supplementary Document). Based on the studies included in the sensitivity analysis, the pooled sensitivity of fast breathing identification by health workers was 78% (95% CI = 72-83) which was almost similar to the results of the primary meta-analysis (where all studies were included); however, the pooled specificity slightly increased to 87% (95% CI = 81-92).

DISCUSSION

This systematic review demonstrated that the performance of health workers in the measurement of RR and identification of fast breathing varied across the studies. Overall performance in classifying fast and normal breathing was moderate, with sensitivity ranging from 61% to 88% and a pooled estimate of 78% from the meta-analysis. As the sensitivity is moderate, a significant number of children may have a missed diagnosis of fast breathing, potentially leading to poor outcomes [44]. Some of these children may also have had other clinical signs of respiratory distress, like lower chest wall indrawing, that could have been identified, resulting in a true pneumonia case detection rate higher than these estimates. Further research is needed to investigate possible causes behind the inconsistency in diagnoses between health workers and reference standards, as well as to elicit the difficulties encountered by the health workers, thus improving sensitivity.

The specificity of the studies ranged from 69% to 91%, with a meta-estimate of 86%, demonstrating consistency in exclusion of a diagnosis of fast breathing pneumonia when the disease is not present. This is potentially encouraging, as it may imply that, if these guidelines are followed and RR counting is consistently applied during patient care, then few children would receive antibiotics unnecessarily, which could mitigate inappropriate use of antibiotics [44]. It also means there is minimal unwarranted distress and economic cost for caregivers who would wrongly believe their child has pneumonia [8].

Although there was a moderate agreement in identifying fast breathing, the agreement in RR count between health workers and reference standards was relatively poor. The level of agreement was inconsistent across the studies. The median agreements were 39%, 47%, and 67% within ±2 bpm, ±3 bpm, and ±5 bpm, respectively. It is worth mentioning that obtaining good agreement on RR counts is challenging, even between experts [45]. The difference in RR counts between two observers often does not change the diagnosis. Therefore, classification of RR into fast and normal breathing would be better than the continuous RR count agreements to evaluate the performance of health workers considering its clinical relevance.

The review found that the agreement in RR count between health workers was poor in children aged 0-2 months compared to the older children. The health workers may find it easier to count RR when it is slower in older children compared to when it is fast in younger children [46]. Interestingly, the review found that, although the specificity of fast breathing identification was higher in children aged 2-59 months, the sensitivity was higher in children aged 0-2 months. However, this finding for identifying fast breathing in newborns was based on two studies only. Sensitivity was also found to be slightly higher in infants compared to older children. More studies evaluating the accuracy of RR measurement and fast breathing identification in newborns and infants would be required to confirm this.

Community-based health workers performed better at counting RR and identifying fast breathing compared to facility-based workers. This might be due to community-based workers are usually recruited and trained for a specific program. They usually assess similar signs and symptoms repeatedly, give more time to do an assessment, develop better skills in assessing those specific signs and symptoms, and thus become more experienced, despite being lower cadres [47]. On the other hand, facility-based workers must deal with different types of patients with a wide range of signs and symptoms. The sensitivity was higher in the studies conducted in the community settings compared to those in facility settings. The crowded and busy environment of the health facilities in LMICs might influence the performance of the health workers [48].

The interval between health worker assessment and reference standard assessment is also important in evaluating the performance of health workers. The review demonstrates marginally higher sensitivity when both assessments were done simultaneously compared to a short or long delay. The RR can change over a period of time and this variability may affect sensitivity and specificity in identifying fast breathing [45]. Therefore, simultaneous measurement of RR by a health worker and a reference standard should be ideal. A short delay is not a valid reference standard for comparing RR but may be fair for comparing a binary pneumonia diagnosis. A prolonged period between the two measurements should be avoided.

The absence of an appropriate reference standard to evaluate the performance of health workers is a challenge. Most of the included studies used manual RR count by an expert as the reference standard. An expert is assumed to be more correct. However, the expert can over-count or under-count breaths. Therefore, using expert counting as a reference standard itself poses challenges due to uncertain accuracy. The possible biases using human expert count as the reference standard includes the difficulty in measuring the RR over the same simultaneous period and inconsistencies in human expert RR counting. One study used capnography as reference, which is an automated method using carbon dioxide (CO2) in exhaled air to extract RR [49]. However, the validity of using capnography in measuring RR in field-setting is yet to be established. The videography of child assessment and interpretation of the videos by an expert panel could be recommended as a reference standard for future studies [50,51].

There were several limitations to this review. First, most of the studies included in this review were conducted in Africa, while only two were conducted in Asia, and one in Oceania. Therefore, the review findings might not be generalizable across LMICs. Second, RR was often measured by health workers as a part of a larger study. The study may not have provided sufficient information about the methods of measurement and comprehensive results. Third, in most studies, a varying level of training was provided to the health worker before their assessment. This could impact the results of this review [52]. This also raises the question of whether the results of these studies assess health workers’ performance in their day-to-day environments instead of their competency after training. Performance of health workers during the study might not accurately reflect their day-to-day performance; it may also decay over time from training. Fourthly, most of the studies used an expert person as the reference standard who observed the assessment performed by the health workers. The performance of health workers might increase due to the observation compared to when conducting their usual day-to-day activities. This means that the findings would reflect a best-case scenario of accuracy and in the real-world context, we might expect it to be even worse [53]. Fifth, different studies used different definitions of RR agreement, ranging from two to five bpm between health workers and the reference standards. Therefore, it was not possible to combine the findings of all studies that reported agreement of RR measurement. Sixth, we have discussed some factors responsible for the variability of performance of health workers across the studies. There could be quite a few more contributing factors. Finally, we could not include some studies in this review that assessed health workers’ performance in this review, including diagnosis and management of pneumonia, which would involve measuring RR and classifying fast breathing. It was unclear whether these outcomes were measured or measured and not reported. Moreover, we could not include some studies in the meta-analysis because TP, FP, FN, and TN data were missing in the reports, or these were not possible to retrieve.

Despite these limitations, this review provides evidence on the need of strengthening the performance of health workers to measure RR and identify fast breathing pneumonia. Counting RR is the cornerstone to the diagnosis of pneumonia in children, but it is rarely practised in the field during real-world care [54]. The performance of health workers could be enhanced by improved training, supportive supervision, ongoing performance monitoring, and feedback [55]. Counting RR manually is challenging often results in inaccurate diagnosis. Therefore, the development of improved pneumonia diagnostic aids, such as a validated automated RR counters appropriate for use by health workers, might improve the diagnosis of pneumonia in LMICs [56]. Appropriate methods including a non-biased reference standard should be used to evaluate the accuracy of health workers’ RR counts. Further implementation research could help define what the best approach for improving their performance.

CONCLUSIONS

This review showed that the accuracy of RR measurement by non-physician health workers varied across the studies. While they could measure RR and identify fast breathing pneumonia with a moderate sensitivity and reasonable specificity, there is still a need for the improvement of RR measurement and identification of fast-breathing pneumonia by these health workers. This could be done through improved training, ongoing supervision, audit of performance, and improved diagnostic aids to measure RR and classify fast breathing accurately. The contribution of well-trained and well-equipped health workers is valuable in LMICs, where it is not always feasible for a child to see a doctor. This should decrease the burden on already scarce doctors and health centres in LMICs and may help reduce morbidity and mortality associated with pneumonia.

Additional material

Online Supplementary Document

Acknowledgments

We are grateful to Ruth Jenkins, Academic Librarian at the University of Edinburgh, for her help in developing the search strategy.

[1] Funding: This research was funded by the UK National Institute for Health Research (NIHR) (Global Health Research Unit on Respiratory Health (RESPIRE); 16/136/109) using UK aid from the UK Government to support global health research. The views expressed in this publication are those of the author(s) and not necessarily those of the NIHR or the UK Government.

[2] Authorship contributions: AMK conceptualized the study, performed database searches; AMK and AOD screened the title and abstract, reviewed the studies, and extracted the data; AMK performed the data analysis and drafted the first version; HC, SC, AHB, EDM, CK, TS, SA, and AOD contributed substantially to the writing of the paper. All authors have read and approved the final manuscript. The RESPIRE collaboration comprises the UK Grant holders, Partners and research teams as listed on the RESPIRE website (www.ed.ac.uk/usher/respire), including Harish Nair and Siân Williams.

[3] Competing interests: Harry Campbell is the Co-Editor in Chief of the Journal of Global Health. To ensure that any possible conflict of interest relevant to the journal has been addressed, this article was reviewed according to the best practice guidelines of international editorial organisations. The authors completed the ICMJE Unified Competing Interest Form (available upon request from the corresponding author) and declare no conflicts of interest.

references

[1] L Liu, S Oza, D Hogan, Y Chu, J Perin, and J Zhu. Global, regional, and national causes of under-5 mortality in 2000-15: an updated systematic analysis with implications for the Sustainable Development Goals. Lancet. 2016;388:3027-35. DOI: 10.1016/S0140-6736(16)31593-8. [PMID:27839855]

[2] I Rudan, KL O’Brien, H Nair, L Liu, E Theodoratou, and S Qazi. Epidemiology and etiology of childhood pneumonia in 2010: estimates of incidence, severe morbidity, mortality, underlying risk factors and causative pathogens for 192 countries. J Glob Health. 2013;3:010401. [PMID:23826505]

[3] . 2016. Estimates of the global, regional, and national morbidity, mortality, and aetiologies of lower respiratory infections in 195 countries, 1990-2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet Infect Dis. 2018;18:1191-210. DOI: 10.1016/S1473-3099(18)30310-4. [PMID:30243584]

[4] H Nair, EA Simões, I Rudan, BD Gessner, E Azziz-Baumgartner, and JSF Zhang. Global and regional burden of hospital admissions for severe acute lower respiratory infections in young children in 2010: a systematic analysis. Lancet. 2013;381:1380-90. DOI: 10.1016/S0140-6736(12)61901-1. [PMID:23369797]

[5] P Geldsetzer, TC Williams, A Kirolos, S Mitchell, LA Ratcliffe, and MK Kohli-Lynch. The recognition of and care seeking behaviour for childhood illness in developing countries: a systematic review. PLoS One. 2014;9:e93427. DOI: 10.1371/journal.pone.0093427. [PMID:24718483]

[6] World Health Organization. Global Health Workforce statistics database. Available: https://www.who.int/data/gho/data/themes/topics/health-workforce. Accessed 23 July 2021.

[7] JI Blanford, S Kumar, W Luo, and AM MacEachren. It’s a long, long walk: accessibility to hospitals, maternity and integrated health centers in Niger. Int J Health Geogr. 2012;11:24 DOI: 10.1186/1476-072X-11-24. [PMID:22737990]

[8] EA Lungu, R Biesma, M Chirwa, and C Darker. Healthcare seeking practices and barriers to accessing under-five child health services in urban slums in Malawi: a qualitative study. BMC Health Serv Res. 2016;16:410 DOI: 10.1186/s12913-016-1678-x. [PMID:27542836]

[9] S Zhang, PM Sammon, I King, AL Andrade, CM Toscano, and SN Araujo. Cost of management of severe pneumonia in young children: systematic analysis. J Glob Health. 2016;6:010408. DOI: 10.7189/jogh.06.010408. [PMID:27231544]

[10] R Izadnegahdar, AL Cohen, KP Klugman, and SA Qazi. Childhood pneumonia in developing countries. Lancet Respir Med. 2013;1:574-84. DOI: 10.1016/S2213-2600(13)70075-4. [PMID:24461618]

[11] C Whidden, J Thwing, J Gutman, E Wohl, C Leyrat, and K Kayentao. Proactive case detection of common childhood illnesses by community health workers: a systematic review. BMJ Glob Health. 2019;4:e001799. DOI: 10.1136/bmjgh-2019-001799. [PMID:31908858]

[12] S Gove. Integrated management of childhood illness by outpatient health workers: technical basis and overview. The WHO Working Group on Guidelines for Integrated Management of the Sick Child. Bull World Health Organ. 1997;75:Suppl 17-24. [PMID:9529714]

[13] S Sazawal and RE Black. Meta-analysis of intervention trials on case-management of pneumonia in community settings. Lancet. 1992;340:528-33. DOI: 10.1016/0140-6736(92)91720-S. [PMID:1354286]

[14] World Health Organization. IMCI information package. Integrated Management of Childhood Illness (IMCI). Geneva: 1999.

[15] World Health Organization. Integrated Management of Childhood Illness: Chart Booklet. World Health Organization: Geneva. 2014.

[16] K Carina, B Kevin, N Rebecca, Q Bassat, Q Shamim Ahmad, and DM Eric. Back to Basics in Paediatric Pneumonia-Defining a Breath and Setting Reference Standards to Innovate Respiratory Rate Counting. J Trop Pediatr. 2021;67:fmaa112. DOI: 10.1093/tropej/fmaa112. [PMID:33280051]

[17] K Källander, G Tomson, X Nsabagasani, JN Sabiiti, G Pariyo, and S Peterson. Can community health workers and caretakers recognise pneumonia in children? Experiences from western Uganda. Trans R Soc Trop Med Hyg. 2006;100:956-63. DOI: 10.1016/j.trstmh.2005.11.004. [PMID:16455119]

[18] D Mukanga, R Babirye, S Peterson, GW Pariyo, G Ojiambo, and JK Tibenderana. Can lay community health workers be trained to use diagnostics to distinguish and treat malaria and pneumonia in children? Lessons from rural Uganda. Trop Med Int Health. 2011;16:1234-42. DOI: 10.1111/j.1365-3156.2011.02831.x. [PMID:21752163]

[19] NP Miller, A Amouzou, M Tafesse, E Hazel, H Legesse, and T Degefie. Integrated community case management of childhood illness in Ethiopia: implementation strength and quality of care. Am J Trop Med Hyg. 2014;91:424-34. DOI: 10.4269/ajtmh.13-0751. [PMID:24799369]

[20] Cochrane Methods Screening and Diagnostic Tests. Handbook for DTA Reviews, Available: https://methods.cochrane.org/sdt/handbook-dta-reviews. Accessed 14 June 2021.

[21] MJ Page, JE McKenzie, PM Bossuyt, I Boutron, TC Hoffmann, and CD Mulrow. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Int J Surg. 2021;88:105906. DOI: 10.1016/j.ijsu.2021.105906. [PMID:33789826]

[22] MDF McInnes, D Moher, BD Thombs, TA McGrath, PM Bossuyt, and T Clifford. Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies: The PRISMA-DTA Statement. JAMA. 2018;319:388-96. DOI: 10.1001/jama.2017.19163. [PMID:29362800]

[23] The World Bank. World Bank Country and Lending Groups. 2018. Available: https://datahelpdesk.worldbank.org/knowledgebase/articles/906519. Accessed 20 July 2021.

[24] PF Whiting, AW Rutjes, ME Westwood, S Mallett, JJ Deeks, and JB Reitsma. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155:529-36. DOI: 10.7326/0003-4819-155-8-201110180-00009. [PMID:22007046]

[25] CM Rutter and CA Gatsonis. A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med. 2001;20:2865-84. DOI: 10.1002/sim.942. [PMID:11568945]

[26] RM Harbord and P Whiting. Metandi: meta-analysis of diagnostic accuracy using hierarchical logistic regression. Stata J. 2009;9:211-29. DOI: 10.1177/1536867X0900900203

[27] Dwamena BA. Midas: a program for meta-analytical integration of diagnostic accuracy studies in Stata. Ann Arbor: Division of Nuclear Medicine, Department of Radiology, University of Michigan Medical School. 2007.

[28] J Lee, KW Kim, SH Choi, J Huh, and SH Park. Systematic Review and Meta-Analysis of Studies Evaluating Diagnostic Test Accuracy: A Practical Review for Clinical Researchers-Part II. Statistical Methods of Meta-Analysis. Korean J Radiol. 2015;16:1188-96. DOI: 10.3348/kjr.2015.16.6.1188. [PMID:26576107]

[29] JJ Deeks, P Macaskill, and L Irwig. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol. 2005;58:882-93. DOI: 10.1016/j.jclinepi.2005.01.016. [PMID:16085191]

[30] K Baker, T Alfvén, A Mucunguzi, A Wharton-Smith, E Dantzer, and T Habte. Performance of Four Respiratory Rate Counters to Support Community Health Workers to Detect the Symptoms of Pneumonia in Children in Low Resource Settings: A Prospective, Multicentre, Hospital-Based, Single-Blinded, Comparative Trial. EClinicalMedicine. 2019;12:20-30. DOI: 10.1016/j.eclinm.2019.05.013. [PMID:31388660]

[31] AH Baqui, SE Arifeen, HE Rosen, I Mannan, SM Rahman, and AB Al-Mahmud. Community-based validation of assessment of newborn illnesses by trained community health workers in Sylhet district of Bangladesh. Trop Med Int Health. 2009;14:1448-56. DOI: 10.1111/j.1365-3156.2009.02397.x. [PMID:19807901]

[32] C Baynes, D Mboya, S Likasi, D Maganga, S Pemba, and J Baraka. Quality of Sick Child-Care Delivered by Community Health Workers in Tanzania. Int J Health Policy Manag. 2018;7:1097-109. DOI: 10.15171/ijhpm.2018.63. [PMID:30709085]

[33] JP Brady, FB Awan, EM Wafula, and FE Onyango. Recognition of illness in very young infants by inexperienced health workers. Ann Trop Paediatr. 1993;13:401-7. DOI: 10.1080/02724936.1993.11747680. [PMID:7506892]

[34] DR Brewster, T Pyakalyia, G Hiawalyer, and DL O’Connell. Evaluation of the ARI program: a health facility survey in Simbu, Papua, New Guinea. P N G Med J. 1993;36:285-96. [PMID:7941757]

[35] CV Cardemil, KE Gilroy, JA Callaghan-Koru, H Nsona, and J Bryce. Comparison of methods for assessing quality of care for community case management of sick children: an application with community health workers in Malawi. Am J Trop Med Hyg. 2012;87:127-36. DOI: 10.4269/ajtmh.2012.12-0389. [PMID:23136288]

[36] KE Gilroy, JA Callaghan-Koru, CV Cardemil, H Nsona, A Amouzou, and A Mtimuni. Quality of sick child care delivered by Health Surveillance Assistants in Malawi. Health Policy Plan. 2013;28:573-85. DOI: 10.1093/heapol/czs095. [PMID:23065598]

[37] JN Kalyango, E Rutebemberwa, T Alfven, S Ssali, S Peterson, and C Karamagi. Performance of community health workers under integrated community case management of childhood illnesses in eastern Uganda. Malar J. 2012;11:282 DOI: 10.1186/1475-2875-11-282. [PMID:22905758]

[38] JM Kelly, B Osamba, RM Garg, MJ Hamel, JJ Lewis, and SY Rowe. Community health worker performance in the management of multiple childhood illnesses: Siaya District, Kenya, 1997-2001. Am J Public Health. 2001;91:1617-24. DOI: 10.2105/AJPH.91.10.1617. [PMID:11574324]

[39] A Langston, A Wittcoff, P Ngoy, J O’Keefe, N Kozuki, and H Taylor. Testing a simplified tool and training package to improve integrated Community Case Management in Tanganyika Province, Democratic Republic of Congo: a quasi-experimental study. J Glob Health. 2019;9:010810. DOI: 10.7189/jogh.09.010810. [PMID:31263553]

[40] MC Mulaudzi. Adherence to case management guidelines of Integrated Management of Childhood Illness (IMCI) by healthcare workers in Tshwane, South Africa. SAJCH. 2015;9:89-92. DOI: 10.7196/SAJCH.7959

[41] EA Simoes, T Desta, T Tessema, T Gerbresellassie, M Dagnew, and S Gove. Performance of health workers after training in integrated management of childhood illness in Gondar, Ethiopia. Bull World Health Organ. 1997;75:Suppl 143-53. [PMID:9529717]

[42] EA Simoes and EJ McGrath. Recognition of pneumonia by primary health care workers in Swaziland with a simple clinical algorithm. Lancet. 1992;340:1502-3. DOI: 10.1016/0140-6736(92)92757-7. [PMID:1361598]

[43] C Sinyangwe, K Graham, S Nicholas, R King, S Mukupa, and K Källander. Assessing the Quality of Care for Pneumonia in Integrated Community Case Management: A Cross-Sectional Mixed Methods Study. PLoS One. 2016;11:e0152204. DOI: 10.1371/journal.pone.0152204. [PMID:27011331]

[44] T Druetz, K Siekmans, S Goossens, V Ridde, and S Haddad. The community case management of pneumonia in Africa: a review of the evidence. Health Policy Plan. 2015;30:253-66. DOI: 10.1093/heapol/czt104. [PMID:24371218]

[45] EA Simoes, R Roark, S Berman, LL Esler, and J Murphy. Respiratory rate: measurement of variability over time and accuracy at different counting periods. Arch Dis Child. 1991;66:1199-203. DOI: 10.1136/adc.66.10.1199. [PMID:1953002]

[46] S Fleming, M Thompson, R Stevens, C Heneghan, A Plüddemann, and I Maconochie. Normal ranges of heart rate and respiratory rate in children from birth to 18 years of age: a systematic review of observational studies. Lancet. 2011;377:1011-8. DOI: 10.1016/S0140-6736(10)62226-X. [PMID:21411136]

[47] K Scott, SW Beckham, M Gross, G Pariyo, KD Rao, and G Cometto. What do we know about community-based health worker programs? A systematic review of existing reviews on community health workers. Hum Resour Health. 2018;16:39 DOI: 10.1186/s12960-018-0304-x. [PMID:30115074]

[48] MC Kok, SS Kane, O Tulloch, H Ormel, S Theobald, and M Dieleman. How does context influence performance of community health workers in low- and middle-income countries? Evidence from the literature. Health Res Policy Syst. 2015;13:13 DOI: 10.1186/s12961-015-0001-3. [PMID:25890229]

[49] C Massaroni, A Nicolò, D Lo Presti, M Sacchetti, S Silvestri, and E Schena. Contact-Based Methods for Measuring Respiratory Rate. Sensors (Basel). 2019;19:908 [PMID:30795595]

[50] Ward C, Baker K, Marks S, Kallander K, editors. Selecting a reference standard for evaluating respiratory rate devices to diagnose symptoms of pneumonia in children under 5: Lessons learned from resource-poor settings in Sub-Saharan Africa and Asia. American Journal of Tropical Medicine and Hygiene; 2017: Am Soc Trop Med & Hygiene 8000 Westpart Dr, Ste 130, Mclean, VA 22101 USA.

[51] JM Ansermino, G Dumont, and AS Ginsburg. How Uncertain Is Our Reference Standard for Respiratory Rate Measurement? Am J Respir Crit Care Med. 2019;199:1036-7. DOI: 10.1164/rccm.201812-2266LE. [PMID:30673505]

[52] L Huicho, RW Scherpbier, AM Nkowane, and CG Victora. How much does quality of child care vary between health workers with differing durations of training? An observational multicountry study. Lancet. 2008;372:910-6. DOI: 10.1016/S0140-6736(08)61401-4. [PMID:18790314]

[53] SY Rowe, MA Olewe, DG Kleinbaum, JE McGowan, DA McFarland, and R Rochat. The influence of observation and setting on community health workers’ practices. Int J Qual Health Care. 2006;18:299-305. DOI: 10.1093/intqhc/mzl009. [PMID:16675475]

[54] E Bjornstad, GA Preidis, N Lufesi, D Olson, P Kamthunzi, and MC Hosseinipour. Determining the quality of IMCI pneumonia care in Malawian children. Paediatr Int Child Health. 2014;34:29-36. DOI: 10.1179/2046905513Y.0000000070. [PMID:24091151]

[55] MC Kok, M Dieleman, M Taegtmeyer, JE Broerse, SS Kane, and H Ormel. Which intervention design factors influence performance of community health workers in low- and middle-income countries? A systematic review. Health Policy Plan. 2015;30:1207-27. DOI: 10.1093/heapol/czu126. [PMID:25500559]

[56] C Ward, K Baker, S Marks, D Getachew, T Habte, and C McWhorter. Determining the Agreement Between an Automated Respiratory Rate Counter and a Reference Standard for Detecting Symptoms of Pneumonia in Children: Protocol for a Cross-Sectional Study in Ethiopia. JMIR Res Protoc. 2020;9:e16531. DOI: 10.2196/16531. [PMID:32238340]

Correspondence to:
Ahad Mahmud Khan, MBBS, MPH
Centre for Global Health Research
Usher Institute
30 West Richmond Street
The University of Edinburgh
EH8 9DX
Edinburgh
United Kingdom
[email protected]