The 2014-2016 Ebola epidemic had a debilitating impact on health systems in Sierra Leone, Liberia, and Guinea [1,2]. High rates of Ebola infections among health workers  led to fear of contracting Ebola in health facilities and derailed confidence in health services, including immunization services . In Sierra Leone, vaccination coverage of diphtheria-tetanus-pertussis vaccine (DTP-3) reached a high of 92% in 2013 before the Ebola epidemic but declined to 84% by the end of the health emergency in 2016 . National vaccination coverage of the first dose of measles-containing vaccine (MCV-1) was estimated to be 81% based on survey data, but with substantial variation across districts . Multiple outbreaks of measles occurred after the Ebola epidemic ended .
Beyond emergency contexts, caregivers may delay or refuse vaccines for their children due to complex individual, cultural, socioeconomic, and vaccination-related reasons [8–10]. Numerous community engagement efforts that involved working with affected communities to deeply understand issues and develop solutions were implemented in Sierra Leone during and after the Ebola epidemic to strengthen health systems and restore public confidence in health services, including childhood immunization [11–15]. Globally, major efforts are underway to strengthen community engagement to enhance confidence in childhood immunization , especially in low- and middle-income countries (LMICs) [17,18] and places recovering from disruptive health emergencies .
During and after major health emergencies, like the 2014-2016 Ebola epidemic and the current COVID-19 pandemic , childhood immunization services are often disrupted due to mitigation efforts such as restrictions in movement, fear of getting infected in clinics, and the shifting of limited resources to the health emergency . In these emergency contexts, community engagement using dialogue, role modeling of promoted behaviors, and collective action planning is critical to safeguard confidence in immunization services, respond to misinformation about vaccines, and foster optimal vaccination uptake . In Sierra Leone, we assessed potential drivers of vaccination uptake after the Ebola epidemic ended, including the potential role of perceived community engagement in planning the immunization program at the community level.
METHODS AND MATERIALS
We conducted a population-based household survey with caregivers of children in a birth cohort of 12 to 23 months in four districts in Sierra Leone that was designed to allow the Sierra Leone Ministry of Health and Sanitation (MoHS) to understand drivers of vaccination and under-vaccination in low performing districts. These four districts had the lowest vaccination coverage of DTP-3 in their respective geographic regions (Kambia district in the Northern region, Kono district in the Eastern region, Moyamba district in the Southern region, and Western Rural in the Western region) . We used multistage sampling to randomly select households, children, and caregivers. The sub-district sampling was done based on guidelines established by the World Health Organization . The survey was conducted in February 2019.
Sierra Leone has a civil society platform to strengthen community engagement in the planning and delivery of childhood immunization services . The immunization civil society platform is an umbrella entity that comprises more than 100 organizations including community-based organizations, non-governmental organization (local and international), teachers associations, and local news media organizations. The Government of Sierra Leone included community engagement in the planning of health services as a core strategy in the post-Ebola recovery plan . Priority activities included support for community-based networks to promote childhood immunization and to bring community stakeholders into the planning of immunization services , including through on-the-ground support from civil society. Moreover, community health workers (CHWs) have had a longstanding role in supporting childhood immunization in Sierra Leone, and were strongly leveraged in the post-Ebola recovery period .
The primary outcomes of interest were caregivers’ vaccination confidence and their children’s vaccination uptake. The primary exposure of interest was perceived community engagement in childhood immunization from the perspective of caregivers. To understand potential drivers of vaccination more comprehensively, we also assessed vaccination decision making, accessibility of childhood immunization services, preferred communication channels, and trusted communicators.
Perceived community engagement
We measured perceived level of community engagement by asking caregivers: “From your observation, how would you rate the level of community involvement and participation in the planning of vaccination programs in your community?” Response options were low, medium, or high.
Items on vaccination confidence were informed by the Caregiver Vaccination Attitudes Scale that was validated in Ghana . We used other theories to develop additional items on vaccination attitudes and experiences. For instance, perceived threat of vaccine-preventable disease and self-efficacy were informed by the Health Belief Model . Additional items were developed to capture descriptive community norms around childhood vaccination and experiences during vaccination clinic visits. As part of a parallel urban needs assessment in Sierra Leone , we conducted cognitive testing of the confidence items via in-depth interviews with a convenience sample of five parents. Cognitive testing provided qualitative insights regarding how participants understood the questions and their interpretations of the response options. This process led to the finalization of 19 items on vaccination confidence that were included in the questionnaire.
The questionnaire assessed vaccination uptake using the child’s home-based record (HBR) and through recall by the child’s primary caregiver when HBR was unavailable . The child’s receipt of the following vaccines and related doses was assessed: Bacillus Calmette-Guerin (BCG) vaccine against tuberculosis; the first, second, and third dose of pentavalent vaccine (penta-1, penta-2, penta-3, respectively) against diphtheria, tetanus, pertussis, hepatitis B and Haemophilus influenzae type b; and MCV-1 against measles. Although the immunization schedule includes additional vaccines, the vaccine doses we assessed reflect the five scheduled immunization visits for infants in Sierra Leone – BCG at birth, penta-1 at 6 weeks, penta-2 at 10 weeks, penta-3 at 14 weeks, and MCV-1 at 9 months .
We randomly selected a total of 72 clusters (15 to 21 clusters per district) from a sampling frame of geographic enumeration areas based on the 2015 census . Within each cluster, we randomly selected 10 eligible households and randomly selected one child from each household using simple random sampling via a mobile application. The corresponding primary caregiver was approached for their voluntary informed consent to an interview. We determined a target sample size of 720 caregiver-child pairs based on the following assumptions/parameters: 70% expected coverage of penta-3, desired precision of ±6.5%, α level of 0.05, and a design effect of 2.5.
Training and data collection
We trained a local team that collected the data in local languages. The questionnaire was piloted in a conveniently selected community that was not part of the survey. Data collection took place in February 2019. A digital version of the questionnaire was programmed in Open Data Kit via KoboCollect version 1.2 (www.kobotoolbox.com) and installed on Android-based tablets.
To reduce the 19-items on vaccination confidence into a conceptually meaningful scale, we conducted exploratory factor analysis (EFA) using a principal components extraction method. An oblique Promax rotation was applied to account for the correlation between factors . We assessed the internal consistency reliability of the reduced items based on their combined Cronbach’s alpha coefficient. The mean score on the confidence scale was calculated and dichotomized; high confidence was defined as a score greater than the mean (coded 1) and low confidence was defined as having a score less than or equal to the mean (coded 0). For the vaccination uptake outcome, we created a composite binary variable wherein a ‘fully vaccinated’ child was defined as one who had received BCG, penta-1, penta-2, penta-3, and MCV-1; ‘under-vaccinated’ was defined as missing any of these vaccine doses.
We calculated proportions and 95% confidence intervals (CI) for point estimates in univariable and bivariable analyses, accounting for complex survey design. We examined the differences in point estimates between the caregivers of fully vaccinated and under-vaccinated children using a design-based F statistic that corrects the Pearson χ2 test for the complex survey design. Modified Poisson regression models with robust variance estimation [32,33] were fitted using generalized estimating equations with exchangeable correlation structure to account for the clustered data . The use of modified Poisson regression modeling enabled us to directly calculate the adjusted prevalence ratios (aPR) .
In the first model, we examined the association between perceived community engagement (low, middle, or high) and expressing high vaccination confidence. In the second model, we examined the association between perceived community engagement and having a fully vaccinated child. We adjusted all models for the child’s birth order (coded 1 for the first child, 2 for the second child, 3 for at least the third child), where the child was delivered at birth (coded 1 for home-based delivery and 2 for delivery in a health facility), retention of the child’s vaccination card record (coded 0 for not retained and 1 for retained), parental education level (coded 0 if both parents had no education, 1 if one parent had some education, and 2 if both parents had some education), and parental religious affiliation (coded 1 if both parents were Christian, coded 2 if both parents were Muslim, coded 3 if parents had mixed-faith). These demographic covariates were included in the models because they have been shown to be consistently associated with vaccination uptake across diverse settings in West Africa and elsewhere [36,37]. All data were analyzed in Stata version 15 SE (StataCorp, College Station TX, USA). All statistical testing was two-sided and a P-value <0.05 was considered statistically significant.
The survey was approved by the Ethics and Scientific Review Committee in the Sierra Leone Ministry of Health and Sanitation. The Human Subject Office in the Center for Global Health at the U.S. Centers for Disease Control and Prevention approved the assessment as a non-research public health activity. All participants provided written or thumb-printed informed consent.
The final sample comprised of a birth cohort of 621 age-eligible children and their primary caregivers, which represented 91% of the 684 caregivers that were approached by the data collection teams (Figure 1). Nineteen percent of the children were delivered at home and 46% were at least the third child to the mother. Almost half of the mothers had some education (49%). Home-based vaccination card record was retained and available for 79% of the children (Table 1). Mothers were predominantly the primary caregivers of the children (90%), followed by siblings (5%), fathers (3%) and other relatives (2%) (data not shown in table).
Figure 1. Flow diagram of the sampling of caregiver-child pairs included in the analysis–multistage cross-sectional cluster survey, Sierra Leone, 2019.
Table 1. Sociodemographic characteristics of sampled children and their caregivers–multistage cross-sectional cluster survey, Sierra Leone, 2019
*Unweighted frequencies and percent distributions of sample characteristics. Father’s education is missing 21 responses: 4 in Kambia, 8 in Kono, 4 in Moyamba, and 5 in Western Rural. Retention of vaccination card is missing 2 responses: 1 from Kono and 1 from Western Rural.
Decision-making power related to vaccination
Most caregivers expressed that the child’s father held the final decision to vaccinate the child (56%), followed by the mother (37%) and other relatives (7%). In bivariable analysis, our findings suggest that the person who has the final decision-making power over the child’s vaccination was associated with vaccination uptake (P=0.04). For example, when mothers were the decision-makers, 80% (95% CI = 68%-89%) of the children were fully vaccinated versus 69% (95% CI = 60%-77%) when fathers were the decision-makers and 56% (95% CI = 38%-72%) when other relatives were (Table 2).
Table 2. Associations of decision making, accessibility, preferred information channels, and trusted communicators related to childhood immunization with having fully vaccinated children–multistage cross-sectional cluster survey, Sierra Leone, 2019
CI – confidence interval
*Proportions and 95% confidence intervals are weighted to account for the complex survey design.
†For analytical purposes, ‘fully vaccinated’ was defined as receiving BCG, penta-1, penta-2, penta-3, and MCV-1; under-vaccinated was defined as missing one or more of the aforementioned vaccine doses. Vaccination status was sourced from the home-based card record (HBR) for 488 children and from caregiver recall for 133 children whose HBRs were unavailable.
‡Due to the complex sampling design, to get a valid P value, the uncorrected χ2 statistic was converted to an F statistic.
Accessibility to vaccination services
Half of the caregivers (52%) reported that it took too much time to get to their usual vaccination site, and 59% expressed that too much time is spent waiting for the child to be vaccinated at the health facility. Caregivers perceived that health workers expected some money (36%) or non-cash items (7%) in return for vaccination. Among the sub-sample of caregivers who perceived some expectation of monetary or non-cash items (n=193), most of them expressed that they found the amount of money or non-cash items to be reasonable (62% and 84%, respectively). These aspects of immunization service accessibility were not found to be associated with vaccination uptake in bivariable analysis (Table 2).
Preferred communication channels
The most preferred communication channels were through health facilities (55%) and household visits by CHWs (25%). In bivariable analysis, communication channel preference was associated with vaccination uptake (P=0.01). For example, caregivers who preferred getting immunization information from health facilities were more likely to have fully vaccinated children compared to caregivers who preferred radio programs (82%, 95% CI = 74%-88%, versus 59%, 95% CI = 48%-70%).
Health workers were largely viewed as the most trusted communicators of immunization information (67%), followed by spouses (15%), and CHWs (8%). Preference for trusted communicators was not associated with vaccination uptake (P=0.15) (Table 2).
Perceived community engagement
Overall, 6% of respondents perceived low level of community engagement, 41% perceived a medium level of engagement and 53% perceived a high level of engagement. In bivariable analysis, perceived community engagement was associated with vaccination uptake (P<0.01). For example, children whose caregivers perceived high level of engagement were 82% (95% CI = 72%-88%) fully vaccinated compared to 63% (95% CI = 53%-71%) of children whose caregivers perceived medium level of engagement and 36% (95% CI = 22%-53%) of children whose caregivers perceived low level of engagement (Table 3).
Table 3. Associations of perceived community engagement, perceived areas of improvement, and measures of vaccination confidence with having fully vaccinated children – multistage cross-sectional cluster survey, Sierra Leone, 2019
CI – confidence interval
*Measured with a 3-point Likert item asking participants: “From your observation, how would you rate the level of community involvement and participation in the planning of vaccination programs in your community?” Response options were low, medium, or high.
†Proportions and 95% confidence intervals are weighted to account for the complex survey design.
‡For analytical purposes, ‘fully vaccinated’ was defined as receiving BCG, penta-1, penta-2, penta-3, and MCV-1; under-vaccinated was defined as missing one or more of the aforementioned vaccine doses. Vaccination status was sourced from the home-based card record (HBR) for 488 children and from caregiver recall for 133 children whose HBRs were unavailable.
§Due to the complex sampling design, to get a valid P value, the uncorrected χ2 statistic was converted to an F statistic.
‖The vaccination confidence scale score was dichotomized into a binary variable to indicate low vaccination confidence (≤ mean score) versus high vaccination confidence (> mean score).
Perceived priority areas of improvement
About one third of respondents wanted more vaccination sites closer to the community (35%) and enhanced community engagement and education (32%). In addition, improving interactions between vaccinators and caregivers was cited by 11% of respondents. In bivariable analysis, perception of the priority areas of improvement was not associated with vaccination uptake (P = 0.29) (Table 3).
Psychometric properties of the Childhood Vaccination Confidence Scale used in the assessment are provided in Table S1, Table S2, and Table S3 in the Online Supplementary Document. In summary, the 19 items on vaccination confidence were reduced to a 12-item scale based on results from the exploratory factory analysis. Two factors were retained in the final scale, namely ‘confidence’ (10 items total) and ‘congruence with faith’ (2 items total) (Table S1 in the Online Supplementary Document). The 12-item scale demonstrated very good internal consistency (Cronbach’s α = 0.94). The scale score was positively associated with having a fully vaccinated child after adjusting for demographic covariates such that every 1 unit increase in the scale score was associated with a 26% increase in the likelihood of the child being fully vaccinated (Table S3 in the Online Supplementary Document).
When the scale score was dichotomized, 78% of caregivers scored above the mean and were considered as having high confidence in vaccination. In bivariable analysis, among caregivers with high confidence in vaccination, 78% (95% CI = 70%-85%) of their children were fully vaccinated compared to 53% (95% CI = 41%-64%) among caregivers with low confidence (Table 3). For example, children whose caregivers thought vaccines are ‘very much’ safe were 76% (95% CI = 68%-82%) fully vaccinated compared to 48% (95% CI = 37%-59%) of children whose caregivers thought vaccines are ‘somewhat’ safe. Children whose caregivers thought vaccination is ‘very much’ compatible with their religious beliefs were 79% (95% CI = 71%-85%) fully vaccinated compared to 51% (95% CI = 40%-63%) of children whose caregivers thought vaccination is ‘somewhat’ compatible with their religious beliefs. Children whose caregivers felt that community discourse about vaccination is ‘mostly positive’ were 79% (95% CI = 70%-85%) fully vaccinated compared to 24% (95% CI = 6%-59%) of children whose caregivers felt it is ‘mostly negative’ and 57% (95% CI = 49%-64%) of children whose caregivers felt it is ‘mixed’ (Table 3).
About 98% of children in our sample received BCG, 97% received penta-1, 94% received penta-2, 81% received penta-3, and 84% received MCV-1. In total, 71% of the children in our sample were considered fully vaccinated and 29% were under-vaccinated. Self-reported vaccination refusal was low (3%) but self-reported vaccination delay was more frequently reported by caregivers (19%). Nearly all children received their last vaccine at a health facility (99%) compared to via community outreach (1%) (Table 4).
Table 4. Childhood vaccination uptake, self-reported refusal, and self-reported delay– multistage cross-sectional cluster survey, Sierra Leone, 2019
CI – confidence interval
*Proportions and 95% confidence intervals are weighted to account for the complex survey design.
†Vaccination status was sourced from the home-based card record (HBR) for 488 children and from caregiver recall for 133 children whose HBRs were unavailable.
Association between perceived community engagement and vaccination confidence
In multivariable analysis, caregivers who perceived high community engagement were nearly three times more likely to express high vaccination confidence (adjusted prevalence ratio (aPR) = 2.60; 95% CI = 1.67-4.04) as compared to caregivers who perceived low community engagement. A caregiver was more likely to express high vaccination confidence when both parents have some education versus no education (aPR = 1.19; 95% CI = 1.02-1.40). Other demographic covariates were not associated with vaccination confidence (Table 5).
Table 5. Multivariable correlates of expressing high vaccination confidence and having a fully vaccinated child – multistage cross-sectional cluster survey, Sierra Leone, 2019
PR – prevalence ratio; CI – confidence interval
*Measured by asking participants: “From your observation, how would you rate the level of community involvement and participation in the planning of vaccination programs in your community?” Response options were low, medium, or high.
†The vaccination confidence scale score was dichotomized into a binary variable to indicate low vaccination confidence (≤ mean score) versus high vaccination confidence (> mean score).
‡ Defined as a child who received Bacillus Calmette-Guerin vaccine, three doses of pentavalent vaccine, and first dose of measles vaccine based on the home-based card record (HBR) for 488 children and from caregiver recall for 133 children whose HBRs were unavailable.
§PR, prevalence ratio from modified Poisson regression model with robust variance estimation by using generalized estimating equation; models adjusted for child’s place of delivery, child’s birth order, parents’ education, parents’ religion, retention of vaccination card.
‖Due to the complex sampling design, to get a valid P value, the uncorrected χ2 statistic was converted to an F statistic.
Association between perceived community engagement and vaccination uptake
In multivariable analysis, caregivers who perceived high community engagement were nearly twice as likely to have children who were fully vaccinated compared to those who perceived low engagement (aPR = 1.67, 95% CI = 1.18-2.38). Demographically, a child was more likely to be fully vaccinated when the caregiver retained the child’s vaccination card versus not retaining the card (aPR = 1.82; 95% CI = 1.48-2.23), the child was delivered in a health facility versus at home (aPR = 1.28; 95% CI = 1.06-1.54), and when both parents have some education versus no education (aPR = 1.33; 95% CI = 1.16-1.53) (Table 5).
Three years after major disruptions in the childhood immunization program that were precipitated by the 2014-2016 Ebola epidemic in Sierra Leone, nearly all caregivers in our assessment expressed that vaccines are good, safe, and beneficial in preventing diseases in children. In districts with low vaccination coverage, compared to caregivers of under-vaccinated children, those with fully vaccinated children were more likely to encourage vaccination, perceive religious compatibility with vaccination, feel respected by the vaccination staff, and prefer to receive immunization information from health facilities. In multivariable analysis, caregivers who perceived a high level of community engagement were nearly three times more likely to express high vaccination confidence and nearly twice as likely to have children that were fully vaccinated compared to caregivers who perceived low level of engagement. In addition, we validated the use of a vaccination confidence scale for the first time in Sierra Leone, which to our knowledge is also the first validated vaccination confidence scale in a low-income country .
Dropout between vaccine doses is a major contributor of under-vaccination in LMICs  that propagates health inequities . In these settings, community engagement efforts that make caregivers feel that their communities are listened to in the planning of the childhood immunization program may boost confidence and motivate completion of the infant immunization schedule. Examples of community engagement activities in the Sierra Leonean context includes use of civil society organizations to engage in dialogue with community leaders to get them to serve as advocates for immunization in their communities; leveraging of religious leaders to promote the benefits of immunization through sermons; organizing community outreach activities jointly with CHWs and immunization staff to identify and vaccinate children who have missed scheduled vaccine doses, especially in geographically hard-to-reach areas .
Community engagement is increasingly acknowledged as a critical component to build vaccination confidence and achieve optimal population-level vaccination uptake, including for vaccines outside of the routine childhood immunization program such as COVID-19 vaccination. However, quantifiable evidence of the causal effects of community engagement to promote immunization remains limited globally [41–46]. A randomized control trial in India has been registered to examine the impact of community engagement on vaccination uptake (43), but results are not yet available. In addition, there is a published protocol for a systematic review on the use of community engagement to increase vaccination uptake in LMICs (42), but without any published results as of date. In the interim, our present results provide cross-sectional evidence showing that community engagement is strongly associated with vaccination confidence and vaccination uptake in Sierra Leone. Our findings are consistent with a descriptive observational study that showed increases in vaccination uptake following enhanced community engagement in Ethiopia (41). Although there is no one-size-fits-all approach to community engagement, the lack of standardized principles and approaches has made it challenging to rigorously evaluate and compare the effects of community engagement within and across LMICs. In 2020, UNICEF developed a set of minimum quality standards for community engagement that can be used to guide the development and evaluation of community engagement interventions . Future assessments of community engagement in immunization programs would benefit from adapting these standard indicators to enable cross-country comparisons of the effects of specific attributes of community engagement on immunization outcomes.
Constraints in accessing immunization services have been documented in urban areas in Sierra Leone  and other settings in sub-Saharan Africa . In our assessment, bringing additional vaccination sites closer to the community was recommended as the most important intervention area to improve childhood immunization in the community. One way to operationalize this recommendation is by conducting more frequent community outreach sessions to reach under-vaccinated children. A separate assessment in Sierra Leone uncovered that community outreach sessions were not conducted four times monthly as recommended by WHO for the African region, and that CHWs were underutilized in defaulter-tracking efforts . Enhancing the frequency and quality of the outreach visits and using community engagement principles may help improve vaccination confidence and uptake.
Despite governmental efforts to remove cost barriers for all caregivers through the Free Healthcare Initiative in Sierra Leone, our assessment showed that payments to health workers during vaccination visits is a common expectation among caregivers. While this practice may discourage caregivers who are unable to pay and compromise the trust in health workers [50,51], in our sample, we did not find any differences in vaccination uptake between caregivers who expected to provide informal payments versus those who did not expect to do so. A prior assessment in Sierra Leone found that informal payments may be precipitated by under-staffed health facilities needing to pay ‘volunteer’ health workers who are not formally on the government’s payroll . Involving communities in the planning of the immunization program and instituting mechanisms to sustainably get community feedback may contribute to ensuring accountability in service delivery, and therefore boost trust in and ownership of the services .
Our study has a few limitations. First, all measures were based on self-report. Implementation of various health promotion activities in these districts may have prompted socially desirable responding. Related, we do not have verifiable data on actual implementation of community engagement activities promoting childhood immunization in these districts. Finally, we cannot infer a causal relationship between the perceived level of community engagement and vaccination confidence and uptake in our cross-sectional study.
Our assessment identified a range of potential drivers of childhood immunization after a complex health emergency in four districts in Sierra Leone. We found that caregivers who perceived high level of community engagement were nearly three times more likely to express high vaccination confidence and their children were nearly twice as likely to be fully vaccinated. The exploratory cross-sectional evidence we have provided may inform future longitudinal assessments to investigate the potential causal effect of community engagement on vaccination confidence and uptake.
We acknowledge several colleagues from the U.S. Centers for Disease Control and Prevention for reviewing the manuscript: Dr. Abigail Shefer, Dr. Laura Conklin, and Dr. Heidi Soeters.
Disclaimer: The findings and conclusions in this paper are those of the authors and do not necessarily represent the official position of the U.S. Centers for Disease Control and Prevention.
Data availability: Available upon request to the corresponding author.