One-third of the global disease burden from HIV, tuberculosis and malaria, and some of the highest levels of maternal, newborn and child deaths are to be found in countries impacted by fragility, conflict and violence . Despite persistent conflict and poverty, substantial improvements in health outcomes have occurred in Afghanistan since 2001, including a 50% decrease in child mortality and a 54% decrease in maternal mortality [2,3]. However, the context has remained fragile due to escalating violence since 2014. Over 10 000 civilians were killed or injured due to conflict each year from 2015-2019, and the proportion of the population falling below the national basic-needs poverty rate grew from 37% in 2012 to 55% in 2017 [4–6].
This challenging context required an innovative approach to health service delivery. The “System Enhancement for Health Action in Transition” project (SEHAT) supported delivery of primary and curative care services at the primary and secondary level between 2013-18 through a contracting out model . Non-governmental organization service providers (SPs) were contracted by the Ministry of Public Health (MOPH) to deliver the MOPH’s Basic Package of Health Services (BPHS, which specifies essential primary care services to be delivered) and Essential Package of Hospital Services (EPHS, which specifies essential hospital services) [8,9]. This approach allowed for improved harmonization in service delivery activities among the many donor parties under the leadership of MOPH. SEHAT supported a substantial expansion in service delivery for many indicators, including a 45% increase in deliveries occurring in a health facility, a 176% increase in Caesarian sections, and a 44% increase in outpatient visits for child morbidity . However, contraceptive use and vaccination rates stagnated, and levels for these indicators were particularly low in conflict-affected provinces .
A series of consultations examining the barriers to health care delivery and lessons learned from the five years of SEHAT implementation were initiated, culminating in a Presidential Summit on Health Care in Afghanistan in 2017, attended by President Ashraf Ghani. While the successes of the existing system were noted, it was identified that an improved system was needed which would clearly define success in service delivery and monitor the progress of SPs against that definition. The creation of a payment system linking payments to more specific performance indicators was critical to this process. A new pay-for-performance model – called Sehatmandi – was introduced in 2018 in response to these lessons, funded by the Afghanistan Reconstruction Trust Fund and the World Bank . This paper describes the design of Sehatmandi and early results of implementation.
Design of Sehatmandi
Under SEHAT, SPs were supervised by the Grants and Contract Management Unit of the Ministry of Public Health, whose main role is centered on procurement of SP services and contract compliance. With Sehatmandi, responsibility for oversight of SPs instead shifted to a broader group within the MOPH with a clear mandate for performance management. This group is coordinated by the Performance Management Office (PMO). SP performance is monitored by the PMO according to a prespecified set of criteria defined as part of the Standard Operating Procedures. Performance on these indicators is summarized into a score that is assessed every quarter at the sub-national level and every six months at national level. SPs that fail to obtain a satisfactory score are subject to sanctions and potentially a loss of contract. This change in management approach has been described as a shift from “contract management” to “performance management”. A comparison of the SEHAT and Sehatmandi models is available in Table 1.
Table 1. Comparison of the design of the SEHAT and Sehatmandi health projects*
ARTF – Afghanistan Reconstruction Trust Fund, BPHS – Basic Package of Health Services, EPHS – Essential Package of Hospital Services, GFF – Global Financing Facility, HMIS – Health Management Information System, IDA – International Development Association, MOPH – Ministry of Public Health, NGO, non–governmental organization
*Differences between projects are highlighted in bold.
Three alternative models of strategic purchasing were considered for the design of Sehatmandi (Table 2). All models relied on the introduction of SP payments being linked to service delivery volume and service-specific tariffs. Under one model, tariffs would be set by MOPH and SP bidders would compete on the cost of a lump sum portion. In a second model, tariffs would be proposed by the SP bidders, with the lump sum portion capped at 30% of the total contract price. The final model option allocated a lump sum associated with a baseline level of services, and services delivered beyond that threshold would be paid based on the tariff. Considering the relative advantages and disadvantages, the first model was used. Under this model, the base tariff was set for each service according to the findings from either an external costing study (for eight services) or an internal MOPH evaluation (for three services) . Tariffs were then adjusted for each province to account for differences in the cost of service delivery by dividing the cost per capita of the SEHAT project contracts in each province by the cost per capita in the external costing study.
Table 2. Comparison of strategic purchasing options for health care system in Afghanistan
MOPH – Ministry of Public Health
Competitive contract bids submitted by potential SPs comprised a technical component (weighted at 70%) and a cost component (weighted at 30%). Under the cost component, SPs bid competitively only on the lump sum proportion of the contract. The objective of this approach was to give SPs greater control over their delivery models, to encourage them to consider the relationship between their expenditure and their performance, and to incentivize them to find efficient, locally responsive ways to deliver more services. Contracts were awarded to cover both BPHS and EPHS services on a per-province basis, for 31 out of 34 provinces (in the remaining three provinces, MOPH directly manages services).
After contracts were awarded, the lump sum payments to SPs were divided equally into installments which are disbursed regularly throughout the contract period. Pay-for-performance payments are calculated based on the number of services reported through the Health Management Information System (HMIS) and verified by a third-party monitor (TPM; see Table 3 for an example of how payments are calculated). HMIS reports are submitted on a monthly basis by each health facility, aggregated at the provincial level, and then submitted to the central MOPH HMIS team on a quarterly basis (details on the HMIS reporting system are available in Appendix S1 in the Online Supplementary Document). Every six months, the TPM takes a sample of health facilities in each province to verify reported service delivery data (see Appendix S2 in the Online Supplementary Document for information on HMIS data verification and accuracy). The count of services reported to HMIS is compared to the registers for all 11 pay-for-performance services, and then a sample of those services are further audited by contacting the patients to confirm whether they in fact received the services. This verification is then used to revise any payments to the SPs which have already been made on the basis of their self-reporting. In order to ensure spending on the Sehatmandi project did not outstrip the available resources, a cap was set for each service in each province for the maximum number of services that could be reimbursed. Caps were set based on historical trend data from household surveys and HMIS reports.
Table 3. Example of service provider payment calculation for pay-for-performance services using national base tariff
USD – United States dollars
RESULTS FROM FIRST YEAR OF IMPLEMENTATION
Trends in service delivery were evaluated using HMIS data from 2017-2019 (see Appendix S3 in the Online Supplementary Document for a description of the evaluation methodology). The volume of services delivered in 2019 (under Sehatmandi) compared to 2018 (under SEHAT) increased for 9 out of 10 payment indicators with data for both years (data for the 11th payment indicator, growth monitoring, was only collected in 2019; Table 4), although the growth in service volume varied widely (-4% to +49%). Furthermore, the rate of change in 2018-19 was higher than the rate of increase from 2017-18 for 8 out of 10 services (Figure 1, Table 4). A second comparison was made between Sehatmandi providers in the 31 contracted-out provinces compared to the three provinces where MOPH directly manages services. Since MOPH-managed providers were not receiving pay-for-performance payments, then it is plausible that differences in the rate of change from 2018-19 between these two types of SPs could be attributable to the introduction of the Sehatmandi model. The rate of change of service volume was higher in Sehatmandi provinces relative to MOPH-managed provinces for 8 out of 10 services (Table 5).
Table 4. Improvements in pay-for-performance services before and after the introduction of Sehatmandi among Sehatmandi service providers in contracted-out provinces*
n/a – not applicable
*Health management information system data submitted by service providers.
Figure 1. Improvements in pay-for-performance services before and after the introduction of Sehatmandi among Sehatmandi service providers in contracted-out provinces.
Table 5. Improvements in pay-for-performance services comparing Sehatmandi providers in 31 contracted-out provinces and MOPH-managed facilities in 3 provinces*
n/a – not applicable
*Health management information system data submitted by service providers.
The risk of SPs inflating reported service count data are mitigated by high penalties for inaccurate reporting backed by independent, random audits. In the first year of implementation, the TPM completed all data collection activities for both HMIS verifications and health service quality despite security challenges. HMIS data accuracy (as assessed by an “HMIS Verification Index”, which indicates the proportion of services reported by SPs to HMIS that are subsequently verified) during Sehatmandi’s first 9 months of operation found that 5 services had accuracy greater than 90%, and the remaining 4 services had accuracy above 75%. Comparisons of HMIS data in the pre- and post-Sehatmandi period also seem to be relatively unaffected by misreporting (see Table S1 in Appendix S2 in the Online Supplementary Document).
The introduction of pay-for-performance contracts did not appear to negatively impact the volume of non-payment indicator services delivered by the same SPs. Six non-payment services (selected to represent service delivery that is not financially incentivized by the 11 payment indicators) had a greater rate of change in the post-Sehatmandi period compared to pre-Sehatmandi (Figure 2, Table 6). This evidence suggests that the introduction of payment indicators has not harmed non-payment indicators, and in fact may have benefitted them.
Figure 2. Improvements in non pay-for-performance services among Sehatmandi providers in contracted-out provinces before and after the introduction of Sehatmandi.
Table 6. Improvements in non pay-for-performance services among Sehatmandi providers in contracted-out provinces before and after the introduction of Sehatmandi*
*Health management information system data submitted by service providers.
Finally, financial incentives to increase service volume does not appear to have led to a significant deterioration in quality of services (as measured by the Balanced Scorecard assessment [BSC]). The BSC measures indicators of service quality according to 5 domains (including client satisfaction, human resources, physical capacity, quality of provider interactions, and provider management) and has been consistently administered by a TPM across all SPs since 2004. These indicators are then compiled into a total score scaled from 1-100. The total score in 2018 was 59.3 and in 2020 was 58.5 [14,15]. Since scores between 2011 and 2017 had varied between 55.0 and 63.5, the findings suggest that there was no adverse effect on service quality.
It is important to note that the non-randomized implementation of the Sehatmandi program prevents an unqualified causal interpretation of the associations presented here. In theory, there could have been other changes that were correlated with the introduction of Sehatmandi and instead resulted in increased service delivery volume. Furthermore, little data are available on the accuracy of the HMIS reports prior to the introduction of Sehatmandi; yet data from 2019 shows accuracy to be reasonably high for most indicators. Despite these caveats, the consistency and magnitude of the observed association make a plausible case for the beneficial impact of Sehatmandi. Increases in service delivery volume were consistently seen when comparing pre- vs post-Sehatmandi trends, SP- vs MOPH-managed provinces, and for both payment and non-payment services. Furthermore, the magnitude of association was considerable (median increase in service volume of 10 percentage points). Taken together, the authors believe these results are worth strong consideration by policy makers globally.
The introduction of the pay-for-performance model has made it clear that a performance-based system of payments and a pre-specified set of metrics for supervision is feasible even within a conflict-affected setting. Indeed, it may have a number of advantages. Whereas decision-making had often been made previously based on anecdotal evidence and political considerations, the Standard Operating Procedures have helped to lessen these concerns. Since SPs know what criteria they will be evaluated against, they can concentrate their efforts on achieving those objectives. With payments linked to performance rather than simply budget reimbursement, SPs have the flexibility (and incentive) to adapt resources in response to the fragile, changing context.
The introduction of performance-based contracts has also stimulated increased learning opportunities for SPs to improve service delivery. The establishment of the PMO has resulted in increased supervision (all provinces were visited by PMO staff in the first six months of Sehatmandi) with opportunities to troubleshoot issues. MOPH has also convened SPs to engage in learning sessions where they discuss their experiences and learn from other providers. SPs are motivated to participate in these activities because there is a financial reward to be gained if their newly applied knowledge results in expanded service delivery.
SPs clearly modified their behavior in response to the pay-for-performance incentive. In fact, many SPs began incorporating pay-for-performance components into staff salaries. This has reportedly served to help reduce absenteeism. However, it is also important to note that health facility staff need to have their salaries protected from impacts on service volume that are outside of their control, such as interference by anti-government armed groups and mismanagement by SPs. As a result, the MOPH has mandated that only 20% of staff salaries may be conditioned on service delivery volume.
In order to further increase service delivery, SPs have developed numerous innovative strategies. Some have addressed supply-side constraints, such as extending the opening hours for clinics along with shift-working for staff. Others have sought to increase demand; some examples include paying bonuses to community health workers who refer patients for services and providing transport for women to and from Caesarian section operations. The flexibility of the performance-based payment system was meant to incentivize exactly this kind of creativity.
One challenge in the implementation of the project is ensuring adherence to the Standard Operating Procedures (SOPs) by MOPH staff. The SOPs are comprehensive in their coverage of performance issues, so adequate training is required to ensure that both members of the PMO and other departments within MOPH are aware of the protocols that should be applied to a given situation. Consistent and fair application of the SOPs is expected to improve trust and an effective working relationship between MOPH and SPs. It is therefore important for there to be oversight of MOPH itself to ensure it is adhering to the SOPs. MOPH and financing partners have agreed to introduce a new layer of performance reporting and to begin holding regular reviews to provide this oversight.
As seen in Figure 1, some services have not grown substantially after the introduction of Sehatmandi. For some of these services, such as vaccinations, it is possible that the tariffs are not high enough to cover the cost of service delivery or to incentivize additional investment. On the other hand, for certain services, service volumes have exceeded the payment caps set in the contract. Since demand is clearly higher than the caps for these services, it is important not to create a perverse incentive for SPs to constrain care. Tariffs must be iteratively reviewed to ensure adequate performance for all services.
While the results of the BSC indicate there was no negative impact of Sehatmandi on service quality, it also shows that service quality has not improved. A critical next step will be to drive enhancements in service quality. While SPs are penalized if service quality falls below a set threshold, the current financial structure does not incentivize quality to increase.
The available data – while subject to limitations – suggest that the Sehatmandi strategic purchasing model is associated with bolstered health service delivery in Afghanistan, despite high levels of poverty and conflict. The approach described here additionally relies on a performance management approach stewarded by MOPH with a well-defined SOP to ensure transparency and accountability. It is also driven by robust data collection and the use of data in decision-making. Other conflict-affected settings may wish to consider implementing a pay-for-performance model based on these findings.