Designing and Evaluating a Health System Resilient to Extreme Weather Events in Rural Madagascar

Study population

Ifanadiana District is located in southeastern Madagascar, in a region identified as vulnerable to climate change [13]. The district is divided into 15 communes and 195 fokontany (the smallest administrative unit), with public health services overseen by the Bureau de Santé du District (BSD). The population of approximately 200,000 people is distributed across rural villages, although the population density is higher at the center of each commune and along the main paved road, which runs east to west across the district (Figure 1). Each commune is served by at least one primary health center (Centre de Base Niveau 2, CSB2), and six of the larger communes contain a second primary health center that provides more limited services (Centre de Base Niveau 1, CSB1) (Figure 1). Each fokontany contains a community health site, staffed by two community health workers, who provide integrated community case management for children under five years and limited maternal and reproductive health services [14]. The District hospital provides secondary care and is located in Ifanadiana, the district capital, with tertiary care provided via referral to the regional hospital 85 km away in the regional capital of Fianarantsoa.

The health conditions in the district are characterized by a high prevalence of infectious diseases and relatively low access to care, although coverage rates have risen since 2014 following the start of the HSSi [15]. A biannual longitudinal cohort study has been conducted since 2014 as part of the HSSi, enabling close monitoring of population health [16]. District‑wide prevalence of diarrheal disease in children under 5 ranged from 9 to 19% between 2014 and 2018, with certain fokontany experiencing prevalence rates as high as 70% [17]. Malaria is endemic in the district and the prevalence in children under 15 years ranged from 0 to 80% across fokontany in 2021 [18, 19]. The 2023 survey found that nearly 50% of the population who reported illness in the prior month sought care, up from 25% in 2016 [15, 20]. Financial and geographic barriers are the most reported reasons for not seeking care [20]. Given the lack of transport infrastructure and the poor quality of roads, more than three‑quarters of the population live further than a 1 hour walk from a CSB, and only 13% of the population lives within two hours of the District hospital [21, 22].

Health system strengthening intervention (HSSi)

Pivot, a non‑governmental organization (NGO), began a partnership with the Madagascar Ministry of Public Health (MoPH) to build a district‑level model health system in Ifanadiana District (Vatovavy) in 2014 via an HSSi. The partnership focuses on service delivery and integrated science, tackling issues of readiness, quality of care, and social protection across community, primary, and secondary care levels of the public health system [23]. The intervention aims to increase health system functioning and resilience to serve a population faced with high rates of poverty in the context of a changing climate [24, 25]. While the HSSi does not include programs specifically targeting climate change, mitigating climate impacts on health is one of its goals, given the population’s vulnerability to extreme weather events.

The HSSi was based on similar initiatives undertaken as part of the multi‑country Population Health and Implementation Training partnerships program [26], particularly the partnership in Rwanda [27]. Broadly, the intervention includes the six components of health system strengthening established by the World Health Organization [28]: service delivery; health workforce; information systems; medicines and supplies; financing; and governance. It focuses on the health district as the unit of study and employs multiple, continuous data streams to monitor and evaluate the intervention, drawing on an implementation science approach [29]. The HSSi differs from similar interventions, particularly in its approach to data and evaluation. In addition to operational data, a longitudinal cohort allows the collection and evaluation of population‑level metrics and a multidisciplinary research team is embedded within the organization to produce high‑quality science to both inform operations and contribute to broader scientific advancement. The HSSi began in 2014 in four communes of Ifanadiana and expanded to two more communes before 2017, encompassing one‑third of the district’s population, referred to as the historic‑HSSi zone in this study (Figure 1). It was gradually expanded across the rest of the district by 2021, corresponding to the recent‑HSSi zone (Figure 1). The whole district was covered by the HSSi beginning in July 2021, and began expanding into two new districts in 2024. Details on the exact programs of the HSSi are provided in Table 1.

Rapid, local adaptation is key to the HSSi. Service availability and readiness are addressed through context‑specific infrastructure, such as signal‑boosting telecommunications antennas in remote villages, and customized protocols to meet patient needs, such as the pilot of a proactive community care program [14]. In addition to the formal program design, local adaptation is also encouraged on an everyday basis through collective values that are established within the partnership, including “care for the patient,” “solidarity,” “curiosity”, “bias towards action,” and “embracing complexity” [30]. The collective values of the HSSi partnership are similar to the idea of the intangible software of a health system in complex adaptive system approaches in that it characterizes the relationships, communication practices, and norms [31–33]. To better respond to patient needs, the HSSi encourages the health district to be dynamic, enabled by a rigorous system of program monitoring and evaluation that tracks several hundred indicators of population health and health system functioning, as well as close partnerships with patients, care providers, and community leaders. For example, when data revealed significantly lower rates of maternal health service coverage in more geographically isolated communities, new programs were created to better engage community health workers (CHWs) and traditional birth attendants in maternal health [23]. In addition, maternal waiting homes were constructed to provide lodging for expectant mothers who live further from the CSB. The HSSi is, therefore, the combination of the resource and programmatic support relevant to the WHO HSS components and the more abstract set of collective values that enable rapid, local adaptation.

Analysis

We used mixed‑methods to document if and how the HSSi served as a form of climate‑change adaptation in the context of Cyclone Batsirai. We analyzed the difference in multiple health system indicators via time series analyses, used satellite imagery to assess the impact of Batsirai on transport infrastructure, and relied on archival records and targeted discussions to provide context and essential details about how the health system performed during that period. Archival documents from the time before and after landfall by Batsirai included meeting minutes from internal daily emergency meetings, daily and weekly reports on health system infrastructure collected by the MoPH and Pivot, daily reports of the needs of displaced persons at all temporary shelters across the district, and internal and external evaluation reports of the response. Co‑authors reviewed documents and jointly established a consensus of HSSi activities at that time.

We conducted statistical analyses of the monthly time series of select indicators using interrupted time series analysis, which account for underlying long‑term trends to allow for the identification of differences due to specific events, from January 2021 to December 2023 from the district health information system relevant to infectious disease and health system performance to compare the historic‑ and recent‑HSSi zones. These zones were determined by how long they had benefitted from the HSSi, which was implemented gradually across the district. The historic‑HSSi zone corresponds to the six communes benefitting from the HSSi before July 2021, and the recent‑HSSi zone corresponds to the nine communes where the HSSi began in July 2021 (Figure 1). We therefore considered two separate geographic zones of the district in the analysis to better compare the impact of the historic‑HSSi and the recent‑HSSi on the health system’s ability to adapt to and recover from the cyclone (Table 1, Figure 1).

We identified eight indicators of infectious disease and health system performance: reported malaria case rates; malaria rapid detection test (RDT) positivity rates; reported diarrheal disease case rates; the proportion of consultations diagnosed with diarrhea; outpatient consultation rates; referral rates; annual childhood routine vaccination coverage (first dose of diphtheria tetanus toxoid and pertussis (DTP3)); and the availability of UHC‑tracer medicines. Indicators related to malaria and diarrheal disease were chosen to represent infectious disease burdens. Consultation rates represent the number of outpatient consultations seen at a health clinic per 10,000 population per month. Referral rates are the number of patients referred to the District hospital per 10,000 population per month and are dependent on a functioning road transport network. UHC‑tracer medicine availability is the proportion of days per month that a health clinic reported having a nationally determined standard of essential medicines available (see Supplementary Table S2 for the full list). Monthly data on each indicator were collected by Pivot’s Monitoring and Evaluation team at the CSB level from January 2021 to December 2023. We applied a multi‑group interrupted time series analysis to each indicator via a generalized linear mixed‑model that controlled for long‑term and seasonal trends in the data. The HSSi zone, the cyclone period (defined as February and March 2022), and the interaction between the two were included as fixed effects, while the year, commune, and month of year were included as random effects. RDT positivity rates and the proportion of consultations diagnosed with diarrhea were modeled via Beta distributions with a log‑link. The availability of UHC‑tracer medicines was modeled via a Gaussian distribution and a log‑link. Childhood vaccination coverage was estimated as an annualized percentage, rounded to the nearest percent, and modeled via a negative binomial distribution and log‑link. All other indicators were transformed to a rate per 10,000 individuals and modeled via a negative binomial distribution and a log‑link. All models were fit via the glmmTMB package [34] and model residuals were tested for uniformity and over‑dispersion via the DHARMa package [35] in R v. 4.4.0 [36]. Rate ratios (e.g. the difference in an indicator between two groups) were calculated for each indicator to facilitate comparison across indicators.

Because Cyclone Batsirai arrived during an ongoing HSSi, we undertook a supplementary analysis using longer term data to explore whether Cyclone Batsirai disrupted multi‑year trends and negatively impacted the ability of the HSSi to improve the health system. This included the indicators mentioned above, with the addition of consultation rates of CHWs and severe malnutrition case intake rates at CSB2s, collected monthly from January 2017 to December 2023. The interrupted time series analysis included time in months since January 2017, the HSSi zone, a binary variable representing the post‑cyclone period, months since the arrival of Batsirai (February 2022), the time since the intervention in the recent‑HSSi zone, the interaction between time and the HSSi zone, and the interaction between the post‑cyclone period and the HSSI zone as fixed effects. We included commune and month of the year as random effects and an AR(1) autoregressive temporal model for each commune to control for temporal autocorrelation in the data. Because the recent intervention began only eight months before the arrival of Batsirai, we were unable to include an interaction between the HSSi zone and the months since Batsirai without introducing singularity into the model. The results of these models are therefore primarily to explore how Batsirai disrupted long‑term trends in each HSSi zone, not to statistically test the difference in this disruption between the two zones. Indicators were transformed and fit as described above for the short‑term analysis.

We used satellite imagery to identify areas impacted by cyclone‑related flooding, particularly the surrounding road networks between CSBs and the district capital, where the District health office is based. We collected Sentinel‑1 radar imagery at the Analysis Ready Data level from the nearest satellite passage dates before (January 27, 2022) and after (February 8, 2022) Cyclone Batsirai. Analysis Ready imagery was accessed at Digital Earth Africa (https://www.digitalearthafrica.org/) and included radiometric terrain correction and normalization of backscatter values. We calculated the normalized difference ratio between the two radar images on the VV and VH polarization independently and applied a threshold of −0.75 to identify pixels that were flooded in the post‑cyclone image, following Johary et al. [37]. We combined this information with a road network of Ifanadiana collected as part of a community cartography effort that contains more than 20,000 km of roads and footpaths in the district [21]. We identified the optimal route between each CSB and the district capital along roads that are passable by motorcycle or car, which represents the route used to distribute medicines and supplies and refer patients to the District hospital. We then noted sections of each route that were within 50 m or 100 m of flooded areas to identify disruptions to the transportation network post‑cyclone.

Use of aggregate monthly consultation counts for this study was authorized by the Madagascar Ministry of Health and the Regional Health Direction of Vatovavy Region. It was deemed non‑human subject research by Harvard University’s Institutional Review Board.

Health system preparations

Preparation for the arrival of Batsirai began a week before its predicted arrival, and included all levels of the health system. A multi‑sectoral meeting was held in Antananarivo, the national capital, between partners at the MoPH, the National Office for Risk and Disaster Management (Bureau Nationale de Gestion des Risques et des Catastrophes), internal technical and financial partners (e.g. UNICEF, Médecins sans Frontières), and local and regional NGOs, including Pivot. The goal of this meeting was to plan for the response and prepare financial and physical resources that might be required. A preexisting Memorandum of Understanding (which details the roles and responsibilities of the MoPH/Pivot collaboration), and relationships that developed from a history of regularly held planning meetings as part of the HSSi, ensured district representatives were active collaborators at the central and local levels. As part of this effort, an emergency stock of medicine established as part of the HSSi was drawn on to supply the mobile health clinics that would serve displaced residents of Ifanadiana. In addition, an alternative supply of medicine was available to limit stock disruptions at CSBs. In the period following the cyclone, CSBs witnessed an increase in the availability of UHC‑tracer medicines compared to the three‑year average (Figure 2, Table 2), suggesting this was a success.

Figure 2

Within the district, staff were mobilized at both the CSBs and MoPH and Pivot central offices to prepare for the response. Teams were formed following existing operational roles to leverage existing expertise and relationships and ensure a dynamic response. For example, to enable the distribution of over 1,000 food and hygiene kits before February 5, the supply team organized the purchase and creation of the kits, the transportation team organized vehicles for their distribution, and the M&E team closely tracked the arrival of displaced persons and kit distributions to plan for future deliveries. These teams were a part of Pivot’s operational workforce, in addition to MoPH staff, and had experience in organizing and improving supply chain management in partnership with MoPH staff. With a well‑trained workforce and a history of collaboration, the operational teams were able to rapidly adjust to this new responsibility. The distribution of emergency kits continued throughout the response, ultimately distributing 2,361 kits across the district.

Medical and operational staff were deployed to more isolated regions of the district prior to February 5, and remained there during the response to support community‑based responses. Not only did this ensure close monitoring of the populations’ needs and health, but the physical placement of personnel at each site enabled an approach that centered on, and was tailored to, the needs of the community and patients. Health staff accompanied village leaders to conduct education campaigns about the coming cyclone, distribute food kits, and encourage those in vulnerable housing to relocate to temporary shelters. This collaboration was enabled by long‑standing informal and formal relationships between local communities and health staff, encouraged by the HSSi’s collective values of “solidarity” and “care for the patient.” These preparations, particularly the organization of temporary shelters, were a success. Because of outreach to identified at‑risk households, families were supported to move to temporary lodging. As a result, despite homes being destroyed and landslides recorded, no deaths were directly attributed to Cyclone Batsirai in the Ifanadiana District.

Immediate health system response

The immediate response focused on supporting those who were directly impacted by the cyclone and ensuring that routine care was able to continue, particularly for infectious diseases such as malaria and diarrheal disease [38]. This was done via the deployment of mobile clinics, which provided daily, on‑site health and social consultations at temporary shelters and the quick re‑opening of CSBs. Mobile clinics were staffed by MoPH clinicians, and medical supplies were drawn from standard MoPH stock and the HSSi emergency fund. Cases were reported to the District office daily via text message and eventually integrated into the standard monthly reports of each CSB. This was facilitated by the HSSi’s focus on communication, which included the establishment of alternative channels of communication via satellite phones and the creation of ad hoc committees and response teams. There were outbreaks of malaria in two communes, but these were quickly reported to the District office during the daily report, and emergency teams were deployed to respond. Neither malaria nor diarrheal disease saw a statistically significant increase in reported case rates or positivity rates following the cyclone (Table 2, Figure 2).

Heavy rains and winds significantly disrupted road accessibility across the district, which hampered the distribution of supplies and the movement of patients. Radar imagery revealed that the primary transportation routes linking CSBs to the District hospital remained impacted by cyclone‑related flooding on February 8, three days post‑Batsirai, and internal reports cite that only 5 of 15 communes were accessible on this date (Figure 3). The primary paved road in the district was severed on the route towards the regional capital of Fianarantsoa in Vohiparara, cutting off the region from road transport with the rest of the country for several days (Figures 1A and 3). A relay ambulance system, similar to that already in use in the HSSi program, was established to connect ambulance routes on either side of this roadblock. This system served to deliver supplies and medical workers to the district and to deliver patients in need of tertiary care to the hospital. A “work‑for‑food” program was put in place to re‑establish key transport routes that had been destroyed, including removing trees from roads and rebuilding a bridge (Figure 1B). This program was unique in that it brought together partners who traditionally did not collaborate, including the military police, the local government, and local actors working on a range of issues including biodiversity protection, education, and health. This collaboration was emblematic of the sense of solidarity felt among partners at this time of crisis. Although technically outside the scope of health service delivery, repairing the road network immediately removed a significant barrier to accessing health facilities, with positive effects on consultation rates, and contributed to food security during this time. Both zones witnessed an increase in consultation rates following Batsirai, relative to long‑term trends (Table 2, Figure 2).

Figure 3

Health facility infrastructure was similarly impacted. At the District hospital, the temporary ward that was constructed for COVID‑19 patients was blown away. Three months after the outbreak of the Omicron variant in nearby South Africa, this represented a dangerous confluence of crises. The loss of roofs and solar panels led to the loss of electricity and consequent loss of vaccination cold chains at three CSBs. Although clinic heads worked with communities to move refrigerators to more secure locations in private houses, vaccination coverage at CSBs decreased in the period following the cyclone (Table 2, Figure 2). This decrease may have been due to the destruction of the stock following the cyclone or the lack of reinforcement of vaccination programs at that time. Vaccination data are also of particularly low quality during this time period, with 20 of 21 CSBs in the district missing data on polio vaccination coverage in February 2022, suggesting preventative care programs, such as vaccination, were particularly impacted by Cyclone Batsirai.

Long‑term impact

Cyclone Batsirai arrived in the midst of an ongoing HSSi, and there was concern among partners that it would disrupt the intervention itself, rolling back advances that had been made since 2014 [15]. In fact, a sense of ownership of the success of the HSSi, and an associated responsibility for it, was one of the motivations cited for such a strong disaster response by personnel. Our supplementary analysis found little evidence for a significant impact of Batsirai on long‑term trends in indicators (Figure 4, Supplementary Table S1). Of the indicators with the most positive trends pre‑Batsirai—notably consultation rates, malaria case rates, and referral rates—only the increase in consultation rates slowed post‑February 2022. However, it is difficult to statistically disentangle this from the expected plateauing of consultation rates over time (Figure 4). Those indicators which were most impacted in the short term by Batsirai, particularly vaccination coverage and the availability of UHC‑tracer medications, have modestly increased in the years since Batsirai, suggesting they are not just recovering from the disruption but continuing to improve at rates seen pre‑Batsirai (Supplementary Table S1, Figure 4).

Figure 4

Cyclones can impact food security in both the short and long term by destroying crops, disrupting supply chains, and increasing prices [39]. To manage short‑term food insecurity, food kits were distributed at temporary sites and within communities before and immediately following the cyclone. Very few households in Ifanadiana have refrigerators or the capacity to store food, making them extremely vulnerable to disruptions to food supplies. While the distribution of food kits may fall outside of a conventional health sector activity, it was recognized as an immediate need of the population to prevent a larger health crisis, and HSSi programs were able to adapt to serve this need. The distribution of food kits in the weeks following Batsirai helped reduce the risk of immediate starvation after the cyclone. However, in some communes, nearly 90% of rice and manioc fields, the population’s primary food sources, were destroyed by Batsirai, increasing the risk of long‑term food insecurity. Madagascar was already in the midst of a malnutrition crisis [40], and Batsirai further aggravated this. Severe malnutrition case intake of children across the district increased from an average of 32 monthly admissions from January 2021 to January 2022 to an average of 92 from April 2022 to December 2023 (Figure 4, Supplementary Table S1).