Skip to content
ALL Metrics
-
Views
-
Downloads
Get PDF
Get XML
Cite
Export
Track
Research Article

Strengthening district health management and planning: an evaluation of a multi-country initiative in Eastern and Southern Africa

[version 1; peer review: 2 approved with reservations]
* Equal contributors
PUBLISHED 16 May 2024
Author details Author details

Abstract

Background

District health management and leadership capacity is increasingly recognised as critical to health system performance. District health managers play a central role in effective implementation and tailoring of national health policies, and they need adequate skills and support. The District Health System Strengthening Initiative (DHSSi) was implemented over 2019-22 in Malawi, Kenya, Tanzania and Uganda to strengthen sub-national capacity for evidence-based planning and a broader set of management competencies needed to operationalize district plans. To support learning and adaptation, a three-year evaluation was conducted to assess progress against outcomes and to inform ongoing implementation and future investment.

Methods

The evaluation used a mixed-methods theory-based approach. Evaluation questions were structured using criteria of relevance, coherence, efficiency, effectiveness, sustainability and impact, and corresponded to outputs, outcomes and impacts in the DHSSi Theory of Change. The evaluation was conducted over three annual rounds, and combined data from document review, observation, interviews and group discussions with cross-country, national and district stakeholders, and analysis of secondary quantitative data.

Results

Experience of DHSSi implementation indicated the relevance of supporting district management and planning, with gaps in current capacity, government interest, and indications of the potential to improve district practice. DHSSi experience also highlighted challenges to effective achievement of stronger capacity, and to translation of skills to improved management practice and, ultimately, to improved service delivery. A range of factors beyond skills affected district capacities, motivation and opportunities for effective planning and management, including areas such as funding, planning templates, health information systems, staff turnover, partner alignment and political structures.

Conclusions

Strengthening sub-national health management is an important agenda in efforts to expand high-quality primary health care. Future initiatives should prioritize detailed understanding of the range of influences on district management, and work to strengthen the enabling environment for effective district practice.

Keywords

Population health management, leadership, Eastern and Southern Africa, Primary Health Care, Health Systems Evaluation, UNICEF

Introduction

Primary health care (PHC) has once again become a rallying cry of the global health community, formalized in a PHC global declaration launched at the fortieth anniversary of the Alma-Ata declaration in late 20181. In Sub-Saharan Africa, efforts to ensure high quality PHC are a cornerstone for achieving SDG 3 - ensuring healthy lives and promoting wellbeing for all at all ages.

Lessons from the first PHC movement, efforts to achieve the Millennium Development Goals, and recent health emergencies such as COVID-19 underscore the importance of strengthening health systems to achieve PHC and SDG 3 goals2. Within the health systems strengthening agenda, there has been increasing focus on ensuring adequate capacity among sub-national health management teams to operationalize national PHC strategies so that they respond to local conditions and community needs. Indeed, health management, leadership and governance capacity at the sub-national level (referred to here as the district, county or equivalent administrative levels), where health policies are implemented and plans are operationalized, is widely cited as critical to health system performance37.

Concepts of sub-national management, leadership and governance overlap and the same individuals – in this case health managers - often play roles in all three areas4,811. While leadership relates to vision and priorities, management involves marshalling resources to achieve these ends, and governance considers how rules and their enforcement distribute responsibilities and shape interaction between actors9,12,13. Leadership is now widely acknowledged as a critical health management competency given the complexity of managing primary health care systems, marked by an array of fluctuating organizational, political, epidemiological and partnership dynamics, often with limited resources14,15. Not only must sub-national health managers have the technical know-how to plan, organize, control, direct and staff their teams; they must also be strategic, politically savvy and persuasive to get things done16,17. Decentralization, long in motion, continues to increase the importance of subnational leadership and management capacity in many countries as responsibility for planning, resource management, quality assurance, emergency response and citizen engagement is devolved18,19. Despite this increased responsibility, many health managers are clinicians with little exposure to health management training either through pre-service education or in-service programs4,9,20.

Many efforts to strengthen sub-national health management and leadership in low- and middle- income contexts during earlier PHC strengthening efforts were inadequate, short lived and ineffective21,22. Effective investment in this area is complex. As noted by Frenk, “probably the most complex challenge in health systems is to nurture persons who can develop the strategic vision, technical knowledge, political skills, and ethical orientation to lead the complex processes of policy formation and implementation23. To elevate the importance of this agenda, there have been calls for the professionalization of health management, to ensure health managers are adequately prepared and that management skills are valued2427. The process of professionalization involves working with state actors to develop and institutionalize management standards, training pathways and professional associations and a constituency that demands healthcare management28. Though this investment in health management as a specific set of skills and the reinforcement of these competencies is necessary, improving management practice also requires that managers operate in a supportive environment with systems and processes that facilitate and incentivize productive management action3,26.

The District Health System Strengthening Initiative was a US$10 million programme to strengthen sub-national health planning and management in four countries in Eastern and Southern Africa (Malawi, Kenya, Tanzania and Uganda), implemented between 2019 and 2022 with funding from the Bill & Melinda Gates Foundation (BMGF). Ministries of health (MoHs), local governments; UNICEF; the Foundation for Professional Development (FPD); country implementing partners1; worked to identify context appropriate pathways to strengthen and professionalize subnational health management, using annual planning and routine reviews as an entry point while building programs to develop wider management competencies29.

To support learning and adaptation, a three-year evaluation was conducted by Oxford Policy Management, to assess progress against outcomes and inform ongoing implementation and decisions on future investment and scale-up. In this article, we describe the intervention and context, lessons learned and implications for practice. We aim to contribute to the evidence on effective approaches to developing health management in sub-Saharan Africa, which is currently growing but limited9,25,30.

Intervention & context

DHSSi developed one over-arching, broadly defined theory of change (ToC) (Figure 1) that was used as a frame of reference to support country-specific program design. The ToC reflected a traditional program theory pipeline logic model specifying intervention activity areas, outputs, outcomes and impact31. UNICEF country offices (COs) and Ministries of Health (MoH) used the ToC to develop country implementation plans, which varied according to context and implementation strategy. Five counties in Kenya, 5 districts in Malawi, 4 districts in Tanzania and 10 districts in Uganda were targeted for sub-national planning and management capacity building. District selection varied by country according to MoH and UNICEF priorities.

a5a78ed0-ab04-498c-931a-07a9969181ea_figure1.gif

Figure 1. DHSSi Theory of Change.

Health performance in Malawi, Kenya, Tanzania and Uganda varies, but significant further progress is needed to meet SDG 3 in each country (Table 1). Health financing environments also differ between the countries, with external aid still constituting a significant portion of health budgets in some contexts (Table 1). According to national policy, each country has decentralized their health system, such that operationalization of policy and management of performance takes place at the sub-national level. However, the degree of sub-national autonomy for key functions such as prioritization, financing, staffing and supply procurement varies by country (Table 1).

Table 1. Population, health and health financing context in Kenya, Malawi, Tanzania and Uganda.

KenyaMalawiTanzaniaUganda
Population & healthPopulation (millions) (2021)*53.00619.8963.58845.854
Universal Health Coverage Service Index ˟56484650
Under-five mortality rate (2021) ´
(SDG 3 country target = 25)
37.241.947.142.1
Maternal mortality ratio (2020) ˇ
(SDG 3 global target = 70)
530381238284
Life expectancy (years) (2020)*63646663
Health financingCurrent health expenditure (CHE) per capita in US$ (2020)°83333934
Primary health care expenditure per capita in US$ (2020)°55211822
Primary health care expenditure as % of CHE (2020)°64694565
Domestic General Government Health Expenditure (2020) as percent of CHE°47364317
External Health Expenditure (Aid etc) (2020) as percent of CHE°18363341
DecentralizationNumber of sub-national units 47 counties29
districts
184
districts
136
districts
Timing of decentralization policyDevolution in 2013Health decentralized in 2005First policy in 1972 and advanced in 1990sDecentralizat-ion formalized in 1997
Extent of functional decentralization in health sectorRapid devolution with significant functional autonomy at county-level and evolving capacity.Slow process, with more devolution in 2019/20. Some key functions such as H.R. and supplies still largely centralized.Devolution approach, but still significantly dependent on centralized resources and HR processes for skilled health workers.Rapid proliferation of districts from 45 to 136. Center maintains some control through conditional grants and supply management.

Sources:

*United Nations Population Division. World Population Prospects: 2022 Revision

˟ World Health Organization. Global Health Observatory. Accessed at: https://www.who.int/data/gho/data/indicators/indicator-details/GHO/uhc-index-of-service-coverage

´ Estimates generated by the UN Inter-agency Group for Child Mortality Estimation (UN IGME) in 2023, accessed at http://data.unicef.org

ˇ Trends in maternal mortality 2000-2020: Estimates by WHO, UNICEF, UNFPA, World Bank Group and UNDESA/Population Division. Geneva: World Health Organization; 2023. Licence: CC BY-NC-SA 3.0 IGO.

° World Health Organization. Global Health Expenditure Database. Accessed at: https://apps.who.int/nha/database/ViewData/Indicators/en

In each country, UNICEF had, to varying degrees, supported some prior sub-national planning capacity development. In Uganda, UNICEF had worked with the MoH on subnational planning since 2011 and conducted a randomized control trial on an initial iteration of the approach that demonstrated positive impact32. In Malawi, UNICEF had worked intermittently with the Ministry of Health and Population on sub-national planning since 2012. In Kenya, UNICEF and county governments began experimenting with aspects of this approach in following devolution in 2013 and in Tanzania the work was more nascent, with minimal engagement prior to DHSSi.

DHSSi expanded upon UNICEF’s historical district health system strengthening approach33, which focuses on improving evidence-based planning (EBP) and monitoring. This approach employs a continuous improvement cycle (diagnose, intervene, verify and adjust) drawn from management theory and models. For the diagnose and verify steps, a bottleneck analysis (BNA) approach coupled with a ‘five whys’ causality analysis is employed to identify and prioritize the most significant health system bottlenecks in a specific sub-national jurisdiction. The emergent bottlenecks are monitored to determine whether they are reduced through district action, which is formalized in annual operational plans34. DHSSi worked to institutionalize this approach through the use of routine health information captured in the DHIS2 (health information system platform) and other health information systems. This was UNICEF’s first significant attempt to rely exclusively on routine information for bottleneck analysis, rather than applying rapid surveys. At national level, institutionalization was supported through inclusion of the EBP process in national health planning guidelines; integration of a BNA application in DHIS2 to expedite analysis through data synthesis and visualization; and the development of training packages and cultivation of national and district-level trainers on the EBP approach. This was complemented by targeted work to support EBP in selected districts including workshops to build district capacity and facilitate EBP as part of annual planning, and support for District Health Management Teams (DHMTs) to review annual plans and engage partners (these subnational teams are known as District Health Teams in Uganda, Council Health Management Teams in Tanzania and County Health Management Teams in Kenya, but we use the acronym DHMT throughout for simplicity).

Under DHSSi, the governments’ planning process was leveraged as an entry point for engagement, with two other major components added. First, DHSSi aimed to expand the remit of capacity building of sub-national health teams to address a broader set of management competencies required to meaningfully operationalize health plans, such as leadership, stakeholder engagement, financial and human management. To do so sustainably, UNICEF worked with national-level governments to root this in broader professionalization of health management agendas. This involved working with MoHs to confirm management strengthening as a priority and identify appropriate stewards for this role within MoHs; and then collaborating with the MoH on activities such as establishing cross-departmental, cross-ministry and partner task forces on health management professionalization; determining requisite management competencies linked to job expectations through needs assessment; identifying the best ways to nurture these competencies, including via mapping existing management capacity building initiatives; and developing training curricula and materials, with specific activities varying between DHSSi countries. In Malawi and Tanzania, this resulted in the development and roll out of a training course for sub-national health management teams and in Malawi, some experimentation with schemes to reinforce learning through follow-up coaching, though this work was nascent. Second, DHSSi aimed to address localized or systemic barriers to the adoption and use of good management practices and behaviors related to the political economy of such actions. Factors originally hypothesized to affect management practice included: 1) the institutionalization of effective practices in government systems35; 2) the availability of adequate decision space22,36,37; and 3) accountability, both vertical within the system, and social accountability with communities38. DHSSi conducted a problem-driven political economy analysis to further study the specific dynamics in Kenya, Malawi and Uganda (Tanzania declined participation) and generate potential solutions to limiting factors. However, time constraints mostly inhibited active implementation of new strategies in this area.

DHSSi viewed work to strengthen subnational management as a long-term agenda that requires significant time to contextualize, root and scale, and originally planned a longer intervention. DHSSi took place during the emergency phase of the COVID-19 pandemic globally and in Eastern and Southern Africa. This posed a major challenge to the timely implementation of all originally forecast activities as both MoH and UNICEF staff were diverted to support national COVID-19 emergency responses. The ongoing evaluation generated useful insight to support program adjustments during this changing context, and to inform future intervention design.

Methods

Ethics

The study was approved by national ethics review committees in all countries: Africa Medical Research Foundation (AMREF) Africa Ethics and Review Committee (P838-2020, Aug 11, 2020) and National Commission for Science Technology and Innovation (249062, Nov 9, 2020) in Kenya; Makerere University, School of Social Sciences Research & Ethics Committee (MAKSS REC 03.20.402, July 23, 2020) and the Uganda National Council for Science and Technology (SS613ES, Nov 23, 2020) in Uganda; the National Committee on Research in the Social Sciences and Humanities (NCST/RTT/2/6, Oct 19, 2020) in Malawi; the Commission for Science and Technology (2020-100-NA-2020-024, Feb 20, 2020) and the National Institute for Medical Research (NIMR/HQ/R.8a/Vol. IX/3522, Sept 28, 2020) in Tanzania; and Oxford Policy Management Ethical Review Committee (ERC-A4009, July 7, 2020). Approvals were renewed before commencing year two and three fieldwork.

Consent

All participants provided informed consent before commencing interviews or group discussions. For Malawi, Tanzania and Uganda, the ethics committee approved use of verbal consent because the research was low-risk, with questions focused on professional rather than personal issues and relatively senior participants. In Kenya, written consent was obtained in line with ethics committee preferences. Where consent was verbal, this was recorded by the evaluator on an interview notes template.

Evaluation design and scope

The DHSSi evaluation was co-designed by UNICEF and OPM. OPM implemented the evaluation, under the management of the Evaluation Section of UNICEF’s regional office, with technical input provided by an Evaluation Reference Group comprising UNICEF staff, government representatives and senior subject experts. The OPM evaluation team included national researchers in each DHSSi country and a central technical team of five (including KG and GA). All team members had a background in health research and evaluation, and all were trained to masters or doctoral level. The team included male and female researchers. Data were collected and analyzed annually, within interim reports and recommendations shared over the course of three years. Annual findings and recommendations were reviewed and discussed by DHSSi teams and contributed to updated annual implementation plans.

The evaluation used a mixed-methods theory-based approach. Evaluation questions were structured using the Organization for Economic Co-operation and Development (OECD) Development Assistance Committee (DAC) criteria of relevance, coherence, efficiency, effectiveness, sustainability and impact, and corresponded to outputs, outcomes and impacts the DHSSi ToC. Within the theory-based approach, DHSSI’s contribution was assessed by examining evidence for the presence of outputs and outcomes along the ToC chain, the processes through which DHSSi contributed to outcomes, and the conditions that affected progress, as well as stakeholder perceptions regarding the significance of and role played by DHSSi.

The evaluation focused on core DHSSi components of EBP in all countries, efforts to develop an agenda and curricula for management capacity building, and district management training where this was implemented; other activities under DHSSi – such as support for district staff recruitment in Uganda or research on community governance in Tanzania – were either minimally examined or not covered by the evaluation due to restrictions on scope. The evaluation covered DHSSi implementation from January 2019 up to around July 2022.

Evaluation methods

For each evaluation round, the OPM evaluation team collected data at national and district levels, covering all DHSSi districts in Kenya, Malawi and Tanzania, and three of ten DHSSi districts in Uganda. Methods comprised document review, interviews, group discussions, observation, and analysis of secondary quantitative data.

Document review provided information on DHSSi activities and district planning practice, and included DHSSi programme documents; district government documents, including annual district health workplans; and national documents such as policy guidelines and working group minutes. Document review was conducted by the central evaluation team and country researchers.

Interview participants were selected purposively in discussion with UNICEF, based on involvement with DHSSi activities. National interviews included UNICEF staff, DHSSi implementing partners, and representatives from the MoH or other relevant ministries engaging with DHSSi. At district level, DHMT members were interviewed in three districts per country, usually a senior DHMT member (such as the District Health Officer), and a member responsible for maternal and child health. Additional interviews were conducted for some evaluation rounds in accordance with priority information needs agreed with UNICEF. For example, in year one, interviews also included national and district development partners or health NGOs, to understand coherence and stakeholder engagement in district planning, and members of the wider local government (the central district council or executive that oversees all sectors), to understand their views on relevance. In year three, additional interviews were conducted with the regional health management teams and zonal health resource centres in Tanzania and with zonal officers in Malawi, as DHSSi developed a focus on these regional/zonal structures for sustainability. At international level, interviews were conducted with UNICEF’s regional office, FPD, the BMGF (in year two) and the UNICEF headquarters (in year three) on areas such as learning and DHSSi alignment with organizational strategies. See Table 2 for the annual number and distribution of interviews. A small number of repeat interviews (2–3 a year) were conducted to provide additional information, primarily with UNICEF country offices or MoH staff who were heavily engaged in DHSSi. Interviews generally lasted 1–1.5 hours, with group discussions taking around 2.5–3 hours.

Table 2. Evaluation interview sample.

NationalDistrictCross-countryTotal
Year 12765193
Year 21824345
Year 32321347
Total681107185

Note: Table figures represent the number of interviews. Many interviews included more than one person, so the total number of interviewees is higher.

Group discussions were held with DHMT members (or planning team members in Malawi) in each district, to discuss district planning and management practice, experience of DHSSi activities, and in year three, any changes in prioritized bottlenecks.

Participants for interviews and group discussions were initially approached by email or phone, depending on their usual communication channels. A small number (around 1–2 people per year) declined to participate because they felt they had insufficient involvement with DHSSi to comment, and some participants were unavailable due to travel or other commitments; in these cases, alternative participants with similar positions were identified (for example, other DHMT members).

Interviews and group discussions followed a semi-structured topic guide, developed based on document review and evaluation questions. This guide was iteratively adapted in each country during data collection to build on emerging insights and experience with the questioning. Time was taken before each interview or group discussion to introduce the evaluation purpose and establish trust. Almost all group discussions took place in person, and interviews were remote or in-person depending on local COVID-19 guidelines and respondent preferences. In-person discussions were held in offices or quiet nearby locations, depending on participant preference. Only the participants and researchers were present. All sub-national interviews and group discussions and most national interviews were conducted by the evaluation team country researchers, with a small number of national interviews (particularly with UNICEF country offices) conducted by the central evaluation team or jointly between the central team and country researchers. Cross-country interviews were conducted by the central OPM evaluation team.

The year two and year three evaluation rounds included observation of one day of district planning or review activities, to provide additional insights into district practice and DHSSi implementation. Observation checklists were tailored to the specific activity, and detailed notes were taken during observation.

Detailed training and cross-team discussion were conducted in advance of each data collection round to ensure research tools were contextually relevant and to support consistent, high-quality data collection. This discussion included reflection on positionality and personal assumptions about likely effectiveness, to support openness in data collection.

Analysis

During data collection, the OPM evaluation team conducted regular debrief discussions to support data quality and build on emerging issues. Frequent themes and new insights were discussed during these debriefs to assess whether the data were adequate and to identify any key gaps for remaining data collection.

Interviews and group discussions were audio recorded and transcribed, and notes were taken during interviews and discussions to support ongoing reflection. Transcripts were then analysed using a framework analysis approach39, using Excel matrices based on the evaluation questions to summarize and combine data from document review, interviews, group discussions and observations. The framework matrix column structure equated to a typical qualitative analysis coding tree: columns corresponded to the overall evaluation questions and specific information needs under each question, for example, with columns on timing, reach, content and strength/sufficiency of DHSSi support under an evaluation question on efficiency. Columns were adapted based on emerging insights, and additional 'other' columns were used to record any data that did not fit a pre-identified theme. Detailed issues under each evaluation question were then identified based on the data for later writing. The framework analysis was conducted by two people in the central team and by country researchers for Kenya and Tanzania, with distribution of work varying between years.

To examine changes in district planning and management practice, the evaluation employed the Planning and Management Assessment Tool (PAMAT), a rubric developed by the evaluation team to score district planning and management practice in domains related to key practices targeted by DHSSi, including problem analysis and prioritisation, stakeholder engagement, review and implementation of plans. Scores were agreed by the evaluation team based on information from document review (particularly district annual plans and reports from district quarterly reviews), interviews and group discussions.

Quantitative methods involved analysis of routine indicator data to assess changes in bottlenecks prioritized by DHMTs each year, and assessment of change in coverage of priority health interventions based on nine core coverage indicators (such as ANC visits). This assessment of impact was designed primarily to inform results framework reporting and to serve as a baseline for future evaluation, because changes in coverage were not expected within the initial three-year DHSSi timeframe.

Transcripts were not returned to participants due to tight evaluation timeframes, as well as recognition of the complexities and potential disadvantages of this approach. Preliminary findings from each evaluation round were discussed with UNICEF, to ensure accuracy, obtain any missing information and refine recommendations, and evaluation reports were reviewed by the Evaluation Reference Group, which included the MoH in each country. In some countries, UNICEF also shared draft findings with implementing partners or DHMTs for additional feedback. A summary of annual findings was also made available to all participants each year.

Limitations

The evaluation faced constraints related to data availability and feasible scope of data collection. For example, quantitative analysis faced gaps and inconsistencies in DHIS2 data and in the data on prioritized bottlenecks, and document review was hindered by difficulties with complete and timely document provision. Observation provided important insights, helping to triangulate information in documents and stakeholder interviews, but was only done for one day per country, and in some cases, activity timing meant it was not possible to observe the most relevant activities. These limitations were considered in the analysis and reporting.

Results

Experience of DHSSi implementation indicated the relevance of supporting district management and planning capacities. However, DHSSi experience also highlighted challenges to effective implementation, including achievement of stronger capacity, and translation of skills to improved management practice and, ultimately, to service delivery improvement. We describe stakeholder views on the DHSSi activities, areas of progress and challenges encountered. We look first at the DHSSi components on broader management skills and the enabling environment, and then at the specific work to strengthen EBP.

Development of leadership and management capacity

Across countries, national and district governments saw work to develop subnational management skills as highly relevant. Good leadership and management were considered critical for effective health service delivery, and a gap, with staff in district management positions often lacking training in the management skills and knowledge needed to perform their roles.

  • It generated a lot of excitement because this is addressing an actual gap … if health managers do not have the skills they need to make appropriate decisions, then all the other inputs in health infrastructure, health products, technical human resources for health - all these may not amount to the outcome that we need, which right now is achieving universal health care. Kenya MoH 01

In some contexts, subnational management was considered an area with limited support from other donors, so UNICEF support was particularly welcome (for example, in Malawi). Where there were several existing management capacity building initiatives, UNICEF support for harmonization was considered valuable to enhance coordination; for example, in Kenya, DHSSi conducted a partner landscaping, established a Technical Working Group to bring together MoH departments and partners, and developed a harmonized healthcare management framework and curricula.

While activities to strengthen and professionalize district management capacity were valued in all contexts, each country experienced delays, and progress varied. In Kenya, Malawi and Tanzania, following initial landscaping and stakeholder engagement, management training needs assessments were undertaken and a training curriculum was developed. District management training was then implemented in Malawi during the second year of DHSSi, and in Tanzania during the final months of year three. There was also initial work on management coaching for DHMTs in Malawi, including training of coaches and visits to some districts. In Uganda, DHSSi initially focused on district recruitment in response to staffing gaps among the district health team and frontline health workers. A need to strengthen management training was only identified during year two, leaving limited time for implementation. In Kenya, delays in developing the harmonized curriculum meant training for county teams was not possible within the DHSSi project period.

Speed of progress of management professionalization and capacity development was affected by a range of issues. Some were internal, such as mixed performance among implementing partners and management capacity being a newer area of work for UNICEF that required significant initial investment in understanding country needs and engagement with new stakeholders. Other issues included changes in government focal points and limited government availability during the onset of COVID-19 (a particular challenge in Kenya), constraints on availability of other stakeholders whose input was needed to ensure a shared agenda, and variation in the initial status of government plans for management training. For example, in Malawi, the MoH had already identified a need to strengthen training for DHMTs and developed plans for this; DHSSi built on this existing agenda, which led to a high degree of MoH engagement and faster progress.

Although time for implementation of management capacity building was limited, early experience in Malawi indicated potential to strengthen district practice. DHMT members in each focus district reported improvements in DHMT practice following leadership and management training and coaching visits, for example in areas such as DHMT teamwork and coordination, supervision, and relationships with clinical teams. Experience in Malawi also indicated the importance of continued support following training for effective and sustained improvement: all DHMTs indicated areas where they hoped to make changes but where little progress was made, or areas where initial improvements were not sustained. DHMTs saw the coaching and supervision visits as important to motivate continued action: “when you know that supervisors are coming, you try to do something” (Malawi DHMT 01). Coaching visits were, however, paused following initial implementation, as UNICEF, government and partners considered that further work was needed to define an appropriate coaching approach. There were several areas of uncertainty, for example regarding the most appropriate personnel to undertake coaching, considering skills, credibility with DHMTs and time available; the appropriate format for visits and balance between technical mentoring and broader coaching; and systems that could be sustained by government, considering the resources needed for further training and coaching visits.

Experience with management capacity building in Malawi also highlighted the influence of the enabling environment, wider systems and other initiatives. For example, some changes that DHMTs hoped to make depended on action by the district council or national government. Other initiatives also supported or incentivized progress, including assessment of relevant practice through the Local Authority Performance Assessments (which affect council funding and assess areas such as DHMT supervision of health facilities), and other partner initiatives on management capacity, which contributed to skills development.

Linked with these wider systems and the enabling environment, a political economy analysis (PEA) study was designed to generate insight on enablers and constraints to effective district practice and inform strategies to address barrier40. Delays with the PEA, in part due to COVID-19, hindered use of findings to design or adapt DHSSi activities within the implementation period. The PEA did, however, contribute to increased awareness and thinking within Ministries and UNICEF about areas for future action, for example, on ways to strengthen district health financing and public financial management.

Strengthening capacity for Evidence-Based Planning

DHSSi work on EBP responded to the importance of effective annual planning and review for district management, including use of evidence. As with the work on broader management capacity, national and district governments saw DHSSi support for EBP as relevant to address gaps in district planning, in particular strengthening skills and knowledge for data use and implementation of national planning guidelines; increasing awareness of the importance of planning, and providing funding for district planning processes (including development of plans, quarterly reviews and stakeholder engagement meetings).

UNICEF’s bottleneck and causality analysis methodology for EBP was applied to varying extents in all districts and seen as enabling more effective identification and prioritisation of activities that responded to real needs. For example, DHMTs in Uganda described this method as increasing analysis and data use, contributing to a move away from planning based on routine activities and previous plans: it helped DHMTs to “think wider and broader” (Uganda DHMT01) and to “plan based on the priority of the district, not just what we think should be done” (Uganda DHMT02). The key contributions of DHSSi EBP support varied between districts, countries and years, depending on aspects such as the focus of DHSSi activities (for example, the level of support for stakeholder engagement and reviews varied between countries), and district priorities and gaps (for example, whether funding for planning workshops was available from other sources, and the extent to which skills and methods were new or already established).

While governments described significant improvement, the extent to which DHSSi brought a change in district plans varied between countries, years and districts, and in many cases, the degree of improvement seen in planning documents was more limited than suggested by DHMT respondents (as assessed through document review and use of the PAMAT). For example, there were continued gaps in quality and completion of plans, and in some cases, plans deteriorated between years, for example due to a change in district planning staff. There were also gaps in application of certain EBP steps in some contexts, including limited use of BNA (for example, for just one intervention), and in some cases little integration of BNA within plans, with activities developed through EBP not clearly included in final plans and budgets.

Implementation of plans also remained a challenge. Lack of documentation meant implementation levels could not be rigorously assessed, and DHMT estimates of implementation levels sometimes varied widely within districts. However, DHMTs often estimated that less than 70% of planned activities were implemented, with estimates of less than 50% in several cases. Furthermore, where documentation was available, this often suggested lower implementation rates than those estimated by DHMTs.

Several factors affected progress on EBP, including DHSSi scope and processes, and district and national conditions. In relation to DHSSi activities, a range of issues were apparent. For example, late timing of DHSSi EBP workshops or insufficient follow-on support sometimes hindered use of the analysis for planning and budgeting, and limited reach of some EBP workshops meant only some of the people responsible for planning were familiar with BNA.

Beyond DHSSI, multiple other conditions at district and national levels affected progress in application of EBP. A persistent concern for national and district stakeholders was scope to apply EBP given challenges with current health information systems. Constraints included a lack of DHIS2 data for some programme areas, varied data quality, and difficulty in drawing data from different information systems (such as those for human resources and medicines and supplies), as well as gaps in DHMT access to DHIS2 and in DHIS2 functionality. BNA sometimes helped to highlight data gaps and prompt improvements; for example, in Tanzania, there were indications that BNA contributed to increased focus on data quality among district teams. There were also examples of working around data gaps, notably institutionalized use of a qualitative approach to bottleneck analysis when DHIS2 data were unavailable in Malawi. However, data gaps continued to hinder use of routine data for EBP and also delayed progress on configuring the BNA DHIS2 App. Several stakeholders considered that more work was needed to strengthen underlying health information systems before BNA using this routine data could be conducted effectively.

Fit of EBP, and planning more generally, with DHMT time and skills also proved challenging. Hectic workloads and staff turnover, combined with BNA being an unfamiliar approach that required significant training and support, reduced effective use of BNA. DHMT confidence with BNA and the wider EBP process grew through refresher training and reinforcement over time, but staff turnover hindered cumulative and sustained skills development. Time-consuming and in some cases complicated government planning processes and tools were also a challenge given DHMT time constraints and varied DHMT familiarity with planning, contributing to incomplete and error-prone plans. For example, some planning templates required large amounts of background data, or suitable IT hardware, software and computer skills. Government planning templates also affected integration of the EBP analysis within the plan, as templates for some countries did not include sections for all EBP steps.

Inclusion of activities identified through EBP (and of other planned priorities) in the final district activity plan and budget was also affected by alignment of planning and budgeting systems. In Tanzania, district government budgets are based on the Consolidated Council Health Plan (CCHP) and submitted through the same system, so when solutions from EBP are included in the plan, they are also in the budget. Elsewhere, there were gaps in alignment due to different systems for submitting the plan and budget, and regular delays that meant plans were finalized too late to inform budget submissions. In addition, particularly in Malawi, plans included a long list of activities without secured funding (included to demonstrate need with the aim of attracting partner funds during the year), with widespread comments that district government funds did not cover all high priority activities.

Alignment of partner funding also affected inclusion of activities from EBP and other district priorities in the final district plan and budget. Partner flexibility to follow district plans and priorities varied between districts and partners, but was often considered inadequate by DHMTs. This contributed to partner-funded activities that DHMTs saw as lower priority being included in district plans, and lack of funding for some district priorities. DHMTs considered that more partner involvement to jointly determine problems and prioritize solutions would strengthen EBP and other analysis for planning and encourage alignment of partner funds. DHSSi sought to increase partner involvement, particularly in Malawi and Uganda via partner mapping and stakeholder meetings. However, the effects were limited by lack of DHSSi funding for these activities in some years, by partners not attending workshops or returning mapping templates, and by partner funding often being decided by national rather than district staff. Recognizing the latter, there was preliminary work at national level to support partner alignment in some countries. Several DHMTs also saw EBP as helping to mobilize partner funding by providing evidence of the rationale for proposed activities.

National guidelines and associated district decision space also affected inclusion of activities identified through BNA in the final activity plan and budget. This was a particular issue in Tanzania, where scoring criteria for CCHPs and set ceilings for different budget components affected prioritization. Some DHMTs saw these guidelines and criteria as reducing scope to follow district priorities, and as potentially discouraging use of EBP. Decision space was also affected by political interests, such as the influence of county political structures in Kenya. Perceived flexibility to work within national guidelines or to negotiate with political interests varied between districts.

Many of these factors also affected implementation of plans, including insufficient alignment of partner funds and limited government budgets. Implementation was often hindered by insufficient or delayed government and partner funding, as well as rising costs, a particular issue in 2022 due to high inflation. Insufficient funding in turn reduced motivation to invest time in EBP:

  • We knew for sure that whatever we are going to put in the plan, it won't be funded. So, it's like we are just wasting our time. (Malawi district planning team).

Implementation was also affected by the degree of DHMT focus on the plan during the year. This varied between countries. For example, in Tanzania and Kenya, some DHMTs indicated accountability for implementation, with the staff responsible required to report on planned activities, and assessment of implementation in annual contracts or performance appraisal. In contrast, DHMTs in Malawi reported insufficient awareness of the final plan among programme staff responsible for implementation, and lack of attention to the plan in regular DHMT budgeting and decision-making. This was due in part to a hectic and unstable district context, which led to ‘firefighting’ rather than following plans, low district government budgets that primarily covered administrative costs with little room for planned service delivery activities, and financial reporting systems that were not linked to the plan. DHSSi sought to strengthen attention to the plan and monitoring of activities though support for quarterly reviews, and in Malawi, via development of a monitoring tool to track implementation. This support was welcomed by district teams as increasing focus on the plan and enabling identification of action to support implementation. However, the level of DHSSi support and funding for reviews varied between countries and districts, and was in some cases limited. As well as gaps in frequency, government quarterly reviews in some contexts (particularly Uganda) did not directly review the plan, instead focusing on performance of key indicators related to results-based funding. To strengthen implementation and so the relevance of developing plans, governments suggested greater focus on monitoring and review in future.

A further influence on both development and implementation of plans was the level of DHMT motivation to invest time in EBP. Motivation was reduced by limited funds to implement plans and also by the disconnect between planning and funding systems; for example in Uganda, where district budgets did not require development of plans. In contrast, motivation was sometimes increased by positive experience of EBP helping to justify activities and so secure partner funding, and by the perceived value of EBP for improving service delivery indicators that affected national performance assessments, notably the District League Table in Uganda.

DHMT leadership was also important for development of plans, partner engagement, and implementation. For example, there were cases where senior DHMT members encouraged a focus on planning, and where limited interest or understanding among senior DHMT managers reduced effort or opportunities to plan. Strong leadership was also important to encourage partner attendance at planning meetings and to promote partner alignment with district priorities. There were also examples of effective leadership supporting implementation, for example by encouraging district staff to monitor planned activities, or by identifying efficiencies or raising additional funds to address resource constraints. The significance of leadership supports the expectation within the DHSSi ToC that effective leadership and management can facilitate implementation of plans, but with further work needed to strengthen leadership practice.

A range of further issues affected quality of planning and implementation, including capacity building and support from other partners, and COVID-19, which sometimes reduced engagement in planning meetings and often hindered implementation of district plans.

Sustainability and scalability

DHSSi aimed to institutionalize EBP and the professionalization of management within government systems, and to support scale up. In relation to institutionalization, the EBP method is included in national planning guidelines in Malawi, Tanzania and Uganda (and in draft guidelines in Kenya). However, further work is needed to ensure a coherent approach across relevant planning and budgeting guidance. For example, in Tanzania, BNA is part of CCHP guidelines, but assessment of CCHPs by regional and national government does not clearly consider use of BNA, and the focus on compliance with set national priorities and ceilings potentially discourages EBP, as noted above. Budgeting systems also need alignment, for example confirming links between plans and systems for government and result-based funding in Uganda.

The degree of government ownership varied between countries and DHSSi activities. For example, ownership for both EBP and management activities was high in Malawi, with clear MoH direction to secure further resources. However, in some contexts, there were concerns within government about insufficient MoH leadership and little evidence of MoH effort to support scale up. Government ownership of management activities was in general more evident than support for further work on EBP. Scale up of EBP and to a large extent management training were seen by government as requiring partner funds.

Some strategies for sustained support to EBP and management capacity were identified in all countries, and to varying extents implemented. Examples include training of trainer (ToT) approaches, plans for use of regional networks to support districts, and scale up through other UNICEF funding, including to other countries. Pre-service training and linking training with Continued Professional Development were also considered, both areas for future work. In part due to an initial expectation that DHSSi activities would take place over a longer timeframe, strategies for sustainability were not always discussed at the outset of DHSSi, particularly for EBP, and some steps were only initiated at a late stage.

At district level, some DHMTs were confident in their skills to continue EBP, but there are some skills gaps and staff turnover may reduce skills without systems for ongoing support. In addition, DHMT time for EBP has relied on DHSSi-funded workshops. Most DHMTs considered that they could find other sources of workshop funding, including from other partners, government or results-based financing, and some suggested conducting EBP through small meetings at the district health office to reduce the need for funds. However, some DHMTs emphasized that EBP would only be possible with further partner funding. Several DHMTs also noted that continued use of EBP requires DHMT motivation, with some pointing to the importance of leadership and demand from higher levels to motivate sustained EBP practice.

Discussion

Supporting sub-national health management is an important but challenging component of health system strengthening for PHC. DHSSi experience confirmed the relevance of work in this area, with gaps in management and leadership capacities across countries, government interest in investment, and indications of the potential to improve management skills, motivation and practice, a finding supported by other health management evaluations and reviews7,4143

DHSSi experience also suggests considerations for future efforts to strengthen subnational management. The findings highlight the critical importance of district and national conditions that affect district health management, both supporting and hindering effective practice. A range of factors beyond skills affected district capacities, motivation and opportunities for effective planning and for implementation of skills gained through management training. These included availability and timeliness of funding; simplicity and coherence of planning templates and guidelines; quality of health information systems; staff turnover and staffing levels; partner alignment; emergencies; decentralization arrangements; political structures; and the influence of other initiatives and systems designed to support effective management. Many of these factors have been identified in other research on efforts to strengthen district planning and management, including aspects such as uncertain resourcing, decentralization and decision space, staff turnover, high workloads, and a supportive working environment7,37,4446. The 2007 WHO Framework for Strengthening Health Leadership and Management also highlights the need for not just appropriate competencies, but also an adequate number of managers, functional support systems, and enabling working environments47.

The range of influences at individual, organizational and system levels and interdependencies between them underscores the complexity of work to strengthen management and the need for long-term, holistic approaches that extend beyond individual skills4850. The influence of this wider set of factors suggests that future initiatives should prioritize a focus on the enabling environment, working to address barriers and make the environment in which managers work more conducive to effective practice. While strengthening the enabling environment was one of three major components of the DHSSi initiative, in practice, this area received less attention, partly due to late timing of the PEA and also due to the complexity of many of these issues48. The need to consider district and national conditions and to address the enabling environment highlights the importance of thorough situation analysis. Effective contextualization is emphasized as a critical condition in other reviews of efforts to strengthen district management as well as health system strengthening initiatives more widely7,41,42,46,49. DHSSi undertook an initial landscaping exercise, but more extensive situation analysis, and earlier, more rapid PEA, could help to identify a broader range of factors and ensure interventions respond to various contextual considerations. Broader situation analysis could also enable more understanding of current strengths and weakness in district management and planning, and of ways that systems could be changed to mitigate constraints and enable effective performance47. In relation to planning, this might have meant, for example, more focus on understanding and tackling constraints in areas such as ownership of plans, funding, decision space, and time-consuming planning templates, rather than an initial focus on EBP training to improve data use47.

Situation analysis should consider areas such as government priorities; alignment with other partner initiatives; and the various government systems that affect district capacity and motivation for effective planning and management, such as systems for financing and performance management. Further development of support to subnational management should consider effective links with these wider systems and initiatives to promote coherence and identify opportunities for complementarity that can support sustainability and scale51. As well as frameworks for PEA and health system situation analysis, this situation analysis could draw on generic frameworks that highlight factors affecting implementation of new interventions. For example, the Consolidated Framework for Implementation Research is a widely used, overarching framework developed through review of evidence on effective implementation52. The Framework is designed to guide evaluations of complex interventions, but it can also be used prospectively to inform design51. Many of the identified factors are potentially relevant for national and district adoption of management practices, for example, perceived complexity, impact on work processes, evidence of effectiveness, and organizations’ absorptive capacity.

While long-term work to address systems factors is critical for effective subnational management performance, DHSSi experience also indicates the need for attention to more immediate aspects of programme design and implementation, such as timing of activities, their reach, selection of implementing partners, and adequate funding. Similarly, as well as identifying relevant contextual conditions through situation analysis, DHSSi experience indicates the need for programme design to fully respond to identified conditions, including action to address identified challenges (such as staff turnover) or make use of opportunities. These systems enablers and constraints could be considered within country-specific ToCs, including identifying action and rethinking programme design to address high-risk assumptions53. Experience from DHSSi implementation and the evaluation also indicated learning around the appropriate evaluation approach. Complex health system interventions often require iterative learning and adaptation40. Effective and efficient programme adaptation requires ongoing monitoring and evaluation to inform decision making and new directions, with learning embedded throughout54,55, and sufficient flexibility and resourcing to respond to emerging information needs and changing programme activities55.

The evaluation also raised areas where further research could inform future initiatives to strengthen district health systems. One area relates to examining the effects of increased district planning and management capacity on health system bottlenecks and service delivery. This could include using discussion and evidence to clarify expected pathways from stronger management practices to health system service delivery and coverage outcomes, linked with developing and evaluating detailed theories of change49. In the context of DHSSi, the evaluation provided preliminary information on change in prioritized bottlenecks and factors affecting this, but additional data collection and analysis would be needed, and some effects may take longer to materialize56.

Conclusion

Strengthening sub-national health management to enable the expansion of high-quality primary health care in Eastern and Southern Africa is highly relevant, but there is still much work to be done to elevate the importance of this agenda; determine appropriate institutional arrangements for its stewardship; and identify appropriate entry points and strategies for management strengthening, guided by country circumstances and opportunities. UNICEF and other health actors have an important role to play in elevating the importance of strong subnational management and supporting countries to design approaches to professionalize health management to achieve population health goals. The DHSSi experience underscored that entry points and strategies for health planning and management strengthening must be highly contextualized to local health systems and political dynamics, evolve over time and address enabling conditions. One noteworthy enabler is the availability of adequate financing for primary health care, an essential requirement for effective district performance57.

Comments on this article Comments (0)

Version 1
VERSION 1 PUBLISHED 16 May 2024
Comment
Author details Author details
Competing interests
Grant information
Copyright
Download
 
Export To
metrics
Views Downloads
Gates Open Research - -
PubMed Central
Data from PMC are received and updated monthly.
- -
Citations
CITE
how to cite this article
Rogers BM, Gooding K, Appleford G et al. Strengthening district health management and planning: an evaluation of a multi-country initiative in Eastern and Southern Africa [version 1; peer review: 2 approved with reservations]. Gates Open Res 2024, 8:38 (https://doi.org/10.12688/gatesopenres.15007.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.

Comments on this article Comments (0)

Version 1
VERSION 1 PUBLISHED 16 May 2024
Comment
Alongside their report, reviewers assign a status to the article:
Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

Are you a Gates-funded researcher?

If you are a previous or current Gates grant holder, sign up for information about developments, publishing and publications from Gates Open Research.

You must provide your first name
You must provide your last name
You must provide a valid email address
You must provide an institution.

Thank you!

We'll keep you updated on any major new updates to Gates Open Research

Sign In
If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password.

The email address should be the one you originally registered with F1000.

Email address not valid, please try again

You registered with F1000 via Google, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Google account password, please click here.

You registered with F1000 via Facebook, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Facebook account password, please click here.

Code not correct, please try again
Email us for further assistance.
Server error, please try again.