Skip to content
ALL Metrics
-
Views
-
Downloads
Get PDF
Get XML
Cite
Export
Track
Research Article

Team- and individual-level motivation in complex primary care system change: A realist evaluation of the Salud Mesoamerica Initiative in El Salvador

[version 1; peer review: 2 approved with reservations]
PUBLISHED 31 Oct 2018
Author details Author details

Abstract

Background: We study the role of individual and team-level motivation in explaining large-scale primary care performance improvements in El Salvador, one of the top-performing countries in the Salud Mesoamerica Initiative.
Methods: Case study with outlier sampling of high-performing, community health teams in El Salvador. Design includes scoping review of literature, document review, non-participant observation, and qualitative analysis of in-depth interviews following a realist case study protocol.
Results: The interplay between program interventions and organizational, community and policy contexts trigger multi-level motivational mechanisms that operate in complex, dynamic fashion. Interventions like performance measurement and team-based, in-kind incentives foster motivation among individual members of high-performing teams, which may be moderated by working conditions, supervision practices, and by the stress exerted by the interventions themselves. Individuals report a strong sense of public service motivation and an overarching sense of commitment to the community they serve. At the interpersonal level, the linkage between performance measurement and in-kind incentives triggers a sense of collective efficacy and increases team motivation and improvement behaviors. The convening of learning forums and performance dialogue increases the stakes for high-performing teams, helps them make sense of performance data, and leads to performance information utilization for healthcare improvements. Closeness to communities creates strong emotional linkages among team members that further increases collective efficacy and social identity. Such changes in individuals, team, and organizational behaviors can contribute to improved delivery of primary care services and explain the gains in performance demonstrated by the program.
Conclusions: This case suggests that primary care systems that rely on multi-disciplinary teams for the provision of care can benefit from performance measurement and management interventions that leverage individual and team-level motivation. Realist evaluation can help prioritize policy-relevant research and enhance the design and evaluation of large-scale performance reforms in primary care systems in low- and middle-income settings.

Keywords

Realist evaluation, Primary Care Systems, Performance Measurement and Management Systems, Workforce Motivation, El Salvador, Salud Mesoamerica Initiative

Introduction

Calls have been made for acquiring a better understanding of the ways in which various policy and management interventions contribute to improving the performance of primary care delivery systems, particularly in low- and middle-income countries (LMIC) where such improvements might be necessary conditions for the achievement of universal health coverage in the age of the Sustainable Development Goals1,2. High-performing primary care systems can be not only the first point of contact for continuous, coordinated, comprehensive and people-centered health services, but can also provide critical preparedness and response to domestic and global public health threats3.

Despite growing interest in improving the performance of primary care systems, gaps in evidence and practice are particularly acute in the area of performance measurement and management (PMM) of primary care systems in LMIC4. In particular, the linkages between workforce motivation and performance improvement in LMIC primary care systems remain poorly understood. Recent global health research has favored studies that address the effectiveness of specific health interventions while insufficiently addressing the effects of cross-cutting, PMM interventions on organizational performance516. Furthermore, experimental studies infrequently address the role of context in moderating the motivational effects of PMM interventions in the public-sector in general1720, and in LMIC primary care systems in particular21,22.

This paper presents results from an ongoing realist evaluation of the Salud Mesoamerica Initiative (SMI). This is the first of a series of papers presenting results from such evaluation. Here, we report on the role of motivation on the performance of El Salvador’s primary care system.

Background

Performance refers to the results generated by an organization, and measured against its intended goals and targets. In the private sector, it can refer to profits, efficiency, effectiveness, quality, market-share, and customer satisfaction. In public sector organizations, the definition has shifted as the role of the State in the delivery of public services has evolved23. Governments’ focus has shifted from inputs and compliance with standards, to quantity and quality of outputs, productivity, and efficiency and, more recently, to outcomes and policy impacts23,24. PMM systems can inform the introduction of clinical, managerial, programmatic and policy changes and thus contribute to the production of socially valued outcomes such as equity and health gains4. Forty years of research on PMM systems in public and private organizations has shown that such systems can be effective, even though unintended or undesirable effects can also occur18,2532.

The literature on PMM interventions in primary care systems in LMICs was reviewed as part of the ongoing preparation of an evidence gap map, and major gaps were summarized in the evaluation protocol4. To address these gaps, one of the authors (WM) adapted a PMM framework4 originally developed by Pollitt to study organizational performance in the public sector33. The framework in Box 1 highlights the elements, inter-relations and inter-dependencies that are typical of public-sector PMM systems and that usually determine the production of health outcomes (or not).

Box 1. Performance measurement and management (PMM) system framework

The elements of a PMM system are: 1) An institutional and programmatic context in which health policies, programs and activities are implemented and in which organizations, actors and stakeholders interact; 2) A local, socio-economic context in which services are provided and where provider-community interactions occur; 3) One or more performance management interventions; 4) A process for measuring the results arising from the implementation of the policies, programs and activities referred to in 1 and 3 above; 5) A sense-making process that allows the transformation of raw performance data into performance information; 6) A process of dissemination of performance information among system actors and stakeholders with the intent of making it actionable; 7) The adoption of performance information (use, misuse or non-use) by system actors and stakeholders (a proximal outcome); 8) Implementation of planned action; and, 9) Other intermediate and distal outcomes derived from the implementation of clinical and/or managerial improvements including process and quality improvements; and population-level health and equity outcomes.

The relationship between system elements above, generate inter-dependencies that lead to non-linear effects, delays, and feedback. System actors’ agency and their interactions with the context are sources for such system behaviors and contribute to the PMM system’s short- and long-term outcomes. The production and accumulation of desirable social and health effects will require continuous repetition of the cycles of measurement, performance information adoption, and implementation for organizational learning and improvement to occur and, particularly, for the sustaining of performance improvements through time.

To categorize the various PMM interventions and outcomes of relevance to this evaluation, we also relied on an ongoing systematic review of the gaps in evidence in the literature4, and a taxonomy developed by the Cochrane Collaboration’s Effective Practice and Organization of Care (EPOC)34. Box 2 summarizes the categories of interventions and outcomes as per the PMM framework in use.

Box 2. Performance measurement and management intervention and outcomes

The three main categories of performance-inducing interventions are:

Implementation strategies, such as, in-service training; supervision; continuing education; clinical practice guidelines; reminders; clinical incident reporting; local opinion leaders; continuous quality improvement.

Accountability arrangements, including audit and feedback; public release of performance information; social accountability

Financial arrangements, such as performance-based financing (including supply-side results-based financing); pay for performance and other provider incentives (in-kind and financial)

Potential outcomes from the interventions above include:

Provider and managerial outputs and outcomes, such as workload; work morale; stress; burnout; staff turnover; sick leave; absenteeism

Patient outcomes, including health status outcomes; health behaviors; unintended patient outcomes

Organizational outcomes, such as quality of care and process improvements; adherence to recommended practice guidelines; patient satisfaction; perceived quality of care; workforce retention; changes in organizational culture; unintended organizational outcomes

Population health outputs and outcomes, such as utilization of services; coverage of interventions or services; access to primary care services; health equity effects; adverse health effects or harm; unintended population health outcomes

Social outcomes, like community participation; equity effects; unintended social outcomes

The Salud Mesoamerica Initiative (SMI)

SMI is a results-based, health system-strengthening initiative that provides funding to participating governments for the achievement of population-level outcomes in reproductive, maternal, neonatal, and child health35. Participating governments agree with the Inter-American Development Bank (IADB) to implementing up to three consecutive 24-month phases, aimed at achieving a series of progressively complex health targets. Country performance frameworks target the poorest, hardest-to-reach municipalities and have an initial focus on inputs and processes, while progressively prioritizing health outputs and outcomes.

Governments contribute domestic funds a-priori and SMI matches these contributions with grant financing. In the contract between SMI and the government, the former commits to reimbursing half of the initially invested domestic funds contingent on the achievement of the agreed-upon performance targets36. A supply-side financial incentive was established, according to which a government has to achieve 80% or more of the targets in its performance framework to be eligible for the reimbursement of half of the initial domestic contribution.

SMI’s Salvadorian phase 2 program include interventions that are common to all participating countries and, also, unique PMM interventions. Of relevance to this evaluation was the inclusion of an in-kind, team-based incentive program implemented between 2015 and 2017, and involving the 75 Community Health Teams (CHTs) participating in SMI.

Participating CHTs agreed to a set of eleven (11) maternal, neonatal and child health targets, linked to the performance framework agreed between SMI and the Ministry of Health (MOH). CHTs that succeeded in achieving 80% or more of their performance targets were eligible for an in-kind incentive from a list of pre-established goods valued at up to $1,000. The payment rule was based on a sliding scale, with the awards varying depending on the number of targets achieved.

A cluster-randomized impact evaluation was conducted to experimentally evaluate the effects of in-kind incentives on CHTs performance37. SMI collected household and facility survey data every six months. Results from the impact evaluation will be published separately by the IADB. In response to SMI’s measurement of CHT performance, the MOH increased its own internal processes for CHT performance verification and supervision. The collection and analysis of performance proxies became a continuous process, and the regular monthly meetings between regional MOH offices and CHTs became a routine source for data analysis, internal accountability, action planning, and course-correction.

Performance results from the four waves of impact evaluation data collection were disseminated every six months, among all CHTs participating independent of experimental assignment. Specially convened meetings were hosted by the MOH and attended by central- and regional-level officers from the MOH, CHT coordinators, and SMI experts.

Program performance is externally verified by the University of Washington’s Institute for Health Metrics and Evaluation (IHME) at baseline and before the finalization of each phase. IHME uses quantitative measures of performance obtained through household and facility surveys38. After each round of performance measurement, results are aggregated and disseminated in each country through dialogue workshops involving the government, IADB and IHME. Further information about SMI’s original design and theory of change can be found in the evaluation protocol39.

Having consistently achieved its performance targets, El Salvador’s SMI program has been a major success to date. As a result, the MOH has received financial incentives for $2,6 million in phases 1 and 2. Table 1 lists targets and the results obtained to date.

Table 1. El Salvador, performance framework and results to date.

IndicatorBaseline 2011
(% and
confidence
interval if
applicable)
Phase 1
target
(C.I.)
Phase-1
result
2014
Phase 1
Score
Phase 2
target
(C.I.)
Phase 2
result
2017
Phase 2
score
Number of Family and Specialized Community
Health Teams (CHT) established
3768590---
Number of families ascribed to Family CHTs14,68138,66159,4950.1---
Number of CHTs with required inputs for
antenatal care
48%
(35-62%)
6898%
(90-100%)
0.1---
% of CHTs with stocks of four family
planning methods (injectables, barriers, oral
contraceptives, and intrauterine devices)
19%
(10-31%)
6592%
(81-98%)
0.1---
% of CHTs with the required inputs for infant
care
36%
(24-50%)
5892,2%
(81-98%)
0.1---
% of CHTs with refrigerator or cool box for
proper vaccine storage
43%
(29-57%)
6578%
(65-89%)
0.1---
Introduction of a national policy for the
distribution of powdered micronutrients for
children aged 6–23 months
NoYesYes0.1---
Issuing a regulation for the adequate dosage
of therapeutic zinc for the treatment of diarrhea
in children of less than 5 years (20 mg of Zinc
during 10–14 days in each diarrheal episode).
NoYesYes0.1---
% of pregnant women recorded in the antenatal
registry that had antenatal care performed by a
medical doctor or a nurse before 12 weeks.
6777%64.9%
(59-71%)
0---
% of children less than 1 year registered in the
health information system in less than 8 days
after birth
516190.1%
(87-93%)
0.1---
% of children 6–23 months with hemoglobin
levels < 110g/L
47%
(44-50%)
N. A.--36.5%47.3%
(36-59%)
0.1
Modern contraceptive use53.9%
(51-57%)
N. A.--60.5%75%
(70-79%)
0.1
% of antenatal care visits with qualityN. A.N. A.--62.5%63.9%
(58-70%)
0.1
% of antenatal visits during the first trimesterN. A.N. A.58.7%
(51-66%)
62.5%63.9%
(58-70%)
0.1
Most recent delivery in the last 2 years, as
institutional delivery, and attended by skilled
personnel
85.7%
(82-88%)
N. A.--94.2%98.3%
(96-99%)
0.1
%Postpartum visit one week after delivery81.2%
(78-84%)
N. A.--91.6%62.6%
(53-71%)
0
% Children 12–23 months immunized with
measles vaccine, according to vaccination card
65.5%
(61-70%)
N. A.--73.6%91.1%
(83-95%)
0.1
% Children 12–59 months who received two
doses of anti-helminth medications during the
last year
36.2%
(33-39%)
N. A.--56.4%40.5%
(35-46%)
0
Mothers that provided oral rehydration salts
and zinc to their children in the last episode of
diarrhea
4.5%
(3-7%)
N. A.--24.4%33.7%
(21-49%)
0.1
Full birth plan at CHTs (second delivery)N. A.N. A.--70%86.3%
(82-90%)
0.1
Total score (A score of 0.8 or more triggers a
financial incentive or performance tranche for
the MOH)
---0.8--0.8

Source: IHME and IADB

Study setting

El Salvador is a low-income country in Central America. According to the World Bank, annual real growth in gross domestic product (GDP) was 1.9% in the period between 2010 and 2016, the lowest in Central America. Since the end of the civil war in 1992, El Salvador has made continuous progress in human development, increasing access to public services among the poor, reducing inequality by about 5 percentage points between 2007 and 2016, and increasing income in the lowest income quintile40. Health indicators have also shown major improvements in the last decade with institutional deliveries and immunization coverage reaching 98 and 93%, respectively. Sustained pro-poor social policies in the country helped the country achieve Millennium Development Goal #4 (MDG 4) for under-5 mortality reduction41. Also, El Salvador is one of twenty countries undertaking accelerated implementation of the 2030 global agenda for the Sustainable Development Goals.

In 2009, the government of El Salvador reformed the public health system by introducing changes to the Ministry of Health’s (MOH) model of primary care delivery. Existing facilities were reorganized into integrated networks of healthcare delivery; free services were introduced; and the provision of comprehensive primary care services through Community Family Health Teams (CHT) became the central mode of service delivery. Rural CHTs include a medical doctor, two nurses, and three community health promoters with a nominal catchment of 3000 individuals for the entire primary care team. CHTs identify, enumerate and assess risks in their assigned catchment areas and are responsible for providing a package of approximately 300 primary care-related interventions such as health promotion, prevention, and treatment of common conditions42. By the end of 2015, El Salvador had established 747 CHTs. In El Salvador, SMI started implementation in 2011 in 14 of the poorest rural municipalities where 75 CHTs operate. Table 2 provides details about the linkage between the PMM framework and the SMI program in El Salvador.

Table 2. SMI interventions by level of implementation in El Salvador.

Framework componentCentral-level of the Ministry of HealthLocal-level (Primary care delivery system
in 75 municipalities)
1-   Institutional and policy
context (policies, programs
and activities) and primary
care system actors and
stakeholders
Health reform of 2009 (comprehensive,
integrated primary care); Health policy priorities
(maternal, neonatal, and child health); focus on
equity and poverty reduction.
Model of care emphasizes integrated
delivery of primary care through community
health teams embedded in the municipalities
they serve; free of charge services; and,
community participation.
2-   Local socio-economic
context
Geographical targeting of the poorest
municipalities in the country.
Rural, hard-to reach communities; extreme
poverty; prevalent security and violence;
3-   Performance measurement
and management
interventions
External verification of country program
performance; and, supply-side incentives to the
MOH through results-based financing.
During phase 2, introduction of community
health team-based, in kind incentives.
4-   Performance measurementIHME collects household and facility-based
surveys across the country every 24 months.
A randomized-controlled trial to evaluate the
impact of the in-kind incentives measured
CHT performance.
Household and facility-based surveys
collected in all 75 participating CHT every
6 months for 2 years.
5-   Sense-making process
(turning data into information)
SMI, IHME and the MOH defined criteria for
data analysis and synthesis.
SMI and the MOH defined data collection
and analysis methods. A set of 10
performance targets included in each CHT
performance framework.
6-   Dissemination of
performance information
Results presented every 24 months in meetings
attended by policy-makers, SMI and IHME.
Achievement of 80 per cent or more of the
agreed-upon targets triggers a performance
payment (50 per cent of the government’s
financial contribution to SMI).
Results presented every 6 months in
meetings attended by medical directors of
participating CHTs (treatment and controls
alike). Awards and in-kind incentives
presented to CHT achieving 80 per cent or
more of the agreed-upon targets.
7-   Adoption of performance
information (use, misuse or
non-use)
Performance information used in support of the
design of the subsequent SMI phase.
Unknown. To be characterized during the
evaluation.
8-   ImplementationUnknown.To be characterized during the
evaluation.
Unknown. To be characterized during the
evaluation.
9-   OutcomesEl Salvador has successfully achieved
80 per cent or more of the agreed-upon targets for
SMI’s phases 1 and 2.
Unknown. To be characterized during the
evaluation.

Methods

Methodological approach and study design

This study used a realist evaluation approach, which provides a valuable standpoint for evaluating policies and programs that attempt to introduce changes in complex adaptive social systems43,44 such as a PMM system45. Realist evaluation is a type of theory-driven inquiry based on the premise that an evaluation needs to answer, “what worked, how, in what circumstances, and for whom” complementing the more conventional evaluation question “Did the program work?43. The rationale for using a realist approach in this study was described in a previously published study protocol39. The appeal we saw in realist evaluation, compared to other possible approaches is its explicit foundations in critical realism, an epistemology that contends that the interactions between context and program interventions generate change through the triggering of underlying, usually hidden, causal mechanisms.

To characterize high-performing primary care systems at scale, we purposefully identified and studied positive outliers, a methodological approach that “can reveal a great deal about intense manifestations of the phenomenon of interest”46. Such study design also aligns well with the realist evaluation proposition that contexts can trigger to-be-identified mechanisms that, in turn, interact with program interventions and contribute to generating outcomes at various levels.

This paper reports the results of an ongoing evaluation that is concurrent with the implementation of SMI’s second and third phases. The research questions addressed here are: (1) What are the effects of using supply-side incentives at the primary care delivery level, on the performance of the primary care delivery system? How are those effects produced? Under what contextual conditions? And, (2) What are the effects of high-stakes external measurement on performance itself? How are those effects produced? What contextual factors contribute or hinder those effects? In this paper, we report program effects only from the perspective of CHTs; policy-level perspectives will be presented elsewhere.

The steps followed in this evaluation were detailed in the study protocol39. They included, first, the development by the research team of a preliminary program theory (PT), followed by the identification of the research methods best suited to test and revise the preliminary PT. Data is then collected and analyzed; the process ends with a synthesis of findings and the generation of a refined PT. Each of these steps will be briefly described below.

Preliminary PT development

The preliminary PT (Figure 1) that underpins this evaluation shows hypothesized causal linkages between context, SMI’s interventions, causal mechanisms, and program outcomes. The initial PT resulted from a scoping review of the literature on PMM and public sector motivation9,1820,23,24,28,47,48, performance information utilization29,4953, diffusion of innovations theory applied to health services delivery5458, socio-cognitive theory5961, self-determination theory6271, neo-institutional theory72,73, and recent evidence on the performance and motivation of frontline workers in LMIC10,7476. Additional information was obtained through SMI documents review and a program stakeholder workshop held in early 2017. The preliminary PT is described in detail in the study protocol in Box 139. Table 3 provides a summary of the data sources used to address specific elements of the preliminary PT and a general schematic of the sequencing and timing of the evaluation components.

495331cd-3302-4e15-9d5a-895814a8820a_figure1.gif

Figure 1.

Table 3. Sources for and timing of the development of the preliminary PT and data collection.

Preliminary PT components and
domains of interest
In-depth
interviews
(July-November
2017)
Non-participant
observation
(May 2018)
Stakeholder
workshop
(March 2017)
Document
review
(Ongoing)
Contextual factors
Institutional and policy context
System antecedents
Organizational environment
Interventions
Mechanisms
Individual and inter-personal motivation
Collective-level mechanisms (network
effects)
Outcomes
Non-health effects
Health effects

In a work environment, motivation is defined as an “individual's degree of willingness to exert and maintain an effort towards organizational goals76.” Highly motivated workers report higher levels of job satisfaction, which can lead to lower levels of health worker attrition, ultimately contributing to the retention and continued presence of skilled and experienced workforce in a health system. Motivation can be instrumental or extrinsic, driven from contingent rewards (both financial remuneration and non-monetary recognition), or intrinsic, driven from an individual’s inherent desire to perform a task for its own sake77,78. Usually a combination of these two facets determine an individual worker’s motivation to perform.

Research in economics, psychology, organizational behavior, and public administration, among other fields, assume that incentives and rewards serve as powerful motivators for the achievement of desirable behaviors among utility-maximizing, rational individuals79,80. This approach has fueled the design of various accountability-driven interventions and many interventions. Many public-sector reforms in LMICs and various global health partnerships have been influenced by this approach. However, theoretical and empirical developments in public administration research suggest that workforce motivation can also be explained by intrinsic motives such as public service motivation, a socially learned set of preferences that can be present among individuals working in the public sector9,81. To characterize workers behaviors in primary care settings in LMICs, we focused on the theory of self-determination, a macro theory of human motivation that has been used to study motivation in the workforce including in a previous realist evaluation in Uganda67,71. The theory has good cross-cultural validation and has demonstrated that individual workers who satisfy internal needs for competence, autonomy and relatedness feel intrinsically motivated and can develop self-motivated behaviors66,8284.

Data collection methods

As detailed in the protocol39, realist evaluation is method-neutral. The methods that optimally addressed the evaluation questions that this paper focuses on, under real-world operational constraints included: document review, key-informant interviews, and non-participant observation. We used a purposeful, outlier sampling46 in which a case is defined a top-performing CHT in the SMI phase 2 program in El Salvador. For the purposes of this study, we selected positive outliers, namely the top four (4) performing CHTs as identified by independent performance measurements conducted by SMI in September 2015; March 2016; September 2016; and February 2017.

Interviews were conducted with MOH policy and program experts, and with providers of care in the four participating CHTs (doctors, nurses and health promoters). To acquire information about SMI’s design, implementation and evaluation not likely to be apparent from program documents, we also interviewed IADB and IHME experts. A total of 38 in-depth interviews were conducted between July and November 2017. Interviews were conducted by one member of the research team while a second investigator acted as observer. Table 4 summarizes the profiles of the respondents.

Table 4. Profile of interview respondents.

Type of respondentNumber of in-depth
interviews
WomenMen
Community health team members1385
Central-level policy-makers862
IADB experts1147
IHME experts633
Totals382117

Specific issues addressed in CHT interviews included, (1) Respondent perceptions of the SMI interventions under study; (2) Emotional and cognitive reactions triggered by the use of MOH-level financial incentives, in-kind team incentives, and external measurement of CHT performance; (3) Knowledge of, and perceptions about, context-specific factors that hinder or contribute to workers individual and team-based behaviors; (4) Descriptions of program ancillary components that could explain program effects; and, (5) Unintentional and undesirable effects. Interview guides were published with the study protocol39.

Information about the dissemination of results from the external verification of performance was obtained by observing IHME’s presentation of results of overall program performance measurement at the end of SMI’s second phase. One of the authors (CI) attended the workshop that took place in El Salvador during May 2018.

Data analysis

Interviews were first transcribed verbatim in Spanish and were then translated into English. A predefined coding system based on the preliminary PT was developed a priori and used for coding by three of the authors (WM, SW and SM) using NVivo 12. Inductive codes also emerged from the data, further enhancing the coding process. Context-Mechanism-Intervention-Outcome patterns across interviews were synthesized into themes using analysis memos that identified salient domains and recorded exemplary quotes.

We used several criteria to identify emergent themes. First, the frequency of respondents’ perceptions was the starting criterion to identify salient areas; these were consolidated as themes based on their relevance and importance to the study questions and preliminary PT, through discussions and consensus-building among the researchers. We also merged the perspectives shared by central-level policy-makers with the accounts shared by frontline workers to compare, contrast, and ultimately converge perspectives from these two organizational levels of the health system.

Once thematic analysis was completed, the themes emerging from the transcripts were arranged in tables according to their reference to context, interventions, causal mechanisms, and outcomes identified by each individual respondent. We used the resulting tables to identify hypothetical Context-Mechanism-Intervention-Outcome configurations that could plausibly explain the observed program results. Findings were also triangulated by considering data from different sources (IHME external verification data, the results from CHT performance measurement, observation notes, and secondary document analysis). The refined PT was synthesized as narratives and figures. Additional details about the data analysis strategy were included in the study protocol39.

To increase the validity of the study in terms of reflexivity, credibility and confirmability, and to enhance the trustworthiness, transparency, and accountability of the research, investigators maintained ‘personal biases memos’ that made explicit all self-identified biases and pre-conceptions that may have effects on the research process85. Also, all analytic decision notes and memos, biases memos, document analysis syntheses, interview guides, research team meeting agendas and minutes, and analysis outputs including coded transcripts, conceptual frameworks, and tables have been preserved to provide a verifiable audit trail.

Ethical considerations

The study’s protocol was reviewed and declared exempt by the George Washington University’s Institutional Review Board (study number 041733). The Ministries of Health of El Salvador and Honduras were informed of the proposed research by the IADB and provided written approval for the research activities.

Respondents were given the choice to provide consent verbally on tape before the interviews, or in writing. To maintain anonymity, respondents reserved the right to review the study outputs and withdraw consent if necessary.

Results

Study findings are presented in three sections: the context; the mechanisms; and, the context-mechanism-outcome configurations. We then summarize the refined PT.

Context

Based on the evidence from the achievement in the four high-performance CHTs sampled, several contextual factors appear to have been conducive to the successful implementation of SMI in El Salvador.

The convergence of domestic political and policy priorities, a conducive global environment, and SMI’s focus on performance measurement created a unique window of opportunity for country ownership to take hold. The election in 2009 of the first-ever, left-wing President in El Salvador generated a favorable political environment for SMI implementation. The incoming administration’s priorities opened a window of opportunity for institutional reforms in the health sector, leading to the formulation of new rules for increasing affordability, equity, access, and coverage of publicly provided health services and to the scaling-up of comprehensive primary health care services in the poorest municipalities in the country. Central to this process were the enhancement of the supply of public services through CHTs and the introduction of free primary care services.

Upon SMI launch, the MOH internal organizational environment was characterized by a positive tension for change and a sense of urgency. The Salvadorian reform’s focus on primary health care delivery and SMI’s pro-performance stance supported the perception at high-levels that the government’s and SMI’s priorities were strategically aligned. This was particularly evident in areas such as targeting the poorest rural municipalities, improving equity gains, and expanding coverage for maternal, neonatal and child health services. These factors may have also provided a positive tension for change and a sense of urgency on the part of the government, leading to their full engagement with SMI and agreement to attempt the achievement of high-stakes performance targets. The global focus on achieving the Millennium Development Goals further reinforced SMI’s and the Salvadorian government’s high priority for maternal and child health.

SMI implementation leveraged existing MOH structures, practices and routines, increasing country ownership and generating complementarities between SMI implementation and the MOH’s internal environment. Since SMI has been implemented directly by the MOH, the program has leveraged existing MOH administrative and managerial systems. For example, the MOH has a long-standing tradition of hosting regular, monthly meetings among primary care stakeholders (e.g., regional offices and CHTs) for supervision and coordination purposes; these meetings were also regularly used to transmit top-down instructions and directives. These routines were adopted and adapted by the MOH and SMI and transformed into continuous, ongoing spaces for performance dialogues, a central aspect to SMI’s approach.

SMI implementation exposed weaknesses in MOH management systems early on, which have been reduced during implementation and no longer hinder CHT performance. Early failures in supply management and logistics led to stock-outs that negatively affected CHTs’ ability to perform. Such experiences led to the identification and explicit discussion of the inter-dependencies between CHT performance and health system deficiencies. Improvements in administrative processes ensued and stock-outs and poor availability of supplies were addressed.

Mechanisms

Causal mechanisms are defined as the actions and reactions, or behaviors, triggered among program actors by the interactions between the context and SMI’s interventions. Extrinsic and intrinsic motives serve as mechanisms for transforming individual and interpersonal behaviors among CHTs and other stakeholders. The mechanisms hypothesized in the preliminary PT, can be positive sources of motivation as well as negative, balancing causes of de-motivation. Table 5 presents exemplar quotes about the mechanisms identified.

Table 5. Mechanisms, selected quotes.

MechanismExemplar quotes
Positive sources of individual
motivation include:

Extrinsic sources of
motivation:
   -   Wages and benefits
   -   A sense of job security
in a public sector
organization
   -   Team-based incentives

Intrinsic sources
   -   Sense of competence,
in completing tasks and
succeeding at what they
do
   
-   Sense of relatedness,
to community and other
team members
   -   
Sense of autonomy when
planning executing team
goals
As one health promoter notes: “Well, in the first place I needed to work. And I liked this.
I had about five years of experience working in the rural area. And when I had the opportunity
of getting a job in the Ministry, (…) it was easier for me because I knew about community
organization (…), and about community work.”

A team member reported: “The salary is something symbolic. The benefits and the employment
are guarantees. There are guarantees. You have to value that.”

A community health team member highlighted: “In the case of Mesoamerica there is a specific
organization of the indicators that you’re going to work with; no doubt with staff sacrifice.
Because this implies a high amount of work. t’s true. We do have good indicators. But it is
also a great human sacrifice on the part of the staff to achieve it. And the staff also makes a
commitment.”

A participant described the emotions triggered when positive performance was recognized
at high-stakes performance dialogues: “To have that kind of profound reaction of satisfaction
with yourself for the work you do; when you feel that satisfaction you feel it deep inside. It is
because there is an answer to something that I’m doing; that motivates me.“

A health promoter reported: “We don’t have a work shift. They may call us in the middle of the
night if there is a pregnant woman. We say: ‘We’re going on the ambulance right away.’ But if
there are no ambulances available, we have to find a plan B so that she can get in time to give
birth in the hospital. So, our satisfaction is that we have accomplished our mission of fulfilling
our people’s needs.”

Participating in high-stakes focusing events trigger deep emotions when successful
performance is publicly recognized: “When we go to the meetings, and I love that.... when we
are all there, they say, ‘We are going to give this diploma to the teams that made a great work.’
They call them up, and it looks like they are graduating from something. And my colleagues
become really motivated. And maybe you can hear from those who didn’t receive anything
phrases like ‘I will propose something for the next meeting because I want to be there.’

Being recognized with the team-based in-kind incentives is reported as: “That's like a bonus for
what we were already doing. So, let's try harder. Since they are recognizing our work, and they
want to recognize it, well, then it was like a motivation, an extra incentive to strive more, right?
To work.”

A doctor reported: “ I feel important. I definitely feel important. I'm important for this institution
because of my work. I feel satisfied and proud of what I'm getting, because they have even
given written acknowledgments under my name, and I feel very important.”
Team-based, interpersonal-
level motivation
  -  Team efficacy
  -  Inter-team learning and
knowledge-sharing
  -  Community recognition
  -  Closeness to communities
  -  Social capital and trust
A health promoter stated: “We have our accountability too. At the beginning of the year, we
present our annual operational plan. So, the community knows about the work we will be
doing. What’s the plan; what the intervention plans in the community are about. In the last
quarter, between October and December, it is mandatory the submission of the accountability.
‘Look, we’ve achieved this. In this community you helped us with this, the population. We were
able to do this, we still have to do this, and these are our weaknesses.’ So, in that way we are
improving.”

A participant in an inter-organizational meeting of CHTs described the dynamics that occur:
“For example, with the other teams in <masked> and <masked> it was like a challenge. The
incentive is a motivation to reach the goal, right? Because I am getting recognition. It can be a
public acknowledgment, it can be an acknowledgment in terms of resources for the team itself.
So, that was a challenge for them too. They took it well. In fact, both teams sat together. They
designed their own strategy to quickly achieve the indicators. It even motivated them to work,
to create ideas, strategies, and to look for them.”

Regarding the usefulness of performance dialogue, a participant reported: “They are very
helpful. It is also a recognition. Because we are taken into account; and they say ‘Hey, you
have achieved this before the rest who have not yet succeeded. Share this. Share your
experiences .... the strategies.’ Well, for one, they are taking us into account and we are
important. We are someone who can be a leader. And we are relating to others, and of course
that makes us feel proud.”

A medical director described the team common motivation for working at the community-level:
“How the team has been transformed! The team that works on the same objective. And is
available and if necessary on a weekend. My purpose in life is clear, right? So, we are here to
help. And this is it! It has been the best opportunity to prove that. My purpose is to serve.”

A medical doctor reported: “We have meetings where we present all the results at the regional
level. For example, we go to the meeting. I see it more as an issue of sharing experience with
peers. Bur for us in <municipality masked> in the department of <masked>, we present the
results. But I don’t see that we compare ourselves to others. It’s more about what things are we
doing well, and what things they can do well, and what things can we learn from other peers.”

A team member discussed external recognition: “Anyone who sacrifices himself and tries
hard, needs to be recognized for the work. And work that is done every day. That was an extra
motivation for everyone. For the teams; that they said they were going to be recognized for
their performance, only.”

A health promoter commented: “Well, I know those people. They may be my relatives,
friends, neighbors. So, that’s a key element to the improvement of the health conditions in the
population. That’s what makes you feel, in many cases, empowered and able to gather the
courage, regardless of each person’s personal situation.”

Relations between the team and community members are described by a team member:
“I think we are a part of their family, because we are already recognized, we are constantly in
their homes, especially where there are children under 5, pregnant women, older adults. We
are closer to them and they don’t… that is, they recognize me, and the promoters of course,
they nurses also. “

A policy-maker commented: “Truly, the health staff in these towns over here, they’re like a
fictional character, really. A character they recognize, you see? And you can be recognized
if you do well when it comes to delivering the work; you do your job well. And this recognition
translates into unconditional support towards your efforts, they help you with every one of the

tasks you're going to do…”

A community health team member stated: “You work closely with local organizations, and that

makes your job easier. For example, when you go to vaccinate, it’s not the promoter that goes

there alone with his thermos. The health committee itself, the women’s committee say, ‘Get

me a bag and we’ll start a cleaning campaign.’ Or, ‘Let’s pick up the trash in order to prevent

diseases in this area.’ That makes the promoter not being alone in the community. He has to

have that support, so his job becomes easier at the community level.”
Potential sources of
demotivation
   -   Poor work-life balance
   -   Dissatisfaction with lack
of supplies
   -   Stress and anxiety
A front-line worker commented: “You may have problems with your family, or economic
problems. There may be problems related to the weather. And in community health, we say
that we are 24/7. Like ATM machines.”

A respondent indicated: “Another thing is to find, for example, that you don't even have a car to
go fetch a patient with. Or you don't have things to give them (patients). This happened to us in
the first measurement. We had no oral rehydration salts, we had no zinc, no micronutrients for
five months! We ran out of everything! And three indicators failed. But that is not up to us. It’s
up to the regional health office. I can’t be measured if I don’t have these (supplies).”

Positive sources of individual motivation (extrinsic and intrinsic). Our results suggest that the Salvadorian reform and SMI’s pro-performance stance may behave as complementary sources of positive, intrinsic motives as well as external drivers of performance, respectively. Furthermore, findings suggest that the intrinsic motivation for joining CHTs are complex and may therefore be triggered by different motives and preferences. When asked about their reasons for working at the community level, CHT respondents identified the acquisition of a sense of competence and the satisfaction of a desire for providing public services, while also acknowledging the benefits from wages and job security.

Extrinsic motives play an important role for individual CHT members who recognize the inherent value of public-sector job security. The joint effect of extrinsic motives like salary, job security and the SMI-specific in-kind incentives for CHT was important for all respondents. A desire for additional bonuses and other forms of external recognition was reported by the majority of CHT respondents.

All frontline workers interviewed expressed a sense of commitment and sacrifice to the communities they serve; to the health of families in their catchment area; and to the achievement of their collective, CHT-specific performance targets. Such degree of commitment plausibly indicates the existence of high levels of intrinsic self-motivation among CHT members that may operate at individual as well as interpersonal levels. Work as a front-line, primary care provider satisfies for most respondents one or more of the following: a sense of competence in their abilities and skills; a sense of autonomy in executing their tasks; and a sense of relatedness to their colleagues and the communities they serve.

Credible data support the qualitative findings regarding commitment and positive motivations expressed by CHT respondents. These include CHT-specific performance information collected by the MOH on monthly cycles of measurement, and the analysis and action-planning reinforced by SMI interventions such as quarterly supervision visits; regular focusing events where teams’ performance is reviewed between CHTs, the MOH, and SMI; and, high-level dialogue meetings where MOH, SMI and IHME discuss the results of the external measurement of system-wide performance.

An intrinsic motive frequently expressed by respondents is the sense of competence obtained from accomplishing their team tasks and thus being able to verify for themselves positive effects on local beneficiaries. Physical and emotional closeness between front-line workers and the community in which they live, come from, and serve, likely increases their social identity and the significance of their jobs, fulfilling a sense of competence and relatedness, which can contribute to the expressions of job satisfaction that emerged frequently in the interviews.

Team-based, interpersonal-level motivation. Our findings suggest that team-level mechanisms play a major role in modifying interpersonal motivation and supporting team cohesion. CHT respondents recognized that SMI introduced major, and positive sources of team motivation. One of the most valued sources of motivation was the sense of contribution, collaboration, and relatedness to other CHTs that was found through the convening of regular performance dialogue meetings. Furthermore, respondents reported a sense of team-based, collective efficacy mediated by the Salvadorian reform requirement that CHTs engage local communities in developing their annual operating plans as well as in accounting their performance results. These are sources of inter-personal or team-based motivation, as the effects they trigger can only be accounted for by collective actions executed by the entire CHT. Such mechanisms are reinforced by the sense of trust that communities develop in high-performing CHTs, thus increasing their members’ individual and team-based social status.

Furthermore, PMM interventions and ancillary components introduced by SMI are implemented in within- and between-teams’ social environments such as ongoing, internal team meetings in each facility, and external learning events convened by SMI and hosted by the MOH. CHTs from different geographical areas get together to learn from each other, share knowledge and experiences, and design improvement plans. The data suggests that such group dynamics can help CHTs make sense of the performance data, leading to the actual utilization of performance information to plan improvements and process innovations. It is thus plausible that these dialogue spaces among peers may trigger social learning mechanisms and set the basis for the organic diffusion of ideas, innovations and behaviors.

When describing the reasons why they enjoy working for their communities, CHT team members consistently reported a sense of empowerment, commitment, team efficacy, and community service. The closeness between team members and beneficiary communities is also a source of inter-personal motivation. Highly-effective CHTs can attain a degree of social recognition that may only be possible in community-embedded teams. Furthermore, for individuals with high levels of public service motivation, energized by fulfilling their sense of competence, autonomy and relatedness, social recognition may further reinforce their intrinsic, pro-social attitudes.

Potential sources of de-motivation. Despite their strong motivation, CHT members oftentimes work against their own personal interests even when this generates undesirable effects for themselves or their families. A frequent theme when describing the challenges of providing outreach services is one of sacrifice which may be both a source of stress as well as an internal drive to help others.

Performance measurement also triggered feelings of stress and anxiety particularly during the early stages of SMI implementation when, according to CHT respondents there were shortages of supplies and other inputs required for meeting CHT performance targets. Also, while respondents in all four CHT reported participating in, and being energized by their involvement in meetings and learning activities, their responses indicated certain preferences and dislikes. For instance, the least appealing activities for many respondents were those that required inter-team comparisons, poorly-delivered feedback by supervisors, and meetings where differentials in goal achievement among CHTs were publicly highlighted.

Refined Program Theory

We identified plausible causal linkages between the relatively small, but well-targeted supply-side incentives brought by SMI at the central level of the MOH; the high-stakes generated by SMI’s measurement of CHT performance every six months; and in-kind, team-based incentives. Findings suggest that SMI interventions and ancillary components; the dynamic institutional environment generated by the health reform; and, the socio-economic contexts in which Salvadorian front-line workers operate, contributed to triggering intrinsic and extrinsic motivation in the four high-performing CHTs studied. Such gains in motivation appear to have been built upon high levels of pre-existing public service motivation among front-line workers and on extrinsic motives such as wages and the expectation of job security and promotions. The introduction by SMI of performance measurement and in-kind, team-based incentives, appear to have reinforced self-motivation among individual team members, also contributing to the generation of team-based motivation.

The learning events scheduled during SMI implementation created a space in which front-line workers, mid-level managers, MOH policy-makers, and SMI staff engaged in enriching performance dialogues and learning and knowledge-sharing. This may have generated several effects. First, it may have focused the attention of CHT and other system actors on making sense of performance data and using performance information to plan future improvements. These behaviors may have also been reinforced by 1) the pro-social, other-oriented and intrinsic satisfaction derived from the sense of competence, autonomy and relatedness described by community health team members; 2) the external motivation triggered by in-kind team-based incentives; and, 3) the extrinsic and powerful incentives generated by increased community recognition and social stature among front-line workers. Second, performance dialogue meetings may have also created safe inter-organizational spaces where social connection, learning, and knowledge-sharing could happen. Finally, these meetings acted as focusing events where high-performing teams were recognized and rewarded, and where dialogue across MOH organizational levels created opportunities to change internal conversations between managerial and operational actors, from conventional top-down control, towards emergent horizontal and, possibly, bottom-up performance management and organizational learning activities. Were this last to be sustained through time, it could set the basis for the emergence of a culture of autonomous, self-directed improvement among CHTs, further reinforcing the positive motivational mechanisms described above.

Our findings identified elements that were not originally included in the preliminary PT that enrich the emerging, refined PT. First, the PT identified theories of human behavior and motivation that were mostly focused on individual-level motives, such as social-cognitive and self-determination theories59,61,65,82,83; it also highlighted interpersonal mechanisms mostly transmitted through diffusion and social learning channels. Second, despite the mention of unintended or negative outcomes in our preliminary PT, the evaluation protocol did not expand on what those effects could be, except as it pertains to gaming, shirking and other negative effects.

The refined PT (Box 3 and Figure 1) now includes team-based motivation as a specific type of interpersonal motivation, informed by collective efficacy theories86 as well as by theories of identity87 and social identity8891. The refined PT also contains unintended, demotivating effects that PMM interventions could induce in frontline workers such as stress, anxiety, and poor work-life balance.

Box 3. Refined program theory as a context-mechanism-intervention-outcome configuration

In contexts where primary care teams and beneficiaries are in close proximity, where front-line workers have high levels of public service motivation, and where primary care provision is embedded in organizational environments that provide job security, wages, and opportunities for promotion; continuous support in the form of appropriate working conditions and supplies; that offer opportunities for recognition linked to providers’ performance; where supervision is supportive and includes actionable feedback; and where the domestic policy and institutional context provides autonomy and clear mandates for the delivery of primary care services;

Where program interventions include, in kind, team-based incentives; independent, rigorous, continuous team-based performance measurement; and, continuous, inter-organizational (i.e., inter-team) and intra-team, performance dialogue and other focusing events that raise the stakes and make performance results visible for providers and communities; the following mechanisms can be triggered:

  • - At the individual-level – the satisfaction of a sense of competence, relatedness, and autonomy; reinforced by wages, job security, and potential for promotions; and balanced by poor supervisory practices, provider stress and poor work-life balance;

  • - At the team- or inter-personal level – by the satisfaction of sense of competence and autonomy (i.e., collective efficacy) and the recursive use of performance information to design, test and evaluate rapid cycles of improvement. Also, at the team level, by the identification of team members with a collective sense of social identity; reinforced by community recognition and increased stature in the community; by individual-level intrinsic motivation; and, by team-awards. Such mechanisms can be balanced by lack of support in the form of appropriate working conditions and supplies, supportive supervision, stress and poor work-life balance; and,

  • - At the inter-organizational level, by the diffusion of lessons and sharing of knowledge through participation in performance dialogues, and collaboration with other teams in high-stakes, focusing events;

Leading to the following outputs and outcomes:

  • - Among individual providers: Workload; work morale; stress; burnout; staff turnover; sick leave; absenteeism.

  • - At the team-level: Quality of care and process improvements; adherence to recommended practice guidelines; patient satisfaction; perceived quality of care; workforce retention; unintended organizational outcomes.

  • - At the patient- and community levels: Utilization of services; coverage of interventions or services; access to primary care services; health equity effects; adverse health effects or harm; unintended population health outcomes

Finally, the use of powerful supply-side financial incentives by SMI at the higher echelons of the MOH may trigger additional mechanisms and outcomes (positive and negative; intended, and otherwise) that need to be further researched and that will be the subject of a separate paper. We expect the PT to continue to be adjusted as new findings emerge from the realist evaluation of SMI in El Salvador and Honduras.

Discussion

To date SMI and the Salvadorian MOH have achieved major gains in primary care performance at the national as well as the primary care delivery levels. Changes in individual- and team-level provider beliefs and behaviors are associated with increased motivation and productivity. The refined PT highlights causal pathways that can plausibly explain how team-based performance improvements can accumulate and lead to system-wide performance gains. Results also provide an in-depth look at the role played by motivated, primary care teams in explaining SMI’s program success in El Salvador.

Our findings suggest that the individual and interpersonal motives that underlie the high performance of CHTs is complex and dynamic, and that PMM interventions implemented by SMI influence these complex motivations in ultimately positive ways. There is also some indication from the data that other interventions implemented as part of SMI contributed to CHT performance improvement, such as national level financial incentives, by creating a positive environment for focusing on performance and embracing data to problem-solve. Findings also indicate that the organizational arrangement of primary care around community-embedded teams by the health reform effort that preceded SMI has been a complementary and necessary contextual foundation for the effective implementation of SMI’s PMM interventions. The interplay between these two factors was not accidental, but the product of deliberate actions by the stakeholders involved. Such interactive and responsive approach to program design and implementation is one of the key contributing factors for the triggering of individual and team-based interpersonal motivation which, in turn, is one of the main causal mechanisms driving the high performance observed in these municipalities.

An over-arching sense of commitment was expressed by all front-line respondents that can be plausibly explained by the reinforcing effect of different individual and interpersonal mechanisms. This sense of commitment has its roots in the Salvadoran Health Reform, which has not only brought these teams in close proximity to communities, but has also increased team commitment and contributed to the autonomy, competence and relatedness that individuals need to stay motivated. Closeness to communities has increased social standing and status for CHT members, as well as community recognition, and appears to be an additional source of motivation for the entire team, a finding supported by recent evidence21,92,93.

At the individual level, respondents reported extrinsic and intrinsic motives that combined in positive, reinforcing manner. Such a mix of financial and non-financial incentives can enhance performance among community-level, frontline workers13. The data also indicate that front-line workers value fair wages, a sense of job security, the provision of in-kind incentives and awards, and community recognition. SMI PMM interventions supported these extrinsic and intrinsic motives through the team-based performance incentives and the recognition given during performance dialogue meetings. Intrinsic motives that prevail are those that satisfy the needs for competence, autonomy and relatedness as described by Self-Determination Theory62,64,66,71,83,84 and as hypothesized in our preliminary PT39.

Motivated individual frontline workers have coalesced into highly effective teams whose purpose and action were further reinforced by their sense of commitment to achieve community-based health targets and by the supportive environment generated by the MOH and SMI. This included the creation of “safe” spaces for dialogue about performance, using data collected both by SMI, IHME, and the MOH, and a slow change from punitive to supportive attitudes when performance was below expectation, and to recognition when it met or exceeded expectations.

SMI interventions also raised the stakes among front-line teams. In addition to the external verification of performance conducted by IHME every two years, four waves of data collection were also conducted by SMI, as part of the impact evaluation of the in-kind, team incentives PMM intervention. This data collection initiative not only increased the frequency of external performance measurements and data availability to the CHTs, but may have amplified emotional and cognitive effects on CHTs49,94,95, thus triggering inter-personal mechanisms that improved team performance, in response to the continuous performance measurement.

Team motivation was also sparked by the social, participatory dissemination activities that are part of the SMI PMM interventions. SMI provided feedback on team performance achievements and in-kind incentives to successful CHTs during regular high-stakes meetings, which are often called performance dialogues in the public management literature24,96,97 and serve as spaces for learning and knowledge-sharing. Our results indicate that reflection, planning and action-oriented social activities took place during these meetings, and that these events helped CHTs make sense of performance data and triggered the utilization of performance information by the teams, an important organizational level outcome18,33,49,53,98. The “extra dose” of team-based performance measurement, the associated recognition of high-performing teams, and the ongoing performance dialogues, were identified by respondents as important sources of mostly positive team-based motivation.

Some of our findings on team-based motivation are promising and require further research. The theories of motivation reviewed for this evaluation mainly focused on individual-level mechanisms. However, our preliminary PT hypothesized a multi-level approach to performance improvement that a priori identified individual, interpersonal, and inter-organizational mechanisms of system change. Our findings indicate that team-based motivation, and not just individual motivation, may play a larger role in explaining performance improvement than initially expected. In turn, performance dialogues triggered diffusion effects that suggest team-level and inter-organizational mechanisms that enabled higher performance. The role of other SMI interventions, especially of the national level financial performance incentives, likely contributed to the creation of the spaces for performance dialogue and the enabling environment for the diffusion effects; however, this study was not designed to investigate the relationships among the different SMI interventions, in particular their relative contribution to performance improvement effects on primary care provision by CHTs; future research could focus on this question.

Our findings underline some mechanisms that could trigger negative, unintended effects among front-line workers. In particular, the sense of sacrifice and stress from performance measurement among over-worked providers, while being positive for intrinsically motivated individuals, may become a source of work-life imbalances, burnout, and job dissatisfaction in others. This is an area with major policy implications that also requires further research7,99.

The findings here help characterize the complex proximal and intermediate program effects at the level of community health teams (process improvements, outputs, and organizational effects) that had not been previously recognized by program designers. This will help SMI consider new metrics and causal pathways that can further complement future program adjustments and improvements to their evaluation agenda.

Strengths of this study include the use of program theory to both hypothesize and then characterize the causal linkages among several concurrent “interventions”: 1) a health system undergoing a major reform; 2) the actions and reactions triggered by SMI PMM interventions among autonomous individuals and teams of primary care providers; and, 3) the proximal and intermediate, provider, and organizational-level effects that have been triggered in this successful program. The use of a realist evaluation approach to explain program outcomes mitigates the limitations of conventional case studies inasmuch as the refined PT contributes explanations that can be tested and further refined44. Other strengths include the use of systematic procedures to limit researcher bias (reflexivity, audit trails, etc.), and the triangulation of multiple data sources, which is essential for rigorous case study100.

There are also several limitations in the study. Due to budgetary and operational limitations, we did not address primary care beneficiaries’ perceptions of the services provided by the teams that were studied. We may have therefore missed relevant perspectives about the quality of interactions between providers and their clients, and the mechanisms thus triggered, or not. Such demand-side perspective is also relevant given the importance that providers’ descriptions of community recognition and closeness had in our results, an area that needs to be further characterized. Finally, we are unable to assess which of the two types of external measurement (IHME every 24 months with financial incentives at the national level, and SMI's own external measurement of CHT performance every 6 months) has greater influence on primary care performance. This is consequence of the qualitative, single-country case study design, which, while best suited for developing program theory, cannot assess the relative impacts of these two very different forms of external measurement on various dimensions of primary care performance.

For this study we purposively selected the highest performing districts in El Salvador to identify what motivations and mechanisms, at different levels, were likely driving good performance, and how SMI PMM interventions were contributing to the results (listed in Table 1). Our data shows clear patterns of motivational mechanisms at individual, interpersonal team, and organizational levels, and clearly outlines the dynamic between the PMM interventions and the ongoing/preceding health reforms, and how these have worked together in high performing municipalities in El Salvador. We would have liked to confirm our findings, using the same recognized methodological approach46, in lower performing districts, assessing context, intrinsic and extrinsic motivations at individual and team levels, and the implementation quality of the SMI PMM interventions in those districts. This would have validated the presence or absence of the context-mechanism-outcome pathways identified in our refined PT.

Additional research in El Salvador SMI program may include further characterizing the role that CHTs’ “embeddedness” in local communities has upon team performance and health outcomes; studying organizational performance in under-performing teams; and testing and validating scales of self-determination, collective efficacy and social identity among frontline workers. While there is a long tradition of team research in organizational science101, management studies102, and team science103,104, team behavior and dynamics has been scarcely investigated in the healthcare contexts105 of LMICs. Addressing these gaps may be of relevance as universal coverage models of primary care, like the Salvadorian CHT, inform reforms in other LMICs.

Conclusions

To date, SMI and the Salvadoran MOH have achieved major health gains at the national and municipal levels, where primary care services are actually delivered. SMI’s PMM interventions have reinforced high performance by community health teams, by providing additional pro-performance and pro-social motivation, and modifying the intrinsic and extrinsic motivation among individual workers and teams, changes that started with the Salvadorian health reform. SMI interventions that have contributed to reinforcing motivation include the continuous, external measurement of CHT performance; the use of in-kind, team-based incentives; and, high-stakes performance dialogue and learning platforms that are regularly scheduled.

The aim of this realist evaluation is to identify plausible causal explanations for SMI-induced performance improvements at a “whole-system level”56,57 in the context of a low-income, high-performance country like El Salvador. We found that changes in individual- and team-level beliefs and behaviors are associated with increased motivation and productivity, and were supported by SMI PMM interventions. Our refined PT identifies causal pathways that plausibly explain how team-based performance improvements can accumulate and lead to system-wide performance gains. Results also provide an in-depth look at the contributions of motivated, primary care delivery teams, which also help explain SMI’s program success in El Salvador. However, SMI interventions occurring at higher-levels in the health system (e.g., policy-makers and mid-level managers) will likely have triggered additional mechanisms whose interplay with the context may have also contributed to the observed system behavior (i.e., high-performance) as hypothesized in the preliminary PT. Subsequent papers will address change at those other levels in the health system.

The findings here can inform policy design in El Salvador and other LMIC countries with similar contexts, when health system stakeholders need to consider ways to effectively engage teams of primary care providers to improve the universal delivery of public, primary care services. The insights obtained can help SMI consider program adjustments and enhance their evaluation agenda. Findings reported here can also help inform policy design in El Salvador and other LMIC with similar contexts when considering system-wide approaches that effectively engage primary care providers in achieving the lofty goals of universal health coverage and the sustainable development goals. Our findings can also contribute to ongoing and future policy work by LMIC agencies and global health funders interested in achieving high-quality health systems106 through the use of PMM interventions. Finally, the use of a theory-driven approach to program evaluation can help define research agendas on PMM in SMI as well as in other LMIC.

Data availability

This manuscript contains examples of de-identified data acquired through interviews and recorded in transcripts and context-mechanism-outcome configurations. In order to protect participants’ confidentiality, additional qualitative data is available upon request from the corresponding author. The majority of respondents in the study work in small local area settings, with a risk of identification by others. As per the study protocol, participants consented to participate in the study under the guarantee of confidentiality. Qualitative data can be obtained by contacting Wolfgang Munar, MD, SM, Milken Institute School of Public Health. Access to this dataset will be provided only upon submission of a specific research proposal.

Comments on this article Comments (0)

Version 1
VERSION 1 PUBLISHED 31 Oct 2018
Comment
Author details Author details
Competing interests
Grant information
Copyright
Download
 
Export To
metrics
Views Downloads
Gates Open Research - -
PubMed Central
Data from PMC are received and updated monthly.
- -
Citations
CITE
how to cite this article
Munar W, Wahid SS, Mookherji S et al. Team- and individual-level motivation in complex primary care system change: A realist evaluation of the Salud Mesoamerica Initiative in El Salvador [version 1; peer review: 2 approved with reservations]. Gates Open Res 2018, 2:55 (https://doi.org/10.12688/gatesopenres.12878.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.

Comments on this article Comments (0)

Version 1
VERSION 1 PUBLISHED 31 Oct 2018
Comment
Alongside their report, reviewers assign a status to the article:
Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

Are you a Gates-funded researcher?

If you are a previous or current Gates grant holder, sign up for information about developments, publishing and publications from Gates Open Research.

You must provide your first name
You must provide your last name
You must provide a valid email address
You must provide an institution.

Thank you!

We'll keep you updated on any major new updates to Gates Open Research

Sign In
If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password.

The email address should be the one you originally registered with F1000.

Email address not valid, please try again

You registered with F1000 via Google, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Google account password, please click here.

You registered with F1000 via Facebook, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Facebook account password, please click here.

Code not correct, please try again
Email us for further assistance.
Server error, please try again.