Keywords
quality improvement, improvement collaborative, community-based family planning, community health workers
-
Views
-
Downloads
How to cite this article
Kim C, Kirunda R, Mubiru F et al. A process evaluation of the quality improvement collaborative for a community-based family planning learning site in Uganda [version 1; peer review: 2 approved]. Gates Open Res 2019, 3:1481 (https://doi.org/10.12688/gatesopenres.12973.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
Export Citation
Sciwheel
EndNote
Ref. Manager
Bibtex
ProCite
Sente
▬
✚
Research Article
A process evaluation of the quality improvement collaborative for a community-based family planning learning site in Uganda
[version 1; peer review: 2 approved]
PUBLISHED 04 Jun 2019
This article is included in the International Conference on Family Planning gateway.
Abstract
Corresponding author:
Christine Kim
Competing interests:
No competing interests were disclosed.
Grant information:
This work was supported by the Bill and Melinda Gates Foundation [OPP1181398]. We are grateful to the United States Agency for International Development (USAID) and the Boren Fellowship for their financial support to conduct this process evaluation and to USAID for funding the APC project implementation. APC is a cooperative agreement funded by the U.S. Agency for International Development under Agreement No. AID-OAA-A-12-00047, beginning October 1, 2012. APC is implemented by JSI Research & Training Institute, Inc., in collaboration with FHI 360.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Copyright:
© 2019 Kim C et al.
This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
How to cite: Kim C, Kirunda R, Mubiru F et al. A process evaluation of the quality improvement collaborative for a community-based family planning learning site in Uganda [version 1; peer review: 2 approved]. Gates Open Res 2019, 3:1481 (https://doi.org/10.12688/gatesopenres.12973.1)
First published: 04 Jun 2019, 3:1481 (https://doi.org/10.12688/gatesopenres.12973.1)
Latest published: 29 Aug 2019, 3:1481 (https://doi.org/10.12688/gatesopenres.12973.2)
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
There is a newer version
of this article available.
of this article available.
Abbreviations
APC, Advancing Partners & Communities Project; CBFP, Community-based Family Planning; CHW, Community Health Worker; FGD, Focus group discussion; FP, Family Planning; HMIS, Health Management Information System; IDI, In-depth interview; LS, Learning Session; MOH, Ministry of Health; PDSA, Plan-Do-Study-Act; VHT, Village Health Teams; QI, quality improvement; QIC, Quality Improvement Collaborative
Introduction
The integration of volunteer community health workers (CHWs), or Village Health Team (VHT) members in Uganda, into family planning (FP) service delivery is one of several proven high-impact practices in family planning and is called community-based family planning (CBFP)1. A review of CBFP programs over the last three decades found that community distribution of FP was necessary in country contexts where clinics and hospitals could not meet the FP needs of communities2. High-quality FP services have been associated with increased FP service demand and use, resulting in improved health outcomes for women, including reductions in unmet need for FP and rates of unintended pregnancy, abortion, morbidity and mortality3. Quality improvement collaboratives (QIC) have been adopted widely as an effective team-based learning and improvement approach4. QICs have been implemented across sub-Saharan Africa, focusing on iteratively testing changes in service delivery and analyzing their effects on processes and outcomes5,6, often using the plan-do-study-act (PDSA) quality improvement model (Figure 1).
Figure 1. FHI 360 Quality Improvement Model.
These quality improvement initiatives are often clinical and disease-focused (e.g., HIV/AIDS), driven by clinical processes (e.g., appropriate treatment, diagnosis) and have occurred at the health center level7,8. Evaluations of quality improvement initiatives in Sub-Saharan Africa are limited, with examples from Uganda and Tanzania on maternal and newborn healthcare6,9 and from northern Ghana on maternal and child health10.
In Uganda, the total unmet need for FP is 28.0%11. This unmet need is highest among the two poorest wealth quintiles and among women 25 to 35 years of age11. Among current female users of FP, only 61.7% were counseled on side effects and 60.8% were told of other methods11. To our knowledge, there is no documentation in the published literature or the Ugandan national health strategies of QICs targeting CBFP service delivery models.
While a variety of QICs have been used in healthcare, it is often unclear what makes them successes or failures12–14. Process evaluations are particularly useful tools in the QIC context, because they allow for detailed understanding of activities actually performed and participants’ experiences, which informs revision and adaptation of the intervention— a natural and iterative process within the QIC model15,16. The Bruce Framework includes six elements that are essential for high quality FP services: 1) choice of methods, 2) information provided to clients, 3) service provider competence, 4) interpersonal relations, 5) continuity and follow-up mechanisms, and 6) appropriate constellation of services17.
This process evaluation was conducted in a QIC’s learning site in Busia district and one of the scale-up districts, Oyam. We analyzed the trends in FP uptake and client retention by VHTs and aimed to identify the factors that were supportive of the CBFP QIC implementation, as perceived by the collaborative actors and in relation to the Bruce Framework.
The QIC
In 2014, Uganda’s Ministry of Health (MOH) selected Busia district to serve as a CBFP learning site for Uganda. Three health centers with established CBFP programs comprised the QIC learning site. A more detailed description of the QIC we evaluated is provided elsewhere18,19.
The first step of the QIC (see Figure 2) was to conduct a collaborative site assessment, which looked at the overall CBFP system, training, supervision, referrals, supplies, infection prevention, waste management and reporting at the district, health center, community health worker and client levels in the learning site’s catchment area. Upon completion, a dissemination meeting was held to achieve understanding of the strengths and areas for improvement, as well as potential solutions to be included in a QI Charter, which would state the aim of the improvement effort and provide a roadmap for the QIC. Subsequently, in collaboration with clients and other local stakeholders, the collaborative members (e.g., midwives, in-charges of health facilities (usually physicians), VHTs and district health officials) developed, implemented and tested innovative change ideas using the PDSA method.
Figure 2. Collaborative Improvement Model.
Approximately every six months, learning sessions were convened for collaborative members to share with and learn from peers who had successfully implemented or replicated change ideas. The first learning session (LS1) was internal, for collaborative members of the learning site (Busia) and a few new health centers in that district that were preparing for scale-up. For LS2, a delegation from the scale-up district (Oyam) came to the learning site to take learning back to their district. LS3 took place in Oyam district with a delegation from the learning site and two additional districts where the collaborative was being scaled-up. This process evaluation was completed before LS3 and subsequent scale-up plans to other districts.
For each LS, the agenda was tailored to the dynamics and the context of each group. This process of adapting the learning sessions with new materials and new roles for participants was designed to continually challenge the QI teams and enhance Busia’s role as a learning site. In addition, clients were engaged throughout the QIC and helped generate change ideas to ensure a client-centered approach. The QIC aligned with all five objectives of the MOH’s Health Sector Quality Improvement Framework and Strategic Plan (2015/2016-2019/2020)20.
Methods
We used a descriptive mixed-methods process evaluation design, including the following components: desk review of program documents, extraction of program monitoring data, and qualitative research methods. We collected data from two program districts: Busia (learning site) and Oyam (scale-up).
Study setting
Busia district is in the eastern region of Uganda. Busia has been implementing CBFP since 2008. As a border district with Kenya, Busia is densely populated, with both rural and peri-urban areas, has one of the highest adolescent pregnancy rates in Uganda and a total fertility rate of 6.0. Oyam district is a relatively new rural district in the northern part of Uganda, with a total fertility rate of 6.3, which is higher than the national average of 5.421. CBFP was introduced to Oyam in 2015.
Sample population
The target populations for this study were the core members of the QI teams: VHTs, midwives and their clients. QI teams are composed of midwives, district health managers, health center in-charges and VHTs. QI team members from Busia were eligible if they had been a member of the QI team at their health center and had attended at least one learning session, followed by implementation of at least one PDSA cycle. To select participants for focus group discussions (FGDs) and in-depth interviews (IDIs), we purposively identified a subset of pilot and scale-up health centers in Busia and Oyam districts. VHTs were short-listed by health center based on eligibility criteria and selected using a random number generator from the list for up to 7 to 10 VHTs per health center.
Using a random number generator, we randomly selected male and female clients at least 18 years of age for FGDs and couples for IDIs who were recorded in the VHT registers during the implementation period (June 2015-December 2016). IDIs were conducted with one midwife and in-charge from each health center. All but two of the health centers in Busia had only one midwife and one in-charge. The midwife responsible for supervision and coaching of VHTs providing CBFP was interviewed at the health center with more than one midwife. Representatives from the MOH at the district and central levels who were familiar with the CBFP QIC were interviewed. All participants located in the two districts were contacted by mobile phone and invited to participate. The MOH representative at the central level was contacted by email. We also sent an email invitation to an NGO representative from a health facility QI program who did not respond. No invited participants refused to participate or dropped out. The total sample size for each participant group is listed in Table 1.
Data Collection
Review of documents. We reviewed program documentation to understand the processes and evolution of the QIC that was implemented by health centers in Busia district from March 2015 to May 2017 and in Oyam district from July 2016 to May 2017. We identified what change ideas were implemented over time and challenges reported in order to understand the context of the trends shown in the data. These documents included: collaborative site assessment reports at baseline, QI indicator data, routine client satisfaction surveys and coaching visit reports, district charters, trip reports, learning briefs, and other program documents.
Quantitative data. We tracked the following three indicators using information from the VHT client registers collected by the program from January 2015 to March 2017: number of female clients on FP returning to the VHTs, total number of VHTs’ female clients on FP, and number of couples counseled on FP by the VHTs. All data were de-identified, then aggregated by health facility and month.
Qualitative data. Local research assistants conducted IDIs and FGDs with clients and VHTs in each participant’s language of preference: Luganda, Samia, Ateso, Langi or English. One interviewer conducted IDIs and a team of two (one facilitator and one note taker) conducted the FGDs. All interviews with midwives, in-charges and government officials were done in English. Interviews were digitally voice recorded upon receipt of each participant’s informed consent to proceed with the interview, field notes, and the voice recordings. Each IDI took approximately 30 minutes, and each FGD took no longer than 45 minutes (per Ugandan IRB requirements). Interviews and discussions were conducted using a semi-structured guide covering topics relevant for the target participant group, such as key features of QI, what worked, challenges, the adaptation process, sustainability and institutionalization and perceptions of the quality of services (see Extended data22). We conducted interviews over a two-week period in May 2017. We aimed to reach apriority thematic saturation and data saturation as described by Saunders et al.23. All interviews and discussions were conducted at a quiet and private location at the health centers. All audio records of interviews were transcribed into English and de-identified.
Data analysis
We cleaned the quantitative data using Stata 14 and collapsed each indicator into monthly sums by district24. We obtained descriptive statistics (counts and means per indicator) and plotted the data on run charts in Microsoft Excel. Run charts are simple to construct and are a useful tool for learning about the performance of a QIC. Our run charts demonstrate over a 28-month period whether we have non-random patterns (“signals”), using the median of the total data points collected during that timeframe as a centerline. The median, which indicates the point at which half the data are either above or below the line, is not influenced by extreme values. In this process evaluation, the question we sought to answer was not whether any changes observed were statistically significant, but to determine whether they represented sustainable improvement over time. To do so, we used runs charts to identify trends and shifts25,26.
We used NVivo 11 to conduct a content analysis of the qualitative data transcripts27 and developed an integrated approach to coding28. After reviewing all the transcripts to gain an understanding of and context for the data, we developed a coding structure deductively based on the previous review of program documents and overall review of the data. Two researchers (CK and RK) worked independently to code the transcripts. After initially coding a few transcripts, the two coders discussed their approaches and agreed upon a coding structure of major themes. Inter-coder reliability and agreement were checked by comparing the analyses of the two coders, assessing the overall level of agreed codes, and discussing areas of disagreement. We used NVivo’s querying capabilities to assess the frequency of codes, assessing codes by attributes and co-occurring thematic codes. We extracted illustrative quotes for each major theme.
Ethical considerations
Participants were informed about the aim of the research project and gave consent to participate. The District Health Quality Improvement Officers leading the health operations in the two districts were part of the team and approved the publication. During the informed consent process, respondents were informed that any information we include in reports would not identify them. Participants did not review the data and provide feedback.
All study documents were reviewed, approved, and categorized as exempt from research status, as it was deemed as posing no more than minimal risk, by FHI 360’s Protection of Human Subjects Committee (Project #: 1035307), the Research Ethics Committee of The AIDS Support Organization in Uganda (Ref. #: TASOREC/15/17-UG-REC-009) and the Uganda National Council for Science and Technology (Protocol #: SS 4267). District offices and health facilities were informed of data collection plans prior to interviews and focus group discussions through phone calls and emails by APC staff. Verbal informed consent was obtained from all interview and focus group discussion participants to minimize links between participants, other people’s knowledge of any participants’ participation in the study, and information they provide. All participants were given paper copies of the informed consent with information on whom to contact with further questions in either English or a local language of their choice.
Results
Trends in key indicators
There was an increase in the number of female FP clients returning to a VHT for follow-up and resupply in Busia and Oyam districts from December 2014 to March 2017 (Figure 3). In Busia, a positive trend (defined as five data points in one direction26) is shown from March 2016 to March 2017, with 14 data points above the median line of 436 clients. Oyam shows a similarly positive trend, with 12 data points above the median line of 138 clients. These positive trends occurred as several changes were being introduced and tested, including home visits by VHTs, use of expert clients, and couples visiting other couples. A similar trend was seen in the number of female FP clients, with a median of 694 for Busia and 228 for Oyam (see Underlying data29). Successfully engaging men was one of the greatest challenges. However, after targeted sensitization and education efforts by VHTs to specifically engage men, we saw dramatic improvements well above the median (25 couples in Busia, 11 couples in Oyam) in the number of couples counseled by VHTs starting in October 2016 (Figure 4).
Figure 3. Number of family planning (FP) clients returning for follow-up, December 2014 - March 2017.
Annotated run chart with examples of change ideas implemented over time. aTalking homes: displaying FP posters and providing confidential space in Village Health Team (VHT) home for services. bMale action groups: groups of men formed to support community mobilization and dissemination efforts typically found in Northern Uganda. These men were selected by community members based on trust and respect.
Figure 4. Number of couples counseled, December 2014- March 2017.
Annotated run chart with examples of change ideas implemented over time. aMeeting men in Malwa/beer drinking joints: Village Health Team (VHT) members meet with men at popular drinking joints to talk to them about family planning (FP) services using their elevator speeches. bMeeting men at moto-taxi stands: VHTs meet with male motorbike taxi drivers at their waiting stands to talk to them about FP services using their elevator speeches.
Characteristics of respondents
We interviewed six key informant groups across the community, health center, district and central levels. Table 2 shows the average age of the VHTs who participated in the FGDs was 44.2 years, with an average of 12.7 years of experience as VHT, of which 3.8 years included CBFP experience. The average number of clients per month for VHTs in both districts was 14.7. The Busia VHTs had more years of experience overall (13.8 years vs. 10.8 years) and in CBFP (4.9 years vs. 1.8 years) compared to those in Oyam. Midwives had an average of 7.9 years of work experience, but three out of the four midwives interviewed had been at their designated health centers for less than a year, indicating the endemic challenge of staff turnover. Clients’ average number of years on an FP method was 2.6 years, with the most commonly reported method choice being an injectable (e.g., DMPA-IM, DMPA-SC).
Table 2. Characteristics of respondents.
Learning site’s experience with a QIC
The key features of the QIC in Busia were the application of the PDSA model to test and implement change ideas, supervision and mentorship of VHTs by midwives, monthly meetings and collection and use of data by VHTs. Factors supporting QIC implementation include all factors that emphasized or maximized making changes in VHTs’ practices or their learning. There were differences in how the PDSA was described by participants, particularly between the VHTs and higher-level staff (e.g., midwives, in-charges, district QI focal persons and district health officers). Higher-level staff referred explicitly to “PDSA,” while most VHTs and some midwives described the process of looking at the data, assessing where there are gaps and finding solutions to improve performance. VHTs focused on specific change ideas that they had implemented as part of their CBFP routines. Participants in every higher-level cadre noted being surprised that VHTs could learn to use the PDSA model, which had mainly been used by health workers for clinical processes:
“With the VHTs it was very interesting that in the beginning, I could not believe that they could apply the PDSA […] it was very important because […] after identifying specifically what their communities are, they could identify their change ideas, go try them, then give feedback themselves. They have applied and then they say this one has worked out, we want to continue with it. […] Others say they are not working, they have modified them, others they have abandoned.” – Busia district official
The district official saw that the PDSA process allows different communities to adapt ideas to their contexts, test them on a small scale, and then abandon or adapt and scale them up. The use of data and visuals, such as run charts (Figure 3, Figure 4), to track progress gave VHTs a stronger evidence base for deciding whether to retain, modify or abandon their change ideas:
“To know that we have worked hard, we look at our run chart to see the indicators — whether we have dropped or increased. If we dropped, we look at our change ideas and modify them to see a change from dropping to increasing for the next month.” – Busia VHT
Participants also mentioned the value of having more structured monthly meetings and the importance of supervision and mentoring by midwives for improving quality. All midwives reported meeting monthly with the VHTs prior to the QIC, but only so the VHTs could submit their monthly health management information system reports. These meetings are now being used to discuss achievements and challenges, study the data, learn and encourage one another. Responses indicate that quality improvement activities can be integrated into existing meetings:
“These monthly meetings started before the QI approach anyway, but with the QI approach, it has helped because there were certain areas that before we were not covering. But from the time we were implementing QI, we improved on those areas. We realized that the VHTs were not giving enough information on family planning, as in they were not giving information on all available methods, but only ones they are giving. But now they are able to give all those information.” – Oyam midwife
“Our coming here [to the health center] monthly has helped us to know what challenges our fellow VHTs are facing in their communities and what challenges I am facing, and when we sit and discuss, we come up with solutions to those challenges.” – Busia VHT
The changes the VHTs considered most effective were adequate counseling with job aids, home visits/client follow-up, sensitization and mobilization and male involvement activities, described in Table 3.
Table 3. Change ideas considered most effective by Village Health Teams.
Factors cited as contributing to the maintenance and sustainability of QIC were the inclusion of local leaders in the first LS in Busia (which included training with guidance materials/job aids) and ensuring client confidentiality. By engaging clients in the assessment and each LS throughout the QIC, VHTs became more aware of the importance of trust and confidentiality.
Experience with the QIC in scale-up
During a joint LS in Busia, Oyam midwives and VHTs were paired with their counterparts from Busia. The Busia midwives and VHTs shared their QIC experiences, including what worked and their implementation of the QIC. The lessons from that experience most commonly mentioned by Oyam respondents in the FGDs and IDIs were mentorship by midwives, data collection and use, infection prevention and using the PDSA process.
Busia midwives shared their client registers, run charts, mentorship schedules, approaches to monthly meetings, use of a supervision checklist, and how they delegated responsibilities to higher-capacity VHTs at their health centers (i.e., compiling monthly VHT reports) to reduce workloads.
“From Busia I learned that the other midwife was having monthly meetings, and another thing she was doing … mentoring. Now I always sit down with them at the end of each week to see what they are not performing and what each one is not doing well, and I mentor.” – Oyam midwife
“Her [Busia midwife’s] ideas helped me a lot. I used to do data collection alone. But now my VHTs will make the summary. That has really helped me a lot, because I was even breaking. But now, I feel relieved.” – Oyam midwife
Busia VHTs showed the Oyam VHTs their VHT client registers and how they entered data. They also discussed how they conducted adequate counseling with the job aids, their change ideas and building relationships with their clients. Oyam participants then visited a VHT’s home to observe how privacy was achieved, storage of supplies and infection prevention. Observing how Busia VHTs maintained infection prevention seemed to make a lasting impression on Oyam VHTs:
“Before we had gone to Busia, after administering the drug to the clients we disposed of it not in the way we were supposed to […] but after we went to Busia we learnt that we should dispose them [cotton wool, needles] off separately.” – Oyam VHT
“They also gave information to prepare a point for washing hands, one at the entrance of latrines and another near the place where I administer medication to my clients.” – Oyam VHT
When they started implementing the QIC, the Oyam team members found that some of what they had learned from Busia needed to be adapted to the Oyam context:
“We went to Busia and learned much about the PDSA cycle, where we also identified our own problems, our own weakness, our own areas where we are not performing well, and we’ll bring a change idea. I also think what worked well in Busia may also work well with us, but we believe that communities vary. We don’t have the same cultural practices and beliefs, because it is the culture and belief of the community that determines their uptake of the services that we offer to them. So, we also want to reserve our own ideas. But what we learned from them is how to go on with the PDSA cycle. Even more new ideas will work.” – Oyam district official
Oyam VHTs and midwives were inspired by many good practices in Busia but had the freedom to adapt those ideas. They also showed that effective mentoring is possible with simple supervision tools and practices, such as checklists and spot checks. They reported frequent peer-to-peer learning between VHTs and midwifes. Overall, their responses indicate that well designed learning exchanges are a critical step in the scale-up process.
Perceived effects of QIC
The greatest perceived direct effects of the QIC were related to improved client satisfaction and trust, changes in the ways the teams worked and improved quality of VHT services. All VHTs and midwives reported increased client satisfaction and trust in the CBFP services delivered by the VHTs due to the improved quality of those services, which they said had resulted in increased numbers of FP clients and better client retention. Midwives and VHTs both reported improvements in VHTs’ ability to counsel clients on side effects and refer them to the health center when necessary for long-acting and permanent methods, serious side effects and/or complications.
“[Unnecessary] referrals were minimized because most of them were running to me, ‘ah, the VHT gave me depo, I’m bleeding, I’m feeling like this…’ those side effects, so when they emphasized critical counseling, we’re seeing less at the [health center] because they explain to the client, you are most likely to get such and such side effects, but if you get such and such side effects, or your body is not yet used so you might feel like this, so they came to understand that now.” – Busia midwife
“The work the VHTs have been doing is great, because if I get a complication on the method I am using for family planning, she would help us, because coming to the health center was a far distance and her being in the community helps us go to her for medication and saves us the cost of transport and medicine to use on time, because sometimes it finds you without any funds to come to the health center, so we call her when we need her.” – Busia client
An emphasis on teamwork allowed for shifts in responsibility, freeing midwives to focus on complicated cases. Both VHTs and midwives described improved relationships with one another, describing how VHTs are more likely to seek the help of midwives and midwives take greater initiative in supporting and mentoring their VHTs through phone calls, spot checks at their homes and one-on-one supervision sessions.
Some indirect effects of the QIC were also reported by health center staff. These include changes in the overall health center’s QI efforts and integration of FP QI practices in other health service areas.
“[The QIC is] more visible in maternity. Everything is well displayed, it’s a talking environment, people are talking about what is happening there. Most colleagues that go there are appreciating that [QI] is being practiced. So, I think [CBFP QIC] has spilled over […] they have made sure that other areas also get quality improvement.” – Busia in-charge
“These days we integrate our work with the change ideas, because we are used to it and have been able to kill two birds with one stone […]so it is not limited to only FP.” – Busia VHT
Challenges to implementation of the QIC
Table 4 highlights the main challenges to implementation in each district. In Oyam, the primary challenges to implementation revolved around the complexity of the QIC in the beginning, but over time VHTs reported becoming more comfortable with the process. By comparison, Busia VHTs had fewer concerns about complexity and expressed greater confidence in their understanding of the indicators and data; they also had several more years’ experience implementing CBFP. The Busia VHTs’ main concerns were related to having a supportive environment to implement quality CBFP services. The differences in expressed challenges demonstrate an expected pattern in pilot (Busia) and scale-up (Oyam) QI teams. Pilot teams typically received more capacity building and support directly from QI specialists, as opposed to scale-up sites where the focus is on peer-to-peer learning. Therefore, the pilot teams were more familiar with the terminology and methods of measuring progress and were able to move to the next stage of discovering system challenges.
Table 4. Main challenges.
Participants from both districts mentioned the challenge of transportation costs and the limitations of the quarterly travel stipends provided by the project. The travel stipend was provided for VHTs to attend monthly meetings at health centers with the midwives. Some VHTs spend hours traveling from their villages. Many suggested that the stipends were not sufficient; VHTs incur additional costs when they conduct home visits in their large catchment area, and midwives incur costs conducting supervisory spot checks at VHTs’ homes. Frequently these costs are out-of-pocket expenses.
Another challenge was the availability of stock and supplies. As the number of FP clients increased, VHTs had difficulty projecting commodity requirements and would take either too much or too little. Finally, VHTs said that engaging men in FP was challenging, but necessary. VHTs reported not having enough time to counsel women adequately, because their husbands were not aware that they were using FP. Clients were often rushed and/or unable to return on time for fear of being caught by their husbands. VHTs also reported that men often believe in myths and misconceptions, are more difficult to meet during seasons of planting and harvesting when they work in the fields and can be abusive toward VHTs for introducing their wives to FP methods without their consent.
Sustainability and institutionalization
Sustainability and institutionalization of the QIC, with Busia continuing to serve as a learning site, is essential for longer-term change in the delivery of quality FP services. Three areas arose in the interviews highlighting the potential sustainability and institutionalization of the QIC: 1) the integration of QI into other service areas; 2) district-level plans and support for QI in general and for CBFP; and 3) motivation of QI teams, especially VHTs and midwives. All health center staff and district officials from both districts expressed the utility of applying the PDSA approach to all health service areas and engaging VHTs in the process. Health workers reported integrating QI with other maternity services, immunization, HIV care and treatment, and health promotion and sanitation. A new project has been introduced specifically for QI in immunization, which has benefited from the training the VHTs had received. Busia proposed a budget for CBFP in its district costed implementation plan for FP. However, financing for QIC-specific activities centrally and at the district levels remains a challenge, and there are concerns about ensuring a continuous supply of FP commodities. Thus, while there is strong support and commitment for Busia to continue as a CBFP QIC learning site, sustaining it without committed government or project resources will be a challenge:
“The idea of having a learning center or learning district is very important, especially in a resource limited setting. When you introduce something, people think it cannot work, but when you tell them that another district with the same environment has begun to do it, then they can think, okay, we can do it. So, it’s very important. That’s why we put it in the [national QI] framework.” –MOH official
“When [FHI 360] leaves us, it doesn’t mean they will collect the materials or knowledge that they have given us. The knowledge remains. I [am] happy about the mechanisms that they have given us, and will remain in us, for example, using the PDSA. As volunteers we can still come together, plan what we are going to do. […] The knowledge…shall maintain.”- Oyam VHT
Discussion
Global evidence has shown that CHWs are competent providers of CBFP services, including injectable contraceptives30–32. However, competence does not equate to quality health service provision. Greater efforts are needed to ensure that CHWs are continuously supported, mentored and monitored for improved service delivery to make effective gains in decreasing the unmet need for FP and increasing the access and use of modern contraceptive methods. This process evaluation shows that the QIC approach applied largely at the health facility level can be adapted and applied to community-based service delivery models.
Table 5 summarizes how the QIC approach supports and emphasizes the Bruce Framework components for quality FP services. QIC efforts to address the six elements of the Bruce Framework were not dependent on highly innovative or complicated change ideas. Rather, VHTs were provided a safe space to thoughtfully discuss their challenges and achievements, identify what they were doing right or incorrectly and support one another to try new ideas to achieve their improvement goals. The activities described are iterative and require time for learning and building trust in relationships among team members and patience for mentoring, with a long-term goal of continuous improvement through an institutionalized collaborative process.
Table 5. How collaborative improvement supports Bruce Framework components.
This is the first process evaluation of a community-based QIC model in CBFP in Uganda. Bazos et al. conducted a proof-of-concept study of a microsystems QI approach to strengthen routine immunization in Uganda33. While the QI approaches are similar in their emphasis on identifying local solutions, Bazos et al. focused their approach on the routine immunization system, which is facility-focused, with immunizations provided by clinic-based providers. Tancred et al. implemented a community-level QI program with village volunteers in neighboring districts in Tanzania and Uganda for maternal and child health services9. The QIC described in this paper focuses on FP, with the VHTs serving as the core service delivery providers at the community level. In addition, while some health centers had established a facility QI team through previous QI projects, this program was the first to implement a QIC with VHTs at the core, extending PDSA cycles to service delivery by community-based providers. Engaging VHTs as integral players in the QIC strengthened the linkages between the health services provided in communities and those provided in health centers.
Ludwick et al. evaluated CHW performance in Uganda and found that factors related to supportive supervision and relationships with other health workers may be strongly associated with variances in performance outcomes34. Studies have highlighted the importance of supportive supervision, supervisor legitimization of CHW roles to communities and positive interactions between CHWs and health workers and their linkages to CHW motivation and performance35,36. The QIC is an approach that may support improved overall CHW performance. With the QIC, a culture of mentorship has emerged that replaces the formal hierarchical midwife-VHT relationship, with peer-to-peer learning accelerating learning and scale-up from district to district. QICs focus on improvements based on collection and reflective analysis of data and testing of small, feasible changes, rather than a didactic pedagogical approach.
Pilot sites experienced accelerated scale-up of the approach to other sites through the deliberately designed peer-to-peer knowledge and experience transfer process. At the same time, there was room for adaptation of change ideas to the new contexts, as was shared by Oyam key informants. Our process evaluation showed that there was flexibility in the QIC scale-up process – through the evolution of change packages – contributed to enhanced peer learning, ownership of the model and successful scale-up. This focus on local adaptation to each scale-up site is a unique characteristic of this QIC program as change packages are often static from the first site and tend to be scaled more prescriptively. In addition, all change ideas were locally driven and culturally appropriate because they were identified by VHTs, who are closer to communities than health providers. The emphasis on trust between VHTs and clients in our findings shows that community-based QICs can be successful in being client-centered.
Some of the challenges that were voiced by participants can be addressed by adjusting tools and terminology for VHTs and clients, being more gradual in capacity building of new teams and simplifying the change identification process. However, the challenges also show the limits of the improvement process. Availability of supplies, for example, is a systems issue that cannot be resolved at the community level or by improving service delivery. On the other hand, progress achieved in male engagement shows that a collaborative can tackle even complex sociocultural factors.
Working with CHWs on a QIC presents different challenges as compared to working at the facility level. In Uganda, VHTs are volunteers, with low levels of literacy and no previous data analysis experience. However, their enthusiasm and creativity proved important to improving processes. We also demonstrate that despite their literacy levels, VHTs given adequate support through mentorship and learning sessions are capable of implementing the PDSA concept, data analysis and continuous improvement. To strengthen the implementation of this CBFP QIC and other community-based QICs, we provide program recommendations in Box 1.
Box 1. Program recommendations
Embed the quality improvement collaborative (QIC) approach in the Village Health Team (VHT) curriculum.
Incorporate recording, plotting, and interpretation of data into VHTs’ and midwives’ training.
Engage clients as key stakeholders in the improvement effort.
Conduct inter-facility and inter-district joint learning sessions to increase VHTs initiative and desire to perform better and to accelerate adaptation of proven change ideas.
Recognize the contributions of VHTs and provide them with transport stipends to enable them to submit reports regularly.
Include budgets for community-based family planning (CBFP) and QI initiatives in FP costed implementation plans.
Have an integrated QI approach at facility level, so that the CBFP QI efforts complement other QI projects for non-FP services.
Consider stock availability and building the capacity of VHTs and midwives to forecast their supply needs as quality improves and demand for services increases.
Develop an adaptable resource package based on Busia’s experience as a learning site for other implementing partners and the government to adopt or adapt for future QIC programs. The resource package should be housed in a central institution, such as the Ministry of Health’s QI coordination committee, which coordinates the national QI strategy.
Results from this process evaluation are not generalizable, particularly as it is largely qualitative. Selection of health centers and VHTs from Busia and Oyam districts to participate in the QIC was not random, resulting in possible selection bias. Busia district was considered a high performing CBFP district lending it to serve as the QIC learning site, but not representing an average performing district in the country. The data used in this evaluation may also be at risk of measurement error, as they were reported by individuals. However, the many lessons learned and the program description may be useful in supporting other programs interested in replication and adaptation efforts.
Conclusions
This process evaluation shows that it is feasible to implement the QIC approach to improve service delivery by VHTs. Our data suggests that such QICs can increase uptake of FP and improve retention of clients on FP methods.
Data availability
Underlying data
Due to data protection issues, publicly sharing data from the qualitative interview transcripts is not feasible. While identifying names have been removed from the transcripts, based on the information provided in the interviews, it would be possible to identify the participants given the position of the respondent, their health center, and shared experiences. We also state in the IRB-approved and waived verbal consent forms that we would share results only, that only study team members would have access to the interview data, and no information identifying the participant would be shared. Additionally, we do not explicitly mention sharing the underlying data from the interviews. While intermediary transcript data maybe not be able to be fully de-identified without compromising anonymity or the quality of the transcripts, depending on the purpose of the request (for example, a request by researchers or implementers of community-based QI for the purpose of informing QI program scale-up), we may be able to discuss the access and use of the qualitative data, if approved by all relevant parties (IRB, program staff, and research staff). This would be done on a case by case basis – the corresponding author can be contacted for initiating discussions on data access and use.
Zenodo: Program data - quality improvement for CBFP. http://doi.org/10.5281/zenodo.264762429
This project contains the following underlying data:
Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).
Extended data
Zenodo: In-depth Interview and Focus Group Discussion Guides. http://doi.org/10.5281/zenodo.264762922
This project contains the following extended data:
- 1_QI_Interview guides_VHTs Busia.docx
- 2_QI_Interview guides_VHTs Oyam.docx
- 3_QI_Interview guides_Midwives Busia.docx
- 4_QI_Interview guides_Midwives Oyam.docx
- 5_QI_Interview guides_Incharges Busia.docx
- 6_QI_Interview guides_Incharges Oyam.docx
- 7_QI_Interview guides_DHO Busia.docx
- 8_QI_Interview guides_DHO Oyam.docx
- 9_QI_Interview guides_MOH representative.docx
- 11_QI_Interview guides_MaleClients Busia.docx
- 12_QI_Interview guides_FemaleClients Busia.docx
- 13_QI_Interview guides_CouplesClients Busia.docx
Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).
Download
Export To
metrics
| Views | Downloads | |
|---|---|---|
| Gates Open Research | - | - |
|
PubMed Central
Data from PMC are received and updated monthly.
|
- | - |
Citations
CITE
how to cite this article
Kim C, Kirunda R, Mubiru F et al. A process evaluation of the quality improvement collaborative for a community-based family planning learning site in Uganda [version 1; peer review: 2 approved]. Gates Open Res 2019, 3:1481 (https://doi.org/10.12688/gatesopenres.12973.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.
Alongside their report, reviewers assign a status to the article:
Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations -
A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions
Competing Interests Policy
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Examples of 'Non-Financial Competing Interests'
- Within the past 4 years, you have held joint grants, published or collaborated with any of the authors of the selected paper.
- You have a close personal relationship (e.g. parent, spouse, sibling, or domestic partner) with any of the authors.
- You are a close professional associate of any of the authors (e.g. scientific mentor, recent student).
- You work at the same institute as any of the authors.
- You hope/expect to benefit (e.g. favour or employment) as a result of your submission.
- You are an Editor for the journal in which the article is published.
Examples of 'Financial Competing Interests'
- You expect to receive, or in the past 4 years have received, any of the following from any commercial organisation that may gain financially from your submission: a salary, fees, funding, reimbursements.
- You expect to receive, or in the past 4 years have received, shared grant support or other funding with any of the authors.
- You hold, or are currently applying for, any patents or significant stocks/shares relating to the subject matter of the paper you are commenting on.
Stay Updated
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Register with Gates Open Research
Already registered? Sign in
Comments on this article Comments (0)