Client-oriented, provider-efficient services (COPE ^®) is a quality improvement methodology first developed in 1995 by EngenderHealth to address clients’ rights to health services and provider needs to deliver quality services ^{12, 13} . Since that time, numerous iterations of COPE for different technical areas have been tested and published, including adaptations to improve the quality of services in reproductive health, HIV care and treatment, male circumcision, and abortion care, among others ^{14– 18} . The COPE ^® for Contraceptive Security methodology and tools are used by frontline health and logistics personnel to identify and implement low-cost, local solutions to address problems related to contraceptive supply ^{19, 20} . The process incorporates staff accountability and linkages with district supervision systems and community and local leadership, as needed, creating ownership in improving quality and strengthening systems for sustainability.

The COPE for CS process begins with an exercise conducted by trained facilitators to orient facility teams on the activity. Once staff agree to tackle the issues under consideration, facility teams complete a series of self-assessment guides on issues ranging from stock management, reporting, requisition, transportation, warehousing and personnel. Problem identifications lead to staff developing action plans to address local bottlenecks and issues that the facility and district can try to address themselves, and formation of a COPE for CS committee to oversee implementation and follow-up of their action plan. A job aid is available to foster continued reflection and reanalysis of issues during implementation and for use in district supervision ²¹ . Intended for global use, COPE for CS was originally designed and tested in Tanzania from 2011 to 2013, where results showed statistically significant improvements in contraceptive availability and increases in family planning use after more than one year ^{11, 22} .

In Malawi, the Ministry of Health (MOH) made remarkable progress in improving family planning access over the past 15 years. Modern contraceptive prevalence among married women increased from 28% in 2004 to 42% in 2010, and again to 58.1% by 2015–2016 ^{23– 25} . The MOH made strides toward achieving contraceptive security at the national level, while noting that contraceptive security is weaker at district and lower-level health facilities. In 2014, the MOH Directorate for Reproductive Health (DRH) requested assistance to address contraceptive security at the last mile. Challenges identified at the local level included: lack of trained providers (especially for long-acting reversible contraception (LARCs) and permanent methods); unclear roles and responsibilities for logistics management; and, lack of training in requisitioning and ordering (which can lead to stock-outs of contraceptives and related supplies) ²⁶ . Box 2 shows the programmatic process for COPE for CS introduction and implementation in the two districts as well as 2015–2016 implementation in additional districts and facilities at the request of MOH/DRH.

To understand the intervention, it is necessary to understand the structure of the COPE for CS process. Site facilitators, trained in the methodology and tool, lead a facility team through an initial exercise and their continuous quality improvement efforts. COPE for CS is designed to be easy to implement without outside technical assistance and is adaptable for local facility contexts. The site facilitators, in coordination with facility leadership, determine the length of their initial COPE for CS facility exercise based on workflow at the facility, usually consisting of several hours in the late afternoon when client flow is slower over two to three days. In larger facilities, staff from multiple departments are asked to join the exercise, including those outside family planning and/or logistics management. For example, facilities are encouraged to invite cleaning personnel who use supplies for infection prevention and guards who may be a client’s first point of contact at the facility. In smaller facilities, the entire staff may work on COPE for CS together for the duration of the work. Whatever staff configuration is chosen, the goal is to have teams with first-hand operational experience with different types of challenges within the facility and who want to participate in problem solving with their colleagues, generating a shared sense of ownership for results ¹⁹ .

The COPE for CS tool includes 10 self-assessment guides containing a series of questions, based on international standards, regarding the quality of services, systems, and procedures (see Box 3) ¹⁹ . Site facilitators and teams review the guides during the initial exercise and can complete individual assessments as a team, in small groups, pairs, or as individuals, depending on their preference and the staff participating. Each guide includes instructions on which type of staff, by function, is best placed to respond to its questions. The guides are designed to be flexible and adaptable to the facility team’s needs. Staff can write in issues that are not directly raised by a guide and are relevant, and can choose to skip questions they do not find relevant to their context. In addition, the team is not required to complete all 10 assessments at one time; rather, they can prioritize which guides to use and at which points in their exercises/process. After completing guide(s), the facility team reviews and identifies issues at the site as revealed by assessment questions.

Following discussion of any one of the self-assessment guides, facility teams develop an action plan to consolidate and prioritize recommendations. Action plans identify problems related to CS; identify the root cause(s) of each problem; propose action items that are realistic, measurable, attainable, and address the root cause(s); assign an individual facility staff member responsibility for each action item; ensure a time-bound goal for completion of each action item; and provide space to comment on action item status and result. Box 4 shows an excerpt from a study facility’s action plan.

COPE for CS emphasizes targeting problems with root causes at the facility level and within facility control. Facilities are also encouraged to include action items that address problems at the district level, if teams can identify a pathway through which the facility may affect change. For example, a facility requiring more trained staff in logistics management, or more trained providers in FP provision, may advocate to district-level management to assign additional personnel to the facility in need. National problems identified are not recommended for inclusion as facility action items.

In addition to completing self-assessment guides and developing action plans, facility teams form a COPE for CS Committee to ensure follow-up and monitor action plan progress. Opportunities to discuss action plan progress include: regular staff meetings, special committee meetings, and during district supervision. COPE for CS Committees may decide to conduct additional full team COPE for CS exercises and continue to complete self-assessments as new issues arise. The COPE for CS job aid is another resource for staff to revisit key self-assessment issues in an abbreviated manner.

Committee members are encouraged to post the facility’s action plan in a visible place where all staff and the public can see it to show the site is dedicated to quality improvement, to encourage accountability and transparency, and to monitor progress against goals. Committees and facility leadership are encouraged to share their action plans with local stakeholders, health advisory committees, politicians, implementing partners and community organizations, as an advocacy tool to request assistance and resources.

The introduction and scale-up projects supported training of trainers, site facilitator trainings, facility exercises, and limited follow up visits, to check on action plan progress and provide space for COPE for CS Committees or facility leadership to ask questions about the methodology or seek implementation guidance as needed.

However, the intervention did not include additional inputs to improve contraceptive security at the 60 sites. The COPE for CS initiative purposefully did not provide technical assistance for clinical or logistics training, for example, nor for the procurement of contraceptives or required equipment and supplies. The idea is for facility staff to look for local solutions. The 10 supported districts received varying levels of support for health services, including FP, from other multilateral agencies and partners. Following COPE for CS exercises and action plan development, the facilities may clearly articulate their needs to district leadership who coordinate donor funding in the decentralized Malawian healthcare system.

We obtained data for this analysis through facility surveys designed to collect facility-level data on performance related to contraceptive security.

In 2014 the Malawian MOH/DRH purposively selected 18 facilities from 2 districts (Salima and Magochi) to participate in the study. In 2015, the MOH/DRH selected 42 additional sites in 8 additional districts (Balaka, Chikwawa, Lilongwe, Mzimba North and South, Nkhotakota, Ntchew, and Thyolo) for the scale-up phase, increasing the total number of study facilities to 60. The MOH/DRH considered districts for inclusion if they reported stockouts in a high percentage of facilities and had previously submitted requests for assistance with contraceptive security. Within these districts, the MOH/DRH and district medical officers selected which facilities would introduce the COPE for CS methodology using the same criteria.

Data collection occurred in three-waves as described in Figure 2. For all facilities, baseline data collection occurred before the start of the intervention (in 2014 for preliminary facilities and 2015 for scale-up facilities). Data was also collected at preliminary facilities in 2015 as a midline assessment. Endline data collection occurred for all sites in 2016. In the 18 preliminary facilities the COPE for CS intervention was introduced between the 2014 and 2015 assessments and continuously implemented through 2016. In the scale-up facilities, the COPE for CS intervention was introduced between the 2015 and 2016 assessment points.

We used a standardized, facility questionnaire to assess each of the dimensions of contraceptive security, as defined by COPE ^® for contraceptive security: An assessment guide (RESPOND, 2013). These dimensions included: 1.

Organization and staffing;

EngenderHealth staff trained the data collection team on survey administration. For each of the survey rounds, a one-day data collector training was held during which data collectors reviewed the questionnaire and staff provided instructions on its administration. This training also instructed and assessed understanding of standard precautions for protecting human subjects, and included role-playing exercises.

Data collectors conducted the facility survey in English using a paper questionnaire. Interviewers identified the Facility In-Charge or their designate in each of the facilities to obtain permission to conduct the assessments. Following informed consent procedures, they administered the facility questionnaire. The facility survey tools are included as extended data to this paper ³⁷ .

One data collector implemented the facility assessment at each facility. At the conclusion of each assessment, data collectors sent the completed questionnaire to EngenderHealth project staff, who reviewed the questionnaire to ensure it was complete.

A project staff member trained in appropriate coding and data entry techniques entered data from the physical questionnaires into a Microsoft Excel (Microsoft Corporation, Redmond, Washington, USA) database. A project manager then reviewed data entry, comparing the physical questionnaires to the database and documenting any discrepancies and subsequent discussion and resolution.

The 2014, 2015 and 2016 survey protocols received ethical approval from an EngenderHealth review board. External review was not obtained given that the research did not meet the threshold to quality as research conducted among human subjects. The data were primarily collected to inform program decision making and questions were not focused on individual perspectives; rather, they focused on health facility capacity, staffing, stock and related issues ²⁷ . Data collectors conducted facility assessments only after administering a standard informed consent form. We employed standard measures to maintain confidentiality and anonymity for the facility staff respondent. All information collected was strictly confidential and used only for study purposes. Respondent names were not stored with the final clean data.

Input variables include those related to how the intervention was implemented, such as the number of days spent on exercises, frequency of group discussion of the COPE for CS action plans, and the number of COPE for CS committee meetings in each facility.

We developed variables pertaining to action plan quality, content, implementation, and commitment. Project staff assessed action plan quality according to a scoring rubric developed a priori (see extended data ²⁸ ) according to several quality dimensions including: whether the problems were clearly identified, whether the action plan identified root causes, whether the action plans offered attainable/realistic solutions, whether individuals were assigned responsibility, and whether items that were identified were time bound. Staff conducted action plan content mapping to determine whether a facility identified items relating to staffing, LMIS, procurement/requisition, inventory control procedures/receiving supplies, warehousing and storage, transport and distribution, financing/budgeting, planning, and medical equipment/instruments/expendable supplies. Box 5 further details the development of variables related to action plan quality.

We measured commitment to COPE for CS action plans by the frequency of group discussion of the action plan and the number of COPE for CS committee meetings. Project staff assessed completion of items in a facility’s action plan based on facility reporting. After reviewing reported plan updates, project staff calculated the number of action plan items that were not initiated, initiated but not yet complete, and completed at endline.

Outcome variables are divided between intermediate and ultimate outcomes. Intermediate outcomes relate to changes in facility performance according staffing, LMIS, supplies/equipment, storage, and procurement, and were developed based on existing literature and expert consultation ^{20, 29, 30} . We developed a detailed composite score in order to assess changes in facility performance ( Box 6). Ultimate outcomes pertain to performance measures that may be indicative of improvement in overall facility performance. These indicators were adapted from the RHSC’s harmonized list of CS indicators, including number of emergency orders for contraceptives that a facility placed in the three months prior to assessment and the number of contraceptive methods available at a facility on the day of the survey ^{31, 32} . We defined method availability as whether the facility had the commodity in stock, all required method-specific equipment, and a provider who is trained in provision (and removal, if applicable) of a given method.

Due to the possibility of confounding variables to influence both the implementation of the intervention and the outcomes of interest, we stratified results according to a variety of important facility-level variables on which data were available including facility location (urban/rural), facility type (health center or hospital), region, and baseline facility performance.

The data available on implementation provide important insight on which components worked well and which could improve in the future. The majority of facilities implemented the intervention according to plan, meaning they held exercises, developed action plans, displayed action plans in a visible location in the facilities, assembled CS committees, had broader group discussions among their staff. Despite the leeway given to facilities to implement in the way that they thought to be most helpful, we observed little variation in implementation. For example, nearly all facilities held the initial exercise in three days. Additionally, the facility action plans tended to be of high quality based on the a priori rubric developed to evaluate them. Given the limited variation observed in quality, however, it is difficult to assess the relationship between the quality of a facility’s action plan and changes in CS performance. Future research may build on the quality rubric presented here to be more discerning. One issue to note is that the quality rubric does not consider whether a facility distinguished areas of low baseline performance and specifically identified action plan items related to those areas.

The data also point to areas for future implementation research and possible improvement. Facilities identified few action plan items in the domains of procurement and equipment, despite low performance in these areas. It is possible that facilities determined that, to realize results, progress in procurement and equipment would require a longer-term advocacy strategy more dependent on the system as a whole and outside of the control of the facility itself. In the future, working with facilities to better identify actionable areas of the local system in these domains, and supporting sites with advocacy approaches for local financing and community participation with in-kind donations, may be an area for improvement in the COPE for CS exercise and action plan implementation.

Facility performance improved as a whole across CS dimensions between baseline and endline. Given the nature of the intervention, these results suggest that facilities successfully advocated for resources and other inputs needed to improve performance, perhaps including trainings or additional staff assigned to the facility. The data show large improvements in the technical capacity of facility staff – especially in health centers. Additionally, health centers developed staff capacity in IUDs, which goes above and beyond the minimum service requirements for that type of facility. Of note, the projects did not provide trainings or other support to facilities aside from the initial COPE for CS exercise and a brief follow up visit a few months later to discuss action plan progress. Therefore, the changes observed were driven from within each facility.

Additionally, the findings suggest that after participation in the COPE for CS intervention, facilities were better able to accommodate a wide variety of client needs. By endline a large percentage of facilities met requirements to serve clients selecting a new method, or switching from or discontinuing a method requiring removal, as discussed in Table 6. Also, the decreases observed in the number of emergency orders placed may indicate that the CS system within facilities improved as a whole.

More frequent staff discussion of action plans—a proxy for staff commitment—is associated with improved outputs and outcomes, in particular with action item completion. As staff commitment and engagement to facility quality improvement is a key underpinning of the COPE for CS methodology, this finding provides a proof of concept that staff action is a key mechanism of action. Interestingly, we found no evidence for association between the number of official COPE for CS committee meetings and output/outcomes. However, any association between committee meetings and performance improvement is difficult to assess as there was little variation among facilities in the number of these meetings and more data on meeting frequency and timing is needed.

This is an important finding regarding sustainability of the intervention. The sub-analysis of the baseline, midline, and endline using data from the preliminary facilities suggests that improvements tend to be sustained over time (measured up to two years post-initiation), despite there being very limited additional investment by the projects. Of note, the preliminary facilities did hold another project-supported facility exercise at the same time that the scale-up sites held their initial exercises. However, the data do not suggest that holding another formal exercise had a major impact on facility performance. Figure 4 shows that preliminary sites achieved their largest gains between 2014 and 2015, with minimal gains after that point.

An interesting observation in assessing the longer-term data is that there appears to be a slight lag in the improvement of equipment scores. While it is impossible to rule out the potential influence of the second supported exercise, it is possible that equipment performance takes more time to improve. Changes in these dimensions may require larger investment, action at higher-level of the health system that may be difficult for a facility to influence, and/or longer-term advocacy and planning. Future research could examine sustainability of the intervention in a more rigorous way, and explore whether there is a need for additional supported facility exercises.

Harvard Dataverse: Replication Data for: COPE for Contraceptive Security Facility Dataset v 2.0, https://doi.org/10.7910/DVN/7XQL6X ²⁸

This project includes the following underlying data files:

Malawi COPE for CS_baseline database_April 2015_FINAL stata.xls

Harvard Dataverse: Replication Data for: COPE for Contraceptive Security Facility Dataset v 2.0, https://doi.org/10.7910/DVN/7XQL6X ²⁸

This project includes the following extended data files:

COPE for Contraceptive Security in Malawi Facility Assessment Questionnaire

All data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).