Gates Open Res Gates Open Research 2572-4754 F1000 Research Limited London, UK 10.12688/gatesopenres.12944.1 Research Article Articles ‘ It is a question of determination’: a case study of monitoring and evaluation of integrated family planning services in urban areas of Togo

[version 1; peer review: 2 approved]

 Baker Helen Conceptualization Data Curation Formal Analysis Investigation Methodology Supervision Visualization Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0002-2882-6723 a 1 Rochat Roger Conceptualization Methodology Supervision Writing – Original Draft Preparation Writing – Review & Editing 2 Hepburn Kenneth Conceptualization Methodology Supervision Writing – Original Draft Preparation Writing – Review & Editing 1 Hennink Monique Conceptualization Formal Analysis Methodology Supervision Writing – Original Draft Preparation Writing – Review & Editing 2 Thiam Macoumba Conceptualization Investigation Methodology Project Administration Supervision Writing – Review & Editing 3 Guede Cyrille Conceptualization Data Curation Investigation Methodology Project Administration Writing – Review & Editing 3 Koalaga Andre Conceptualization Methodology Project Administration Writing – Review & Editing 3 Amegan Eloi Conceptualization Data Curation Methodology Project Administration Supervision Writing – Review & Editing 3 Fombo Koffi Methodology Project Administration Resources Writing – Review & Editing 3 Ogunbiyi Bolatito Supervision Writing – Review & Editing https://orcid.org/0000-0002-9979-3471 4 Sibley Lynn Conceptualization Formal Analysis Investigation Methodology Supervision Visualization Writing – Original Draft Preparation Writing – Review & Editing 1 Nell Hodgson Woodruff School of Nursing, Emory University, Atlanta, Georgia, 30322, USA Rollins School of Public Health, Rollins School of Public Health, Emory University, Atlanta, Georgia, 30322, USA AgirPF, EngenderHealth, Lomé, Togo EngenderHealth, Washington D.C., 20004, USA helen.baker@emory.edu

No competing interests were disclosed.

 1 5 2019 2019 3 1451 8 4 2019 Copyright: © 2019 Baker H et al. 2019 This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Background: Integrating family planning into postabortion and postpartum services can increase contraceptive use and decrease maternal and child death; however, little information exists on the monitoring and evaluation of such programs. This article draws on research completed by the EngenderHealth’s AgirPF project in three urban areas of Togo on the extent to which monitoring and evaluation systems of health services, which operated within the AgirPF project area in Togo, captured integrated family planning services.

Methods: This mixed methods case study used 25 health facility assessments with health service record review in hospitals, large community clinics, a dispensary, and private clinics and 41 key informant interviews with health faculty, individuals working at reproductive health organizations, individuals involved in reproductive health policy and politics, health care workers, and health facility directors.

Results: The study found the reporting system for health care was labor intensive and involved multiple steps for health care workers. The system lacked a standardized method to record family planning services as part of other health care at the patient level, yet the Ministry of Health required integrated family planning services to be reported on district and partner organization reporting forms. Key informants suggested improving the system by using computer-based monitoring, streamlining the reporting process to include all necessary information at the patient level, and standardizing what information is needed for the Ministry of Health and partner organizations.

Conclusion: Future research should focus on assessing the best methods for recording integrated health services and task shifting of reporting. Recommendations for future policy and programming include consolidating data for reproductive health indicators, ensuring type of information needed is captured at all levels, and reducing provider workload for reporting.

 Postabortion family planning Postpartum family planning Monitoring and evaluation Togo West Africa Gates Foundation OPP1181398 United States Agency for International Development AID-624-A-13-00004 This publication was made possible by the generous support of the American people through the U.S. Agency for International Development (USAID), under the terms of the cooperative agreement AID-624-A-13-00004.The contents are the responsibility of authors and do not necessarily reflect the views of USAID or the United States Government. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Introduction

Interest in providing family planning integrated into other health services has increased in the last five years 1– 3 . While it is well established that integrating services such as family planning into postabortion and postpartum care can increase contraceptive use and decrease maternal and child deaths 2, 4 , little information exists on the monitoring and evaluation of such programs at country or regional levels 5 .

Two studies in Togo, the focus of this paper, on postabortion care (PAC) and record keeping at five participating health facility sites documented that in 2014 the monitoring systems for PAC were informal and non-standardized in four of the five facilities 6 . With further training and support, a follow up study in 2016 in the same sites found standardized PAC registers which were generally filled out at the patient level, but the information was not transferred to the district, regional, or national levels 7 .

In opposition to this lack of standardized monitoring, there is increasing interest in monitoring progress towards international benchmarks such as the Sustainable Development Goals, as well as indicators for health initiatives funded by international organizations. Some of the barriers to timely and accurate monitoring include a lack of funding and resource allocation to monitoring and evaluation 8 , a disconnect between expectations of high quality monitoring systems and the level of detail required in reporting 8– 10 , poor linkages between monitoring systems and points of data generation 11 , a shortage of trained professionals working in monitoring and evaluation 12– 15 , and the large quantities of required health indicators 8 , which often results in duplication of data collection, and frequent underutilization of existing data collection tools 16, 17 .

The best methods for reporting integrated family planning programs in practice are still being developed, especially in areas where health care is highly fragmented by type of service delivery (such as vaccination, maternity care, HIV care) 18– 20 . The published literature lacks information about the processes and strategies for reporting integrated family planning services when implemented 4 .

Project description and study aims

Agir pour la planification familial ( AgirPF) was a 5-year USAID/West Africa project (2013–2018) implemented by EngenderHealth to build capacity in and increase access to family planning, with interest in the integration of family planning into PAC and postpartum services. AgirPF was implemented in the urban and peri-urban areas of five West African countries- Burkina Faso, Cote d’Ivoire, Mauritania, Niger, and Togo, in partnership with ministries of health, the private sector, and non-governmental organizations (NGOs).

In this paper, we describe the extent to which monitoring and evaluation systems of health services, which operated within the AgirPF project area in Togo, captured integrated family planning services. We then examine the influence of values, interests, and power dynamics between key stakeholders on quality of the monitoring and evaluation system and discuss the implications for policy, programming, and research. The paper is based on research that was conducted as part of a larger case study 21 completed by EngenderHealth’s AgirPF project in April-August 2016 with the goal of understanding the status of integrated family planning in urban areas of Togo.

 Methods Study design and setting

For this study we used mixed methods including health facility assessments, health service record review, and key-informant interviews. The study was situated in the three largest urban areas in Togo, Lomé (pop. 956,000), Sokodé (pop. 114,800), and Kara (pop. 110,900) 22 . Between February and June 2016, and prior to data collection for this study, AgirPF initiated training around the integration of family planning with other health services. They trained health care workers at 35 facilities in PAC/postabortion care-family planning (PAC-FP), and health care workers at 7 health facilities in postpartum intrauterine device (PPIUD) insertion.

Procedures

Health facility assessment. We obtained a diverse, purposive sample of 25 health facilities affiliated with AgirPF including university hospitals (n=2), regional hospitals (n=3), district hospitals (n=5), large community health facilities (n=9), a health dispensary (n=1), and private clinics (n=5).

For the assessment, we adapted the Postabortion Care-Family Planning Service Availability and Readiness Assessment 23 and ISSU Enquête Finale au Niveau des Structures de Santé (Final Survey at the Health Facility Level) 2015 24 .The adapted guide contained 194 both open and closed-ended questions. For this part of the health facility assessment, we focused on 19 questions related to monitoring and evaluation of reproductive, sexual, and child health services, and reporting of integrated family planning services.

We trained a data collection team of two nurses and four midwives and pilot tested the guide in July 2016. During the pilot testing and first 3 health facility assessments, the research team discovered that health workers were adapting the government approved and widely used family planning health registers to capture additional aspects of health service integration. This included recording information about what other services the woman received (PAC, postpartum care, immunizations, child health) in addition to whether the woman came on her own or with her husband/male partner. The health facilities also were creating their own unofficial registers to capture information about PAC and general gynecological care.

With this discovery, the health facility assessment team was asked to photograph de-identified filled out health service registers, including family planning, during the health facility assessments. These registers illustrate the different ways in which the registers had been adapted when the official register did not provide a way to capture information needed for reporting to the Ministry of Health and other agencies, including international NGOs.

The team conducted the assessments in person at the selected health facilities. They spoke with a staff member who had been designated by the director of the health facility to participate in the assessment. Since taking photos of the health facilities was not in the original study plan, the data collectors took photographs of different types of available health registers (e.g. not in use or inaccessible), and if the team member had use of a digital camera or smart phone with a camera the day of the health facility assessment. On average the health facility assessments took about three hours to complete by one data collector.

We entered all the closed ended data from the health facility assessment questionnaire into SPSS 24 25 software for cleaning and analysis and compiled the responses from the open-ended questions in tables in MS Word 26 . We used univariate analysis to generate descriptive statistics related to information about reporting on integrated family planning and the services offered at the facility related to integrated family planning. The principal investigator (first author of this article) developed a simple guide to assist with the review and analysis of the photographs by record type.

In-depth interview. We purposively sampled 41 total respondents from diverse professional roles within the Ministry of Health, academic institutions, NGOs/international organizations, and health services. These included faculty at schools of medicine, nursing, and midwifery (n= 3); directors and health workers in health services (n=9 and n=14, respectively); individuals working at reproductive health NGOs/international organizations (n=9); and individuals who worked primarily in reproductive health policy and politics (these included individuals working in the Ministry of Health and NGO workers focused on reproductive health policy) (n=7). This diversity in the sample was intended to elicit a variety of perspectives on reproductive health care from individuals with varying degrees of contact with the AgirPF program activities and trainings related to integrated family planning services. Respondents were recruited by AgirPF through official collaborations with the Togolese Ministry of Health, other reproductive health NGOs, and health facilities.

We developed and pretested five semi-structured interview guides appropriate to the type of respondent. Each of the guides included questions for respondents about multiple different aspects of family planning and integrated family planning in Togo. This paper uses data from the key-informant interviews related to questions about monitoring and evaluation of family planning and integrated family planning.

Five Togolese social scientists and the principal investigator conducted the face-to-face audio-recorded interviews in French. The interviews took place at a time and location chosen by the respondents and took from 30 to 150 minutes to complete. Interview recordings were transcribed verbatim in French by the Togolese social scientists using Express Scribe software version 5.000 27 and then copied into MS Word 2013 26 .

We entered the interview transcripts into Nvivo 11 for analysis 28 and then developed initial codes as well as an initial codebook with the input of Togolese social scientists. The principal investigator then coded all transcripts using the codebook and applied thematic analysis, a rigorous, inductive set of procedures with the goal of identifying and examining themes from textual data in a way which is transparent 29 . Matrices were created in Nvivo to better understand the intricacies of the responses by participant type and location for codes related to monitoring and evaluation of reproductive and sexual health and emerging methods of recording family planning integrated into postabortion and postpartum care. Transcripts were then re-read for further understanding of themes identified, and quotes were chosen to further illustrate selected themes.

 Ethical review and informed consent

The study was approved by the Institutional Review Board at Emory University (eIRB#88781), the EngenderHealth Knowledge Management, Monitoring and Evaluation, and Research unit, and the Togolese Ministry of Health Comité de Bioéthique pour la Recherche en Santé (CBRS) (AVIS N o015/2016/CBRS du 30 juin 2016). Data were collected only after written informed consent was taken from participants using standard disclosure procedures.

 Results

The results are presented beginning with the health facility assessment and health record review followed by the results from the key informant interviews.

Health facility assessment

Description of the facilities. There were 13 facilities in Lomé, four in Sokodé, and eight in Kara, a total of 25 sampled facilities. Eleven of the facilities provided PAC prior to the start of the AgirPF project in 2013. In 2016, 24 of the facilities had some of their health care workers trained by AgirPF in PAC and PAC-FP, 6 had health care workers trained in postpartum family planning (PPFP) and PPIUD insertion. All the health facilities were supported in some way by the AgirPF project. Table 1 shows the types of reproductive and child services available at each health facility with an associated official government issued register or with an unofficial register made by each health facility.

Types of services provided, presence of reporting guidelines, and reporting tools for integrated services.
University Hospital (n=2) Regional Hospital (n=3) District Hospital (n=5) Large community clinic (n=9) Dispensary (n=1) Private (n=5) Total (n=25)
Types of services with associated reporting register
  Family planning 2 3 5 9 1 5 25
   Postabortion * 2 3 5 8 1 5 24
  Prenatal 2 3 5 9 1 5 25
  Labor/delivery 2 3 5 9 1 3 23
  Postpartum 2 3 5 9 1 5 25
  Child health 2 3 5 8 1 4 23
  Vaccination 2 3 5 9 1 5 25
Presence of guidelines for reporting integrated services
  Yes 0 0 2 5 1 2 10
Observed or reported records on family planning integrated into
   Postabortion care * 2 3 4 7 1 4 21
  Immediate postpartum 2 1 4 6 1 3 17
  Postnatal care 0 1 5 8 1 2 17
  Infant care 0 1 2 7 0 1 11
  Vaccination 0 1 5 4 0 1 11

*Italic indicates an “unofficial” register

System of reporting reproductive and child health services

The expected system. The expected flow of information in the system is from the patient to the health facility to the district, regional, national, and international levels of the Ministry of Health ( Figure 1). At the patient level, the health care worker must enter and then re-enter information multiple times about a single patient. For information to flow from the patient to facility levels, the health worker must initially enter information into a patient booklet and, for family planning and vaccination, into a patient form for these services. Then the health worker must file the patient form(s) and enter the information in the patient health booklet and sometimes a patient health form into one or more official health register(s). The PPFP/PPIUD register was the only official register which included information about integrated family planning services. This register also had a corresponding book with carbon copy sheets for the monthly reports sent to the Ministry of Health and the partner organizations.

The process of Reporting Reproductive and Child Health Care in Togo.

The expected data collection steps and tools used in the reporting process to monitor utilization of integrated family planning services in Togo.

The actual system: individual level reporting. If an official register did not exist for a given service, sometimes NGO workers involved in integration projects would tell health workers to create an unofficial register for the service using a notebook and pen, inserting columns for recording information. Unofficial registers were used most often for general gynecological care (which sometimes included PAC/PAC-FP) as well as PAC/PAC-FP registers. There was little consistency across sites – they were idiosyncratic and site-specific in the unofficial records across the sampled facilities with respect to titles and the information captured. Figure 2 shows the different examples of PAC registers in use and a list of commonly included information in the PAC register.

Examples of PAC Unofficial Registers with commonly included reported information.

Common information collected by the Togolese Ministry of Health’s registers and images of the various registers utilized by family planning services reporting system.

Family planning registers were adapted to capture when a client received family planning as part of other health services to enable the health workers to include these data on integrated health services into the monthly service provision reports ( Figure 3).

Ways of adapting the family planning register to record integrated services.

How health workers adapted registers to capture necessary family planning services information necessary to report to the Togolese Ministry of Health and NGOs.

The actual system: facility-level reporting. Within the past two years, the Ministry of Health revised its monthly service provision reporting form, Maternal and Infant Health Report, to include integrated family planning services as outlined in Table 2. However, the existing service registers were not changed accordingly. As a result, the registers did not always have the data needed to complete the new form at the facility level.

Categories of information collected monthly in the revised “Report of Maternal and Infant Health” form.
•    Women receiving family planning by method type •    Women receiving prenatal care •    Women with a referral to a high level of care due for pregnancy services •    Deliveries by method of delivery and complications from delivery •    Women receiving postnatal visits •    Women who received a family planning method immediately after delivery •    Women who received a PPIUD •    Neonatal and maternal deaths •    Women who received abortions by type (spontaneous or provoked) •    Women who received PAC and PAC-FP •    Women who received emergency obstetric care •    Availability of emergency obstetric care •    Availability of staff trained in emergency obstetric care

In addition to reporting at facility level for the Ministry of Health, the health care workers were also required to report monthly to the AgirPF project using AgirPF’s form. The respondents also prepared reports for other national and international organizations when asked.

The greatest discrepancy between the sampled facilities with respect to reporting up the chain ( Figure 1) was when and where the facility level reports were sent. All 21 of the health facilities which sent reports on integrated family planning sent them monthly. Figure 4 shows the recipients of reports sent by the health facilities. The most common responses included NGOs and the district level Ministry of Health; reports could be sent to more than one recipient.

Where reports are sent each month by the sample of health facility.

Where sampled health facilities send monthly monitoring reports and discrepancies between these facilities with respect to reporting up the chain.

 Key informant interviews

Table 3 gives information about the 41 key informants involved in the study. Two of the health faculty were trainers for PAC, PAC-FP, PPFP, and PPIUD for the AgirPF project. One health facility director was trained in PAC/PAC-FP and 3 were trained in PPFP/PPIUD. One of the health care workers was a trainer for PAC and 8 had been trained in PAC. Four of the health care workers had been trained in PPFP/PPIUD insertion.

Characteristics of key informants.
Characteristic Category of informant
Health faculty (n=3) Health facility directors (n=10) Healthcare workers (n=14) NGO workers (n=8) Policy makers (n=7)
Age (Mean, SD) 49.3 (15.0) 45.1(10.8) 38.8(8.7) 53.2(6.2) 50.3(9.1)
Sex (%)   Male   Female 66 33 70 30 14 86 75 25 29 71
Medical training (%) Doctor Physician assistant Midwife Nurse Midwifery assistant No medical training 33 33 33 0 0 0 100 0 0 0 0 0 0 21 65 7 7 0 50 0 12 0 0 38 14 14 57 0 0 14
Location of work (%)   Lomé   Sokodé   Kara   Lomé & Kara 0 33 0 66 50 20 30 0 43 29 29 0 63 25 13 0 71 14 14 0
Type of work place Academic institution University Hospital Regional Hospital District Hospital Neighborhood Clinic Dispensary Private Clinic International Org International NGO National NGO National MOH Regional MOH 100 0 0 0 0 0 0 0 0 0 0 0 0 20 30 10 10 0 20 0 0 0 0 0 0 14 21 14 21 7 21 0 0 0 0 0 0 0 0 0 0 0 0 13 25 63 0 0 0 0 0 0 0 0 0 0 14 0 43 43
Years working in RH (mean, SD) 22.7 (12.7) 107.0 (11.0) 12.0 (7.4) 20.1 (11.3) 19.0 (8.5)

*One key informant was interviewed as both a faculty and health facility assessment.

How family planning registers are being adapted to include integrated family planning data. The health workers interviewed conveyed a great interest in ensuring high quality reporting within the health care registers. What they noted as missing was appropriately formatted registers--both official and unofficial-- which captured all the information required in the facility-level monthly reports sent to the district level of the Ministry of Health and partner NGO projects. This most frequently occurred with the family planning register. One of the educators (02) who also worked as a clinician in a district hospital noted, “ Some indicators that we must track are not noted in some registers, which is worrisome and causes extra work. The registers we have do not conform to the reporting tools. I suggest we review all the registers in use for family planning service providers and make sure they are consistent with the reporting tools. Then reporting would be easy.”

Availability of support for integrated family planning reporting. Individuals involved in reproductive health politics, NGOs, and health facilities directors had varying responses in relation to available support and resources to help implement and record family planning integrated with other health services. All but two respondents to this question felt that further support was necessary and that currently the availability of resources for integrated reporting was lacking, in part because the formats were not as useful as they could be, as described above. A major theme across respondents was how health care workers were already overwhelmed with the amount of required documentation that they were expected to produce. Respondents indicated that future reporting needed to be simplified or it would be necessary to have individuals specifically tasked with reporting. Training of health providers was a prominent request in improved reporting methods.

International NGOs provided health workers with the support for documentation of integrated services. During the PPFP/PPIUD training conducted by Jhpiego and EngenderHealth, health care workers being trained were given hard cover, bound registers approved by the Ministry of Health to record PPFP/PPIUD insertion to use at their respective health facilities as discussed above. One individual involved in reproductive health politics (05) said, “ For PPIUD, Jhpiego provided a register for postpartum care! But postabortion care currently does not even have a proper collection tool. It is handmade and could be used at all levels but is not.”

Uses of family planning data that is reported. There was not consensus as to who used the collected family planning data from the informants. Informants named the Ministry of Health, the Division of Family Health/Division of Maternal and Infant Health/Family Planning, funding organizations, general government, their superior at the health facility, NGOs, United Nations agencies, and/or that it was used at all levels of the health system. Only 6 of 41 informants (three health care workers, three health facility directors) noted that they used the collected data from the health facilities themselves to inform their work and programming.

Challenges and possible solutions associated with reporting integrated family planning services

1. Too many different registers

Adding registers for integrated family planning was the most common way of recording these services. Two informants noted that there were numerous registers for all different types of health care areas and that there may be a point at which the number of registers was becoming excessive. One director (08) noted, “ There are too many registers to fill out, but to get all the integrated services information one is tempted to increase the number of registers. On the other hand, we must rationalize to avoid having too much data to fill in. […] Integrated family planning must also be part of the overall data collection plan that is being worked on at the Ministry of Health.”

Possible solutions suggested for this problem included bringing in support staff to help the health care workers to complete the reporting or, as noted above, revision and streamlining of relevant health reporting tools. One individual involved in reproductive health policy (01) said, “ It could be helpful if there are specialized services for data collection as well.”

2. Lack of a standardized reporting system

The need for standardized tools was noted by many of the informants. Without the proper tools to measure the integration of services it is impossible to know if the services are currently being integrated. An individual from a reproductive health NGO (05) said, “ In terms of integration, data collection needs to have integrated tools, […] which must be at all levels of [patient care] delivery.” Possible solutions for this were given by two informants (a health worker and a director) which included having the government officials learn more about reproductive health work needs and implementing electronic systems. One director wanted to make all the reporting in a computerized format, possibly using electronic tablets for providers to enter all data at the patient level.

3. Difficulty in getting access to the registers

More integration of registers took place in smaller health facilities compared to larger hospitals due to the geography of the hospitals and the separation of departments for PAC, labor and delivery, vaccination, and family planning. In the larger hospitals, the different gynecological services were offered in different areas by individuals trained specifically in that area, so the health care staff who oversee the family planning programs were not the ones who were running the PAC services. The hours of service availability also varied depending on the type of service. It was difficult to get the necessary access to these different areas within the larger health facilities to fill out the appropriate register. Often the person doing the PAC service had to go to the family planning service the following day to enter the information about the patient who had received family planning as part of PAC. This person was not necessarily identified specifically as having received family planning as part of PAC unless someone compared the family planning register with the PAC register and identified the woman by name.

In the smaller health facilities, one or two health care providers were responsible for all reproductive health services. This made it easier for them to record the integrated services since they were the ones providing all the services in one location. This way the registers were kept in one area and the health worker had the ability to choose how that register was used and how integrated services were noted.

According to informants, possible solutions to this included having the family planning register available in common locations which all health care workers have access to or change to electronic report (using computers or tablets) on services at the patient level in a platform such as DHIS2. The NGO workers wanted to have further exploration into how the problem of register location could be improved with greater knowledge of the work flow.

4. Too much to record and too little time to do it

Health care workers in Togo provide hundreds of services each month and are required to also chart these services, often in multiple charts and registers by hand. An individual involved in reproductive health politics (01) said, “[Data collection] is done by service providers and they have a lot of work. The providers have to be providers, and I do not know, computer scientist, logistician, as well - they do everything at the same time.” The informants mentioned the large number of tasks health care workers were responsible for throughout the interviews.

Possible solutions posed to this included having less information for the providers to fill out and adapt the reporting tools to streamline the process. One NGO worker (06) said, “We need to have well-designed reporting tools that are not overloaded because […] when there are too many registers to fill in that can cause fatigue. We must […] adapt reporting tools and train personnel in the use of these tools.”

Why family planning reporting matters. The informants agreed that recording family planning played an important role in influencing future service needs. The areas which were prominent included the necessity to be able to make decisions and predict future service use; the ability to monitor improvements or declines in service, and if established goals were met; a way to justify staffing and expenses; and a way to see if there were current unmet needs or new program needs. An informant who worked at a reproductive health NGO (01) noted that while not always used effectively, “ data allows us to know if we are moving towards the goals.”

While there are many challenges to the reporting system, one facility director (07) made a thought-provoking statement when he said, in relation to improving reporting systems in Togo, “ It seems like dreaming, but it is already done elsewhere, and it is a question of determination – we can do this if enough people think it is useful.”

 Discussion

Reducing maternal and newborn morbidity and mortality is a priority of the Togolese government 30 . Integration of family planning into other reproductive health services may help increase the modern contraceptive prevalence rate and decrease the unmet need for contraception which can contribute to reducing maternal and newborn mortality and morbidity. Quality data (e.g. timely, complete, precise and accurate) are key to being able to accurately measure the success of integrated services 31 .

The study findings highlight vulnerabilities with respect to data quality in each link of the reporting chain at the individual patient and facility levels. These findings are unfortunately common. Numerous studies have highlighted the many challenges to reliable and timely information related to health service and health status of the population including problems with completeness, accuracy, and timeliness in low resource settings 32, 33 , duplicate or parallel reporting systems and lack of capacity for data analysis 34 making planning, monitoring, and evaluation of these programs difficult 35 . In the end, a health monitoring and evaluation system that is designed (unintentionally) to generate poor quality data provides a shaky foundation for health service decision-making, planning, and health policy.

Global discussions around challenges and improvements to reproductive health monitoring and evaluation

Recording information about different types of care in individual registers is not unique to Togo and has been noted in other sub-Saharan African countries 33, 36 . This type of recording makes it difficult to assign individual identifiers and increases staff workload due to the duplication of material in each register and the frequent changes in data entry protocols 36 . Adapting health care registers to capture the needed information on various reporting forms is also found in other African countries and points to the need to improve and make the health reporting system more flexible to capture multiple service provision in one visit.

In addition, while the innovation shown by the individual health workers in adapting the standard family planning form is important to recognize, it is not a long-term solution to improve the quality, ease, or time requirement of reporting. As in many low-resource countries, some areas of health care provision are supported by donor agencies that fund specific programs which often require additional reporting and documentation. This data should ideally be taken from existing data collection systems but often requires the creation of additional, parallel documents 37, 38 .

With all the challenges noted above, there is a need to streamline indicators related to maternal health and contraception. One example of this is the FP2020 initiative and the Track20 project, which monitors progress of achieving the FP2020 goals. These goals include increasing modern method users by an additional 120 million women between 2012 and 2020 in the world’s 69 poorest countries, which includes Togo 39 . The Track20 project aims to reduce the need for heavy reliance on large national household surveys and instead use estimates of data collected through the public and private sector on specified family planning indicators 39 . Track20 uses a set of core indicators which were selected through a systematic, consultative process to allow for data-driven decision making by countries and measurement of how well individual needs are met 39 .

 Future research questions

Numerous areas require further research into assessing and improving the reporting systems of integrated family planning programs. Time allocation studies of health care providers could show the actual burden on health workers for each type of task and can demonstrate specific areas that may be appropriate for streamlining, including technologies and task shifting that could reduce time burdens. A study of task shifting was undertaken in Botswana involving the creation of a new cadre of health worker, the Monitoring and Evaluation District Officer. These individuals were trained on the job for their tasks 40 . After 3 years on the job, data quality had improved, there was increased use of data for disease surveillance, research, and planning, and nurses and other health professionals had more time to focus on the clinical components of their work 40 . If such task shifting were to be scaled up, health care worker efficiency could improve and burnout could decrease.

Further research is needed to investigate the most effective ways to improve monitoring and evaluation systems themselves, especially in relation to integrated reproductive health programs. A study in Mozambique of a Health Management Mentorship (HMM) program to strengthen health systems in 10 rural districts analyzed change in 4 capacity domains after one year of the program: accounting, human resources, monitoring and evaluation, and transportation management. All the domains except for monitoring and evaluation showed improvement over the one-year mentorship program. The authors of this study noted that challenges included constantly changing program targets and objectives, continually being in a “crisis mode” (constantly trying to catch up on reporting or needing reports in a short period of time) which did not allow time to set up efficient systems, and unavailability of key program staff due to the frequent out-of-office trainings 41 .

This finding shows that monitoring and evaluation systems are difficult to improve even with the deployment of additional resources specifically for this purpose due to the inherent constraints. Implementation research to improve the functionality and sustainability of monitoring and evaluation within the health system, perhaps using a collaborative quality improvement approach, should be considered.

 Programming and policy recommendations

Recently, the World Health Organization published results from a five-country intervention to strengthen measurement of reproductive health indicators which aimed to improve national information systems for routine monitoring of reproductive health indicators 42 . Activities within this intervention included revising, standardizing, and making consistent the existing reproductive health indicators gathered through routine systems and building capacity in data collection methods through training and supervision in pilot sites. The country teams reorganized and updated existing monitoring and evaluation frameworks. Challenges encountered even in this focused effort included frequent changes in staffing, delays from administration such as slow response times to updating systems and competing priorities for staff time for implementing reporting improvements. Thus, even with focused intervention it is challenging to streamline and harmonize monitoring and evaluation systems related to reproductive health.

The main recommendations for policy and programming in the Togolese context include consolidating reproductive data for health indicators and reducing provider workload for reporting, especially reporting integrated reproductive health services. This could include adopting electronic data management systems at the health facility level. Currently the largest task of recording integrated family planning is placed on the health care workers, who have adapted health registers to capture the requested information in monthly reports, but this requires extra work, memory, and creativity on the part of the health care worker. When reporting forms are developed they must be standardized to correspond with the associated health register. The number of times the health provider must enter, and re-enter data needs to be reduced.

 Limitations

One main limitation of the health facility assessment included that health facilities in the purposive sample were all affiliated with AgirPF and located in urban areas. Another challenge is that the data are cross-sectional and therefore only provide a snapshot of the current monitoring and evaluation system. Lastly, there were challenges associated with photographing registers; however photographs of available registers were only used to illustrate the kinds of adaptations undertaken by health care workers.

Limitations of the key informant interviews included potentially not understanding all the possible perspectives of the informants, differences in the interviewer’s methods for probing and what areas were focused on in each interview, and potential response bias as the study was conducted under the auspices of AgirPF.

 Conclusions

Monitoring and evaluation systems are fraught with implementation challenges that affect the quality of data used in patient care, planning, and policy, especially in relation to recording integrated health services. This is a reality not only in Togo but also in other countries. There is a need for a concerted, collaborative effort on the parts of national governments and global partners to address challenges to improve monitoring of integrated health services.

 Data availability Underlying data

Due to restrictions of access to data outlined in the research participant agreement approved by the Togolese national ethics committee (CBRS), the datasets generated and analyzed during the study are not publicly available. Readers can access data by contacting the Lillian Carter Center for Global Health and Social Responsibility at the Nell Hodgson Woodruff School of Nursing at Emory University (lcc@emory.edu). Data will only be shared with researchers for reanalysis and grant proposals.

 Extended data

Figshare: Study Documents Agir03. https://doi.org/10.6084/m9.figshare.7823651.v1 21 .

This project contains the following extended data:

Study documents including the protocols, consent forms, health facility assessment forms, and semi-structured interview guides

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

 Acknowledgements

We would like to thank the data collectors involved in this research from the URD in Lomé, the Togolese Ministry of Health, and our partner organizations. We also want to recognize the amazing work of Togolese health care providers who work tirelessly to improve access to health care in Togo.

 Family Planning High Impact Practices: Family Planning High Impact Practices List. K4Health;2017. Reference Source International Federation of Gynecology and Obstetrics (FIGO), International Confederation of Midwives (ICM), International Council of Nurses (ICN), et al.: Post abortion family planning: A key component of post abortion care. Washington D.C;2013. Reference Source United States Agency for International Development (USAID), Australia AID, World Health Organization, et al.: Statement for Collective Action for Postpartum Family Planning.2013. Reference Source World Health Organization, USAID, Maternal and Child Health Integrated Program: Programming strategies for postpartum family planning. Geneva: World Health Organization.2013; 9241506490. Reference Source Pleah T Hyjazi Y Austin S : Increasing Use of Postpartum Family Planning and the Postpartum IUD: Early Experiences in West and Central Africa. Glob Health Sci Pract. 2016;4 Suppl 2:S140–152. 27540120 10.9745/GHSP-D-16-00039 4990157 Fikree F Murgore S Forrester H : Strengthening Postabortion family planning in Togo, Mobilizing Resources. Washington D.C.: Evidence to Action Project;2014. Reference Source Mugore S Kassouta NT Sebikali B : Improving the Quality of Postabortion Care Services in Togo Increased Uptake of Contraception. Glob Health Sci Pract. 2016;4(3):495–505. 27688719 10.9745/GHSP-D-16-00212 5042703 Wisniewski JM Yeager VA Diana ML : Exploring the barriers to rigorous monitoring and evaluation of health systems strengthening activities: qualitative evidence from international development partners. Int J Health Plann Manage. 2016;31(4):e302–e311. 26927839 10.1002/hpm.2339 Chaulagai CN Moyo CM Koot J : Design and implementation of a health management information system in Malawi: issues, innovations and results. Health Policy Plan. 2005;20(6):375–384. 16143590 10.1093/heapol/czi044 Kimaro HC Twaakyondo HM : Analysing the hindrance to the use of information and technology for improving efficiency of health care delivery system in Tanzania. Tanzan Health Res Bull. 2005;7(3):189–197. 16941947 10.4314/thrb.v7i3.14259 Thiam S Kimotho V Gatonga P : Why are IPTp coverage targets so elusive in sub-Saharan Africa? A systematic review of health system barriers. Malar J. 2013;12(1):353. 24090252 10.1186/1475-2875-12-353 3850646 Jacucci E Shaw V Braa J : Standardization of health information systems in South Africa: The challenge of local sustainability. Inform Technol Dev. 2006;12(3):225–239. 10.1002/itdj.20044 Ledikwe JH Grignon J Lebelonyane R : Improving the quality of health information: a qualitative assessment of data management and reporting systems in Botswana. Health Res Policy Syst. 2014;12:7. 24479822 10.1186/1478-4505-12-7 3910237 Mphatswe W Mate KS Bennett B : Improving public health information: a data quality intervention in KwaZulu-Natal, South Africa. Bull World Health Organ. 2012;90(3):176–182. 22461712 10.2471/BLT.11.092759 3314204 Nash D Elul B Rabkin M : Strategies for more effective monitoring and evaluation systems in HIV programmatic scale-up in resource-limited settings: Implications for health systems strengthening. J Acquir Immune Defic Syndr. 2009;52 Suppl 1:S58–62. 19858942 10.1097/QAI.0b013e3181bbcc45 Chan M Kazatchkine M Lob-Levyt J : Meeting the demand for results and accountability: a call for action on health data from eight global health agencies. PLoS Med. 2010;7(1):e1000223. 20126260 10.1371/journal.pmed.1000223 2811154 Kawonga M Blaauw D Fonn S : Aligning vertical interventions to health systems: a case study of the HIV monitoring and evaluation system in South Africa. Health Res Policy Syst. 2012;10(1):2. 22280794 10.1186/1478-4505-10-2 3293072 Adamchak SE Okello FO Kaboré I : Developing a system to monitor family planning and HIV service integration: results from a pilot test of indicators. J Fam Plann Reprod Health Care. 2016;42(1):24–29. 26424662 10.1136/jfprhc-2014-101047 Chabikuli NO Awi DD Chukwujekwu O : The use of routine monitoring and evaluation systems to assess a referral model of family planning and HIV service integration in Nigeria. AIDS. 2009;23 Suppl 1:S97–S103. 20081394 10.1097/01.aids.0000363782.50580.d8 Spaulding AB Brickley DB Kennedy C : Linking family planning with HIV/AIDS interventions: a systematic review of the evidence. AIDS. 2009;23 Suppl 1:S79–88. 20081392 10.1097/01.aids.0000363780.42956.ff Baker H Rochat R Hennink M : Study Documents Agir03. figshare. Fileset.2019. http://www.doi.org/10.6084/m9.figshare.7823651.v1 Central Intelligence Agency, Togo: The World Factbook.2017. Reference Source PAC-FP EngenderHealth: PACFP service availability and readiness assessment.2016. République du Sénégal, Measurement, Learning & Evaluation Project, Global Research and Advocacy Group, Initiative Sénégalaise de Santé Urbaine: Enquê te Finale au Niveau des Structures de Sante 2015.Dakar: Urban Reproductive Health Initiative; 2015. IBM SPSS Statistics for Windows, Version 22.0 Released 2013. [computer program]. Armonk, New York: IBM Corporation;2013. Reference Source Word [computer program]. Redmond, Washington: Microsoft;2013. Express Scribe [computer program]. Greenwood Village, CO: NCH Software, Inc;2014. Reference Source NVivo qualitative data analysis Software [computer program]. Version 11: QSR International Pty Ltd.;2015. Reference Source Guest G MacQueen KM Namey EE : Applied thematic analysis. Los Angeles: Sage Publications;2012. 10.4135/9781483384436 Ministère de la SantéRé publique du Togo: Plan d'Action pour le Repositionnement de la Planification Familiale au Togo. Lomé, Togo: Ministère de la Santé.2013. Reference Source World Health Organization: Improving Data Quality: A Guide for Developing Countries. Geneva: WHO Press;2003. Reference Source Garrib A Stoops N McKenzie A : An evaluation of the District Health Information System in rural South Africa. S Afr Med J. 2008;98(7):549–52. 18785397 Mate KS Bennett B Mphatswe W : Challenges for routine health system data management in a large public programme to prevent mother-to-child HIV transmission in South Africa. PLoS One. 2009;4(5):e5483. 19434234 10.1371/journal.pone.0005483 2677154 Chilundo B Sundby J Aanestad M : Analysing the quality of routine malaria data in Mozambique. Malar J. 2004;3(1):3. 14998435 10.1186/1475-2875-3-3 395838 Lim SS Stein DB Charrow A : Tracking progress towards universal childhood immunisation and the impact of global initiatives: a systematic analysis of three-dose diphtheria, tetanus, and pertussis immunisation coverage. Lancet. 2008;372(9655):2031–2046. 19070738 10.1016/S0140-6736(08)61869-3 Gourlay A Wringe A Todd J : Challenges with routine data sources for PMTCT programme monitoring in East Africa: insights from Tanzania. Glob Health Action. 2015;8:29987. 26715204 10.3402/gha.v8.29987 4695617 Chilundo B Aanestad M : Negotiating multiple rationalities in the process of integrating the information systems of disease specific health programmes. The Electronic Journal of Information Systems in Developing Countries. 2004;20(1):1–28. 10.1002/j.1681-4835.2004.tb00129.x Igira FT : The dynamics of healthcare work practices: Implications for health management information systems design and implementation. Management Research Review. 2012;35(3/4):245–259. 10.1108/01409171211210145 Avenir Health: Track20: Monitoring progress in family planning. N.D. Accessed January 16, 2017. Reference Source Mpofu M Semo BW Grignon J : Strengthening monitoring and evaluation (M&E) and building sustainable health information systems in resource limited countries: lessons learned from an M&E task-shifting initiative in Botswana. BMC Public Health. 2014;14:1032. 25281354 10.1186/1471-2458-14-1032 4192275 Edwards LJ Moisés A Nzaramba M : Implementation of a health management mentoring program: year-1 evaluation of its impact on health system strengthening in Zambézia Province, Mozambique. Int J Health Policy Manag. 2015;4(6):353–361. 26029894 10.15171/ijhpm.2015.58 4450730 Barreix M Tunçalp Ö Mutombo N : Experience from a multi-country initiative to improve the monitoring of selected reproductive health indicators in Africa. Int J Gynaecol Obstet. 2017;137(2):205–212. 28108993 10.1002/ijgo.12105 10.21956/gatesopenres.14048.r27940 Reviewer response for version 1 Hardee Karen 1 Referee https://orcid.org/0000-0003-2545-6269 What Work Association, Arlington, VA, USA

Competing interests: No competing interests were disclosed.

 7 10 2019 Copyright: © 2019 Hardee K 2019 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve

This is an important and well written paper about the complexities of service delivery reporting systems for monitoring integrated family planning services.  The paper uses data from the AgirPF project in three urban areas of Togo to trace the expected and actual system for recording family planning integrated with postabortion and postpartum services in 25 health facilities and through 41 key informant interviews. While the focus is integrated services, the findings are applicable to reporting systems for any services and provide a cautionary note for programming that does not pay adequate attention to or funding for M&E. For example, the finding that the relevant registers for PAC are not located in the same place is instructive for understanding the complexities of reporting. The finding that providers are overwhelmed with so much reporting for different purposes is unfortunately not new. The use of visuals to show the workarounds that service providers do in order to record needed indicators is useful for those working on improving reporting of service statistics. I was struck by Figure 4 showing that most monthly reports are sent to NGOs – are those implementing partners funded by donors?  Are they contributing to the complexity and work for providers by, as the authors say, “If an official register did not exist for a given services, sometimes NGO workers involved in integration projects would tell health workers to create an unofficial register for the service using a notebook and pen, inserting columns for recording information.” 

I did not find the paragraph about FP2020 and Track20 in its current version so relevant to this paper. Service statistics constitute a small component of the data that go into FP2020’s 18 core indicators – and few of the 69 countries have strong enough systems to produce data of sufficient quality to use in the reporting. With that said, I think Track20 is doing interesting work with countries on their routine health systems – particularly to rationalize indicators. Rather than include a summary paragraph about Track20, I suggest that the authors contact Track20 to find out what they are doing related to routine health information systems – so the paragraph can be better tailored to the paper. 

I was surprised to see no mention of the work of MEASURE Evaluation, which has for decades been working on improving M&E systems, including routine health information. As one example, through JSI, MEASURE Evaluation has worked on PRISM (Performance of Routine Information System Management), that would be useful to review. Another resource is RHINO, the Routine Health Information Network ( https://www.rhinonet.org/). And PEPFAR, with its focus on reporting indicators, has generated lessons learned for strengthening routine health information systems. I suggest that the authors check these resources to strengthen the section of the paper on how to make improvements in the reporting system. 

Also, it would be good for the authors to say something about the official process of revising the components of routine health information. For example, who has the authority to make changes to the system? How often are registers reviewed and revised? By who? The authors indicate that some respondents recommended computerized reporting with tablets. Who would have the authority to make this change to the M&E system? Any implementation research carried out to strengthen reporting of integrated family planning services will need to acknowledge the official process for making changes – otherwise, it might just be creating more workarounds.

Is the work clearly and accurately presented and does it cite the current literature?

Partly

If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable

Are all the source data underlying the results available to ensure full reproducibility?

No

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

Family planning implementation science, monitoring and evaluation and policy.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

 10.21956/gatesopenres.14048.r27109 Reviewer response for version 1 T Bertrand Jane 1 Referee Tulane University School of Public Health and Tropical Medicine, New Orleans, LA, USA

Competing interests: No competing interests were disclosed.

 28 5 2019 Copyright: © 2019 T Bertrand J 2019 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve

This article addresses the challenge of monitoring and evaluating integrated family planning services in urban areas of Togo. Specifically, it focuses on the integration of services for family planning, post abortion care, and postpartum care. The findings are based on an assessment of 25 purposively selected health facilities, a review of health services records, and 41 in-depth interviews with different categories of health personnel.

The article is well written and clearly presented. The photos of the registers are useful in reminding readers of the very basic data collection tools available in these facilities. The photos also reflect the attempts by health providers to do the right thing, in the absence of having the appropriate tools to do so.

The paper merits publication because of its systematic examination of the issue of monitoring integrated family planning services. With a new interest in postpartum (and in some countries post-abortion) family planning, this methodological issue is very relevant to those working in family planning. It captures a reality found in many health systems in low-income countries, which often is “understood” but not documented.

The paper effectively communicates the tension between good service provision and timely data collection by low- to middle-level health workers, often poorly paid, who feel overwhelmed by the amount of documentation they are required and expected to produce.

Missing from the paper is a clear statement of what the authors would consider the short list of most valuable indicators to measure integrated family planning. The service providers developed ingenious “work-arounds” to provide information relevant to this question, as they improvised the use of the available forms. If the researchers had full reign over the data collection, what would they recommend? This is particularly relevant, given their citation of the FP 2020 indicators as an example of a standardized set of metrics that countries could then strive to obtain and report.  Table 2 provides a list of 13 indicators that includes integrated family planning services. However, the list is long, and the authors don’t attempt to identify or prioritize those of greatest relevance to the question. For readers looking to better understand best practices in the area of measuring integrated family planning services, it’s important to learn what this handful of indicators most important in measuring integrated family planning services would be.

In a related vein, there is lack of a clear path toward a better system. The authors do provide recommendations for improving the quality of service statistics by capturing the data at all levels and reducing the reporting burden on service providers. Yet the very circumstances that created the problems in the current data collection system are fairly ingrained in weak health systems (in Togo and elsewhere). Can the authors provide concrete examples of where their recommendations have actually paid off in other countries?

The following edits would improve the flow and content of the paper:

Paragraph 3 under the introduction: replace “in opposition to…” with “in contrast to…”

Under procedures/health facility assessment: The authors clearly explain the purpose of the sample of 25 health facilities. It would be useful to have an approximate estimate of the universe of facilities from which the 25 were selected.

Under results/health facility assessment: it is not intuitively clear why the researchers took half as many facilities from Sodoké as from Kara, when in the previous section they explain that the population of Sodoké is slightly larger than that of Kara.

Figure 4. The idea behind this graph and the explanation underneath it do not clearly communicate what it is intended to show. For instance, what the expected/preferred route of transmission?

On page 10, under “uses of family planning data reported,” the authors make the very salient point that “only six of 41 informants… noted that they use collected data from the health facilities themselves to inform their work and programming.” The author should further develop this issue in the discussion.

 With these minor edits, I recommend indexing.

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable

Are all the source data underlying the results available to ensure full reproducibility?

No

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

M&E systems for family planning programs in low-income settings

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.