Gates Open Res Gates Open Research 2572-4754 F1000 Research Limited London, UK 10.12688/gatesopenres.15870.1 Research Article Articles Building a digital supportive supervision system for improving health service delivery in Low- and Middle-Income countries through the collaborative requirements development methodology (CRDM): Experience from Tanzania

[version 1; peer review: 1 approved with reservations]

 German Chrisogone Justine Conceptualization Data Curation Formal Analysis Methodology Validation Visualization Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0002-3812-0310 a 1 Kengia James Tumaini Data Curation Methodology Validation Writing – Review & Editing 2 Mwanyika Henry Data Curation Funding Acquisition Project Administration Supervision Writing – Review & Editing 3 Makaranga Joseph T. Conceptualization Formal Analysis Validation Writing – Original Draft Preparation Writing – Review & Editing 4 Werner Laurie Funding Acquisition Methodology Project Administration Validation Writing – Review & Editing https://orcid.org/0000-0002-0892-2865 5 Benadotte Christina Funding Acquisition Methodology Project Administration Validation Writing – Review & Editing 6 Kisanga Auson Resources Software Writing – Review & Editing 3 Macha Tumainiel Project Administration Supervision Writing – Review & Editing 1 Weja Joel Methodology Supervision Writing – Review & Editing 1 Mugeta Stephano Conceptualization Data Curation Formal Analysis Funding Acquisition Project Administration Resources Supervision Writing – Review & Editing 3 Kinyenje Erick S. Formal Analysis Methodology Validation Visualization Writing – Review & Editing https://orcid.org/0000-0001-8434-2232 1 Marandu Laura E. Formal Analysis Validation Writing – Review & Editing https://orcid.org/0000-0001-6632-644X 1 Mwaisengela Syabo M Data Curation Formal Analysis Methodology Validation Visualization Writing – Review & Editing 1 Ilomo Silvanus Project Administration Resources Software Writing – Review & Editing 1 Seiff Sultana Methodology Software Validation Writing – Review & Editing 7 Kapologwe Ntuli Data Curation Formal Analysis Methodology Project Administration Validation Writing – Review & Editing 1 Hokororo Joseph C. Data Curation Methodology Validation Visualization Writing – Review & Editing 1 Eliakimu Eliudi S. Conceptualization Data Curation Formal Analysis Methodology Project Administration Resources Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0003-2383-9530 1 Sirili Nathanael Conceptualization Data Curation Formal Analysis Methodology Supervision Validation Visualization Writing – Review & Editing 8 Settle Dykki Funding Acquisition Methodology Project Administration Supervision Validation Writing – Review & Editing 5 Rashid Seif Conceptualization Formal Analysis Funding Acquisition Investigation Methodology Project Administration Supervision Validation Visualization Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0009-0001-5361-8731 3 Ministry of Health, Tanzania, P. O. Box 743, Dodoma, Tanzania President’s Office - Regional Administration and Local Government, P. O. Box 1923, Dodoma, Tanzania Data Use Partnership Project, PATH, P. O. Box 13600, Dar es Salaam, Tanzania College of Informatics and Virtual Education, The University of Dodoma, P. O. Box 259, Dodoma, Tanzania PATH, Seattle, USA PATH, Montana, USA e-Government Authority (eGA), President’s Office - Public Service Management and Good Governance, P. O. Box 2833, Dodoma, Tanzania School of Public Health and Social Sciences, Muhimbili University of Health and Allied Sciences, P. O. Box 65015, Dar es Salaam, Tanzania drxgone@gmail.com

Y Deceased.

No competing interests were disclosed.

 5 8 2024 2024 8 74 25 7 2024 Copyright: © 2024 German CJ et al. 2024 This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Background

Supportive supervision is pivotal for both health service providers and managers to improve the quality of services and health outcomes. Tanzania's digital supportive supervision system is called Afya Supportive Supervision System ( AfyaSS ®). The latter was designed and developed using a human-centered approach called Collaborative Requirements Development Methodology (CRDM). This paper describes the experiences of building the digital supportive supervision system through CRDM in Tanzania, a transition from the paper-based supportive supervision system.

 Methods

In 2018, with the support of PATH, the government of Tanzania adopted a participatory human-centered design by carrying out desk reviews of existing guidelines and tools, field visits, stakeholder workshops, and user advisory groups to gather information for developing a digital supportive supervision system. The gathered information was analyzed with the lens of identifying the common challenges and system requirements.

 Results

AfyaSS was successfully developed using CRDM and deployed in all regions across the country. It has consolidated multiple checklists for distinct health domains, and dashboard functionalities to track progress toward health system indicators, objectives, and action plans. As part of the deployment, several resources were developed to aid in the deployment process, such as a comprehensive user manual, facilitator's guide, training slides, and video tutorials. Health workers and managers can be empowered and motivated to implement comprehensive and coherent supportive supervision by using the lessons learned from this digitalization process to transform the current supervision processes to improve the quality of care by offering instruments that promote evidence-based actions.

 Conclusion

A human-centered approach has been shown to be useful in developing digital tools for use in Tanzania’s health system. Tanzania's lessons can be applied in other low- and middle- income countries (LMICs) with similar contexts when considering implementing digital health interventions. While using the human-centered approach, it is crucial to establish a system development roadmap, conduct appropriate training, provide sustained information and communication technology, and management support for unforeseen issues, and ensure ongoing maintenance.

 Collaborative Requirements Development Methodology (CRDM) Supportive Supervision Afya Supportive Supervision System Data Use Data Use Partnership Digital Investment Roadmap Tanzania Gates Foundation INV-006947 This work was supported by the Gates Foundation [INV-006947]. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Introduction

Supportive supervision is a collaborative approach undertaken by both health providers and managers to improve the quality of health service delivery 1 . It involves the provision of support to improve the skills and knowledge of healthcare workers. Supportive supervision has proven potential for promoting quality improvement for structural and process elements in several low-resource settings’ facilities 2– 5 . Improved quality of health services is a prerequisite for moving towards Universal Health Coverage (UHC) and for achieving health-related Sustainable Development Goals (SDG) 6, 7 .

In Tanzania, the health care service delivery system is organized in a pyramid structure of three levels: primary, secondary, and tertiary. The primary level is comprised of the council health management team, district hospitals, other hospitals at the council level, health centres, dispensaries, and the community. The secondary level is comprised of regional referral hospitals, other referral hospitals at the regional level, and regional health management teams. The tertiary level is composed of the zonal referral hospitals and other referral hospitals at the zonal level, specialized hospitals, and national hospitals 8– 11 .

Supportive supervision is conducted by Health Management Teams and health managers at different levels of health service delivery 12 . The President’s Office - Regional Administration and Local Government (PO-RALG) oversees the provision of services at regional and council levels. At the regional level, there are Regional Health Management Teams (RHMTs), and at the council level, there are Council Health Management Teams (CHMTs). The RHMTs oversee the provision of services in the region and provide technical support to CHMTs through supportive supervision. The Ministry of Health (MoH) is responsible for policy formulation, development of sector strategic plans, regulatory function, and quality assurance, and also oversees the provision of services at the national, specialized, zonal, and regional referral hospitals 13 . Historically, the supervision teams used paper-based tools, which had several challenges, including uncoordinated processes, lack of access to reports, and lengthy and unstandardized checklists.

Health sectors in Low- and Middle-Income Countries (LMICs), including Tanzania, are transitioning to a digital model of operation and business. Use of digital technologies in the healthcare system improves supervision practices by allowing access to real-time data dashboards which enable comparison of the performance of facilities and follow-up of the implementation of improvement actions for gaps identified; hence contributing to the improvement of the quality-of-service delivery, and the experience of both service providers and clients worldwide 14– 16 . Tanzania has been using different digital tools for conducting supportive supervision. The Electronic Tool to Improve Quality of Healthcare (eTIQH) was first introduced in 2011 in two pilot councils in Tanzania, Ulanga and Kilombero, and was then extended in 2012 and 2013 to a further six councils in Morogoro, Iringa, and Pwani regions 3, 4, 17 . Although eTIQH was used to assess the quality of primary healthcare provision, health managers used it for supportive supervision purposes.

In 2016, the Government finalized and launched its "Digital Health Investment Road Map: A Journey to Better Data for Better Health" 18 . To implement parts of the 17 costed investment recommendations of the roadmap, the Government of Tanzania, with support from the Bill and Melinda Gates Foundation (BMGF), established an initiative called Data Use Partnership (DUP). DUP identified to strengthen systems for facility performance management and supervision as one of its priority areas 18 The supportive supervision roadmap of 2018 shows how digitization will strengthen coordination of visit scheduling, prioritize and improve data use, link and adapt various supervision checklists, and facilitate implementation of recommendations linked to health facility planning and budgeting tools. The Government of Tanzania through the DUP project, developed a digital supportive supervision platform known as Afya Supportive Supervision System ( AfyaSS ®). After the introduction of AfyaSS, the government expects the health workers to experience supportive supervision as a continuous, coherent, and effective process. The latter is expected to be realized through improved coordination of different facility supervisions and assessments; use of data before, during, and after facility supervisions; harmonized and interlinked supervision tools; consolidated and coherent action plans for improved implementation; and follow-up on supervision recommendations 18 .

 Methods Settings

This developed system is one of the initiatives for improving the quality of supportive supervision in Tanzania. Tanzania Mainland has 26 administrative regions with 184 councils, a total area of 945,087 km 2, and a population of 61,741,120 as per 2022 population census 19 . Tanzania's health system is organized in a pyramidal structure, with the community and primary healthcare (PHC) facilities (dispensaries, health centers, and district hospitals) at the bottom and regional referral hospitals, zonal hospitals, specialized hospitals, and the National Hospital on top 20 . The number of health facilities increased from 8,449 in 2019 to 10,067 by 2022 21 .

 AfyaSS development process and literature review

The Collaborative Requirements Development Methodology (CRDM) was developed by the Public Health Informatics Institute – United States of America (USA) and adapted as a public good for use in public health. CRDM enables public health managers to define and document their requirements in a standardized and collaborative manner 22, 23 . CRDM allows informed decisions through the involvement of the users of the intended system, which may include purchasing, building, or developing health information systems collaboratively with other organizations. In 2018, with the support of PATH, the government of Tanzania adopted a participatory human-centered design by carrying out desk reviews of existing guidelines, strategic documents, and tools ( Table 1) to analyze the existing supportive supervision systems, which was followed by field visits to selected regions, councils, and health facilities to collect the information from stakeholders involved in supervision activities. This was followed by multiple stakeholder workshops, and user advisory groups to gather information for developing a digital supportive supervision system.

Documents reviewed.
Document Specific section
1. Digital Health Strategy, 2019–2024 24 Strategic Priority 6: Improve data use for evidence-based actions at all levels of the health system and the collection of high-quality data and its transformation into valuable information for evidence-based actions across all levels of the health system. Furthermore, under this priority, strategic Initiative one describes the implementation of digital solutions for facility supervision under which AfyaSS was developed.
2. Digital Health Investment Roadmap, 2017–2023 18 • Recommendation number two describes strengthening systems for facility performance management and supervision. • Recommendation number three describes the supportive supervision digitalization roadmap, which outlines the vision, mission, and processes to be undertaken.
3. Health Facility Supervision Digitalization Roadmap, 2018 The document outlined the "as is" and "to be" scenarios for digitizing supportive supervision processes.
4. National Supportive Supervision Guidelines for Quality Health Services, 2017 12 Section 7 and subsection 7.3 highlight the use of electronic devices in supportive supervision to improve data accuracy, simplify data collection and report handling, and provide immediate feedback to facilities and other stakeholders.
5. Health Sector Strategic Plans (HSSP) - IV & V 9, 11 . The importance of comprehensive supervision with harmonized tools was clarified in HSSP IV, Section 5.2 (p. 32), and having online supervision is one of the key messages in the introductory section as emerging strategic priorities in HSSP V (p. xii).

As part of the processes of developing a digital supportive supervision system, we developed health facility digitalization roadmap, system requirements and reviewed different sections of different strategic documents related to data and digital health ( Table 1). The reviews intended to identify key directions and functions using supportive supervision for quality improvement of healthcare services 12, 24 . It has mainstreamed the objective and system requirements to shape the procurement processes, identification and engagement of the external software development company. The external software development company implemented a leading role in the following steps: development of the inception report, development of detailed system design, producing a Beta version of the system, conducting user acceptance testing (UAT), production of a stable system and its source code, and deployment of the software. Details of the steps are described below.

 Steps to conduct the CRDM process

CRDM products include Business Process Analysis, Business Process Redesign, and Requirements Definition. Business Process Analysis is the effort to understand an organization and its purpose while identifying the activities, participants, and information flows that enable the organization to do its work. This phase produces a business process model that can be used to design or redesign business processes. Business Process Redesign entails fine-tuning our understanding of the workflow to define the database outputs required to support the work.

The CRDM consists of the following steps: Model the Business, Requirements Definition, Design and Develop, Test, Train, Deploy, Manage, and Optimize ( Figure 1).

The CRDM and the Information Technology life cycle (adopted from: <xref ref-type="bibr" rid="ref-25">25</xref>).

Model the Business: In this phase, the focus is on understanding the business processes, goals, and objectives. It involves analyzing the current state of the business and identifying areas for improvement.

Requirements Definition: This phase involves gathering and documenting the requirements for the software system. It includes identifying stakeholder needs, defining functional and non-functional requirements, and prioritizing them.

Design and Develop: In this phase, the software architecture and design are created based on the defined requirements. Developers then proceed to build the software according to the design specifications.

Test: Testing is an essential phase where the developed software is thoroughly tested to ensure that it meets the specified requirements and is free of defects. This includes unit testing, integration testing, system testing, and user acceptance testing.

Train: Training is provided to end-users and stakeholders to familiarize them with the new software system. This ensures a smooth transition to the new system and maximizes its effectiveness.

Deploy: The software is deployed into the production environment, making it available for use by end-users. This involves installing the software, configuring it, and ensuring that it works correctly in the production environment.

Manage: Once the software is deployed, it needs to be effectively managed to ensure its continued operation and maintenance. This includes monitoring the software, addressing any issues that arise, and making updates or enhancements as needed.

Optimize: The optimization phase focuses on continuously improving the software system to meet changing business needs and technological advancements. This involves gathering feedback from users, analyzing system performance, and implementing optimizations and enhancements.

In building the digital system, we adopted the CRDM approach 23 . CRDM, as an approach, ensures teams effectively plan, develop, and deliver software solutions that meet the needs of stakeholders and contribute to the success of the business. We documented all phases of the development of the AfyaSS system using the CRDM approach.

 Ethics

The study received approval from the Muhimbili University of Health and Allied Sciences (MUHAS) institutional review board with approval number DA. 282/298/01.C/2020 of February 7, 2024.

 Consent

This study did not involve human subjects and hence no consent was required.

 Findings

The findings are presented according to the CRDM phases outlined in the methodology section.

 Model the business

The MoH established the AfyaSS technical committee to oversee and guide the AfyaSS development process. Members of the committee came from the MoH's Quality Assurance, Policy and Planning, Information and Communication Technology (ICT), Curative Services, and Human Resource Management directorates, as well as PO-RALG. In collaboration with stakeholders, the committee held meetings and workshops and reviewed relevant documents, articles, and tools to develop the supportive supervision business process. To describe the ongoing practices in conducting supportive supervision (or the current and “as-is” process), a desk review of health sector strategies and policies, supervision guidelines, and tools was conducted. We reviewed several supportive supervision tools, including the Star Rating Accelerated Quality Improvement Tool (ACQUIT) and MalariaCareEDS tools which are owned by the Ministry of Health with which the corresponding author is affiliated; the eTIQH tool owned by Swiss Tropical and Public Health Institute that was developed as part of the “Initiative to Strengthen Affordability and Quality of Healthcare (ISAQH)” 4 in which the consent to access and use the tool was obtained from the project manager. The health facility supervision digitalization roadmap described processes for preparation, coordination, planning, conducting, and follow-up of supervision visits.

Furthermore, the MoH, PO-RALG, and stakeholders developed the "to-be" conceptual framework, which can link various types of supervision. The "to-be" conceptual framework included additional components that allowed for further modification of the original model. The "to-be" conceptual framework included three core business processes: before supervision (preparation), during supervision (conducting), and after supervision (follow-up).

In the "to be" model, preparation for supervision visits included a review and comparison of data for all facilities in a council. Furthermore, it allowed joint general and technical supervisions to select focus areas, determining which focus issues will be drilled down into in more detail at each facility (i.e., for which issues detailed questions will be used instead of high-level general questions during the supervision visit).

The community consultation component was also added to the "to be" model while conducting supervision visits. Village/ ‘ Mtaa’ Chairperson, Village/‘ Mtaa’ Executive Officer, health facility governing committee (chairperson or representative), other community leaders, and/or other community groups were consulted about their involvement in and views on the health facility's services. To facilitate better use of the data collected on these checklists, a tagging framework was developed to link common topics across different checklists and tools. The agreed-upon facility supervision business processes were documented in the Facility Supervision Digitalization Roadmap, which included recommendations for process improvements and high-level requirements for integrated facility supervision digital tools.

 Requirements definition for AfyaSS

The Technical Team (composed of MoH, PORALG and other stakeholders), the ICT technical working group (TWG), DUP governance unit (GU), DUP implementation team, and the user advisory group (UAG) were among the important stakeholders that participated in the process of gathering system requirements. With the help of stakeholder meetings, field visits, and interviews, we gathered and verified the requirements. The requirements were connected to three business process mapping (BPM) steps: preparation before performing supervision visits, during supervision visits, and follow-up following supervision visits. There were 17 requirements for the preparation process (before the supervision visit), 30 for the supervision process (during the supervision visit), and 10 for the follow-up process (after the supervision visit). In addition, 62 non-functional system requirements were also developed and recorded (see extended data). The relevant MoH divisions and units subsequently developed and approved the system requirement specification (SRS) document.

 Design and development of AfyaSS

According to the SRS and the supportive supervision digitalization strategy, the AfyaSS system was developed. The design and development adhered to Tanzania Health Enterprise Architecture (THEA) and e-Government Authority (eGA) requirements. For the AfyaSS system's design and development, we hired an external software development company to guarantee responsibility, effectiveness, and control based on robust outsourcing practices that ensure the contracted company has the expertise and skills, is trustful and will complete the project within the shortest time agreed 26, 27 . The design and development milestones included the initial meeting, a detailed system design, a beta system, UAT, a stable system, and the transfer of the software source code.

To establish a shared understanding of the project's goals and objectives and clarify the system requirements, deliverables, and timelines, members of the AfyaSS technical committee, consultants, and business experts attended the inception meeting. The consultant developed the minutes of the inception meeting, which were later approved as a reference by the DUP technical team.

The external software development company created a thorough system design and presented it to the technical committee for review and approval. A prototype for the system's functional and non-functional requirements was created using a thorough system design. After approval, the external software development company worked closely with the UAG and technical committee on all steps until a beta version was acquired. The beta version of the system met important criteria for system configurations, user administration, preparation for and execution of supervision, follow-up, system outputs and reports, and interoperability. Additionally, mobile access features for performing supervision and follow-up were incorporated into the beta version. The features that can be accessed from a mobile device are compatible with Android smartphones and tablets. The system was handed over to MoH management for final review and approval after the beta version was finished.

 Testing of AfyaSS

To verify that the built system operated in accordance with the specifications for running test scripts intended to mimic end-to-end supervision processes, the established UAG team ran the UAT. Users were given testing scripts to test various elements of the system that mimic business processes with anticipated system outcomes, and they were instructed to rate each tested script as Pass or Fail based on the system response.

The system was enhanced based upon the results of the UAT to provide a stable version that has gone through additional testing. The priority level for resolving each issue identified during testing was documented in a shared repository. First, the "high priority" feedback was addressed. Weekly feedback calls were arranged to assess concerns that had been rectified and to involve stakeholders in further testing to guarantee direct user involvement as problems were fixed.

 Deployment process of AfyaSS

To give comprehensive guidance on how to use the system, training materials were developed per the AfyaSS user manual. The training materials included a facilitator's guide, training slides, and video tutorials to aid in conducting the training sessions.

The process for building capacity involved training the trainers of trainees (ToTs) at the national and regional levels, who then cascaded training to peers and lower levels. This training took place both in person and virtually. Thirty-four national ToTs in total, including 22 from the MoH and 12 from PO-RALG, were trained. Fifty-two regional ToTs from all 26 regions of the mainland side of the United Republic of Tanzania, including the persons in charge of quality improvement (QI) and health management information system (HMIS), were trained. Then, the regional QI and HMIS focal persons in each region were responsible for training the RHMTs and CHMTs at the regional and council levels, respectively. Health workers from national, specialized, zonal, and regional referral hospitals (RRHs) were also trained. Following that, cascade training was carried out across the country with the assistance of implementing partners as well as regional and council resources 28 .

To implement the AfyaSS system, the DUP project purchased 900 tablets for offline and online use, which were delivered to all 26 regions (three tablets each) and 184 councils (three tablets each). The software source codes and detailed technical documentation were submitted to the MoH's ICT unit, which will be in charge of system leadership and maintenance. Key aspects of submitted files, folder structure, source collaboration, and issue reporting were outlined.

 Managing the AfyaSS

We formed a user support group comprised of ICT, QI, and HMIS officers at both the local government authority (LGA) and regional levels. They assisted both on-site and off-site. To ensure system administration and maintenance, staff from the MoH's ICT Unit were involved in every stage of system development.

 System optimization for AfyaSS

System optimization is the process of improving the performance, reliability, and efficiency of a system basing on feedback from users. For AfyaSS optimization the following issues that required system enhancement were addressed: system response time, insufficient data use features, system integration, and incorrectly structured checklists.

 AfyaSS system capabilities

Over 200 supportive supervision checklists are included in the designed AfyaSS system to support supervision preparation, resource coordination (including transportation, finance, and visit execution), and follow-up features in the areas of managerial, technical, and general types of supervision visits. It is capable of generating data and has a built-in accountability system such as the Global Positioning System (GPS) to ensure that the scheduled visits take place at the appropriate locations. It also enables the supervisor and supervisee to agree on areas identified as needing improvement and action items to be performed. The system dashboard gives an overview of visits planned, conducted, action items identified, and those implemented at the national, regional, and council levels, and even at the health facility level providing an overview of the country's health system's success in executing supportive supervision activities.

 Lessons learned along the process of application of the CRDM to develop and use the AfyaSS

After the successful development of AfyaSS using the CRDM and one year of AfyaSS implementation, we have encountered some challenges but also learned along the way. Some of the reported challenges include inadequate resources for system dissemination at various levels of health service delivery; poor internet connectivity in some regions and councils for online functionalities of the system; supervision checklists uploaded to systems need to be revised or updated by user directorates, units, programs, and professional councils; and inadequate knowledge among healthcare providers in the use of AfyaSS. Despite the challenges, we have learned that broad stakeholder buy-in and consensus, across different government departments and partners, is critical to scaling up the use of digital solutions like AfyaSS. There is also the importance of basing our work on a thorough understanding of existing work, tools, and documents. Additional requirements that emerged during system development caused delays in completing the system's development, necessitating having a sufficient buffer timeline for system development to account for unforeseen challenges. We also learned that the use of the ToTs approach has been extremely valuable in ensuring that whenever resources are available at the regional level, the available ToTs can provide training and technical support on the use of AfyaSS. We have an optimistic feeling that the ToTs would play a significant role in promoting the implementation of sustainable cascade training on AfyaSS at all levels of service delivery in the country 29– 31 .

 Discussion and conclusion

Innovative digital health systems are a result of the convergence of science and technology in the fast-moving digital era, enabling access to quality healthcare services 32, 33 . We aimed to describe the experiences of building the digital supportive supervision system through CRDM in Tanzania, a transition from the paper-based supportive supervision system.

We demonstrate the usefulness of applying the CRDM, a proven human-centered design approach, to the development of an AfyaSS system for conducting supportive supervision 22, 23, 34 . This approach ensures digital system users are at the centre of the development and has been applied by several LMICs 35– 37 .

Despite the challenges, Tanzania has successfully developed the AfyaSS system as part of the DUP project by applying CRDM. The CRDM has been a useful methodology in LMICs including Tanzania, and the involvement of all relevant stakeholders was critical to the development process's success. Due to the strong development process, the CRDM has proven to be successful in developing AfyaSS. It is also, expected that the AfyaSS will be very instrumental in ensuring robust implementation of supportive supervision at all levels to support healthcare workers improve the delivery of services to their clients as Tanzania embarks on the implementation of the Universal Health Insurance Act No. 13 of 2023 38

Ethics

The study received approval from the Muhimbili University of Health and Allied Sciences (MUHAS) institutional review board with approval number DA. 282/298/01.C/2020 of February 7, 2024.

 Consent

This study did not involve human subjects and hence no consent was required.

 Data availability Underlying data

No data are associated with this article.

 Extended data

Zenodo : Building a digital supportive supervision system for improving health service delivery in Low- and Middle-Income Countries through the Collaborative Requirements Development Methodology (CRDM): Experience from Tanzania 10.5281/zenodo.12807235

The project contains the following data: AfyaSS Development Requirements_Gates.docx

Data are available under the terms of the Creative Commons Attribution 4.0 International

 Acknowledgments

The authors gratefully acknowledge the Bill & Melinda Gates Foundation (BMGF) for providing catalytic support to the DUP project in Tanzania, through which this work was undertaken. We also thank the Tanzanian government for taking the leadership in project implementation.

 Tegegne SG Shuaib F Braka F : The role of supportive supervision using mobile technology in monitoring and guiding program performance: a case study in Nigeria, 2015–2016. BMC Public Health. 2018;18(Suppl 4): 1317. 30541512 10.1186/s12889-018-6189-8 6291920 Marquez L Kean L : Making supervision supportive and sustainable: new approaches to old problems. BJOG. 2002;112(4). Mboya D Mshana C Kessy F : Embedding systematic quality assessments in supportive supervision at primary healthcare level: application of an electronic Tool to Improve Quality of Healthcare in Tanzania. BMC Health Serv Res. 2016;16(1): 578. 27737679 10.1186/s12913-016-1809-4 5064905 Renggli S Mayumana I Mboya D : Towards improved health service quality in Tanzania: appropriateness of an electronic tool to assess quality of primary healthcare. BMC Health Serv Res. 2019;19(1): 55. 30670011 10.1186/s12913-019-3908-5 6341708 Mwendwa P McAuliffe E Uduma O : The impact of supportive supervision on the implementation of HRM processes; a mixed-methods study in Tanzania. Health Syst Policy Res. 2017;4(1). 10.21767/2254-9137.100066 Friebel R Molloy A Leatherman S : Achieving high-quality universal health coverage: a perspective from the National Health Service in England. BMJ Glob Health. 2018;3(6): e000944. 30613424 10.1136/bmjgh-2018-000944 6304094 Asma S Lozano R Chatterji S : Monitoring the health-related Sustainable Development Goals: lessons learned and recommendations for improved measurement. Lancet. 2020;395(10219):240–246. 31767190 10.1016/S0140-6736(19)32523-1 MoHSW: Health Policy (2007).MoHSW.2007. Accessed February 25, 2024. Reference Source MoHCDGEC: Health Sector Strategic Plan July 2021 – June 2026 (HSSP V): leaving no one behind.MoHCDGEC.2021. Accessed February 25, 2024. Reference Source Hokororo J Eliakimu E Ngowi R : Report of trend for compliance of infection prevention and control standards in Tanzania from 2010 to 2017 in Tanzania Mainland. Microbiol Infect Dis. 2021;5(3):1–10. 10.33425/2639-9458.1118 MoHSW: Health Sector Strategic Plan July 2015 – June 2020 (HSSP IV): reaching all households with quality health care. Ministry of Health and Social Welfare (MoHSW).2015. Accessed February 25, 2024. Reference Source MoHCDGEC: National supportive supervision guidelines for quality health services.Dar Es Salaam. MoHCDGEC.2017. Accessed February 25, 2024. Reference Source MoHCDGEC and PORALG: Functions of Regional Health Management System: Roles and Responsibilities of Regional Health Management Team, Regional Referral Hospital Management Team and Regional Referral Hospital Advisory Board.MoHCDGEC and PORALG.2019. Accessed February 25, 2024. Reference Source Labrique AB Wadhwani C Williams KA : Best practices in scaling digital health in Low and Middle Income Countries. Global Health. 2018;14(1): 103. 30390686 10.1186/s12992-018-0424-z 6215624 Avortri GS Nabukalu JB Nabyonga-Orem J : Supportive supervision to improve service delivery in low-income countries: is there a conceptual problem or a strategy problem?. BMJ Glob Health. 2019;4(Suppl 9): e001151. 31673434 10.1136/bmjgh-2018-001151 6797347 Osipov VS Skryl TV : Impact of digital technologies on the efficiency of healthcare delivery.In: Studies in Computational Intelligence.2021;933:243–261. 10.1007/978-981-15-9897-5_12 Renggli S Mayumana I Mboya D : Towards improved health service quality in Tanzania: appropriateness of an electronic tool to assess quality of primary healthcare. BMC Health Serv Res. 2019;19(1): 55. 30670011 10.1186/s12913-019-3908-5 6341708 MoHCDGEC: Tanzania digital health investment road map 2017-2023. The journey to better data for better in Tanzania. MoHCDGEC.2016; Accessed February 25, 2024. Reference Source The United Republic of Tanzania (URT): The 2022 population and housing census: administrative units population distribution report.2022. Reference Source Kapologwe NA Meara JG Kengia JT : Development and upgrading of public primary healthcare facilities with essential surgical services infrastructure: a strategy towards achieving Universal Health Coverage in Tanzania. BMC Health Serv Res. 2020;20(1): 218. 32183797 10.1186/s12913-020-5057-2 7076948 Opuko B : Country focus: Tanzania’s healthcare system.2023; Accessed June 17, 2024. Reference Source Seymour D Werner L Mwansa FD : Electronic immunization registries in Tanzania and Zambia: shaping a minimum viable product for scaled solutions. Front Public Health. 2019;7:218. 31440494 10.3389/fpubh.2019.00218 6693385 Rembert JH Zometa CS O’Carroll PW : Using the collaborative requirements development methodology to build laboratory capacity for timely diagnosis during the zika epidemic in Puerto Rico. J Public Health Manag Pract. 2021;27(3):E143–E150. 32011597 10.1097/PHH.0000000000001130 9059220 MoHCDGEC: Digital health strategy, July 2019 - June 2024, Dodoma, Tanzania. MoHCDGEC,2019; Accessed February 25, 2024. Reference Source PATH & Public Health Informatics Institute: Collaborative requirements development methodology: overview participant’s guide. PATH & Public Health Informatics Institute,2015; Accessed February 25, 2024. Reference Source Designveloper: Software development outsourcing: a detailed guide in 2024.2024; Accessed April 9, 2024. Reference Source Hassan M : Software development outsourcing: positive strategies to elevate your projects.2024; Accessed April 9, 2024. Reference Source Zafar S Sikander S Hamdani SU : The effectiveness of technology-assisted cascade training and supervision of community health workers in delivering the thinking healthy program for perinatal depression in a post-conflict area of Pakistan - study protocol for a randomized controlled trial. Trials. 2016;17(1): 188. 27048477 10.1186/s13063-016-1308-2 4822299 Pearce J Mann MK Jones C : The most effective way of delivering a Train-The-Trainers program: a systematic review. J Contin Educ Health Prof. 2012;32(3):215–226. 23173243 10.1002/chp.21148 Mormina M Pinder S : A conceptual framework for Training of Trainers (ToT) interventions in global health. Global Health. 2018;14(1): 100. 30348183 10.1186/s12992-018-0420-3 6198384 Poitras ME Bélanger E Vaillancourt VT : Interventions to improve trainers’ learning and behaviors for educating health care professionals using Train-the-Trainer method: a systematic review and meta-analysis. J Contin Educ Health Prof. 2021;41(3):202–209. 34292260 10.1097/CEH.0000000000000375 Bhavnani SP Narula J Sengupta PP : Mobile technology and the digitization of healthcare. Eur Heart J. 2016;37(18):1428–38. 26873093 10.1093/eurheartj/ehv770 4914890 Kaboré SS Ngangue P Soubeiga D : Barriers and facilitators for the sustainability of digital health interventions in low and middle-income countries: a systematic review. Front Digit Health. 2022;4: 1014375. 36518563 10.3389/fdgth.2022.1014375 9742266 Baltaxe E Czypionka T Kraus M : Digital health transformation of integrated care in europe: overarching analysis of 17 integrated care programs. J Med Internet Res. 2019;21(9): e14956. 31573914 10.2196/14956 6794072 Zaffran M Vandelaer J Kristensen D : The imperative for stronger vaccine supply and logistics systems. Vaccine. 2013;31 Suppl 2:B73–80. 23598495 10.1016/j.vaccine.2012.11.036 Vielot N : Improving public health preparedness: strengthening biosurveillance systems for enhanced situational awareness. Online J Public Health Inform. 2014;6(1): e48. 10.5210/ojphi.v6i1.5116 4050888 Singletary V Baker EL : Building informatics-savvy health departments: the systems development life cycle. J Public Health Manag Pract. 2019;25(6):610–611. 31569190 10.1097/PHH.0000000000001086 The United Republic of Tanzania (URT): Universal Health Insurance Act No. 13 of 2023.2023; Accessed June 17, 2024. Reference Source 10.21956/gatesopenres.17228.r38151 Reviewer response for version 1 Gómez-Pérez Gloria Patricia 1 2 Referee https://orcid.org/0000-0001-8555-8596 PharmAccess Foundation, Amsterdam, The Netherlands University of Amsterdam, Amsterdam Institute for Global Health and Development, Amsterdam, North Holland, The Netherlands

Competing interests: No competing interests were disclosed.

 25 10 2024 Copyright: © 2024 Gómez-Pérez GP 2024 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve-with-reservations

Dear Editor,

This research article describes the development of a digital supportive supervision system to improve healthcare delivery in Tanzania mainland using a user-centered approach. In the times of internet and the mobile use revolution that has reached low- and middle-income countries, digitalization of health services is the way to go to accelerate UHC and more resilient and stronger healthcare systems. Moreover, digitalization allows access to real time data, and without data, evidence-based decisions cannot be taken.

Dear authors,

I would like to congratulate you and the Minister of Health of Tanzania for this and other efforts towards the use of digital tools to improve the quality of care of the country.

I have nevertheless some questions and comments kindly for your consideration that I hope can help you enrich your manuscript even more:

- The methodology section is very well explained. However, and although it is mentioned that the tablets could be used online and offline, when you describe the software development and characteristics, it would be important to clarify if the software allowed for offline uploading of data. The challenges with internet connection will most likely continue in Tanzania and similar regions, therefore it would be important to state this clearly about the tool if it is the case. Something like: the software allowed for offline data upload.

- Would you please mention in which geographic areas was internet connectivity most frequently a reported problem?

- Would you also please describe what proportion of your targeted healthcare facilities used the tool during the year of testing? You mention the following in the challenges section: " include inadequate resources for system dissemination at various levels of health service delivery." From my experience, primary healthcare centers (PHC) in rural areas struggle more than bigger facilities to dedicate time to new tasks that they perceive no urgent due to limited number of personnel, and also because of lower digital literacy, poor internet connection, etc. Nevertheless, it is of great importance to include PHC in quality improvement efforts, that is why I wonder at which levels of healthcare delivery you encounter more problems.

- Would you kindly describe where is the data hosted, and efforts taken to protect it and guarantee Tanzania's ownership of it?

- Would you kindly clarify if the digital supportive supervision system would be, as I understand, only supporting the supervision, but not replacing any QI methodology each facility might have running at the moment? Or it is maybe the plan to centralize QI implementations?

- In case you have this information, would you kindly describe if the facilities already applying a QI methodology adhere faster and better to the use of the tool? Or if there were no differences? If not studied yet, I would recommend it, this way you know if this is a positive or negative factor in the implementation of the tool that you could take into consideration. It could be that because they are familiar with QI assessments, they do better, or it could be that because they already have a QI methodology in place, this represents and additional time effort for them and hinders their adherence.

- For the reader with few knowledge about digital tools, would you kindly explain what a beta version is.

- Finally, could you clarify if the data collected transforms into a score or report of compliance that is accessible to all users, including access to data from other facilities? This could serve as benchmarking and incentivize to do better when comparing with other facilities.

More thinking about the usability and sustainability of the digital tool more than about the manuscript, kindly for your consideration the following comments:

- Regarding data, and to allow its usage by other countries/healthcare bodies (with required permissions in place) have you structured its collection using a FAIR methodology? My experience is as well that collecting data alone is not enough, but to make sure the data is of good quality is also very important, e.g. using standard terms, reducing the chances of human errors (e.g. typos), etc. This reduces significantly the time invested in cleaning the data for analysis and interpretation and also to avoid that valuable data are lost because is not clear/accurate.

- Since the aim is to supervise/assess various aspects of the quality of care, and after the assessments a quality improvement plan (QIP) would be given, the latter would not guarantee in itself: 1. QIP implementation; 2. follow up assessment to check progress. The main obstacle is the lack of resources for (1) when for instance renewal of equipment is needed, or other expensive investments and the facility doesn't have the means to materialize this; and lack of motivation for (2) when you could not apply the QIP, why would you repeat it if you know not much has changed. In this regard has the MoH planned any kind of support to attain the QIP or created an incentive to: help facilities apply the QIP and increase their quality of care, and to fill in next assessments. Monetary incentives are not sustainable, but for instance discount in purchase of medicines or supplies, or access to loans with lower interest to buy equipment, or free training courses could be tested.

These would be my comments. I hope these are of help to you.

With kind regards, Patricia

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable

Are all the source data underlying the results available to ensure full reproducibility?

Partly

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Partly

Are sufficient details of methods and analysis provided to allow replication by others?

Partly

Reviewer Expertise:

Public Health, Quality of Care, Infectious Diseases

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.