Specifically, the study aimed to answer the following research questions:

A secondary objective, which is exploratory, was to describe selected MNH outcomes for clients in Makueni county during the study’s implementation. This objective was exploratory because a direct relationship between the NOC and the selected outcomes may not be determined.

This study will inform decision-makers locally and globally on whether NOC is feasible and acceptable as a county-level model of care in Kenya with application to similar LMIC settings. We used the SQUIRE 2.0 guidelines from the EQUATOR Network to report this quality improvement study protocol ¹⁶ .

The site for this study was Makueni county, one of the 47 counties in Kenya. Makueni county is situated in South-Eastern Kenya and has a population of approximately 900,000 people, with women of reproductive age representing 23% of the population ¹⁰ . Makueni county health management team (CHMT) provides the oversight and coordinating mechanism for MNH service delivery in the county. The NOC approach was introduced in 60 health facilities that provide MNH services and referrals (51 public, 9 private), across all the six sub-counties of Makueni county, namely Makueni, Kaiti, Mbooni, Kibwezi East, Kibwezi West, and Kilome sub-counties. Purposive sampling in consultation with the Makueni CHMT officials was used to identify the 60 health facilities. The selection was based on the geographical distribution (at least six facilities in each of the six sub-counties in Makueni) and maternity caseloads per month (these facilities reported the highest maternity caseloads in 2020 and represented 17% of the 344 health facilities in Makueni). The health facilities included levels II, III, IV, and V facilities. Level II health facilities are dispensaries, level III are health centres, level IV are sub-county hospitals, and level V is the main county referral hospital.

The NOC intervention for maternal and perinatal health is defined as a collection of public and/or private health facilities and health workers in a defined geographic area that are deliberately interconnected to promote multi-disciplinary teamwork and collaborative learning. Health facilities within the network implement a package of interventions that follows a theory of change model ( Figure 1) with the intent to improve MNH outcomes in both public and private health facilities. The intervention is co-implemented by health facility staff who are employed by the government or private sector and routinely provide MNH services, and county government officials from Makueni CHMT. The NOC package of interventions was designed and based on evidence and learnings from the CHAI scoping study ¹⁴ ( Figure 2) and includes:

Domain I: Agreement and enabling environment

A sequential participatory approach was utilized to build support and buy-in from Makueni county stakeholders. First, several targeted planning meetings including an introductory meeting at the county level with approximately 35 participants, a meeting to identify communication mechanisms and determine data review meeting cadence with approximately five participants, and a meeting to review and finalize NOC materials with approximately five participants were conducted. Participants included several county stakeholders from Makueni CHMT to health facility staff at level II to V facilities. Following the planning meetings, health facilities were introduced to the NOC concept through an orientation meeting with approximately 90 participants drawn from the intervention facilities. During the health facility orientation meeting, mapping of the health facility resources across the county was done. Mapping included adapting, printing, and applying the mapping tool at all 60 intervention facilities within Makueni county. Mapping data were collected by a facility focal person/in-charge, shared with the sub-county reproductive health (RH) coordinator for validation and further shared with the sub-county health records and information officer and county RH coordinator for analysis and results generation. A county level NOC steering committee was then established comprised of Makueni CHMT, health facility focal persons and county records officers. The steering committee conducted a one day meeting with approximately 90 participants drawn from the intervention facilities to review the mapping exercise results, define a referral pathway and develop materials to streamline referrals. Roles and responsibilities of each intervention facility were also defined during this meeting. A half-day meeting that engaged approximately 90 participants drawn from the steering committee, Makueni CHMT, facility focal persons and records officers was then held to enable facilities to sign a collaborative agreement agreeing to participate in the NOC intervention. The finalized NOC materials were distributed during this meeting.

Domain II: Operational standards

The NOC materials including referral pathways were adapted, printed and distributed to each intervention facility during the signing of the collaborative agreement. These included maternal and newborn referral forms, referral registers and facility readiness assessment tool. The facility readiness assessment tools captured, on a monthly basis, information on supplies, guidelines, clinical skills, documentation, and ongoing referral mechanisms.

Domain III: Quality, efficiency, and responsibility

Mentorship support is an important element for building trust and relationships within the NOC, as well as driving improvements in quality of care. In Makueni county, the Makueni CHMT supports need-based mentorships that are informed by data, facility assessments, and mentee requests. Mentors – including clinicians and RH coordinators – were recruited at the county and sub-county levels based on their MNH experience and skills and visited NOC intervention facilities at least once every three months. During these mentorship sessions, facilities review their outcomes and analyze their reporting rates and existing quality indicators, and they address any quality of care issues.

To nurture and facilitate communication and coordination within the NOC, Makueni county selected and used the WhatsApp platform as one of its communication mechanisms. Facilities within the NOC used WhatsApp for consultation, problem solving, referral coordination, and information sharing. The WhatsApp platform was also used for clinical updates as well as identifying challenges and best practices. In addition, health facility focal points used phone call to inform about, manage, coordinate, and account for referrals between sending and receiving facilities.

Domain IV: Learning and adaptation

Data review meetings with approximately 78 participants drawn from the NOC intervention facilities, steering committee members, Makueni CHMT and MNH partners were conducted every three months to review service statistics and referral data across the county. Service statistics were assessed via facility readiness tools, registers and referral forms. The data review meetings also provided a forum for joint problem-solving around challenges faced by health facilities within the network. The research team also provided updates on study progress during the data review meetings.

International Centre for Reproductive Health Kenya was the local research partner responsible for measurement of study outcomes while Jhpiego supported Machakos county in the implementation of the NOC intervention and provided quality assurance.

This was a mixed-methods implementation research study. Both qualitative and quantitative data were collected. Health workers participating in the NOC intervention used their mobile phones to complete a self-administered survey at baseline and then every three months. A subset of those completing phone surveys participated in seven focus group discussions (FGDs) at baseline, midline, and endline. A subset of 24 Makueni CHMT members participated in key informant interviews (KIIs) at baseline, midline, and endline. Additionally, data was extracted from health facility readiness assessment tools, facility maternal and newborn referral forms, and facility referral registers. The study also monitored trends in MNH indicators for Makueni county from the KHIS.

The primary outcome, the impact of NOC on relationships among health facilities and health workers within the county health system, was assessed using a self-administered mobile phone-based questionnaire for health workers, data extracted from health facility referral registers and referral tools, and FGDs for health workers. Secondary outcomes of interest included the acceptability and feasibility of integrating NOC into the county and sub-county health care and referral systems including the cost. These outcomes were measured by program data, FGDs with health workers, and KIIs with Makueni CHMT. Exploratory outcomes included changes in selected MNH outcomes for clients in Makueni county and were measured from KHIS data.

The target sample size for the study was 184 health workers. Purposive sampling was used to identify health workers to participate in the NOC intervention.

Study participants for the phone surveys and FGDs were selected in two stages. In the first stage, the research team engaged the facility in-charges of the 60 NOC health facilities to identify and list all staff working in MNH units and other relevant hospital departments. All health workers who provided MNH services were listed. In the second stage, health workers were selected based on profession (nurse/midwife, doctor, clinical officer, and emergency medical technicians), department (maternity, ANC, MCH, and outpatient), gender, and the number of years of experience in the health facility. The sample size for each level of health facility was determined by the number of health workers providing MNH services.

For the surveys, two health workers were sampled from each of the level II facilities, three from level III, four from level IV, and five from the level V facility with consideration for the increased number of health workforce with an increased level of health facility. For the seven FGDs, seven to ten health workers, a subset of those completing phone surveys, were purposively sampled per sub-county from each of the six sub-counties. An additional seven to 10 health workers were sampled across all the six sub-counties specifically from Newborn Units (NBU). The NBU-specific FGD was included to understand specific health worker experiences when handling unique neonate management challenges.

Makueni CHMT participating in the KIIs were purposively selected from the steering committee with additional non-steering committee members from all six sub-counties of Makueni county.

The research team conducted stakeholder meetings with Makueni CHMT and in-charges of NOC health facilities to share study objectives and seek permission to conduct the study. Four research assistants were trained to provide information on the study to eligible health workers and to Makueni CHMT, and to invite them to participate in the survey, FGDs, and KIIs.

Health workers who agreed to participate in the survey were required to provide written informed consent and a personal mobile phone number issued by the telecommunication provider Safaricom. Safaricom is the largest mobile telecom operator in Kenya with 63.6% market share and 94% 3G network coverage of the population ¹⁷ . Study participants received an orientation on how to respond to self-administered survey questions and submit survey responses through their mobile phones at the time written consent was obtained. The mobile phone number list of all consented participants was shared with the study Information Technology (IT) lead who then imported the mobile phone numbers on an unstructured supplementary service data (USSD) platform developed by Innovee Systems (T) Limited. The IT lead then activated the study time on the USSD platform to run for 21 days. Upon activation, the USSD system sent push notifications - short messages to study participants welcoming them to the USSD Platform and providing participant credentials to log into the survey together with a specific code to access the survey. The survey technology worked with all mobile phones. Participants who logged into the survey but did not complete their responses were sent automated reminders every 48 hours via a push notification from the USSD system until they completed the survey. Each question had a maximum of five possible responses, and each response led to the next.

Health workers who agreed to participate in the FGDs and Makueni CHMT who agreed to participate in the KIIs were also required to provide written informed consent. FGDs and KIIs were conducted in person by a trained researcher at a venue that offered privacy with the aid of interview guides and tape recorders. The interview language was English since all participants had a tertiary level of education. Data collection involved probing to gather information until saturation was achieved. Interview audio recordings were saved on tape recorders.

The study used a questionnaire format that was validated during a previous study on measuring health workers’ communication, collaboration, and trust (Nuño-Solinís et al.) ¹⁸ . The questionnaire was adapted and tailored to the Makueni county context with input from Makueni CHMT and NOC steering committee members. The questionnaire was comprised of 17 questions, which covered four main domains: (i) Perceived health worker confidence to manage MNH emergencies and referrals based on existing resources, (ii) Perceived health worker accountability for the management of MNH complications and ensuring the best outcomes, (iii) Perceived levels of collaboration, communication and trust among health workers and health facilities when managing MNH emergencies and referrals and (iv) Perceived appropriateness and timeliness of referrals. The responses were graded on a five-point Likert Scale from 1 to 5, whereby 1 corresponded to “disagree strongly”, 2 for “disagree somewhat” 3 for “neutral”, 4 for “agree somewhat” and 5 for “agree strongly”. Piloting of the survey questionnaire was done in October 2021 to refine the tool. The questionnaire is available at Open Science Framework ¹⁹ .

The FGDs and KIIs used a semi-structured interview guide on topics including:

1.   Communication, collaboration, and trust between health facilities;

2.   Confidence in the ability to manage complications and the level of accountability in health facilities;

3.   Acceptability and feasibility of integrating NOC into existing county and sub-county health care and referral systems;

4.   Appropriateness of the referrals to other facilities and trust in their capacity to handle cases;

5.   Sustainability of the NOC.

The FGD and KII interview guides are available at Open Science Framework ¹⁹ .

Once the health workers submitted the phone surveys, these were sent to a cloud-based server via the telecommunications provider Safaricom. The data was aggregated there before being downloaded to a secure server for data cleaning and validation checks by the study analyst. Before analysis, data was checked for errors, inconsistencies, completeness, and accuracy. Frequency distributions were computed by age category, gender, duration worked in the facility, cadre, department, and level of health facility. The internal consistency of the items in the questionnaire domains was assessed using Cronbach alpha. Chi-square tests will be computed to assess for associations.

All interview audio recordings were downloaded to a password-protected computer and stored in a project folder. Audios were transcribed and translated where necessary and stored as Microsoft Word documents. Transcribed data was iteratively read through, and themes identified. Data will be analyzed with NVivo 12 (Open access alternative: LiGRE). Charting will be done to link the specific quotes with the themes, and thereafter interpretation will be done. Signed written informed consent for all study participants was filed and kept in lockable cabinets at ICRHK offices.

The study was conducted with ethical approval from the Amref Ethics and Scientific Review Committee (Protocol number 1016/2021 – 10/08/2021), and the National Commission for Science, Technology and Innovation (NACOSTI – 26/08/2021) in Kenya. A non-human subjects research determination was approved by the Institutional Review Board at the Johns Hopkins University in the USA (IRB00017146 – 02/08/2021). All study participants were required to provide written informed consent.

The study team plans to disseminate findings among global, national and sub-national stakeholders through in-country dissemination events and globally through peer-reviewed journal manuscripts and international conference presentations.