Determinants of technology use for a mobile health intervention across public health facilities in rural India: Protocol for implementation research

Introduction

In India, the increased coverage of institutional births has not successfully translated to proportional improvement in maternal mortality rate (MMR) and newborn mortality rate (NMR) (Patel et al., 2015). Evidence suggests challenges of inadequate service availability and facility readiness, along with poor quality of care at public health facilities continue to plague the public health system and impede the desired reduction in NMR and MMR (Friberg et al., 2010; Jayanna et al., 2014; Kaur et al., 2019; Rudan et al., 2010). India has also taken steps to combat NMR and MMR by increasing the investment in its public health apparatus such as; capacity building of healthcare providers, operationalizing healthcare facilities for providing 24x7 basic and comprehensive obstetric care services, name-based web enabled tracking of pregnant women to ensure antenatal, intra-natal and postnatal care, to name a few (GoI, 2014; Press Information Bureau, 2014). Among the multiple strategies for reducing NMR and MMR, use of Information and Communication Technology (ICT) interventions have emerged as a promising solution (Howitt et al., 2012).

India, as has the world, is undergoing an increasing development of ICT solutions for health care access and delivery (Bodavala, 2002). Mobile health (mHealth) interventions, a specific example of ICT led solutions, supported by mobile devices, such as mobile phones, patient monitoring devices, personal digital assistants (PDAs), and other wireless devices (Kay, 2011; mHealth: new horizons for health through mobile technologies., 2011) offer new opportunities for alleviating a range of global health challenges including Maternal, Newborn and Child Health (MNCH) outcomes (Tamrat & Kachnowski, 2012). These interventions may focus on the demand-side influencers, such as client education and behaviour-change communication, or on service delivery personnels, such as health workers at the facility or community level (Abejirinde et al., 2018; Labrique et al., 2013).

Several mHealth interventions have been introduced to improve MNCH outcomes across low and middle-Income Countries (LMICs) (Martinez et al., 2018; Ruton et al., 2018). The Alliance for Saving Mothers and Newborns (ASMAN) program is one such mHealth intervention. ASMAN, a partnership between the Reliance Foundation, Bill & Melinda Gates Foundation, MSD for Mothers, Tata Trusts and The United States Agency for International Development (USAID), was established in 2017 with the objective of improving maternal and neonatal health outcomes across the two northern states of Rajasthan and Madhya Pradesh ((ASMAN), 2019). These two states were chosen due to persistent levels of high NMR and MMR than national average. ASMAN aims at capacity building and use of technology that enable health care workers to improve their performance and provide quality care for intrapartum and early new-born care at selected public health facilities in two states. The ASMAN intervention is implemented by a consortium, with Jhpiego¹ as the implementer of ASMAN activities in public healthcare facilities, Avalon² and Bodhi³ developing the digital content and application for PDAs, and Sambodhi⁴ conducting the external evaluation.

The ASMAN digital platform is a PDA-based (digital tablet in this case) intrapartum and immediate postpartum decision support tool for staff nurses across 81 public health facilities in Rajasthan and Madhya Pradesh. The nurses use this platform to enter real-time data on digitized maternity case sheets that also include a Safe Childbirth Checklist along with provision for vital digital recording. Also, alerts and notifications inform staff about high-risk cases so they can receive timely support. A live dashboard helps health workers and managers monitor all cases in real-time, identify and manage high-risk cases, refer cases to higher centers, and make urgent decisions if necessary. Continual learning that makes use of game-based case scenarios and e-learning modules is also included to keep the nursing staff informed and equipped to handle complicated cases. The final component of this intervention includes setting up remote support centers where specialist doctors help advise or confirm a health worker’s choice of intervention in case of a complication.

There is limited empirical evidence on the effectiveness of mHealth interventions and research often does not investigate the mechanism of action behind the successful adoption of mHealth interventions and its association with positive patient outcomes (Damschroder et al., 2009; Kaplan et al., 2010; Marcolino et al., 2018). A multi-country reviewfound that while mHealth interventions have shown to have positive effects on health outcomes, the “how” behind a successful intervention remains unclear (Cresswell et al., 2013). In the case of India, while mHealth interventions implement technology-enabled solutions, the determinants of their uptake and interaction with public health functionaries at the facility remains unexplored.

A systematic review of mHealth evaluations in India recommended implementation research (IR) as a pragmatic approach to unpacking the black box of mHealth interventions in India (Bassi et al., 2018). IR offers a robust set of tools that are useful in understanding this complex intervention and investigate questions concerning implementation, specifically the act of carrying an intention into effect (Peters et al., 2014; Sturke et al., 2014). The study proposes to conduct IR for ASMAN intervention with the aim to understand the role of contextual factors associated with implementation and adoption of mHealth interventions along with studying the public healthcare provider’s interactions with technology. The IR will also explain the findings from a separate outcome evaluation for ASMAN intervention. The outcome evaluation will collect quantitative data on knowledge and skill of healthcare workers, and intrapartum and newborn practices in the public health facility.

Research objective

The primary objective of the IR study is to investigate the determinants for mHealth uptake in public health facilities in India. Specifically, the IR study aims to answer the following research questions:

Interventions to be measured

The IR aims to assess the implementation dynamics behind all the mHealth intervention activities under ASMAN (Figure 1). Specifically, the IR will investigate the mechanism of action and nuanced interaction between public healthcare service providers and ASMAN interventions such as digitized case sheets, digital vitals recording, remote call-center support, technology learning aids, audio-video tutorials and game-based learning. The IR will also attempt to triangulate the insights with quantitative findings on MNCH outcomes from the separate outcome evaluation.

Figure 1. Processes and intended results for ASMAN intervention.

Methodology

The study proposes a longitudinal mixed-methods multiple case study design with the facility as a unit of analysis to answer the research questions. Case study methods are well-suited for studying implementation processes, which tend to be fluid, non-linear, and context sensitive (Ferlie et al., 2005; Van de Ven et al., 2008). In addition to permitting in-depth analysis of individual cases, case study methods offer analytic strategies for systematically comparing patterns observed across cases (Miles & Huberman, 1994; Yin et al., 1978).

The study will use the Consolidated Framework for Implementation Research (CFIR) as the theoretical guide of measures within the ambit of the design. CFIR is a meta-theoretical framework and includes constructs from a synthesis of existing implementation theories. CFIR is primarily a determinant framework used to understand and/or explain influences on implementation outcomes (Nilsen, 2015). It offers an overarching typology – a list of constructs to promote theory development and verification about what works, why, and where across multiple contexts (refer to Table 1) (Damschroder et al., 2009). CFIR provides a comprehensive general structure for unpacking the complex process of real-world implementation across multiple settings, multiple levels, and multiple phases. At the same time, the use of CFIR can be highly flexible and promotes the selection and use of constructs from within the CFIR that are most relevant for the study settings (Damschroder et al., 2009).

But the CFIR constructs are yet to be contextualized to the Indian setting. Using the longitudinal mixed-methods multiple case study design, the study will collect data sequentially across two phases. In phase 1 of data collection, the study will assess health facilities, using CFIR as a theoretical guide, identifying factors that influence the implementation of ASMAN intervention. In phase 2, health facilities reporting common factors will be followed up to re-confirm the validity of identified factors and their association with MNCH outcomes. Together, the design components will give us a more comprehensive understanding of the unexplored domains of contextual factors influencing the implementation of ASMAN intervention. The design will also align with timelines for the concurrent outcome evaluation.

Sampling plan and participants

Under the ambit of longitudinal multiple case-study design, the study will select public healthcare facilities that are a part of ASMAN intervention and assess them across two rounds of data collection. Overall, the study will cover 8 to 10 ASMAN-implementation districts, and 16 ASMAN-implementation public healthcare facilities across Rajasthan and Madhya Pradesh. A separate outcome evaluation of 49 ASMAN-implementation public healthcare facilities in the same districts, conducted earlier in 2019, will act as the sampling frame for choosing the facilities. The study will adopt a three-step process for facility selection to ensure the representativeness. First, the study will use quantitative results of the 49 facilities on health outcomes as the sampling frame. Second, 16 facilities (8 facilities from each state) will be selected purposively based on their performance in health outcomes. Third, the study will also try to ensure representativeness among facilities, by selecting 2 District Hospitals (DH), 2 Sub-District Hospitals, 2 Community Health Centers (CHCs), and 2 Primary Health Centers (PHCs) in each state (refer to Table 2). Thus, we plan to adopt a maximum variation sampling in order to capture the heterogeneity of facilities and stakeholders and make it more representative of target groups. In phase 1, the selected 16 facilities will be assessed to help contextualize CFIR. Phase 2 will follow up with the same health facilities from phase 1 using the contextualized CFIR constructs and answer the overall research questions.

The study will conduct in-depth interviews (IDIs) with healthcare providers in the selected public health facilities. The data collection will include staff nurses, labor room in-charge, and medical officers or head of the department, previously covered under the outcome evaluation and depending on their availability at the time of interview. Healthcare providers who have not been covered in the previous outcome evaluation will be excluded from the study. Overall, the study plans to conduct 30 to 38 interviews (refer to Table 1). We also plan to interview the program managers at district and state level in the respective state governments (refer to Table 3). From the funding partners, the study will conduct IDIs with the Program Management Unit of ASMAN. From the implementation partners, the study will interviewprogram teams at Jhpiego, Avalon, and Bodhi (Refer to Table 3). Remote Support Center staff will also be interviewed to gain an understanding of their perspective on the ASMAN intervention. We anticipate about 60–65 participants to participate in each phase, although the precise number will be dependent on attaining saturation in addressing the research questions. The qualitative interviews conducted under IR will be supplemented by quantitative interviews with the same participants under the previous outcome evaluation. The survey instrument used during the outcome evaluation will include direct observation of delivery in within the health facility’s labor room, infrastructural assessment of the health facility, and provider’s frequency of using ASMAN technology on a 5-point Likert scale. The outcome evaluation also included the Health ITUES questionnaire consisting of 20 questions rated on a 5-point Likert scale, comprising of four sub-scales: (1) quality of work life, (2) perceived usefulness, (3) perceived east of use, and (4) user control of the ASMAN technology (Schnall et al., 2018).

Measures

The CFIR will guide the IR from an implementation perspective. CFIR is a meta-theoretical framework synthesizing 19 multidisciplinary theories, several frameworks, and 37 constructs used in dissemination and IR. Relevant constructs under CFIR will be used to collect data from the selected health facilities. Since the CFIR constructs lack contextualization to the mHealth context in India, the study uses phase 1 to harmonize them with the ASMAN intervention. In phase 1, the study will develop qualitative interview tools based on CFIR constructs and tailored based on the participant’s roles and responsibilities. Table 4 summarizes some of the key domains and relevant constructs as per the CFIR. The key areas of inquiry for phase 1 is provided in Table 5. In phase 2, the study will employ a mixed-methods explanatory design to re-assessthe same health facilities on the contextualized constructs that have emerged as significant influencers of the implementation during phase 1 (Creswell et al., 2004; Ivankova et al., 2016). The survey will attempt to follow the same respondents in the health facilities. The key areas of inquiry for phase 2 of the study will be developed using findings from phase 1.

From the quantitative side, phase 2 will borrow findings from the outcome evaluation measures and documentary evidence such as intrapartum and newborn practices, facility infrastructure and readiness, knowledge and skill of healthcare providers, and the motivation of healthcare providers and their perception of work climate.

Ethical considerations

Sigma institutional review board, a third-party ethical oversight agency in India, has reviewed and approved the study. All ethical guidelines mandated by Sigma will be followed strictly through the study. Apart from these, the study will seek necessary approvals from participants, state and health departments, and other relevant departments before and during the study rollout. Participants will be assigned random identification codes and will not be identified by their names or any other demographic detail. All information regarding the IR will be disclosed to participants and guardians via an informed consent form before collecting data. A set of trained research investigators will procure the consent. Data collected from the participants will be kept confidential and will only be accessed by researchers for analysis.

Data analysis plan

The study aims to identify determinants of mHealth interventions by attempting to arrive at literal and theoretical replications. Multiple case study design can be used to either predict similar results (a literal replication) or predict contrasting results but for predictable reasons (a theoretical replication) (Yin, 2014). First, the study will conduct a thematic analysis using framework method to analyze the qualitative data captured through IDIs (Gale et al., 2013). The study will employ pattern-matching logic to guide the data analysis (Yin et al., 1978). The qualitative data will lead to a starting list of codes, supplemented with emergent codes as analysis proceeds. Then a within-case analysis of each public health facility in a state will be conducted to assess the degree to which the construct emerges in the data (its “salience”) and positively or negatively affect implementation (its “valence”). The study will also check for construct strength and valence between cases (across facilities in both states). Putting together the results, the within-case analysis of salience and valence will help us identify which contextual factors from the CFIR operate as potential barriers or facilitators to implementation and use.

The within-case analysis will be followed by a between-case analysis to investigate how those potential barriers and facilitators vary by facility type and state. The cases (facilities) will be arrayed into a 3×2 table with facility type by state. Using replication logic, each case will be treated as if it were an experiment where the conditions are either intentionally repeated from one case of the next, or systematically varied. Literal replication occurs when cases in the same cell exhibit similar patterns for theoretically predictable reasons (i.e., they are the same facility type and state). Theoretical replication occurs when cases in different cells exhibit similar patterns for theoretically predictable reasons (i.e., they differ by facility type or state or both). This analysis will indicate whether identified barriers and facilitators are unique to specific cases, common to similar cases, and different among different cases.

Phase 2 will investigate the relationship between contextualized CFIR constructs and MNCH outcomes in the public health facility collected via outcome evaluation, articulated in research question 2 under research objectives. The study will test the hypothetical relationships by using three criteria proposed by Trochim (Trochim, 2007) and Miles and Huberman (Miles & Huberman, 1994). The first criterion will look for the overall covariance of the constructs (e.g., whether facilities exhibiting a strong implementation climate for mHealth also exhibit high implementation effectiveness or positive patient outcomes, i.e., accrual). The second criterion will look for explicit attributions or the identification of plausible mechanisms to link the two constructs (e.g., participants attribute a strong implementation climate to the deployment of appropriate implementation policies and practices). The third criterion will look for indications of temporal precedence for the hypothesized relationship. In this, the study will rely primarily on documentary evidence for establishing temporality but will also consider participants’ accounts of the sequence of events. Together, the study will apply the three criteria across the cases to determine if cross-case variation in implementation is consistent with the hypothesized relationships in the model. Also, the study will create within-case and between-case data displays that cross-tabulate the quantitative and qualitative data to facilitate the use of pattern-matching logic (Miles & Huberman, 1994).

The final product of the analysis will be a theoretically informed and empirically grounded model of organizational mHealth implementation, strongly associated with improved MNCH outcomes.

Discussion

The paper discusses the protocol of an IR study investigating factors that influence the roll-out and adoption of ASMAN mHealth intervention and their association with MNCH outcomes. LMICs globally have attracted investment in mHealth space as a means of alleviating multiple global health challenges, including MNCH (Tamrat & Kachnowski, 2012). In India alone, multiple mHealth solutions have been implemented, aimed at improving healthcare service delivery and patient health outcomes. But despite its programmatic potential, there is a lack of evidence on user acceptability, adoption, and replicability of mHealth solutions (Bassi et al., 2018).

The IR approach aims to unpack the existing heterogeneity within types of public health facilities and across state contexts that are critical to implementing the mHealth solution (Ilozumba et al., 2018). This IR study will identify factors contextual to Indian public health facilities that influence the successful implementation and replication of mHealth solutions such as under ASMAN intervention. Further, the study will investigate whether and how are these factors associated with positive MNCH outcomes.

The IR study has several implications. From an implementation perspective, findings from the study can act as a guide for implementers in public health facilities while introducing technology or inform interim course corrections. From a policy perspective, the findings can contribute to shaping newer perspectives to the designing and evaluation of mHealth based interventions in the Indian context. From a research perspective, the study will add to the existing empirical evidence on mHealth interventions in LMICs in general and India specifically. The findings from the study may also provide the answer to “how” can such interventions be implemented and adopted in different contexts to influence MNCH outcomes.

Limitations

There are several limitations to the methodology. The qualitative interviews to be conducted in phase 1 may not reflect differences across other public health facilities in the entire region. Also, the public health facilities are in northern India and may not reflect the environment in other parts of the country. Another limitation of the study is the possible low construct validity in the selected CFIR constructs. The study will collect a varying number of participants from each health facility based on their availability. Lesser number of participants may affect the construct reliability. For example, too few respondents within a facility may not provide a complete representation of the public health facility.

Data availability

No data are associated with this article.