Simulation and nurse-mentoring in a statewide nurse mentoring program in Bihar, India: diagnosis of postpartum hemorrhage and intrapartum asphyxia

Ethics statement

The study was approved by the Institutional Committee for Ethics and Review of Health Management Research Office of the Indian Institute of Health Management Research in Jaipur, India and the Committee for Human Research at the University of California San Francisco. Study ID# 14-15446.

Study intervention and personnel

The AMANAT program was implemented in four phases in 320 Basic Emergency Obstetric and Neonatal Care (BEmONC) facilities between May 2015 and January 2017. Each phase covered 80 facilities, which were chosen based on facility readiness to implement the program. Since the inception of the AMANAT nurse-mentoring program, PRONTO International has worked with CARE-India to integrate simulation and team-training into the AMANAT curriculum, tailored to address local contextual needs.

During the AMANAT program, 120 mentors (with Bachelor's degree) were trained in simulation facilitation and debriefing best-practices²³. They received two trainings, simulation facilitation and advanced simulation facilitation training that were roughly four months apart. After the first training, a pair of mentors visited one facility per week covering four assigned facilities in about a month. The nurse-mentor pairs repeated their visits every month to their assigned facilities. In other words, a facility received one week of mentoring per month for seven to nine consecutive months. The AMANAT mentors trained the L&D team in the facilities where each team comprises of one to two nurses with or without any other birth attendant. Physicians or specialists are rarely available. The nurses are either Auxiliary Nurse Midwives with a 2-year training on multipurpose community health or General Nurses with a 3-year training in general nursing and midwifery. The details of the AMANAT program and PRONTO’s simulation and team-training have been elaborated elsewhere and are briefly described below^20,21,24.

Embedded in the AMANAT program was PRONTO’s curriculum that included simulations to promote the use of EBPs, improve T&C, and to increase awareness around person-centered maternity care among L&D staff. To improve T&C between providers, PRONTO’s curriculum includes TeamSTEPPS™²⁵ concepts within its simulation exercises resulting in a curriculum, which allowed nurses to practice both technical and non-technical skills in a simulated scenario that replicates a real emergency. Mentor pairs facilitated simulations and debriefs, both of which were video recorded²⁶. In addition to simulation and T&C activities, mentors were encouraged to guide and supervise clinical care of patients in real time from admission to discharge, hereafter referred to as bedside mentoring. Finally, there were regular knowledge reviews (case-based learning) and routine didactic sessions with demonstrations on topics such as the partograph, active management of third stage of labor, infection control, fetal heart rate monitoring, basic resuscitation, golden minute for an asphyxiated newborn, and kangaroo mother care. More contextual information is presented in the Extended data^27,28.

Data sources

For this analysis we used data from two separate sources that are designed, collected and managed by CARE’s Concurrent Measurement and Learning team: (1) the Facility Information System (FIS), a web-based routine monitoring system used to record data on live deliveries and training activities during the weeks of mentoring; and (2) the direct observation of deliveries (DOD) before and after implementation of the AMANAT intervention. A third source, maintained by UCSF/PRONTO, recorded the number of simulations performed, which was used to validate the FIS based simulation counts. Data on select delivery practices were collected on all deliveries, daily, by mentors using observations and registers during their visits to the facilities and subsequently entered into the FIS. For the DOD, clinically trained project personnel visited each facility for a week to observe real deliveries occurring between 9 AM and 5 PM, approximately. The DOD provided comprehensive information on specific EBPs for individual deliveries both before and after the intervention periods. Data for this analysis was obtained from the FIS because it provided sufficient sample size to power the investigation on rare complications. The DOD data was used to generate facility level practice scores to adjust the final models, as described elsewhere²¹.

Clinical outcomes

The two clinical complications investigated in this analysis were diagnosis of maternal PPH and intrapartum (or birth) asphyxia of the newborn as defined by the World Health Organization^29,30. PPH was defined as blood loss associated with obstetric labor or childbirth of more than 500ml for a vaginal delivery. Intrapartum (or birth) asphyxia was defined as failure by the neonate to initiate or sustain breathing at birth.

Statistical analysis

We used two metrices to quantify the association between performance of simulations and diagnosis of complications. The first metric was total counts of maternal or neonatal complication simulations that were conducted in the entire mentoring period in a facility. The facilities were ranked based on the number of simulations performed and grouped into three roughly equal categories (tertiles): bottom-third, middle-third and top-third. The second metric was the ratios of the count of simulations to the count of other activities (T&C, demonstrations of various clinical skills, bedside mentoring and didactic lessons), performed in the facilities. Therefore, we had four ratios – (1) simulations : T&C, (2) simulations : demonstrations, (3) simulations : bedside mentoring and (4) simulations : didactic lessons. Like the first metric, each of these ratios were categorized into three ordered groups (tertiles). For example, a facility in the top-third sim: didactic ratio group spent more time doing simulations relative to didactic sessions than a facility in the bottom-third group (reference group). The four ratios were estimated separately for maternal complication simulations and neonatal complication simulations. We examined both continuous form and categories of simulations. A small percentage (~1%) of the facilities had high values of these ratios, meaning the number of simulations performed were much higher compared to the other activities. To minimize the outlier effect and to avoid imposing linearity in the relationship, we generated tertiles from the continuous ratios, using 33^rd and 66^th percentiles as cut-offs. Programmatically, results in categories are more meaningful to implementors as it will provide a specific number (midpoint of the range) or range of simulations associated with maximum improvement. In contrast, analysis using a linear continuous form (of simulation counts) will not provide a maximum limit, which aside from being infeasible will imply more simulations is better. This might not always be the case as it may take away time from other important activities, including conducting mentored real deliveries.

The analytical strategy used for this analysis is similar to the one previously reported²⁰. Births and complications between the start and end date of each mentoring week for each facility were aggregated. Thus, one row of observation represented one week in one facility. We used negative binomial models appropriate for the count of complications. To account for excessive zeros, i.e., no complications in a facility in a week, we utilized the zero-inflated negative binomial model and reported the incidence rate ratios (IRRs) with 95% confidence intervals (CIs). The IRRs compared the middle-third group of facilities with the bottom-third group and the top-third with the bottom-third, for both metrices. For readers interested in statistical details, we have rationalized the choice of models in the supplement and in a previous publication²⁰. Deliveries were clustered in both time and space, so we used the sandwich variance estimator, which provides appropriate standard errors after accounting for non-independence of observations³¹.

The final models were adjusted for weeks of nurse-mentoring, days per week of nurse-mentoring, total number of births per week, number of T&C activities performed, phase of the AMANAT program, availability of physicians, facility level practice scores and proportion of mentee-sessions attended. Mentee-sessions attended were defined as follows: if facility A had 10 mentees and week 1 of mentoring had 40 total sessions, there will be a maximum of 400 mentee-sessions in that facility(A)-week(1). Of these, if 8 mentees were present for all 40 sessions, 9^th mentee was present for 30 sessions, and 10^th mentee was present for 10 sessions only, this would equate to 90% mentee-sessions attendance [(320+30+10)/400] for that facility(A)-week(1).

Facility level practice scores were generated from the DOD data using a set of 11 intrapartum and 12 newborn care indicators, explained in a prior publication²¹. Individual indicators were assigned a score of 1 if they were performed as recommended; otherwise, a 0 was assigned. Individual delivery scores were rescaled to range between 0 and 100, where 0 refers to none and 100 refers to all EBPs being performed appropriately for the delivery. The indicator-specific scores were aggregated to obtain a delivery score and scores for all deliveries in a facility were averaged to obtain a facility level practice score.

Significance was examined at the 5% (two-tailed) level. As the categories of simulations were ordered, we performed a chi-squared linear test of trend to examine increasing or decreasing trend in the associations³². Analysis was done using STATA 16.1. Patients or public were not involved in the design, conduct, or reporting or dissemination plans of the research.

Postpartum hemorrhage

Relative to the bottom-third facilities that did fewest (~9) maternal complication simulations, facilities in the middle one-third group (~19) diagnosed 26% (IRR = 1.26, 95% CI: 1.00, 1.59) more cases of PPH in real deliveries, while the top one-third (~31 sims) did not show increased diagnosis (Figure 1). This association between maternal complication simulation and PPH diagnosis increased to 1.33 (95% CI: 1.04, 1.70) after co-adjustment for T&C activities, demonstration sessions, bedside mentoring and didactic sessions. Analysis using the ratio metric of the number of maternal complication simulations to the number of T&C activities shows facilities in tertile 2 diagnosed 24% (IRR = 1.24, 95% CI: 1.00, 1.54) more PPH cases, compared to the facilities in tertile 1 (those that performed fewest maternal complication simulations relative to T&C activities) (Figure 1). Facilities in tertile 2, spent about twice as much time performing simulations relative to T&C activities, then the facilities in tertile 1. In contrast, performing more simulations relative to bedside mentoring, decreased diagnosis of PPH because facilities in tertile 3 diagnosed 25% (IRR = 0.75, 95% CI: 0.56, 1.00) fewer PPH cases, compared to facilities in tertile 1 (Figure 1). Facilities in tertile 3 did about 5 maternal complication simulations for every 10 live deliveries, whereas facilities in tertile 1 did about 2 simulations for every 10 live deliveries. The elevated but non-significant point estimates generally suggest that performing maternal complication simulations relative to didactic lessons may increase PPH diagnosis.

Figure 1. Adjusted1 incidence rate ratios for the diagnosis of postpartum hemorrhage with simulation counts or ratios of simulations performed relative to other activities in the AMANAT nurse-mentoring program in Bihar, India (2015 – 2017).

¹ Adjusted for weeks of nurse-mentoring, days per week of nurse-mentoring, total number of births per week, phase of program, availability of physicians, proportion of mentee-sessions attended, facility level practice. ^{2 3 5} These associations represent the added benefit of doing a simulation relative to doing a teamwork activity. Likewise, for a demonstration activity or a didactic session. ⁴ The decreased association for the ratio, simulation: bedside mentoring, shows that a proper mix of the two is more beneficial than just performing simulation. Note: The solid dots represent the associations, and the vertical bars represent the upper and the lower confidence intervals.

Intrapartum asphyxia

For intrapartum asphyxia, facilities in the middle one-third group (did about 9 neonatal complication simulations), diagnosed 25% (IRR = 1.25, 95% CI: 1.04, 1.50) more cases relative to the bottom-third facilities (~4) that conducted fewest neonatal simulations (Figure 2), which changed to 1.22 (95% CI: 1.01, 1.47) after co-adjustment with counts of T&C activities, demonstration sessions, bedside mentoring and didactic sessions. Facilities that did the highest (~ 16) number of neonatal simulations showed no improvement in diagnosis, with no linear trend across groups (Figure 2). Results also suggest that performing neonatal complication simulations relative to T&C activities increase diagnosis of intrapartum asphyxia (Figure 2). Facilities in tertile 2, with a moderate number of neonatal complication simulations performed relative to T&C activities, diagnosed 28% (IRR = 1.28, 95% CI: 1.09, 1.50) more cases of intrapartum asphyxia, compared to the facilities in tertile 1 (Figure 2). Facilities in tertile 2, spent more than twice as much time performing simulations than T&C activities, then the facilities in tertile 1. The elevated but non-significant point estimates suggest performing neonatal complication simulations relative to demonstrations activities may also increase diagnosis of intrapartum asphyxia. In contrast, performing more neonatal complication simulations relative to bedside mentoring, decreased diagnosis of intrapartum asphyxia. For example, facilities in tertile 3 diagnosed 28% (IRR = 0.72, 95% CI: 0.59, 0.89) less intrapartum asphyxia cases, compared to the facilities in tertile 1, with a significant linear decreasing trend (Figure 2). Facilities in tertile 3 did about 1 neonatal complication simulation for every 5 live deliveries, whereas facilities in tertile 1 did about 1 simulation for every 50 live deliveries.

Figure 2. Adjusted1 incidence rate ratios for the diagnosis of intrapartum asphyxia with simulation counts or ratios of simulations performed relative to other activities in the AMANAT nurse-mentoring program in Bihar, India (2015 – 2017).

¹ Adjusted for weeks of nurse-mentoring, days per week of nurse-mentoring, total number of births per week, phase of program, availability of physicians, proportion of mentee-sessions attended, facility level practice. ^{2 3 5} These associations represent the added benefit of doing a simulation relative to doing a teamwork activity. Likewise, for a demonstration activity or a didactic session. ⁴ The decreased association for the ratio, simulation: bedside mentoring, shows that a proper mix of the two is more beneficial than just performing simulation. Note: The solid dots represent the associations, and the vertical bars represent the upper and the lower confidence intervals.

There was no linear relationship between any of the metrices and both outcomes. The full models are presented in supplementary Tables 1 and 2 and the numerical estimates for Figure 1 and Figure 2 are presented in supplementary Tables 3 and 4 (see Extended data^27,28).

Underlying data

Dryad: Data for ‘Simulation and nurse-mentoring in a statewide nurse mentoring program in Bihar, India: diagnosis of postpartum hemorrhage and intrapartum asphyxia’. https://doi.org/10.7272/Q6VQ30X9²⁷.

This project contains the following underlying data:

Extended data

Zenodo: Simulation and nurse-mentoring in a statewide nurse mentoring program in Bihar, India: diagnosis of postpartum hemorrhage and intrapartum asphyxia. https://doi.org/10.5281/zenodo.6584800²⁸.

This project contains the following extended data:

Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).