Accessible metrics of access: Novel tools to measure immunization coverage in rural sub-Saharan Africa

Study overview

This is a three-phase study designed to compare different metrics of healthcare access derived from routine, facility-based data as surrogate measures of immunization coverage. Specifically, we aim to:

In Phase 1, we will determine the accuracy of self-reported household locations. Adult patients or the guardians of pediatric patients presenting to level II and level III health facilities in Bugoye sub-county will estimate the location of their household on open-access satellite imagery (Google Earth, Google, Mountain View, CA) uploaded onto a tablet device, from which study staff will record the latitude and longitude. After the visit is complete, study staff will accompany patients to their home and record the household location using a handheld GPS device. Coordinates derived from the satellite imagery will be compared to the GPS coordinates to estimate the standard error. Phase 1 is expected to take place over a period of one month beginning in September 2019.

In Phase 2, we will ask the guardians of children (age <1 year) presenting to level II and level III health facilities in Bugoye sub-county for the purpose of routine vaccination to complete a brief basic questionnaire and estimate the location of their household on satellite imagery as described above. One day per week we will collect similar information from the guardians of children (age <5 years) presenting for febrile illness and women presenting for antenatal care in order to triangulate our estimates of access across care-seeking indications. The resulting data will be used to estimate the three measures of access, including (i) the CCR, (ii) spatial clusters, and (iii) Bayesian probability. Phase 2 is expected to take place over a period of six months beginning in November 2019.

In Phase 3, anticipated to begin in May 2020, we will test the accuracy of these metrics as surrogate measures of immunization uptake. At the conclusion of the observation period, we will conduct a census of all children aged 12–23 months within the sub-county. At each household with an eligible child, we will ask to review the vaccination card to assess for receipt of all recommended vaccinations. Because vaccination cards are often lost or damaged, we will also perform a rapid diagnostic test for the presence of antibodies to tetanus and collect dried blood spots (DBS) for future testing.

The results of the census will then be compared to the estimates derived from the three metrics to assess both accuracy and feasibility of use. We hypothesize that areas predicted as having poor access will have significantly lower rates of childhood immunization compared to areas having full access.

Study population

Participants will be recruited from the Bugoye Sub-County in the Kasese District of western Uganda (Figure 1). The sub-county is comprised of five parishes and thirty-six villages, with an estimated population of 50,249, approximately one-quarter of whom are children under five years of age⁸. The sub-county spans a rural, highland area of approximately 55 square kilometers along the border with the Democratic Republic of Congo. The geography is characterized by deep river valleys and steep hillsides with elevations up to 2,000 meters near the base of the Rwenzori Mountains. Road access in many villages is extremely limited, especially during the rainy seasons. The climate in Bugoye permits year-round malaria transmission marked by semi-annual transmission peaks, typically following the end of the rainy seasons in May and December⁹.

Figure 1. Map of Bugoye sub-county showing the location of the five parishes and level II and level III health centers.

The sub-county’s primary public health facility is the Bugoye Level III Health Center (BHC). BHC is comprised of: a 25-bed inpatient ward (IPD), where patients can receive intravenous medications; a busy outpatient clinic (OPD) that evaluates 60–80 patients per day; a maternity ward; and a small laboratory, capable of performing point-of-care tests for diseases such as malaria and HIV. Clinical officers, nurses, midwives, and laboratory technicians employed by the Ugandan Ministry of Health staff the health center. There are also level II health centers in each of the five parishes that are staffed by a nurse or midwife and offer basic outpatient services including routine vaccination, and one private-not-for-profit (PFNP) level III health center operated by the Rwenzori Mountaineering Services (RMS).

Patients requiring a higher level of care are usually referred to health facilities in Kasese, which is located approximately 25 kilometers from BHC. Transport typically takes 45 minutes at costs that can represent the weekly income for a local subsistence farmer. For emergency services, such as trauma or obstructed labor, patients are often able to pool resources from friends and family to arrange transport. Even then, our previous work has shown that more than half of pediatric deaths still occur in the home, largely because patients are not able to reach a clinic or hospital¹⁰.

Eligibility criteria

Phase 1: Self-identified vs. GPS-defined household locations

Phase 2: Metrics of Access

Phase 3: Household Census

Exclusion criteria

Sample size & accrual

The accuracy of self-identified household locations has not been previously assessed in rural western Uganda. Therefore, the objective of Phase 1 is to define the mean difference and standard error between self-identified and GPS-defined household locations. To achieve this, we will conduct measurements over a period of two weeks at each of the eight health centers in the sub-county. We anticipate that we will be able to enroll two patients per day at each level II and four patients per day at each level III facility, resulting in a total of 200 participants during the ten clinic days.

In Phase 2, we plan to continually recruit participants over a period of six months from the eight health facilities in the sub-county. This period was chosen in order to assess potential differences in access and/or vaccination uptake between traditional wet and dry seasons. Over this period, we anticipate that we will be able to enroll up to 7,000 participants, including approximately 4,000 presenting for vaccination, 1,500 presenting for febrile illness, and 1,500 presenting for antenatal care.

Lastly, in Phase 3, we will attempt to visit all 6,872 households in the sub-county, as estimated by the most recent sub-county census. We estimate that 15–25% of households will have an eligible child (i.e. age 12–23 months), suggesting that we will enroll upwards of 1,800 children.

Pre-implementation & staff training

A community sensitization event including representatives from the sub-county leadership, village chairmen/chairwomen, and health center in-charges will take place at BHC. The purpose of this meeting is to inform key leaders as to the rationale for and methods of the study. There will be an opportunity for representatives to ask questions and provide feedback.

Study staff will then geocode the location of all village-based community health workers, also known as village health teams (VHTs), using handheld GPS devices. In each village, there are up to five VHTs, each of whom is responsible for providing health prevention and promotion messages to 20 to 30 households within his/her respective village. The location of each VHT will be marked on the satellite imagery for purposes of orienting participants to the area.

Prior to beginning enrollment, all field staff serving as data collectors will undergo a two-day, practical training with the tablet devices to learn how to navigate the satellite imagery on Google Earth and enter participant data into the electronic database. Each staff member must complete at least three directly-observed, simulated encounters with no errors.

Enrollment

In Phase 1, study staff at each facility will approach potentially eligible patients waiting in line to be seen by a clinician. Staff will provide information about the objectives, eligibility criteria, methods, and risks/benefits of the study. There will be an opportunity to ask questions. Individuals agreeing to participate in the study will be asked to provide written consent.

In Phase 2, enrollment will be staggered across sites to collect data from participants with different reasons for care seeking. Three days per week (Mon/Weds/Fri), study staff at each facility will approach potentially eligible patients waiting in line for vaccination of a child. Again, staff will provide information about the objectives, eligibility criteria, methods, and risks/benefits of the study. There will be an opportunity to ask questions. Individuals agreeing to participate in the study will be asked to provide written consent. Similar procedures will be carried out for febrile illness care-seeking (Tues) and routine antenatal care (Thurs) one day per week.

For the household census, survey teams, which will include a member of the study staff and the local VHT, will visit all households in the sub-county. The team will identify an adult caregiver (i.e. mother or father age ≥18 years). If there is no adult present at the time of the visit, the survey team will move to the next household. One attempt will be made to revisit the household. If, in speaking with the adult caregiver, the survey team determines that there is no eligible child (i.e. age 12–23 months) living in the household, the team will move to the next household. Upon reaching an eligible household, staff will provide information about the objectives, eligibility criteria, methods, and risks/benefits of the study. There will be an opportunity to ask questions. Individuals agreeing to participate in the study will be asked to provide written consent.

Data collection

Once consent is provided, study staff will show participants in Phase 1 a satellite image on the tablet device. The staff member will pinpoint the current location and ask the participant to identify the location of his/her household on the satellite image. The participant will touch the screen, which will generate a ‘pin’ with an associated latitude and longitude. The latitude and longitude will be recorded in decimal degree format. The staff member will then re-orient the satellite image so that it is centered on the location of the participants VHT. The participants will again be asked to point at his/her household location and the coordinates will be recorded. At the conclusion of clinical care, study staff will accompany participating individuals home from the health center. Upon reaching the participants household, study staff will record the latitude and longitude using a handheld GPS device. At this time, participation in the study is complete.

In Phase 2, study staff will administer a brief demographic and health questionnaire (available as Extended data¹¹) to participants. Once complete, study staff will show consenting individuals a Google Earth image on a tablet device and repeat the above procedures for identifying the location of the participant’s household. The latitude and longitude in decimal degree format, will be recorded. At this time, participation in the study is complete.

In Phase 3, the survey team will administer a brief demographic and health questionnaire (available as Extended data¹²) and ask to review the eligible child’s vaccination card. Study staff will record the vaccines received. Study staff will perform a finger-prick blood draw, perform a tetanus rapid diagnostic test (SD Bioline Tetanus, Abbott Laboratories, Lake Bluff, IL), and make a DBS (four spots of 50 μl each). At this time, participation in the study is complete and compensation, consisting of a bar of soap and/or a kilogram of sugar, will be provided.

Data security

To minimize the risk of loss of information, each patient will be moved to a private area prior to the informed consent discussion. They will remain in this area while the questionnaire is completed. To minimize the risk of loss of confidentiality, each patient will be assigned a study identification number (SIN) on enrolment. The names of participants will not be recorded in any database. Only study staff will have access to the records. All information will be recorded on an electronic tablet and uploaded to a secure database maintained behind the university firewall (UNC REDCap). Household locations may be visually represented in publication, but these will not be linked to individual data and the scale will not be sufficient to locate individual households. Any paper documents, including consent forms, will be scanned and destroyed upon completion of the study.

Data analysis

In Phase 1, we will assess the accuracy and determine the variance and standard error of self-identified household locations compared to those obtained using a handheld GPS device. We will also compare the accuracy of self-identified locations when both the health center and VHT is used as the reference point on the satellite imagery. The more accurate method will be used in Phase 2.

The data collected in Phase 2 will be used to calculate three surrogate measures of access. First, we will estimate the cumulative case ratio (CCR), defined as the ratio of the observed to the expected number of vaccination-related visits, for each village within the sub-county⁷. This metric assumes that if there were no barriers to care, a similar proportion of children from each village should present to a facility to be vaccinated. If the actual number of children presenting from a village is less than the expected number, then we assume that access to the facility may be limited by geographic and/or socioeconomic factors. The primary advantage of this approach is the ease of data collection analysis, which can be performed on a template spreadsheet at each health facility. We need only to know (1) each child’s village of residence, (2) the total number of children from each village who are vaccinated, and (3) the population of each village, derived from the most recent community health worker census. The disadvantage of this approach is that the spatial resolution is limited, which may mask within-village heterogeneity.

Next, we will define geographic areas, also known as spatial clusters, of low immunization uptake. The approximate geographic coordinates for all children receiving vaccination will be entered into SaTScan, a free, open-access software program. The SaTScan algorithm detects spatial clusters using a moving window approach, comparing the number of observed and expected observations inside the window at each location and identifying areas with significantly high or low rates of vaccination-related visits across a continuous geographic surface (i.e. without prior knowledge of household locations). The primary advantage of this approach compared to CCR is the higher degree of spatial resolution, enabling the user to visualize areas of low uptake that may not be constrained within village boundaries. The disadvantage is the higher burden of data entry, which is likely more appropriate for the district level management team.

Lastly, we will utilize a Bayesian statistical framework to generate a model of the probability of immunization continuously across space. This model will employ the same data as the SaTScan model, but will also allow us to integrate access-related covariates, such as elevation or distance from the nearest road, which may influence immunization uptake. The Bayesian model also can provide ‘smooth’ surface measures of immunization uptake with uncertainty estimates that can be used to visually map access and uptake. This analysis will be conducted using R, an open-access statistical program, which if successful, we plan to operationalize into a downloadable package. While somewhat more complex, this approach has many benefits and may be particularly well suited to national-level program managers.

In Phase 3, we will use the information on immunization rates among children 12-23 months of age collected during the household census to compare the accuracy of the three estimated measures of access. We will calculate error for each measure by subtracting the estimated immunization percent from the observed value. We will evaluate each measure’s overall accuracy (for the entire study region), as well as geographic accuracy using the root mean squared error (RMSE) at the village level.

Ethical considerations

This protocol and the informed consent documents (available as Extended data¹³) have been reviewed and approved by the Mbarara University of Science and Technology Research Ethics Committee (11/11-18), the Uganda National Council of Science and Technology (HS 2617), and the UNC Institutional Review Board (18-2885). Written informed consent will be obtained from all adult study participants and the caregivers of children.

Dissemination

Study findings will be presented to local stakeholders including the Bugoye sub-county leadership, clinical staff from participating health center, and representatives from the Kasese District Health Office, at a formal dissemination event. If our hypothesis is correct and any of the proposed tools demonstrate efficacy for identifying areas of low vaccine uptake, we will coordinate a dissemination event with representatives from the Ministry of Health in Kampala. Results may be presented at professional meetings, and national, regional or international scientific meetings. To accelerate cross-country and peer-to-peer learning, results from the study will be prepared for publication in peer-reviewed journals.

Study status

At the time of initial submission (September 2019), the project was preparing to enroll the first participants in Phase 1.

Underlying data

All data underlying the results are available as part of the article and no additional source data are required.

Extended data

Figshare: Consent Forms. https://doi.org/10.6084/m9.figshare.9793091.v1¹³.

Figshare: Questionnaires – Phase 2. https://doi.org/10.6084/m9.figshare.9793100.v1¹¹.

Figshare: Questionnaire - Phase 3. https://doi.org/10.6084/m9.figshare.9793109.v1¹².

Extended data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).