Gates Open Res Gates Open Research 2572-4754 F1000 Research Limited London, UK 10.12688/gatesopenres.13066.1 Study Protocol Articles Accessible metrics of access: Novel tools to measure immunization coverage in rural sub-Saharan Africa

[version 1; peer review: 1 approved, 1 approved with reservations]

 Boyce Ross M. Conceptualization Funding Acquisition Methodology Project Administration Supervision Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0002-9489-6324 a 1 2 Delamater Paul Conceptualization Formal Analysis Funding Acquisition Methodology Resources Software Supervision Writing – Original Draft Preparation Writing – Review & Editing 3 Muhindo Rabbison Data Curation Investigation Project Administration Supervision Writing – Review & Editing 2 Matte Michael Data Curation Investigation Methodology Supervision Writing – Review & Editing 2 Ntaro Moses Conceptualization Methodology Project Administration Writing – Review & Editing 2 Verity Robert Conceptualization Formal Analysis Funding Acquisition Methodology Software Writing – Review & Editing 4 Mulogo Edgar Conceptualization Project Administration Supervision Writing – Review & Editing 2 Division of Infectious Diseases, University of North Carolina at Chapel Hill, Chapel Hill, NC, 27599, USA Department of Community Health, Mbarara University of Science and Technology, Mbarara, Uganda Department of Geography, University of North Carolina at Chapel Hill, Chapel HIll, NC, 27599, USA Faculty of Medicine, Imperial College London, London, W2 1NY, UK roboyce@med.unc.edu

No competing interests were disclosed.

 17 9 2019 2019 3 1540 13 9 2019 Copyright: © 2019 Boyce RM et al. 2019 This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Immunization rates in most sub-Saharan African countries fall far below stated targets. Measuring access in resource-limited settings, however, is challenging, especially with the data available at the district level, which is the primary administrative division for most immunization programs. Despite calls to improve routine data collection and use, there remains a lack of structured methods and practical tools to target underserved populations. Herein, we describe a prospective study that aims to develop, pilot, and validate a set of user-friendly tools to identify geographic areas with limited access to immunization services and by extension, low immunization coverage. The approach will leverage routinely-collected data from public health facilities combined with novel methods of household mapping to perform spatial analyses using open-access platforms. In addition, we will triangulate the analyses across datasets representing common reasons for care seeking – namely, visits for vaccination, antenatal care, and malaria – to improve the accuracy of our estimates. The ultimate goal of this project is to equip front-line providers and district level program managers with novel tools that facilitate timely and accurate analysis of routinely-collected data to guide immunization efforts.

 Vaccination Immunization Routine data Geographic information systems Sub-Saharan Africa Uganda Gates Foundation OPP1199232 The work was supported by the Gates Foundation [OPP1199232]. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Introduction

Immunization rates in most sub-Saharan African countries fall far below stated targets. In Uganda, for example, national surveys demonstrated that only 55% of children age 12–23 months had received all basic vaccinations, and an even smaller proportion had received vaccinations according to schedule 1 . We believe that the primary determinant of vaccine uptake in rural sub-Saharan Africa is access, a concept that encompasses multiple structural and population-level dimensions beyond simple measures of geographic distance from a health facility 2 . Engaging and targeting populations with limited access is critically important, not only for concepts of equity and social justice, but also because these individuals frequently suffer a disproportionate burden of vaccine preventable disease 3 .

Measuring access in resource-limited settings, however, is challenging, especially with the data available at the district level, which is the primary administrative division for most immunization programs. In the current surveillance structure, peripheral health facilities compile and forward monthly reports summarizing the total number of vaccinations administered to the district health management team. Facility-based data may be useful to track and forecast supply, but yields little useful information on immunization coverage between facility catchment areas, let alone differences within a catchment area. As a result, outreach events are conducted arbitrarily, which represents an inefficient and inequitable use of resources compared to targeted approaches 4 .

In response, numerous reports and consensus guidelines have highlighted the urgent need to improve data collection and use 5 . Despite these calls, there remains a lack of structured methods and practical tools to target underserved populations 6 . Therefore, we propose to develop, pilot, and validate a set of user-friendly tools to identify geographic areas with limited access to immunization services and, by extension, low immunization coverage. The goal is to equip front-line providers and district level program managers with novel tools that facilitate timely and accurate analysis of routinely collected data to guide immunization efforts.

 Protocol Study overview

This is a three-phase study designed to compare different metrics of healthcare access derived from routine, facility-based data as surrogate measures of immunization coverage. Specifically, we aim to:

Aim 1: Estimate the cumulative case ratio (CCR), defined as the ratio of the observed to the expected number of vaccination-related visits, for each village within the sub-county 7 .

Aim 2: Define geographic areas, also known as spatial clusters, of low immunization uptake.

Aim 3: Utilize a Bayesian statistical framework to generate a model of the probability of immunization continuously across space.

In Phase 1, we will determine the accuracy of self-reported household locations. Adult patients or the guardians of pediatric patients presenting to level II and level III health facilities in Bugoye sub-county will estimate the location of their household on open-access satellite imagery (Google Earth, Google, Mountain View, CA) uploaded onto a tablet device, from which study staff will record the latitude and longitude. After the visit is complete, study staff will accompany patients to their home and record the household location using a handheld GPS device. Coordinates derived from the satellite imagery will be compared to the GPS coordinates to estimate the standard error. Phase 1 is expected to take place over a period of one month beginning in September 2019.

In Phase 2, we will ask the guardians of children (age <1 year) presenting to level II and level III health facilities in Bugoye sub-county for the purpose of routine vaccination to complete a brief basic questionnaire and estimate the location of their household on satellite imagery as described above. One day per week we will collect similar information from the guardians of children (age <5 years) presenting for febrile illness and women presenting for antenatal care in order to triangulate our estimates of access across care-seeking indications. The resulting data will be used to estimate the three measures of access, including (i) the CCR, (ii) spatial clusters, and (iii) Bayesian probability. Phase 2 is expected to take place over a period of six months beginning in November 2019.

In Phase 3, anticipated to begin in May 2020, we will test the accuracy of these metrics as surrogate measures of immunization uptake. At the conclusion of the observation period, we will conduct a census of all children aged 12–23 months within the sub-county. At each household with an eligible child, we will ask to review the vaccination card to assess for receipt of all recommended vaccinations. Because vaccination cards are often lost or damaged, we will also perform a rapid diagnostic test for the presence of antibodies to tetanus and collect dried blood spots (DBS) for future testing.

The results of the census will then be compared to the estimates derived from the three metrics to assess both accuracy and feasibility of use. We hypothesize that areas predicted as having poor access will have significantly lower rates of childhood immunization compared to areas having full access.

 Study population

Participants will be recruited from the Bugoye Sub-County in the Kasese District of western Uganda ( Figure 1). The sub-county is comprised of five parishes and thirty-six villages, with an estimated population of 50,249, approximately one-quarter of whom are children under five years of age 8 . The sub-county spans a rural, highland area of approximately 55 square kilometers along the border with the Democratic Republic of Congo. The geography is characterized by deep river valleys and steep hillsides with elevations up to 2,000 meters near the base of the Rwenzori Mountains. Road access in many villages is extremely limited, especially during the rainy seasons. The climate in Bugoye permits year-round malaria transmission marked by semi-annual transmission peaks, typically following the end of the rainy seasons in May and December 9 .

Map of Bugoye sub-county showing the location of the five parishes and level II and level III health centers.

The sub-county’s primary public health facility is the Bugoye Level III Health Center (BHC). BHC is comprised of: a 25-bed inpatient ward (IPD), where patients can receive intravenous medications; a busy outpatient clinic (OPD) that evaluates 60–80 patients per day; a maternity ward; and a small laboratory, capable of performing point-of-care tests for diseases such as malaria and HIV. Clinical officers, nurses, midwives, and laboratory technicians employed by the Ugandan Ministry of Health staff the health center. There are also level II health centers in each of the five parishes that are staffed by a nurse or midwife and offer basic outpatient services including routine vaccination, and one private-not-for-profit (PFNP) level III health center operated by the Rwenzori Mountaineering Services (RMS).

Patients requiring a higher level of care are usually referred to health facilities in Kasese, which is located approximately 25 kilometers from BHC. Transport typically takes 45 minutes at costs that can represent the weekly income for a local subsistence farmer. For emergency services, such as trauma or obstructed labor, patients are often able to pool resources from friends and family to arrange transport. Even then, our previous work has shown that more than half of pediatric deaths still occur in the home, largely because patients are not able to reach a clinic or hospital 10 .

 Eligibility criteria

Phase 1: Self-identified vs. GPS-defined household locations

Adult patients (age ≥18 years) OR adult caregivers of children presenting to level II or level III health center in Bugoye sub-county for outpatient care

Resident of Bugoye sub-county

Phase 2: Metrics of Access

Adult patients (age ≥18 years) OR adult caregivers of children presenting to level II or level III health center in Bugoye sub-county for one of the following reasons:

Vaccination of child <18 months of age

Febrile illness in child <5 years of age

Antenatal care

Resident of Bugoye sub-county

Phase 3: Household Census

Household in Bugoye sub-county with child 12–23 months of age

 Exclusion criteria

Children presenting to health center without caregiver (Phases 1 & 2) or caregiver not present at home at time of visit (Phase 3)

Patients being admitted to inpatient ward (Phases 1 & 2)

Three unsuccessful attempts to obtain finger-prick blood (Phase 3)

 Sample size & accrual

The accuracy of self-identified household locations has not been previously assessed in rural western Uganda. Therefore, the objective of Phase 1 is to define the mean difference and standard error between self-identified and GPS-defined household locations. To achieve this, we will conduct measurements over a period of two weeks at each of the eight health centers in the sub-county. We anticipate that we will be able to enroll two patients per day at each level II and four patients per day at each level III facility, resulting in a total of 200 participants during the ten clinic days.

In Phase 2, we plan to continually recruit participants over a period of six months from the eight health facilities in the sub-county. This period was chosen in order to assess potential differences in access and/or vaccination uptake between traditional wet and dry seasons. Over this period, we anticipate that we will be able to enroll up to 7,000 participants, including approximately 4,000 presenting for vaccination, 1,500 presenting for febrile illness, and 1,500 presenting for antenatal care.

Lastly, in Phase 3, we will attempt to visit all 6,872 households in the sub-county, as estimated by the most recent sub-county census. We estimate that 15–25% of households will have an eligible child (i.e. age 12–23 months), suggesting that we will enroll upwards of 1,800 children.

 Pre-implementation & staff training

A community sensitization event including representatives from the sub-county leadership, village chairmen/chairwomen, and health center in-charges will take place at BHC. The purpose of this meeting is to inform key leaders as to the rationale for and methods of the study. There will be an opportunity for representatives to ask questions and provide feedback.

Study staff will then geocode the location of all village-based community health workers, also known as village health teams (VHTs), using handheld GPS devices. In each village, there are up to five VHTs, each of whom is responsible for providing health prevention and promotion messages to 20 to 30 households within his/her respective village. The location of each VHT will be marked on the satellite imagery for purposes of orienting participants to the area.

Prior to beginning enrollment, all field staff serving as data collectors will undergo a two-day, practical training with the tablet devices to learn how to navigate the satellite imagery on Google Earth and enter participant data into the electronic database. Each staff member must complete at least three directly-observed, simulated encounters with no errors.

 Enrollment

In Phase 1, study staff at each facility will approach potentially eligible patients waiting in line to be seen by a clinician. Staff will provide information about the objectives, eligibility criteria, methods, and risks/benefits of the study. There will be an opportunity to ask questions. Individuals agreeing to participate in the study will be asked to provide written consent.

In Phase 2, enrollment will be staggered across sites to collect data from participants with different reasons for care seeking. Three days per week (Mon/Weds/Fri), study staff at each facility will approach potentially eligible patients waiting in line for vaccination of a child. Again, staff will provide information about the objectives, eligibility criteria, methods, and risks/benefits of the study. There will be an opportunity to ask questions. Individuals agreeing to participate in the study will be asked to provide written consent. Similar procedures will be carried out for febrile illness care-seeking (Tues) and routine antenatal care (Thurs) one day per week.

For the household census, survey teams, which will include a member of the study staff and the local VHT, will visit all households in the sub-county. The team will identify an adult caregiver (i.e. mother or father age ≥18 years). If there is no adult present at the time of the visit, the survey team will move to the next household. One attempt will be made to revisit the household. If, in speaking with the adult caregiver, the survey team determines that there is no eligible child (i.e. age 12–23 months) living in the household, the team will move to the next household. Upon reaching an eligible household, staff will provide information about the objectives, eligibility criteria, methods, and risks/benefits of the study. There will be an opportunity to ask questions. Individuals agreeing to participate in the study will be asked to provide written consent.

 Data collection

Once consent is provided, study staff will show participants in Phase 1 a satellite image on the tablet device. The staff member will pinpoint the current location and ask the participant to identify the location of his/her household on the satellite image. The participant will touch the screen, which will generate a ‘pin’ with an associated latitude and longitude. The latitude and longitude will be recorded in decimal degree format. The staff member will then re-orient the satellite image so that it is centered on the location of the participants VHT. The participants will again be asked to point at his/her household location and the coordinates will be recorded. At the conclusion of clinical care, study staff will accompany participating individuals home from the health center. Upon reaching the participants household, study staff will record the latitude and longitude using a handheld GPS device. At this time, participation in the study is complete.

In Phase 2, study staff will administer a brief demographic and health questionnaire (available as Extended data 11 ) to participants. Once complete, study staff will show consenting individuals a Google Earth image on a tablet device and repeat the above procedures for identifying the location of the participant’s household. The latitude and longitude in decimal degree format, will be recorded. At this time, participation in the study is complete.

In Phase 3, the survey team will administer a brief demographic and health questionnaire (available as Extended data 12 ) and ask to review the eligible child’s vaccination card. Study staff will record the vaccines received. Study staff will perform a finger-prick blood draw, perform a tetanus rapid diagnostic test (SD Bioline Tetanus, Abbott Laboratories, Lake Bluff, IL), and make a DBS (four spots of 50 μl each). At this time, participation in the study is complete and compensation, consisting of a bar of soap and/or a kilogram of sugar, will be provided.

 Data security

To minimize the risk of loss of information, each patient will be moved to a private area prior to the informed consent discussion. They will remain in this area while the questionnaire is completed. To minimize the risk of loss of confidentiality, each patient will be assigned a study identification number (SIN) on enrolment. The names of participants will not be recorded in any database. Only study staff will have access to the records. All information will be recorded on an electronic tablet and uploaded to a secure database maintained behind the university firewall (UNC REDCap). Household locations may be visually represented in publication, but these will not be linked to individual data and the scale will not be sufficient to locate individual households. Any paper documents, including consent forms, will be scanned and destroyed upon completion of the study.

 Data analysis

In Phase 1, we will assess the accuracy and determine the variance and standard error of self-identified household locations compared to those obtained using a handheld GPS device. We will also compare the accuracy of self-identified locations when both the health center and VHT is used as the reference point on the satellite imagery. The more accurate method will be used in Phase 2.

The data collected in Phase 2 will be used to calculate three surrogate measures of access. First, we will estimate the cumulative case ratio (CCR), defined as the ratio of the observed to the expected number of vaccination-related visits, for each village within the sub-county 7 . This metric assumes that if there were no barriers to care, a similar proportion of children from each village should present to a facility to be vaccinated. If the actual number of children presenting from a village is less than the expected number, then we assume that access to the facility may be limited by geographic and/or socioeconomic factors. The primary advantage of this approach is the ease of data collection analysis, which can be performed on a template spreadsheet at each health facility. We need only to know (1) each child’s village of residence, (2) the total number of children from each village who are vaccinated, and (3) the population of each village, derived from the most recent community health worker census. The disadvantage of this approach is that the spatial resolution is limited, which may mask within-village heterogeneity.

Next, we will define geographic areas, also known as spatial clusters, of low immunization uptake. The approximate geographic coordinates for all children receiving vaccination will be entered into SaTScan, a free, open-access software program. The SaTScan algorithm detects spatial clusters using a moving window approach, comparing the number of observed and expected observations inside the window at each location and identifying areas with significantly high or low rates of vaccination-related visits across a continuous geographic surface (i.e. without prior knowledge of household locations). The primary advantage of this approach compared to CCR is the higher degree of spatial resolution, enabling the user to visualize areas of low uptake that may not be constrained within village boundaries. The disadvantage is the higher burden of data entry, which is likely more appropriate for the district level management team.

Lastly, we will utilize a Bayesian statistical framework to generate a model of the probability of immunization continuously across space. This model will employ the same data as the SaTScan model, but will also allow us to integrate access-related covariates, such as elevation or distance from the nearest road, which may influence immunization uptake. The Bayesian model also can provide ‘smooth’ surface measures of immunization uptake with uncertainty estimates that can be used to visually map access and uptake. This analysis will be conducted using R, an open-access statistical program, which if successful, we plan to operationalize into a downloadable package. While somewhat more complex, this approach has many benefits and may be particularly well suited to national-level program managers.

In Phase 3, we will use the information on immunization rates among children 12-23 months of age collected during the household census to compare the accuracy of the three estimated measures of access. We will calculate error for each measure by subtracting the estimated immunization percent from the observed value. We will evaluate each measure’s overall accuracy (for the entire study region), as well as geographic accuracy using the root mean squared error (RMSE) at the village level.

 Ethical considerations

This protocol and the informed consent documents (available as Extended data 13 ) have been reviewed and approved by the Mbarara University of Science and Technology Research Ethics Committee (11/11-18), the Uganda National Council of Science and Technology (HS 2617), and the UNC Institutional Review Board (18-2885). Written informed consent will be obtained from all adult study participants and the caregivers of children.

 Dissemination

Study findings will be presented to local stakeholders including the Bugoye sub-county leadership, clinical staff from participating health center, and representatives from the Kasese District Health Office, at a formal dissemination event. If our hypothesis is correct and any of the proposed tools demonstrate efficacy for identifying areas of low vaccine uptake, we will coordinate a dissemination event with representatives from the Ministry of Health in Kampala. Results may be presented at professional meetings, and national, regional or international scientific meetings. To accelerate cross-country and peer-to-peer learning, results from the study will be prepared for publication in peer-reviewed journals.

 Study status

At the time of initial submission (September 2019), the project was preparing to enroll the first participants in Phase 1.

 Discussion

The proposed study will generate valuable data regarding the association between access and immunization coverage, while at the same time assessing the predictive accuracy of three different models of access. Our approach is unique in the way that we will leverage routinely collected data from public health facilities with novel methods of household mapping to perform powerful spatial analyses using available platforms. In addition, we will triangulate the analyses across datasets representing common reasons for care seeking – namely, visits for vaccination, antenatal care, and malaria – to improve the accuracy of our estimates. If successful, these tools can be operationalized at facility and district levels and studied in a prospective, randomized control trial comparing access-targeted immunization outreach to routine program management to determine if improved data usage can increase coverage rates.

 Data availability Underlying data

All data underlying the results are available as part of the article and no additional source data are required.

 Extended data

Figshare: Consent Forms. https://doi.org/10.6084/m9.figshare.9793091.v1 13 .

Figshare: Questionnaires – Phase 2. https://doi.org/10.6084/m9.figshare.9793100.v1 11 .

Figshare: Questionnaire - Phase 3. https://doi.org/10.6084/m9.figshare.9793109.v1 12 .

Extended data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

 Uganda Bureau of Statistics (UBOS) and ICF: Uganda Demographic and Health Survey 2016. Kampala, Uganda and Rockville, Maryland: UBOS and ICF.2018. Reference Source Levesque JF Harris MF Russell G : Patient-centred access to health care: conceptualising access at the interface of health systems and populations. Int J Equity Health. 2013;12:18. 23496984 10.1186/1475-9276-12-18 3610159 Restrepo-Mendez MC Barros AJ Wong KL : Inequalities in full immunization coverage: trends in low- and middle-income countries. Bull World Health Organ. 2016;94(11):794–805B. 27821882 10.2471/BLT.15.162172 5096343 Uganda Ministry of Health: Uganda national expanded programme on immunization multi-year plan. 2012-2016. Kampala, Uganda: Uganda MoH.2012. Reference Source World Health Organization (WHO): Global vaccine action plan 2011-2020. Geneva: WHO.2013. Reference Source World Health Organization (WHO): Immunization and vaccine-preventable diseases surveillance data management in the WHO African Region: Status, challenges and way forward. Brazzaville, Congo: WHO Regional Office for Africa.2014. Zinszer K Charland K Kigozi R : Determining health-care facility catchment areas in Uganda using data on malaria-related visits. Bull World Health Organ. 2014;92(3):178–86. 24700977 10.2471/BLT.13.125260 3949593 Uganda Bureau of Statistics (UBOS): National Population and Housing Census 2014: Provisional Results. Revised Edition ed. Kampala, Uganda.2014. Reference Source Yeka A Gasasira A Mpimbaza A : Malaria in Uganda: challenges to control on the long road to elimination: I. Epidemiology and current control efforts. Acta Trop. 2012;121(3):184–95. 21420377 10.1016/j.actatropica.2011.03.004 3156969 Nabukalu D Ntaro M Seviiri M : Community health workers trained to conduct verbal autopsies provide better mortality measures than existing surveillance: Results from a cross-sectional study in rural western Uganda. PLoS One. 2019;14(2):e0211482. 30759139 10.1371/journal.pone.0211482 6373919 Boyce R : Questionnaires - Phase 2. figshare. Online resource.2019. http://www.doi.org/10.6084/m9.figshare.9793100.v1 Boyce R : Questionnaire - Phase 3. figshare. Online resource.2019. http://www.doi.org/10.6084/m9.figshare.9793109.v1 Boyce R : Consent Forms. figshare. Online resource.2019. http://www.doi.org/10.6084/m9.figshare.9793091.v1 10.21956/gatesopenres.14197.r28474 Reviewer response for version 1 Travassos Mark A. 1 Referee https://orcid.org/0000-0002-6045-3322 Center for Vaccine Development, University of Maryland School of Medicine, Baltimore, MA, USA

Competing interests: No competing interests were disclosed.

 18 2 2020 Copyright: © 2020 Travassos MA 2020 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve-with-reservations

This study protocol addresses a critical need for immunization programs in resource-limited settings: the development of tools to identify under-immunized populations. To accomplish this, the study will measure the accuracy of self-reported household locations against actual locations in a resource-limited setting and compare immunization uptake rates measured by vaccination card against cumulative case ratio estimates from health facility visits; spatial cluster analyses; and Bayesian probability. The protocol is written clearly and describes how these different approaches will be compared. Additional details to avoid the pitfalls outlined below would be helpful.

Phase 2 and 3 take place in different time periods. While this may be necessary logistically, the lack of simultaneity hampers the transposability of the results for Phase 2 to Phase 3. Vaccination uptake may change in the region from November 2019 to May 2020 due to seasonality, political events, etc. As a result, while Phase 2 may allow for the development of methods for each immunization uptake measure, it would make sense to continue to accumulate this data while Phase 3 is underway so that estimates may be developed at that time as well for comparison.

It is unclear how antibodies to tetanus and dried blood spots will be factored into the study. Are they intended for future validation with methods that have not yet been delineated? It is unclear if a tetanus rapid diagnostic test approximates successful immunization levels with pentavalent vaccine. Is there literature to support this? We have previously used validated serological markers of immunization that may provide some guidance 1 .

Phase 3 appears to rely on vaccination cards as objective measures to determine the immunization status of children. We have previously found that vaccination cards do not accurately predict immunization status 1 , as they may neglect, for example, the results of immunization campaigns or may not be brought to each immunization visit. Immunization records at district health centers should be considered as another measure of immunization status for this study. These logs represent a valuable alternative means of determining immunization status.

Do immunization campaigns occur? If so, Phase 3 should take into account vaccines from campaigns and determine if they are represented on vaccination cards.

Is the study design appropriate for the research question?

Partly

Is the rationale for, and objectives of, the study clearly described?

Yes

Are sufficient details of the methods provided to allow replication by others?

Yes

Are the datasets clearly presented in a useable and accessible format?

Not applicable

Reviewer Expertise:

Assessing vaccine coverage, particularly using serosurveys

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

References : Immunization Coverage Surveys and Linked Biomarker Serosurveys in Three Regions in Ethiopia. PLoS One .2016;11(3) : 10.1371/journal.pone.0149970 e0149970 26934372 10.1371/journal.pone.0149970 10.21956/gatesopenres.14197.r28031 Reviewer response for version 1 Utazi Chigozie Edson 1 Referee WorldPop, School of Geography and Environmental Science, University of Southampton, Southampton, UK

Competing interests: No competing interests were disclosed.

 1 11 2019 Copyright: © 2019 Utazi CE 2019 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve

The paper proposes an interesting prospective study to investigate access to immunization services in a rural area in Uganda. This is a welcome development as most studies on access to immunization services and vaccine uptake are based on retrospective designs utilizing household survey data. The paper is well-written, but I have a few comments as provided below.

 

The authors need to clarify the focus on tetanus vaccine in the abstract and throughout the paper.

In the Introduction, it will be useful if the authors could highlight why facility-based data may not provide useful information on vaccination coverage. Are these due to inaccurate denominators, migration, or people seeking services outside their catchment areas? Suitable references will be needed too.

For study aim 1, the authors should state the source and reliability of the data to be used for calculating the expected number of vaccine-related visits. The accuracy of the CCR metric will depend on this to a large extent.

I am skeptical about the idea of asking the respondents to locate their households using open-access satellite imagery. I think that the success of this exercise will be mostly dependent on the level of map literacy among the respondents. It would be good to know if this has been ascertained before the design of this study. Also, the idea of following the respondents’ home after their visit to a health facility does not seem convincing. What if the patient or the guardian has other engagements after the visit?

The proposed Bayesian analysis to predict vaccination coverage continuously across space could be used to delineate low coverage areas, as has been done in many papers mapping vaccination coverage for individual and multiple countries. However, the authors propose to identify clusters of low coverage using SaTScan. It will be interesting to see whether the low coverage clusters identified align with low coverage areas in the smooth surface of immunization coverage.

It is not possible to produce a coverage map using the same data specified for the SaTScan analysis (i.e. data for all children receiving vaccination), as the authors have indicated. Data for all children, both those receiving vaccination and those not receiving vaccination, will be needed for the Bayesian analysis. This needs to be corrected.

Lastly, I wonder how generalizable the results of this study will be and whether this was taken into consideration in the choosing the study location. It seems to me that the findings might be subject to geographical, environmental and cultural bias. 

Is the study design appropriate for the research question?

Yes

Is the rationale for, and objectives of, the study clearly described?

Yes

Are sufficient details of the methods provided to allow replication by others?

Yes

Are the datasets clearly presented in a useable and accessible format?

Not applicable

Reviewer Expertise:

Spatial and spatiotemporal analysis with an interest in modelling and mapping vaccination coverage

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.