<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.2 20190208//EN" "http://jats.nlm.nih.gov/publishing/1.2/JATS-journalpublishing1.dtd"><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="other" dtd-version="1.2" xml:lang="en">
    <front>
        <journal-meta>
            <journal-id journal-id-type="pmc">Gates Open Res</journal-id>
            <journal-title-group>
                <journal-title>Gates Open Research</journal-title>
            </journal-title-group>
            <issn pub-type="epub">2572-4754</issn>
            <publisher>
                <publisher-name>F1000 Research Limited</publisher-name>
                <publisher-loc>London, UK</publisher-loc>
            </publisher>
        </journal-meta>
        <article-meta>
            <article-id pub-id-type="doi">10.12688/gatesopenres.13211.1</article-id>
            <article-categories>
                <subj-group subj-group-type="heading">
                    <subject>Software Tool Article</subject>
                </subj-group>
                <subj-group>
                    <subject>Articles</subject>
                </subj-group>
            </article-categories>
            <title-group>
                <article-title>
                    <italic>Fpemlocal</italic>: Estimating family planning indicators in R for a single population of interest</article-title>
                <fn-group content-type="pub-status">
                    <fn>
                        <p>[version 1; peer review: 2 approved]</p>
                    </fn>
                </fn-group>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Guranich</surname>
                        <given-names>Gregory</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Software</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Original Draft Preparation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <uri content-type="orcid">https://orcid.org/0000-0003-4611-5607</uri>
                    <xref ref-type="aff" rid="a1">1</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Cahill</surname>
                        <given-names>Niamh</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Methodology</role>
                    <role content-type="http://credit.niso.org/">Software</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <uri content-type="orcid">https://orcid.org/0000-0003-3086-550X</uri>
                    <xref ref-type="aff" rid="a2">2</xref>
                </contrib>
                <contrib contrib-type="author" corresp="yes">
                    <name>
                        <surname>Alkema</surname>
                        <given-names>Leontine</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Conceptualization</role>
                    <role content-type="http://credit.niso.org/">Funding Acquisition</role>
                    <role content-type="http://credit.niso.org/">Methodology</role>
                    <role content-type="http://credit.niso.org/">Project Administration</role>
                    <role content-type="http://credit.niso.org/">Software</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Original Draft Preparation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <uri content-type="orcid">https://orcid.org/0000-0001-8806-5957</uri>
                    <xref ref-type="corresp" rid="c1">a</xref>
                    <xref ref-type="aff" rid="a1">1</xref>
                </contrib>
                <aff id="a1">
                    <label>1</label>Department of Biostatistics and Epidemiology, University of Massachusetts Amherst, Amherst, USA</aff>
                <aff id="a2">
                    <label>2</label>Department of Mathematics and Statistics, Maynooth University, Kildare, Ireland</aff>
            </contrib-group>
            <author-notes>
                <corresp id="c1">
                    <label>a</label>
                    <email xlink:href="mailto:lalkema@umass.edu">lalkema@umass.edu</email>
                </corresp>
                <fn fn-type="conflict">
                    <p>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>24</day>
                <month>2</month>
                <year>2021</year>
            </pub-date>
            <pub-date pub-type="collection">
                <year>2021</year>
            </pub-date>
            <volume>5</volume>
            <elocation-id>24</elocation-id>
            <history>
                <date date-type="accepted">
                    <day>6</day>
                    <month>1</month>
                    <year>2021</year>
                </date>
            </history>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2021 Guranich G et al.</copyright-statement>
                <copyright-year>2021</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <self-uri content-type="pdf" xlink:href="https://gatesopenresearch.org/articles/5-24/pdf"/>
            <abstract>
                <p>The global Family Planning Estimation model (FPEM) combines a Bayesian hierarchical model with country-specific time trends to yield estimates of contraceptive prevalence and unmet need for family planning for countries worldwide. In this paper, we introduce the R package 
                    <italic toggle="yes">fpemlocal</italic> that carries out the estimation of family planning indicators for a single population, for example, for a single country or smaller area. In this implementation of FPEM, all non-population-specific parameters are fixed at outcomes obtained in a prior global FPEM run. The development of this model was motivated by the demand for computational efficiency, without loss of model accuracy, when estimates and projections from FPEM were needed only for a single country. We present use cases to produce estimates for a single population of women by union status or all women based on package-provided data bases and user-specified data. We also explain how to aggregate estimates across multiple populations. The R package forms the basis of the Track20 Family Planning Estimation Tool to monitor trends in family planning indicators for the FP2020 initiative. 
                    <italic toggle="yes">Fpemlocal</italic> is available from: 
                    <ext-link ext-link-type="uri" xlink:href="https://github.com/AlkemaLab/fpemlocal">https://github.com/AlkemaLab/fpemlocal</ext-link>
                </p>
            </abstract>
            <kwd-group kwd-group-type="author">
                <kwd>Family Planning estimation tool</kwd>
                <kwd>global versus local model fitting</kwd>
            </kwd-group>
            <funding-group>
                <award-group id="fund-1" xlink:href="http://dx.doi.org/10.13039/100000865">
                    <funding-source>Gates Foundation</funding-source>
                    <award-id>INV-008441</award-id>
                </award-group>
                <funding-statement>This work was supported by the Gates Foundation [INV-008441]</funding-statement>
                <funding-statement>
                    <italic>The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.</italic>
                </funding-statement>
            </funding-group>
        </article-meta>
    </front>
    <body>
        <sec sec-type="intro">
            <title>Introduction</title>
            <p>The global Family Planning Estimation model (FPEM) combines a Bayesian hierarchical model with country-specific time trends to yield estimates of contraceptive prevalence and unmet need for family planning. It was first designed to produce estimates for women aged 15&#x2013;49 who are married or in a union, referred to here as in-union women, for 195 countries worldwide (
                <xref ref-type="bibr" rid="ref-1">Alkema 
                    <italic toggle="yes">et al.</italic>, 2013</xref>; 
                <xref ref-type="bibr" rid="ref-2">Cahill 
                    <italic toggle="yes">et al.</italic>, 2018</xref>). Subsequently, it has been used for producing estimates for women who are not in a union, referred to here as not_in_union women (
                <xref ref-type="bibr" rid="ref-6">Kantorov&#x00e1; 
                    <italic toggle="yes">et al.</italic>, 2020</xref>). The model accounts for differences by data source, sample population, and contraceptive methods included in the contraceptive use measure. The Bayesian hierarchical structure in the model is used to exchange information across countries regarding uptake of contraceptive methods, the relative share of modern versus traditional methods, and unmet need.</p>
            <p>The local implementation of FPEM is a scaled-down version of the global FPEM, where the family planning (FP) model is fitted at a more local (or population-specific) level. Here we use the term local to refer to either in_union or not_in_union women in a single country or smaller area, referred to here as subdivisions. The distinction between the local FPEM and the global FPEM is that the local version can be run on data from a single population and in the model specification all non-population-specific parameters are fixed at outcomes obtained in the most up to date global FPEM run. The development of this model was originally motivated by the demand for computational efficiency, without loss of model accuracy, when estimates and projections from FPEM were needed only for a single country. Specifically, this local-run option was needed to facilitate the use of FPEM at country-support workshops run by the Track20 project (
                <xref ref-type="bibr" rid="ref-13">
                    <italic toggle="yes">Track20</italic>, n.d.</xref>). These workshops provided technical support to the pledging countries of the Family Planning 2020 initiative for monitoring progress in FP and the global FPEM was too computationally intensive to be useful. </p>
            <p>To illustrate how the local FPEM links to the global FPEM we have provided a flowchart in 
                <xref ref-type="fig" rid="f1">Figure 1</xref> depicting (1) the hierarchical structure in the global FPEM and (2) the dependencies of the country and subnational local FPEM model specifications on the global FPEM output. In summary, in the country-specific implementation of the local FPEM non-country-specific parameters, e.g. the subregional pace of the uptake of contraceptive methods, across-country variances and the error variances and covariances for different data source types, were not estimated but were fixed at the point estimates obtained from a recent global model run. Similarly, FPEM can be fit to data from subdivisions, i.e. subnational populations. In the subnational implementation of the local FPEM, each subnation is considered as a &#x201c;country&#x201d; within the &#x201c;subregion&#x201d; of its respective nation. For instance, in India, States/UTs were considered as &#x201c;countries&#x201d; within the &#x201c;subregion&#x201d; of India. By fixing the subregional parameters at parameter point estimates obtained from the global run, this implementation of the local FPEM was used to obtain estimates of FP indicators for Indian states/UTs (
                <xref ref-type="bibr" rid="ref-8">New 
                    <italic toggle="yes">et al.</italic>, 2017</xref>).</p>
            <fig fig-type="figure" id="f1" orientation="portrait" position="float">
                <label>Figure 1. </label>
                <caption>
                    <title>A flowchart to illustrate the relation between global and local FPEM.</title>
                    <p>Figure taken from 
                        <xref ref-type="bibr" rid="ref-8">New 
                            <italic toggle="yes">et al.</italic> (2017)</xref>, distributed with a CC BY license.</p>
                </caption>
                <graphic orientation="portrait" position="float" xlink:href="https://gatesopenresearch-files.f1000.com/manuscripts/14429/01c8746f-69bf-4176-9c88-6a83306b4d78_figure1.gif"/>
            </fig>
            <p>This paper introduces the R package 
                <italic toggle="yes">fpemlocal</italic>. 
                <italic toggle="yes">fpemlocal</italic> contains R functions and input data to do model fitting using the local implementation of FPEM. The package contains data from UNPD on contraceptive use and population numbers (
                <xref ref-type="bibr" rid="ref-14">United Nations, Department of Economic and Social Affairs, Population Division, 2020a</xref>; 
                <xref ref-type="bibr" rid="ref-15">United Nations, Department of Economic and Social Affairs, Population Division, 2020b</xref>). In addition, the package contains data from Track20 on contraceptive use. In this paper, we present different use cases for how to fit a model to data (UNPD or user-provided) to obtain estimates of contraceptive prevalence and users of contraceptive methods, among in_union or not_in_union women, for a country or subnational area. We also present how to obtain estimates that are aggregated across in_union and not_in_union women, and across geographical regions.  </p>
        </sec>
        <sec sec-type="methods">
            <title>Methods</title>
            <sec>
                <title>Implementation</title>
                <p>
                    <italic toggle="yes">fpemlocal</italic> is an R package that contains R functions to carry out the pre- and postprocessing, and fit FPEM to data for a population of choice. Model fitting here refers to using Markov Chain Monte Carlo (MCMC) sampling to obtain samples from the posterior distributions of all model parameters.</p>
            </sec>
            <sec>
                <title>Operation</title>
                <p>
                    <italic toggle="yes">fpemlocal</italic> is a publicly available R package stored on Github. For usage, an R (
                    <xref ref-type="bibr" rid="ref-11">R Core Team, 2020</xref>) installation (&#x2265;3.4.0) and a JAGS (
                    <xref ref-type="bibr" rid="ref-9">Plummer, 2017</xref>) installation (&#x2265;4.0.0) are required. R can be downloaded on CRAN. JAGS is a program for the analysis of Bayesian models using MCMC. JAGS is written in C++ and is portable to all major operating systems. JAGS is available for download at 
                    <ext-link ext-link-type="uri" xlink:href="https://sourceforge.net/projects/mcmc-jags/">https://sourceforge.net/projects/mcmc-jags/</ext-link>. Note that users will not interact with JAGS directly. Instead, 
                    <italic toggle="yes">fpemlocal</italic> will interface with JAGS through the dependency 
                    <italic toggle="yes">R2jags.</italic> (
                    <xref ref-type="bibr" rid="ref-9">Plummer, 2017</xref>; 
                    <xref ref-type="bibr" rid="ref-12">Su &amp; Yajima, 2020</xref>)</p>
                <p>
                    <italic toggle="yes">fpemlocal</italic> can be installed with one of two methods: (1) direct install from Github repository using the 
                    <italic toggle="yes">install_github()</italic> API from the devtools package (
                    <xref ref-type="bibr" rid="ref-17">Wickham 
                        <italic toggle="yes">et al.</italic>, 2020</xref>); (2) install from the package binary using the base R function 
                    <italic toggle="yes">install.package()</italic>. Package dependencies are listed in the package DESCRIPTION file and will be automatically installed upon installing the main package. There are no minimum RAM, CPU, or HARDDRIVE requirements apart from what is necessary to store model runs, which varies case-by-case.</p>
            </sec>
        </sec>
        <sec sec-type="cases">
            <title>Use cases</title>
            <sec>
                <title>Input data</title>
                <p>The main functionality of 
                    <italic toggle="yes">fpemlocal</italic> is the fitting of the Bayesian FP estimation model to data for a population of interest. We describe its usage in the use cases in this section. Central to all examples are inputs in the form of a contraceptive use survey dataset (referred to as survey data), and a population count dataset (referred to as population data), for the population of interest. 
                    <italic toggle="yes">fpemlocal</italic> contains survey data and population data provided by the UNPD (
                    <xref ref-type="bibr" rid="ref-14">United Nations, Department of Economic and Social Affairs, Population Division, 2020a</xref>, 
                    <xref ref-type="bibr" rid="ref-15">United Nations, Department of Economic and Social Affairs, Population Division, 2020b</xref>). Help files provide the metadata related to these data sets, e.g. 
                    <italic toggle="yes">?contraceptive_use</italic> will display the helpfile for the contraceptive use survey dataset metadata, see 
                    <ext-link ext-link-type="uri" xlink:href="https://github.com/AlkemaLab/FPEMlocal/blob/master/vignettes/package_data.md">&#x2018;package data&#x2019; vignette</ext-link> for details. In summary, the contraceptive use survey dataset includes family planning data by, location, age, and marital status. Data are in the form of aggregated survey responses, i.e., prevalence, their transformations, and sampling errors where available. The function 
                    <italic toggle="yes">impute_user_se</italic> is used for imputing missing sampling errors in custom survey data prior to model fitting, based on the approach outlined in 
                    <xref ref-type="bibr" rid="ref-2">Cahill 
                        <italic toggle="yes">et al.</italic> (2018)</xref> (appendix p. 16). The population count dataset includes population counts by year, location, age, and marital status. 
                    <italic toggle="yes">Fpemlocal</italic> supports the use of external datasets in place of the default package datasets as long as the format follows that of the default data sets (see use case 1.2). </p>
                <p>Additional data provided with the package is used for model fitting and consists of (1) parameter estimates to use in local FPEM, and (2) information on how countries are organized in hierarchical groupings. Parameter estimates are obtained from the most recent UNPD global FPEM runs, currently those of the 2020 revision (
                    <xref ref-type="bibr" rid="ref-15">United Nations, Department of Economic and Social Affairs, Population Division, 2020b</xref>). Information on hierarchical groupings (geographical or otherwise) is provided in the dataset 
                    <italic toggle="yes">divisions</italic>.</p>
            </sec>
            <sec>
                <title>Model fitting</title>
                <p>The primary function of 
                    <italic toggle="yes">fpemlocal</italic>, 
                    <italic toggle="yes">fit_fp_c</italic>, fits the family planning model for a geographical population of interest, for in_union women and/or not_in_union women. Its arguments are summarized in 
                    <xref ref-type="table" rid="T1">Table 1</xref>. The first and primary argument to 
                    <italic toggle="yes">fit_fp_c</italic>, is a contraceptive use survey dataset. If the user does not supply a survey dataset, the package survey dataset is used. Subsequent arguments 
                    <italic toggle="yes">division_numeric_code</italic> and 
                    <italic toggle="yes">is_in_union</italic> are used to filter input data to obtain the inputs and parameter values that are relevant to the population of interest. In particular, the division code and union status will determine the groupings applied when fitting the local FPEM, and the selected survey observations. Argument 
                    <italic toggle="yes">is_in_union</italic> determines if results are to be obtained for in_union, not_in_union or all women. If results are to be obtained for all women, model fits for in_union and not_in_union women are obtained.</p>
                <table-wrap id="T1" orientation="portrait" position="anchor">
                    <label>Table 1. </label>
                    <caption>
                        <title>Argument descriptions for function 
                            <italic toggle="yes">fit_fp_c.</italic>
                        </title>
                    </caption>
                    <table content-type="article-table" frame="hsides">
                        <thead>
                            <tr>
                                <th align="left" colspan="1" rowspan="1">Argument</th>
                                <th align="left" colspan="1" rowspan="1">Data type</th>
                                <th align="left" colspan="1" rowspan="1">Description</th>
                            </tr>
                        </thead>
                        <tbody>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="top">surveydata_filepath</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">Character</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">Path to survey data. Survey data should be a .csv. When left NULL, the function
                                    <break/>will default to the package dataset `contraceptive_use`</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="top">division_numeric_code</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">Numeric</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">A numeric code associated with the country. This code will determine the
                                    <break/>groupings applied when fitting FPEM. See the data from `divisions`</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="top">is_in_union</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">Character</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">Specify the union status of women. Options are in-union, not-in-union, and all
                                    <break/>women. &#x201c;Y&#x201d;, &#x201c;N&#x201d;, and &#x201c;ALL&#x201d; respectively. Default is &#x201c;Y&#x201d;.</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="top">first_year</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">Numeric</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">The first year of model estimates returned. The model will be fit to all data,
                                    <break/>including dates before this date if available.</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="top">last_year</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">Numeric</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">The last year of model estimates returned. The model will be fit to all data,
                                    <break/>including dates after this date if available.</td>
                            </tr>
                        </tbody>
                    </table>
                </table-wrap>
            </sec>
            <sec>
                <title>Model output</title>
                <p>After calling 
                    <italic toggle="yes">fit_fp_c</italic>, the resulting fit object contains posterior samples of all relevant model parameters and essential fit data called core data. Core data is a list containing processed survey data, bias-adjusted survey data, other 
                    <italic toggle="yes">fit_fp_c</italic> inputs, and information about the model fit.</p>
                <p>Model output is supplied to 
                    <italic toggle="yes">calc_fp_c</italic> where quantiles of posterior samples are calculated to produce credible intervals. The estimates from 
                    <italic toggle="yes">calc_fp_c</italic> in conjunction with the fit object returned from 
                    <italic toggle="yes">fit_fp_c</italic> can be supplied to 
                    <italic toggle="yes">plot_fp_c</italic> to generate plots of estimates over time overlapping input data.</p>
            </sec>
            <sec>
                <title>Case 1: Estimating FP indicators for in_union or not_in_union women</title>
                <p>
                    <italic toggle="yes">
                        <bold>Case 1.1: (default case) FP estimation with UNPD package datasets.</bold>
                    </italic> The use case of estimating FP indicators for a country of interest using default UNPD data is given in the 
                    <ext-link ext-link-type="uri" xlink:href="https://github.com/AlkemaLab/FPEMlocal/blob/master/vignettes/in_union_women.md">&#x2018;in union women&#x2019; vignette</ext-link>. This vignette takes less than one minute to run on a machine with an 8 core 3.60GHz CPU and 16GB of RAM. In this use case, the user starts by finding the country code (division numeric code) for the country of interest. Then, the user calls the function 
                    <italic toggle="yes">fit_fp_c</italic> to fit the local FPEM model. The user supplies the country code, the union status, and the years of estimates to be returned (see 
                    <xref ref-type="fig" rid="f2">Figure 2</xref>). The user does not supply survey data. By default, the function loads package survey data. The function 
                    <italic toggle="yes">fit_fp_c</italic> returns posterior samples and core data. After the model is fit, the user calls the function 
                    <italic toggle="yes">calc_fp_c</italic> to calculate family planning indicators. The calculation of some indicators requires population counts. By default, the function 
                    <italic toggle="yes">calc_fp_c</italic> loads package population count data if it is not supplied by the user. Lastly, the user supplies the fit object, the results object, and a vector of indicator names to the function 
                    <italic toggle="yes">plot_fp_c</italic> to plot indicator estimates and survey data (see 
                    <xref ref-type="fig" rid="f2">Figure 2</xref>).</p>
                <fig fig-type="figure" id="f2" orientation="portrait" position="float">
                    <label>Figure 2. </label>
                    <caption>
                        <title>Function calls for use case 1.1, estimating family planning indicators for in_union or not_in_union women for Afghanistan (country numeric code 4) with an illustration of a plot for modern contraceptive prevalence estimates over time from the function 
                            <italic toggle="yes">plot_fp_c</italic>.</title>
                        <p>Results are shown for Afghanistan. Light purple shaded area represents 95% credible intervals and the dark purple area represents 80% credible intervals.</p>
                    </caption>
                    <graphic orientation="portrait" position="float" xlink:href="https://gatesopenresearch-files.f1000.com/manuscripts/14429/01c8746f-69bf-4176-9c88-6a83306b4d78_figure2.gif"/>
                </fig>
                <p>
                    <italic toggle="yes">
                        <bold>Case 1.2: Estimating FP indicators using custom datasets.</bold>
                    </italic> The use case for FP estimating using custom datasets is given in the 
                    <ext-link ext-link-type="uri" xlink:href="https://github.com/AlkemaLab/FPEMlocal/blob/master/vignettes/in_union_women_from_custom_data.md">&#x2018;in union women from custom data&#x2019;</ext-link> vignette. Similar to case 1.1, the user starts by fitting the model with the function 
                    <italic toggle="yes">fit_fp_c</italic>. In addition to the inputs in case 1.1, the user supplies the file path of the .csv file containing the survey dataset. Any missing sampling errors in the input data are imputed automatically with the function 
                    <italic toggle="yes">input_user_se</italic>.</p>
                <p>After the model is fit, the user reads in the .csv file containing the population count data for the population of interest (see 
                    <xref ref-type="fig" rid="f3">Figure 3</xref>). Next, they supply the fit object and the population count data to the function 
                    <italic toggle="yes">calc_fp_c</italic>. Results can be plotted using the 
                    <italic toggle="yes">plot_fp_c</italic> function.</p>
                <fig fig-type="figure" id="f3" orientation="portrait" position="float">
                    <label>Figure 3. </label>
                    <caption>
                        <title>Function calls and illustrative output for use case 1.2: Estimating FP indicators for married women with custom user data.</title>
                    </caption>
                    <graphic orientation="portrait" position="float" xlink:href="https://gatesopenresearch-files.f1000.com/manuscripts/14429/01c8746f-69bf-4176-9c88-6a83306b4d78_figure3.gif"/>
                </fig>
                <p>
                    <italic toggle="yes">
                        <bold>Case 1.3 Estimating FP indicators with custom subnational datasets.</bold>
                    </italic> The user must supply a custom dataset for the use of 
                    <italic toggle="yes">fpemlocal</italic> for subnational estimation, as no default datasets are available. The use case for subnational estimation with custom datasets is given in the 
                    <ext-link ext-link-type="uri" xlink:href="https://github.com/AlkemaLab/FPEMlocal/blob/master/vignettes/subnational.md">&#x2018;subnational&#x2019; vignette</ext-link>. The use case follows that from case 1.2 with the only change being that the user sets the argument 
                    <italic toggle="yes">subnational</italic> equal to 
                    <italic toggle="yes">TRUE</italic> when calling subnational 
                    <italic toggle="yes">fit_fp_c</italic>. Internally, this results in using parameters for model fitting that are relevant for subnational runs, considering each subnation to be a &#x201c;country&#x201d; within the &#x201c;subregion&#x201d; of its respective nation. For example, mean pace of the uptake parameters are obtained from the country of interest (as indicated by 
                    <italic toggle="yes">division code</italic>) as opposed to its larger region. As in case 1.2, the user supplies custom survey data for model fitting and custom population counts.</p>
            </sec>
            <sec>
                <title>Case 2: Estimating FP indicators for all women</title>
                <p>The use case for estimating FP indicators for all women is given in the 
                    <ext-link ext-link-type="uri" xlink:href="https://github.com/AlkemaLab/fpemlocal/blob/master/vignettes/all_women.md">`estimating for all women` vignette</ext-link>. Obtaining results for both in_union and not_in_union women entails running the in-union and not-in-union model. In this use case, the user can supply survey data for in_union and not_in_union women or use the default UNPD data base, and the model is fit to both by specifying argument 
                    <italic toggle="yes">is_in_union = &#x201c;ALL&#x201d;</italic> in the model fit function call. The resulting fit object is a named list of fits (see 
                    <xref ref-type="fig" rid="f4">Figure 4</xref>).</p>
                <fig fig-type="figure" id="f4" orientation="portrait" position="float">
                    <label>Figure 4. </label>
                    <caption>
                        <title>Function calls and illustrative output for use case 2: Estimating FP indicators for all women.</title>
                    </caption>
                    <graphic orientation="portrait" position="float" xlink:href="https://gatesopenresearch-files.f1000.com/manuscripts/14429/01c8746f-69bf-4176-9c88-6a83306b4d78_figure4.gif"/>
                </fig>
                <p>Next, the user supplies the entire list of fits from 
                    <italic toggle="yes">fit_fp_c</italic> to the function 
                    <italic toggle="yes">calc_fp_c</italic>. Like the previous function, 
                    <italic toggle="yes">calc_fp_c</italic> returns a list of results with estimates for in-union women, not-in-union women, and all women. Results can be plotted using the 
                    <italic toggle="yes">plot_fp_c</italic> function.</p>
            </sec>
            <sec>
                <title>Case 3: Aggregating multiple fits and obtaining aggregate estimates</title>
                <p>
                    <italic toggle="yes">fpemlocal</italic> allows users to aggregate estimates from multiple populations to produce estimates that refer to the combined population. Aggregate estimates of family planning proportions - referring to contraceptive use, unmet need, and the no need category - are given by the weighted average of population-specific outcomes, with weights given by the number of women in the respective population. For example, to obtain modern use among in_union women, the weights are given by the number of in_union women in each single population that is combined in the aggregate outcome.</p>
                <p>The case for aggregating multiple fits is given in the 
                    <ext-link ext-link-type="uri" xlink:href="https://github.com/AlkemaLab/FPEMlocal/blob/master/vignettes/aggregating_estimates.md">&#x2018;aggregating estimates&#x2019; vignette</ext-link> and summarized in 
                    <xref ref-type="fig" rid="f5">Figure 5</xref>. First, the user fits FPEM to each population of interest. Next, the user prepares a single population count dataset containing all populations of interest. The user supplies the fit objects and the population data to the function 
                    <italic toggle="yes">calc_fp_aggregate</italic> to obtain aggregate estimates.</p>
                <fig fig-type="figure" id="f5" orientation="portrait" position="float">
                    <label>Figure 5. </label>
                    <caption>
                        <title>Function calls and illustrative output for use case 3: Aggregating multiple fits and obtaining aggregate estimates.</title>
                    </caption>
                    <graphic orientation="portrait" position="float" xlink:href="https://gatesopenresearch-files.f1000.com/manuscripts/14429/01c8746f-69bf-4176-9c88-6a83306b4d78_figure5.gif"/>
                </fig>
            </sec>
            <sec>
                <title>Additional features</title>
                <p>
                    <italic toggle="yes">Automatic fit saving: fpemlocal</italic> includes wrapper functions, built around the functions described so far, to automatically save outputs. These functions may be useful to users fitting the model to multiple countries. The wrapper functions are given by 
                    <italic toggle="yes">fit_fp_c_autosave</italic>, 
                    <italic toggle="yes">calc_fp_cautosave</italic>, and 
                    <italic toggle="yes">plot_fp_c_autosave</italic>.</p>
                <p>
                    <italic toggle="yes">Diagnostics:</italic> The fit function 
                    <italic toggle="yes">fit_fp_c</italic> includes an option to save diagnostic checks in the form of trace plots and convergence checks (
                    <xref ref-type="bibr" rid="ref-16">Vehtari 
                        <italic toggle="yes">et al.</italic>, 2020</xref>). Default settings are used in model fitting based on analysis of these diagnostics.</p>
                <p>
                    <italic toggle="yes">Service statistics data</italic>: Service statistics data as summarized into Estimated Modern Use (EMU) can be included in the model fitting as well (
                    <xref ref-type="bibr" rid="ref-7">Magnani 
                        <italic toggle="yes">et al.</italic>, 2018</xref>).</p>
            </sec>
        </sec>
        <sec sec-type="conclusions">
            <title>Conclusions</title>
            <p>We introduced the R package 
                <italic toggle="yes">fpemlocal</italic>. This package can be used to fit the local Family Planning Estimation Model to populations of interest. The package is used by Track20 for use in country workshops, through an online interface (
                <xref ref-type="bibr" rid="ref-13">Track20, n.d.</xref>) which has informed FP2020 initiative reporting since 2013. The package is used by Track20 which has informed FP2020 initiative 
                <ext-link ext-link-type="uri" xlink:href="http://progress.familyplanning2020.org/sites/default/files/FP2020_2019Report_FINAL_110819.pdf">progress reports</ext-link> (
                <xref ref-type="bibr" rid="ref-10">FP2020, 2019</xref>). Recent additional use of 
                <italic toggle="yes">fpemlocal</italic> includes the assessment of the increase in modern contraceptive use needed to reach demand satisfied targets by 2030 (
                <xref ref-type="bibr" rid="ref-3">Cahill 
                    <italic toggle="yes">et al.</italic>, 2020</xref>).</p>
            <p>
                <italic toggle="yes">Fpemlocal</italic> can serve as an example for other global modeling exercises. Publishing the code in the form of an R package facilitates the production, reproduction, and transparency, of model-based estimates. The local implementation of a global model makes fitting less computationally demanding, thus enabling users with limited computational resources to fit Bayesian models to populations of their choice, as indicated in the case studies.</p>
        </sec>
        <sec>
            <title>Software availability</title>
            <p>Software source code: 
                <ext-link ext-link-type="uri" xlink:href="https://github.com/AlkemaLab/fpemlocal">https://github.com/AlkemaLab/fpemlocal</ext-link>
            </p>
            <p>Archived source code as at time of publication: 
                <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.5281/zenodo.4302624">https://doi.org/10.5281/zenodo.4302624</ext-link> (
                <xref ref-type="bibr" rid="ref-4">Guranich 
                    <italic toggle="yes">et al.</italic>, 2020</xref>).</p>
            <p>License: MIT license</p>
        </sec>
    </body>
    <back>
        <ref-list>
            <ref id="ref-1">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Alkema</surname>
                            <given-names>L</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Kantorova</surname>
                            <given-names>V</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Menozzi</surname>
                            <given-names>C</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>National, regional, and global rates and trends in contraceptive prevalence and unmet need for family planning between 1990 and 2015: A systematic and comprehensive analysis.</article-title>
                    <source>

                        <italic toggle="yes">Lancet.</italic>
</source>
                    <year>2013</year>;<volume>381</volume>(<issue>9878</issue>);<fpage>1642</fpage>&#x2013;<lpage>1652</lpage>.
                    <pub-id pub-id-type="pmid">23489750</pub-id>
                    <pub-id pub-id-type="doi">10.1016/S0140-6736(12)62204-1</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-2">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Cahill</surname>
                            <given-names>N</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Sonneveldt</surname>
                            <given-names>E</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Stover</surname>
                            <given-names>J</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Modern contraceptive, use, unmet need, and demand satisfied among women of reproductive age who are married or in a union in the focus countries of the Family Planning 2020 initiative: A systematic analysis using the Family Planning Estimation Tool.</article-title>
                    <source>

                        <italic toggle="yes">Lancet.</italic>
</source>
                    <year>2018</year>;<volume>391</volume>(<issue>10123</issue>):<fpage>870</fpage>&#x2013;<lpage>882</lpage>.
                    <pub-id pub-id-type="pmid">29217374</pub-id>
                    <pub-id pub-id-type="doi">10.1016/S0140-6736(17)33104-5</pub-id>
                    <pub-id pub-id-type="pmcid">5854461</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-3">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Cahill</surname>
                            <given-names>N</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Weinberger</surname>
                            <given-names>M</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Alkema</surname>
                            <given-names>L</given-names>
                        </name>
</person-group>:
                    <article-title>What increase in modern contraceptive use is needed in FP2020 countries to reach 75% demand satisfied by 2030? An assessment using the Accelerated Transition Method and Family Planning Estimation Model  [version 1; peer review: 2 approved].</article-title>
                    <source>

                        <italic toggle="yes">Gates Open Res.</italic>
</source>
                    <year>2020</year>;<volume>4</volume>:<fpage>113</fpage>.
                    <pub-id pub-id-type="pmid">33117964</pub-id>
                    <pub-id pub-id-type="doi">10.12688/gatesopenres.13125.1</pub-id>
                    <pub-id pub-id-type="pmcid">7578409</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-10">
                <mixed-citation publication-type="journal">
                    <collab>FP2020</collab>:
                    <article-title>FP2020 Women at the center 2018&#x2013;2019</article-title>.<year>2019</year>.
                    <ext-link ext-link-type="uri" xlink:href="http://progress.familyplanning2020.org/sites/default/files/FP2020_2019Report_FINAL_110819.pdf">Reference Source</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-4">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Guranich</surname>
                            <given-names>G</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Wheldon</surname>
                            <given-names>M</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Cahill</surname>
                            <given-names>N</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>fpemlocal: The local implementation of the Family Planning Estimation Model (Version 1.01) [R package]</article-title>.<year>2020</year>.
                    <ext-link ext-link-type="uri" xlink:href="http://www.doi.org/10.5281/zenodo.4302624">http://www.doi.org/10.5281/zenodo.4302624</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-6">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Kantorov&#x00e1;</surname>
                            <given-names>V</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Wheldon</surname>
                            <given-names>MC</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Ueffing</surname>
                            <given-names>P</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Estimating progress towards meeting women&#x2019;s contraceptive needs in 185 countries: A Bayesian hierarchical modelling study.</article-title>
                    <source>

                        <italic toggle="yes">PLoS Med.</italic>
</source>
                    <year>2020</year>;<volume>17</volume>(<issue>2</issue>):<fpage>e1003026</fpage>.
                    <pub-id pub-id-type="pmid">32069289</pub-id>
                    <pub-id pub-id-type="doi">10.1371/journal.pmed.1003026</pub-id>
                    <pub-id pub-id-type="pmcid">7028249</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-7">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Magnani</surname>
                            <given-names>RJ</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Ross</surname>
                            <given-names>J</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Williamson</surname>
                            <given-names>J</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Can Family Planning Service Statistics Be Used to Track Population-Level Outcomes?</article-title>
                    <source>

                        <italic toggle="yes">Glob Health Sci Pract.</italic>
</source>
                    <year>2018</year>;<volume>6</volume>(<issue>1</issue>):<fpage>93</fpage>&#x2013;<lpage>102</lpage>.
                    <pub-id pub-id-type="pmid">29467167</pub-id>
                    <pub-id pub-id-type="doi">10.9745/GHSP-D-17-00341</pub-id>
                    <pub-id pub-id-type="pmcid">5878083</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-8">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>New</surname>
                            <given-names>JR</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Cahill</surname>
                            <given-names>N</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Stover</surname>
                            <given-names>J</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Levels and trends in contraceptive prevalence, unmet need, and demand for family planning for 29 states and union territories in India: A modelling study using the Family Planning Estimation Tool.</article-title>
                    <source>

                        <italic toggle="yes">Lancet Glob Health.</italic>
</source>
                    <year>2017</year>;<volume>5</volume>(<issue>3</issue>):<fpage>e350</fpage>&#x2013;<lpage>e358</lpage>.
                    <pub-id pub-id-type="pmid">28193400</pub-id>
                    <pub-id pub-id-type="doi">10.1016/S2214-109X(17)30033-5</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-9">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Plummer</surname>
                            <given-names>M</given-names>
                        </name>
</person-group>:
                    <article-title>JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling (Version 4.3.0) [R package]</article-title>.<year>2017</year>.
                    <ext-link ext-link-type="uri" xlink:href="https://sourceforge.net/projects/mcmc-jags">Reference Source</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-11">
                <mixed-citation publication-type="journal">
                    <collab>R Core Team</collab>:
                    <article-title>R: A language and environment for statistical computing</article-title>. R Foundation for Statistical Computing, Vienna, Austria.<year>2020</year>.
                    <ext-link ext-link-type="uri" xlink:href="https://www.R-project.org/">Reference Source</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-12">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Su</surname>
                            <given-names>YS</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Yajima</surname>
                            <given-names>M</given-names>
                        </name>
</person-group>:
                    <article-title>R2jags: Using R to Run &#x201c;JAGS&#x201d; (Version 0.6-1) [R package]</article-title>.<year>2020</year>.
                    <ext-link ext-link-type="uri" xlink:href="https://CRAN.R-project.org/package=R2jags">Reference Source</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-13">
                <mixed-citation publication-type="journal">
                    <article-title>Track20</article-title>: (n.d.). Retrieved August 3,<year>2020</year>.
                    <ext-link ext-link-type="uri" xlink:href="http://www.track20.org/">Reference Source</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-14">
                <mixed-citation publication-type="journal">
                    <collab>United Nations, Department of Economic and Social Affairs, Population Division</collab>:
                    <article-title>Estimates and Projections of Women of Reproductive Age Who Are Married or in a Union (2020 revision) [Data set].</article-title>United Nations.<year>2020a</year>.
                    <ext-link ext-link-type="uri" xlink:href="https://www.un.org/en/development/desa/population/theme/marriage-unions/marriage_estimates.asp">Reference Source</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-15">
                <mixed-citation publication-type="journal">
                    <collab>United Nations, Department of Economic and Social Affairs, Population Division</collab>:
                    <article-title>World Population Prospects (2019 Revision) [Data set].</article-title>United Nations.<year>2020b</year>.
                    <ext-link ext-link-type="uri" xlink:href="https://population.un.org/wpp/Download/Standard/CSV/">Reference Source</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-16">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Vehtari</surname>
                            <given-names>A</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Gelman</surname>
                            <given-names>A</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Simpson</surname>
                            <given-names>D</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Rank-Normalization, Folding, and Localization: An Improved R-hat for Assessing Convergence of MCMC.</article-title>
                    <source>

                        <italic toggle="yes">Bayesian Anal.</italic>
</source>
                    <year>2020</year>;<fpage>28</fpage>.
                    <pub-id pub-id-type="doi">10.1214/20-BA1221</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-17">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Wickham</surname>
                            <given-names>H</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Hester</surname>
                            <given-names>J</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Chang</surname>
                            <given-names>W</given-names>
                        </name>
</person-group>:
                    <article-title>devtools: Tools to Make  Developing R Packages Easier  R package version 2.3.0</article-title>.<year>2020</year>.
                    <ext-link ext-link-type="uri" xlink:href="https://CRAN.R-project.org/package=devtools">Reference Source</ext-link>
                </mixed-citation>
            </ref>
        </ref-list>
    </back>
    <sub-article article-type="reviewer-report" id="report30447">
        <front-stub>
            <article-id pub-id-type="doi">10.21956/gatesopenres.14429.r30447</article-id>
            <title-group>
                <article-title>Reviewer response for version 1</article-title>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author">
                    <name>
                        <surname>Li</surname>
                        <given-names>Qingfeng</given-names>
                    </name>
                    <xref ref-type="aff" rid="r30447a1">1</xref>
                    <role>Referee</role>
                    <uri content-type="orcid">https://orcid.org/0000-0002-6390-6921</uri>
                </contrib>
                <aff id="r30447a1">
                    <label>1</label>Department of International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA</aff>
            </contrib-group>
            <author-notes>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>29</day>
                <month>3</month>
                <year>2021</year>
            </pub-date>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2021 Li Q</copyright-statement>
                <copyright-year>2021</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <related-article ext-link-type="doi" id="relatedArticleReport30447" related-article-type="peer-reviewed-article" xlink:href="10.12688/gatesopenres.13211.1"/>
            <custom-meta-group>
                <custom-meta>
                    <meta-name>recommendation</meta-name>
                    <meta-value>approve</meta-value>
                </custom-meta>
            </custom-meta-group>
        </front-stub>
        <body>
            <p>The authors present an R package named fpemlocal, a local implementation of a previously published global Bayesian hierarchical model. Similar functions are already available through an online version, but this R package may be more convenient for users who prefer the R interface and have unstable internet access. With the instructions from the article and README on GitHub, the package is user-friendly overall. Some suggestions for further improvement: 
                <list list-type="order">
                    <list-item>
                        <p>For use case 1.1, perhaps the package can directly include the results from a global run. The package only needs to extract the results for the country of interest to the user, instead of fitting a single-country model for the given country. Theoretically, by fixing non-country-specific parameters at the global FPEM estimates, the single-country implementation compromises the model accuracy. For use cases 1.2 and 1.3, the substantial saving in computational load overwhelms the slight loss in accuracy. But that does not apply to case 1.1.</p>
                    </list-item>
                    <list-item>
                        <p>Customize figure appearance. Current output figures are nice and clear, but they require some final minor changes if the user wants to include them in a presentation or a report. For example, replace variable names on axis titles with properly capitalized words. An easy way to achieve that is by changing the default setting and adding those arguments to the plot_fp_c function.</p>
                    </list-item>
                    <list-item>
                        <p>The users may need&#x00a0;more instructions on preparing&#x00a0;data for cases 1.2 and 1.3. The instructions should include column&#x00a0;definitions&#x00a0;and clarify&#x00a0;which columns are mandatory.&#x00a0;</p>
                    </list-item>
                </list>
            </p>
            <p>Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?</p>
            <p>Yes</p>
            <p>Is the rationale for developing the new software tool clearly explained?</p>
            <p>Yes</p>
            <p>Is the description of the software tool technically sound?</p>
            <p>Yes</p>
            <p>Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?</p>
            <p>Yes</p>
            <p>Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?</p>
            <p>Yes</p>
            <p>Reviewer Expertise:</p>
            <p>demographic and statistical modeling; public health</p>
            <p>I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.</p>
        </body>
    </sub-article>
    <sub-article article-type="reviewer-report" id="report30449">
        <front-stub>
            <article-id pub-id-type="doi">10.21956/gatesopenres.14429.r30449</article-id>
            <title-group>
                <article-title>Reviewer response for version 1</article-title>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author">
                    <name>
                        <surname>Stevens</surname>
                        <given-names>Oliver</given-names>
                    </name>
                    <xref ref-type="aff" rid="r30449a1">1</xref>
                    <role>Referee</role>
                    <uri content-type="orcid">https://orcid.org/0000-0001-6842-9434</uri>
                </contrib>
                <aff id="r30449a1">
                    <label>1</label>Department for Infectious Disease Epidemiology, School of Public Health, Imperial College London, London, UK</aff>
            </contrib-group>
            <author-notes>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>Collaborator with Leontine Alkema. Sought expert advice on demographic modelling work. No shared funding, no shared publications in past three years. I confirm that this has not affected my ability to write an objective and unbiased review of the article.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>4</day>
                <month>3</month>
                <year>2021</year>
            </pub-date>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2021 Stevens O</copyright-statement>
                <copyright-year>2021</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <related-article ext-link-type="doi" id="relatedArticleReport30449" related-article-type="peer-reviewed-article" xlink:href="10.12688/gatesopenres.13211.1"/>
            <custom-meta-group>
                <custom-meta>
                    <meta-name>recommendation</meta-name>
                    <meta-value>approve</meta-value>
                </custom-meta>
            </custom-meta-group>
        </front-stub>
        <body>
            <p>This article reports the implementation and use of the 
                <italic>fpemlocal </italic>package, a slimmed down version of FPEM for estimating family planning indicators at the single country level. Both the manuscript and the package itself are clearly laid out and easy to follow. The model fits quickly on a domestic laptop and performs well against survey data tested.</p>
            <p> </p>
            <p> Fixing the model hyperparameters to global posterior point estimates is a good solution to reduce computational burden. It may be useful to country team users to include the globally fitted FPEM results as a 
                <italic>fpemlocal </italic>package dataset with which 
                <italic>fpemlocal</italic> results may be overlaid.</p>
            <p> </p>
            <p> Some minor comments: 
                <list list-type="bullet">
                    <list-item>
                        <p>Functions would benefit from improved help text, including examples with working arguments. Without the examples in the manuscript which future users may not consult, it was not immediately clear how functions should be called.</p>
                    </list-item>
                </list> fit_fp_c() 
                <list list-type="bullet">
                    <list-item>
                        <p>It would be useful to state which arguments for fit_fp_c() are mandatory and which are optional/autofilled by the package.</p>
                    </list-item>
                    <list-item>
                        <p>Following fitting, all the model internals are placed individually into the global environment. Please nest as a list within the fit object.</p>
                    </list-item>
                </list> plot_fp_c() 
                <list list-type="bullet">
                    <list-item>
                        <p>plot_fp_c() requires ggnewscale as a dependency which is not installed with the package.</p>
                    </list-item>
                    <list-item>
                        <p>Help text for plot_fp_c argument "results" 
                            <italic>Results data from&#x00a0;
                                <ext-link ext-link-type="uri" xlink:href="http://127.0.0.1:13094/help/library/fpemlocal/help/fpem_calculate_results">fpem_calculate_results</ext-link>
                            </italic>, but should read "results data from calc_fp_c()"</p>
                    </list-item>
                    <list-item>
                        <p>Plots from plot_fp_c(): 
                            <list list-type="bullet">
                                <list-item>
                                    <p>Axes and legend labels should be more human readable - e.g. "Modern Contraceptive Use" vs "contraceptive_use_modern"</p>
                                </list-item>
                                <list-item>
                                    <p>Please display confidence intervals on survey estimates</p>
                                </list-item>
                                <list-item>
                                    <p>Not clear what group_type_relative_to_baseline represents</p>
                                </list-item>
                            </list> </p>
                    </list-item>
                </list> Future work may look to implement a spatial model when single country fits are done at the subnational level - beyond the scope of this software report.</p>
            <p> </p>
            <p> The subnational vignette, though setting subnational = TRUE, doesn't use subnational data &amp; accordingly doesn't generate subnational results, so it's difficult to comment on its usage.</p>
            <p>Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?</p>
            <p>Yes</p>
            <p>Is the rationale for developing the new software tool clearly explained?</p>
            <p>Yes</p>
            <p>Is the description of the software tool technically sound?</p>
            <p>Yes</p>
            <p>Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?</p>
            <p>Yes</p>
            <p>Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?</p>
            <p>Yes</p>
            <p>Reviewer Expertise:</p>
            <p>Infectious disease epidemiology/demography</p>
            <p>I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.</p>
        </body>
    </sub-article>
</article>
