Setting the stage for strengthened annual monitoring of Family Planning program performance at the National and Subnational level in Myanmar

Khaing Nwe Tin; Jessica Williamson; Emily Sonneveldt

doi:10.12688/gatesopenres.13012.3

Home Browse Setting the stage for strengthened annual monitoring of Family Planning...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Research Article

Revised

Setting the stage for strengthened annual monitoring of Family Planning program performance at the National and Subnational level in Myanmar

[version 3; peer review: 2 approved]

Previously titled: Setting the stage for strengthened annual monitoring of family planning program performance at the state/national level in Myanmar

Khaing Nwe Tin ¹, Jessica Williamson², Emily Sonneveldt³

PUBLISHED 18 Dec 2019

Author details Author details

¹ Department of Public Health, Ministry of Health and Sports, Myanmar, Nay Pyi Taw, 15011, Myanmar
² Track20, Avenir Health, Washington DC, USA
³ Track20, Avenir Health, Connecticut, USA

Khaing Nwe Tin
Roles: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Jessica Williamson
Roles: Methodology, Visualization, Writing – Review & Editing

Emily Sonneveldt
Roles: Resources, Supervision, Validation

OPEN PEER REVIEW

REVIEWER STATUS

This article is included in the International Conference on Family Planning gateway.

Abstract

Background: Although Myanmar has made good progress in family planning by increased contraceptive prevalence rate (CPR) from 41% in 2007 to 52.2% in 2016, it remains lower than the target of 60% by 2020. There are also huge disparities sub-nationally, ranging from 25% to 60%. While there is a strong need to monitor the progress of family planning program regularly at the national and sub-national level, Myanmar has limited surveys, data quality and methodological issues in its Health Management Information System (HMIS), and a scattered rollout of the Logistic Management Information System (LMIS).
Methods: To identify viable options for annual monitoring, four data sources: modelled contraceptive prevalence rate for modern methods (mCPR) estimates from Track20’s Family Planning Estimation Tool (FPET); method-specific prevalence from the 2015-16 Myanmar Demographic and Health Survey (DHS); mCPR estimates and method prevalence from HMIS and estimates of modern method use (EMU) based on commodity consumption data from LMIS, were used to compare for the years 2015-2017. Estimates of mCPR from HMIS were tested for accuracy based on whether they fell within the 95% confidence interval of mCPR estimates from the FPET for the corresponding years. EMU from LMIS was also tested for those years and States/Regions where available.
Results: For annual tracking of mCPR, direct estimates of HMIS were considered; they were much higher than those of the DHS survey and were not matched by FPET results, except in Chin and Kayin. To monitor the method mix, HMIS data can be used as these are similar pattern with DHS in both national and State/Regional level except Chin and Kayin. LMIS could be used in annual tracking when there are high reporting rates and valid information of consumption.
Conclusions: Track20’s FPET is the method of choice to get valid information for annual monitoring of family planning program.

Keywords

Family Planning, monitoring, Myanmar

Corresponding author: Khaing Nwe Tin

Competing interests: No competing interests were disclosed.

Grant information: Publication of this research was funded by the Bill and Melinda Gates Foundation (OPP1181398).
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2019 Nwe Tin K et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Nwe Tin K, Williamson J and Sonneveldt E. Setting the stage for strengthened annual monitoring of Family Planning program performance at the National and Subnational level in Myanmar [version 3; peer review: 2 approved]. Gates Open Res 2019, 3:1541 (https://doi.org/10.12688/gatesopenres.13012.3) First published: 18 Sep 2019, 3:1541 (https://doi.org/10.12688/gatesopenres.13012.1) Latest published: 18 Dec 2019, 3:1541 (https://doi.org/10.12688/gatesopenres.13012.3)

Revised Amendments from Version 2

I have changed the words in title as "Setting the stage for strengthened annual monitoring of Family Planning program performance at the National and Sub-national level in Myanmar" instead of Setting the stage for strengthened annual monitoring of family planning program performance at the state/national level in Myanmar because readers can misinterpret the State as national. Actually, in my paper, State means that sub-national like Region. In Myanmar there are 9 States and 8 Regions at sub-national level and the study also focuses both National and sub-national (State/Regional) level. So, I changed it to be more clear to readers.

See the authors' detailed response to the review by Janine Barden O'Fallon
See the authors' detailed response to the review by James F. Phillips

Introduction

As family planning is an evidence-based intervention for improving the maternal and newborn health, as well as a cost-effective powerful tool for development, Myanmar committed to the Family Planning (FP) 2020 initiative in 2013 aiming to improve women and children’s health through increased access to the quality family planning services without any disparities^1,2. In order to achieve the objectives of the Myanmar FP 2020 commitment; Contraceptive Prevalence Rate (CPR) must be over 60% by 2020. Myanmar has been endeavoring with strong coordinated efforts among the different sectors: public, private, UN agencies, INGOs, NGOs and donor agencies. Additionally, the family planning program has been implemented under the guidance of the Reproductive Health policy (2002), Five years Reproductive Health Strategic Plans (RHSP) and Costed Implementation Plan for FP 2020^1,2.

Another driving force for family planning program is high Maternal Mortality Ratio (MMR) in Myanmar. According to UN interagency estimates, the Myanmar MMR has reduced from 453 per 100,000 live births in 1990 to 178 per 100,000 live births in 2015; however, this figure was the second highest among ASEAN countries and did not meet the 2015 Millennium Development Goal³. As of the 2014 census, the MMR in Myanmar was 282 deaths per 100,000 live births⁴ Aiming to reduce maternal morbidity and mortality, family planning service has been accorded as a priority issue in the basic Essential Package of Health Services of Myanmar National Health Plan (2017–21)⁵.

Myanmar’s family planning programme started in 1991 as a public sector pilot in one township, and then progressively extended to 163 out of 330 townships in 2014. Before 2011, the government had no specific financial allocation for reproductive health commodities, including contraception, and heavily relied on UNFPA supplies. From 2011, the government increased the health budget, allocated a budget for contraceptive commodities and invested more in the family planning program, to allow it to provide more contraceptives, both short- and long-term, free of charge in all public facilities since 2012².

Although various inputs have been used in the Myanmar Family Planning program and the contraceptive prevalence rate (CPR) has increased from 41% in 2007⁶ to 52.2% in 2016⁷, it is estimated to be slightly lower than the target of 60% by 2020⁵. At the same time, an unmet need for family planning has been reduced from 19% in 2007⁶ to 16% in 2016⁷, still falling short of the 2020 target of an unmet need of less than 10%⁵. Therefore, continuous monitoring plays a role in providing the useful inputs for the directions or strategies of program implementation.

In order to monitor the progress of family planning program, Myanmar uses the FP indicators (CPR, method mixed prevalence, unmet need for family planning and demand satisfaction) mainly from the limited available sources; first ever Demographic and Health Survey (DHS) (2015–16) and Health Management Information System (HMIS), as well as commodity consumption data from Logistic Management Information System (LMIS). HMIS was developed as paper based in 1995 and is now transformed to an electronic system of HMIS as District Health Information System (DHIS) 2. It is the responsibility of Myanmar Ministry of Health and Sports to monitor the performance of all public health facilities while it accounts for local annual census for family planning data and covers both public and private⁸. In Myanmar, LMIS for reproductive health commodity (RHCLS: Reproductive health commodity logistic system) was initiated as national program in 2016 in selected regions and has been rolled out phase by phase. It provides the information related with stock status as well as commodity consumption data; commodity distributed to the clients, from different public health facilities⁹. However, systematic tracking of annual family planning progress with valid data source has not yet been set yet.

Rationale

According to the Myanmar DHS (2015–16), contraceptive use is growing nationally, but there are disparities in use among different states and regions, from the lowest prevalence in Chin State (25%) up to 60% in Yangon and Bago region⁷. Given the wide variability in contraceptive use and the performance of the FP program by state/region, there is a strong need for valid information about contraceptive use for better annual tracking. Currently, there is limited information on contraceptive use available for the regular monitoring and evaluation as Myanmar has had limited surveys, data quality and methodology issues exist in the Health Management Information System (HMIS), and slow and scattered rollout of the Logistic Management Information System (LMIS) means these service statistics data have limited application for State/Regional routine monitoring. The national estimates of mCPR for same period, 2016, are quite different among DHS survey (51.3%)⁷, result from the Family Planning Estimation Tool (FPET) http://www.track20.org/pages/resources/track20_tools.php; a web application developed by Track 20 project/Avenir Health that uses statistical modeling by incooperating with all available survey and service statistics data to produce annual estimates for key family planning indicators, (50.8%)¹⁰ and HMIS (61.3%)¹¹ (Figure 1). This discrepancy between the different data sources led the program to consider the most reliable data source for both national and subnational annual monitoring on family planning. Also, lack of studies for systematic analysis of the family planning data source called to do this study to set the annual family planning monitoring system in Myanmar.

Figure 1. Comparing National mCPR estimates of DHS, FPET, and HMIS.

Objectives

The objective of this study is to review existing sources of data on the modern contraceptive prevalence rate (mCPR) and method-specific prevalence in Myanmar to identify viable options for annual monitoring of the family planning program at the national and State/Regional level.

Methods

Data sources

In order to understand what data may best serve annual monitoring of the performance of the family planning program in Myanmar, both at the national and State/Regional level, four data sources of modern contraceptive use were used to compare:

1. Modelled mCPR estimates (and confidence intervals) from Track20’s FPET tool (requires free registration), based on nationally and State/Regionally representative surveys¹⁰ for Myanmar: Myanmar DHS (2015–16), Multiple Indicator Cluster Survey (2010), Fertility and Reproductive Health Survey (1991,1997, 2001 & 2007) (While FPET tool can be incooperated with both survey and service statistics data, FPET run for Myanmar could not use the service statistics data due to data validity.)
2. mCPR and method-specific prevalence from the 2015–16 Myanmar DHS⁷.
3. mCPR estimates and contraceptive method prevalence from Myanmar’s HMIS system¹¹, based on local annual census conducted by midwives on contraceptive use among married women in their catchment areas; which is different from other routine HMIS indicators and collected once a year at the end of calendar year for the information of contraceptive use including type of methods used, and
4. mCPR from Estimates of modern method use (EMU) tool based on contraceptive commodity consumption data from Myanmar’s LMIS system¹² were used to compare (Table 1). While FPET can be incorporated into service statistics data, it could not be put with the direct estimates of mCPR, other commodity or visit data from service statistics. EMU excel can convert service statistics data to estimated method use (which is a proxy for mCPR) and it can be used as input in the Family Planning Estimation Tool (FPET). EMU tool was developed by Track 20 team and available at its website http://www.track20.org/pages/our_work/innovative_tools/SS_to_EMU_tool.php. In this study, EMU tool was used to provide EMU (Estimated method use-a proxy of mCPR) by converting of commodity consumption data to compare with the mCPR from HMIS and FPET tool.

Table 1. Data sources for comparison.

Source	mCPR	Years	Method use	Year
FPET	X	2015–2017
HMIS	X	2015–2017	X	2016
LMIS	Limited regional and time trend availability
DHS			X	2016

While DHS data are the gold standard for monitoring of FP indicators, it does not provide information for annual monitoring. Between the years of surveys, to monitor the annual progress, Track 20 project proposes a FPET tool which is based on a Bayesian, hierarchical model that fits curves to historical data (all available survey and service statistics). The model fits a logistic growth curve to CPR data for all methods to determine the long term trend in contraceptive use and adds a time-series model with autocorrelation to capture country-specific deviations around the long term trend. Since FPET gives almost the same result as the DHS (Figure 1), it is the most reliable data source for annual estimates so far.

In addition to FPET, to identify the most valid data source and to test whether the routine service statistics data can be used or not for annual monitoring, contraceptive use data from service statistics (HMIS and LMIS) were used to compare, along with the confidence intervals from FPET for three consecutive years from 2015 to 2017 as FPET data were assumed that the most reliable data source apart from DHS. If the specific data were within the 95% of confidence interval of FPET, it was considered as the accurate data in this study to be used for annual monitoring of family planning.

Firstly, estimates of mCPR from the HMIS at both the national and State/Regional level were tested for accuracy based on whether they fell within the 95% confidence interval of mCPR estimates of FPET for the corresponding years. The consistency between two sources was matched for each three years. Then, the method-specific prevalence data for both national and State/Regional level from HMIS were compared with DHS data for 2016 only. As the FPET could not provide the method mix data, this data could be compared with DHS. Finally, another service statistic data source; commodity consumption data from LMIS, were used to compare for mCPR along with FPET for its accuracy. The mCPR extracted from estimates of method use (EMU) excel tool; by converting the inputs of commodity consumption data, from LMIS were also tested for those years and in States/Regions where the LMIS was available.

Ethics approval and consent to participate

This paper is a secondary analysis of the four different sources of data: contraceptive prevalence rate from Family Planning Estimation Tool of Track 20, 2015–16 Myanmar Demographic and Health Survey (MDHS), Health Management Information System (HMIS) data of Department of Public Health and Logistic Management Information System (LMIS) data of RH commodities from Maternal Reproductive Health Unit of Department of Public Health. Ethics approval for Myanmar DHS was obtained from the Ethics Review Committee of the Department of Medical Research, Ministry of Health and Sports, Myanmar and the secondary data analysis for this study was done after obtaining the permission from the Department of Public Health, Ministry of Health and Sports, Myanmar.

Results

Comparing mCPR from HMIS and FPET

In comparing HMIS estimates to the point estimates and confidence intervals from FPET, among the total 17 States and Regions, years of consistency for mCPR between these data for three consecutive year could be observed in Figure 2 and Figure 3. The bar shows the HMIS data and FPET data points including upper and lower border (95% CI) were shown on that bar for each three years. Among 17 State/Regions, only two States (Chin and Kayin) produced HMIS-based estimates consistent with FPET results for three consecutive years (e.g. fell within the 95% CI of FPET estimates). Another two (Ayeyarwaddy and Kayah) were consistent with FPET for two of the three years of available HMIS data. Only one year of matching HMIS and FPET results were found in Mandalay, Sagaing, Tanintharyi and Yangon regions (Figure 2). In the other ten States/Regions, estimates of mCPR from HMIS were not within the CI of FPET results for any of the years available (Figure 3). In general, the HMIS results were most consistent with FPET results in 2015, with six of 17 Regions falling within the CI; this dropped to five in 2016 and three in 2017.

Figure 2. Comparing the HMIS and FPET results of mCPR by Regions of at least one year consistent estimates.

Figure 3. Comparing the HMIS and FPET results of mCPR by Regions of zero years consistent estimates.

At the national level, only 2015 estimates from HMIS fell within the CI of the FPET mCPR estimates. In general, as at the national level, the HMIS estimates of mCPR appear to over-estimate prevalence when compared to FPET and DHS (Figure 4).

Figure 4. Comparing National mCPR of HMIS and FEPT estimates.

Comparing the method prevalence/mix from HMIS and DHS

In comparing method prevalence, different contraceptive methods (injection, pills, IUD, implants, condom and female sterilization) from HMIS data were compared with the DHS for both national and State/Regional level. Although the mCPR from HMIS data showed consistently higher prevalence compared to the DHS, the same patterns in method mix were observed between the two data at national level, with the exception of female sterilization, which appears to be under-reported in the HMIS system. Across both data sources, injectables were indicated as the most common method in use, making up more than half of all use, followed by pills, used by about a quarter of all married users of modern contraception (Figure 5).

Figure 5. Comparing the modern contraceptive method mix.

It was observed that there were similar patterns of method mixes at the national level and most of the 17 States/Regions, except Chin and Kayin State. In Chin, rates of use of the long-term methods IUD, implants, and sterilization were considerably higher than the other areas in DHS data. In HMIS data, only IUD and implant were found as higher proportion than that of other State/Regions. Also in Kayin, the higher use of pills than injections was found only in the DHS, while the injection method was the highest proportion in national and other areas in both DHS and HMIS data.

Comparing the data of LMIS with HMIS data and FEPT results

Regarding estimates of modern method use (EMU, comparable to mCPR) from LMIS commodity consumption data, the LMIS data were found to be quite low in comparison to the HMIS and FPET estimates, except in Southern Shan State. For 2017, in Southern Shan State, while there was a >90% reporting rate, EMU from LMIS data were nearly identical to the mCPR of HMIS (63% vs 64%); however, both values are not within 95% CI of FPET estimates.

Discussion and conclusion

Access to FP contributes up to a 44% reduction in maternal deaths and a 21% reduction of deaths in children under age 5¹³, therefore, Myanmar needs to endeavor towards strong efforts to increase access to family planning services in order to tackle the high MMR issue. In the meantime, Myanmar is also trying to achieve the FP 2020 targets; therefore, setting the systematic approach for continuous monitoring is important to provide the valuable information to be considered for future strategies.

DHS is the gold standard method of tracking the family planning program; however, annually tracking the national and subnational progress level for equitable access to family planning services is required. When considering the data source for the annual tracking of mCPR, (either estimates from FPET or routine service statistics), the use of direct estimates of HMIS should be considered carefully as it is much higher than the DHS survey and not matched with FPET results except in Chin and Kayin States. It might be due to data quality issue and related with performance of data collectors. Therefore, it should be explored in detail why HMIS shows a high result with 5% annual growth, through reviewing the methodology and validating the data quality.

Currently, there is limited information from documented references about why HMIS overestimates and grows 5% annually, however, in my working experience, there is doubt in the quality of data collected by midwives. According to the methodology, it is necessary to ask all married women under their jurisdiction areas. This is difficult to cover especially in big cities, per-urban as well as hard-to-reach areas. Therefore, it is possible that it includes only the information of women who received family planning services from them and is probably higher prevalence than those who could not access health services. However, these are just only the opinions of authors and need to explore systematically and should improve based on that findings.

While the methodology for data collection is quite unusual as a local annual census, it is needed in order to be consistent with other HMIS indicators through the register/record for family planning information and integrated into electronic HMIS (DHIS 2) to get the valid data of family planning service visits and contraceptive users at least for public sectors. Considering the monitoring of the methods mix, the series of HMIS data are similar patterns with DHS at both national and State/Regional level, except Chin and Kayin.

Therefore, HMIS data can be used for monitoring of the method mix, except in Chin and Kayin although it is limited in use for tracking of mCPR. The LMIS data could be used for annual tracking of mCPR when there are high reporting rates and valid information of consumption. Thus, LMIS are needed to strengthen in data validity as well as area coverage.

These findings are consistent with other international literature. According to the analysis of three service statistics data (number of: contraceptive commodities distributed to clients, family planning service visits, and current contraceptive users) from 22 countries in Africa and Asia, it was observed that none of these three data were accurate to be used when comparing with United Nations Population Division World Contraceptive Use annual mCPR estimates as the “gold standard.”

Although service statistics data for family planning are insufficient accuracy to use stand-alone like survey data, data on contraceptive commodities distributed to clients, family planning service visits, and current users can be used accurately when combined with survey data in FPET tools to estimates the trends of annual mCPR between the years of surveys¹⁴.

Limitation of study

This study is only secondary data analysis, therefore, the reasons behind the inaccuracy of service statistics data could not explain. It covers only Myanmar data so detail results might be limitation to be applied for other countries context although it is consistent with findings of analysis in 22 countries which is service statistics data were not sufficient in accuracy to be used stand alone for annual tracking.

Recommendations

In order to get valid information for annual monitoring of Myanmar’s family planning program, Track20’s FPET tool is the method of choice for now. Although service statistics data are insufficient accuracy to use stand-alone like survey data, valid and qualified service statistics data could be used through in-cooperating into FPET tools together with survey data to provide more precise estimation of annual mCPR trend. Therefore, it is important to get valid information of some important service statistic data related with family planning such as family planning service visits, current users and contraceptive commodities distributed to clients.

In future, there is a need to:

- strengthen of HMIS data through reviewing the methodology of data collection for family planning information with further primary research and validating the data quality;
- establish the Family Planning register/record and integrate into electronic HMIS/DHIS 2;
- strengthen the LMIS for reproductive health commodity in terms of data quality as well as area coverage.

Data availability

Underlying data

The Demographic and Health Surveys dataset analyzed during the current study (Myanmar 2015–16) is available in the MEASURE DHS repository (http://www.measuredhs.com). Access to the dataset requires registration, and is granted to those that wish to use the data for legitimate research purposes. A guide for how to apply for dataset access is available at: https://dhsprogram.com/data/Access-Instructions.cfm.

The HMIS data were requested from the HMIS unit of Department of Public Health, Ministry of Health and Sports, Myanmar, available at https://mm.dhis2.net/hmis/dhis-web-commons/security/login.action. Access to this data is restricted to protect the identities of the subjects; researchers wishing to apply for access should send an email to the Deputy Director General of HMIS, Dr Thet Thet Mu at thetthetmu@mohs.gov.mm, including justification for why access should be granted.

The contraceptive commodity consumption data for LMIS for RH commodities were requested from the Maternal and Reproductive Health Unit of Department of Public Health, Ministry of Health and Sports, Myanmar and John Snow International (JSI); an international organization that are providing technical support for LMIS, also supported in accessing the data. Data can be visualized here. To protect the identities of the subjects, those wishing to gain access to the data should submit an official request to the Director of Maternal and Reproductive Health Division of Ministry of Health and Sports, Myanmar, via ayechum_yi@mm.jsi.com or khaingnwetin@gmail.com, including justification for why access should be granted.

Acknowledgments

I would like to express my sincere gratitude to the Track20 project, including facilitators, for giving the opportunity to conduct this analysis and kind support throughout the analysis and report writing. I also thanks to the Ministry of Health and Sports, Myanmar, for approval to carry out this study and to use the data of HMIS and LMIS.

Faculty Opinions recommended

References

1. Ministry of Health and Sports: Five-Year Strategic Plan for Reproductive Health (2014-2018). Nay Pyi Taw, The Republic of the Union of Myanmar. 2013. Reference Source
2. Department of Public Health: Costed Implementation to Meet FP2020 Commitments Myanmar 2014. Nay Pyi Taw, The Republic of the Union of Myanmar. 2015. Reference Source
3. Global Health Observatory data repository | By category | Maternal mortality - Data by country. In: WHO. Accessed 12 Aug 2018. Reference Source
4. Department of Population, Ministry of Immigration and Population: The 2014 Myanmar Population and Housing Census. Department of Population, Ministry of Immigration and Population, Nay Pyi Taw. 2015. Reference Source
5. Ministry of Health and Sports: Myanmar National Health Plan (2017-2021). Nay Pyi Taw, The Republic of the Union of Myanmar. 2016. Reference Source
6. Department of Population and UNFPA Myanmar: Country Report on 2007 Fertility and Reproductive Health Survey. 2009. Reference Source
7. Ministry of Health and Sports: Myanmar Demographic and Health Survey 2015-16. Nay Pyi Taw, Myanmar ICF, Ministry of Health and Sports, Nay Pyi Taw, The Republic of the Union of Myanmar. 2017. Reference Source
8. Ministry of Health and Sports: Health Management Information System data dictionary. 2018.
9. Maternal and Reproductive Health: Reproductive Health Commodity Logistic System report. 2017.
10. Maternal and Reproductive Health: Family Planning Estimation Tool results. 2017.
11. Ministry of Health and Sports: Health Management Information System data. 2015–17.
12. Ministry of Health and Sports: Reproductive Health Commodity Logistic System data. 2017.
13. Starbird E, Norton M, Marcus R: Investing in Family Planning: Key to Achieving the Sustainable Development Goals. Glob Health Sci Pract. 2016; 4(2): 191–210. PubMed Abstract | Publisher Full Text | Free Full Text
14. Magnani RJ, Ross J, Williamson J, et al.: Can Family Planning Service Statistics Be Used to Track Population-Level Outcomes? Glob Health Sci Pract. 2018; 6(1): 93–102. PubMed Abstract | Publisher Full Text | Free Full Text

Comments on this article Comments (0)

Version 3

VERSION 3 PUBLISHED 18 Sep 2019

Author details Author details

¹ Department of Public Health, Ministry of Health and Sports, Myanmar, Nay Pyi Taw, 15011, Myanmar
² Track20, Avenir Health, Washington DC, USA
³ Track20, Avenir Health, Connecticut, USA

Khaing Nwe Tin
Roles: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Jessica Williamson
Roles: Methodology, Visualization, Writing – Review & Editing

Emily Sonneveldt
Roles: Resources, Supervision, Validation

Competing interests

No competing interests were disclosed.

Grant information

Publication of this research was funded by the Bill and Melinda Gates Foundation (OPP1181398).
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Article Versions (3)

version 3

Revised

Published: 18 Dec 2019, 3:1541

https://doi.org/10.12688/gatesopenres.13012.3

version 2

Revised

Published: 10 Dec 2019, 3:1541

https://doi.org/10.12688/gatesopenres.13012.2

version 1

Published: 18 Sep 2019, 3:1541

https://doi.org/10.12688/gatesopenres.13012.1

© 2019 Nwe Tin K et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
Gates Open Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Nwe Tin K, Williamson J and Sonneveldt E. Setting the stage for strengthened annual monitoring of Family Planning program performance at the National and Subnational level in Myanmar [version 3; peer review: 2 approved]. Gates Open Res 2019, 3:1541 (https://doi.org/10.12688/gatesopenres.13012.3)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 3

VERSION 3

PUBLISHED 18 Dec 2019

Revised

Views

Reviewer Report 18 Dec 2019

Janine Barden O'Fallon, Carolina Population Center, Maternal & Child Health Department, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

Approved

https://doi.org/10.21956/gatesopenres.14256.r28386

Thank you for making these revisions. I had some difficulty ... Continue reading

CITE

Report a concern

Author Response 18 Dec 2019

Khaing Nwe Tin, Department of Public Health, Ministry of Health and Sports, Myanmar, Nay Pyi Taw, 15011, Myanmar

18 Dec 2019

Author Response

Dear Janine,

Thank you for your comments. Please allow me to respond for the limitation section. As this study is secondary data analysis, it could not explain the definite reasons why ... Continue reading Dear Janine,

Thank you for your comments. Please allow me to respond for the limitation section. As this study is secondary data analysis, it could not explain the definite reasons why the HMIS data are inaccurate and much higher than FPET and DHS.

Another point is that although the finding in this study (service statistics data are inaccurate to use stand alone to monitor FP annually) is consistent with the analysis of 22 countries (Ref), this study was used only Myanmar data so the application of the findings might be limited to the other specific country context.

Ref: Magnani RJ, Ross J, Williamson J, et al.: Can Family Planning Service Statistics Be Used to Track Population-Level Outcomes? Glob Health Sci Pract. 2018; 6(1): 93–102. PubMed Abstract | Publisher Full Text | Free Full Text

Thanks again for your time and best regards,

Khaing
Dear Janine,

Thank you for your comments. Please allow me to respond for the limitation section. As this study is secondary data analysis, it could not explain the definite reasons why the HMIS data are inaccurate and much higher than FPET and DHS.

Another point is that although the finding in this study (service statistics data are inaccurate to use stand alone to monitor FP annually) is consistent with the analysis of 22 countries (Ref), this study was used only Myanmar data so the application of the findings might be limited to the other specific country context.

Ref: Magnani RJ, Ross J, Williamson J, et al.: Can Family Planning Service Statistics Be Used to Track Population-Level Outcomes? Glob Health Sci Pract. 2018; 6(1): 93–102. PubMed Abstract | Publisher Full Text | Free Full Text

Thanks again for your time and best regards,

Khaing
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 18 Dec 2019

Khaing Nwe Tin, Department of Public Health, Ministry of Health and Sports, Myanmar, Nay Pyi Taw, 15011, Myanmar

18 Dec 2019

Author Response

Dear Janine,

Thank you for your comments. Please allow me to respond for the limitation section. As this study is secondary data analysis, it could not explain the definite reasons why ... Continue reading Dear Janine,

Thank you for your comments. Please allow me to respond for the limitation section. As this study is secondary data analysis, it could not explain the definite reasons why the HMIS data are inaccurate and much higher than FPET and DHS.

Another point is that although the finding in this study (service statistics data are inaccurate to use stand alone to monitor FP annually) is consistent with the analysis of 22 countries (Ref), this study was used only Myanmar data so the application of the findings might be limited to the other specific country context.

Ref: Magnani RJ, Ross J, Williamson J, et al.: Can Family Planning Service Statistics Be Used to Track Population-Level Outcomes? Glob Health Sci Pract. 2018; 6(1): 93–102. PubMed Abstract | Publisher Full Text | Free Full Text

Thanks again for your time and best regards,

Khaing
Dear Janine,

Thank you for your comments. Please allow me to respond for the limitation section. As this study is secondary data analysis, it could not explain the definite reasons why the HMIS data are inaccurate and much higher than FPET and DHS.

Another point is that although the finding in this study (service statistics data are inaccurate to use stand alone to monitor FP annually) is consistent with the analysis of 22 countries (Ref), this study was used only Myanmar data so the application of the findings might be limited to the other specific country context.

Ref: Magnani RJ, Ross J, Williamson J, et al.: Can Family Planning Service Statistics Be Used to Track Population-Level Outcomes? Glob Health Sci Pract. 2018; 6(1): 93–102. PubMed Abstract | Publisher Full Text | Free Full Text

Thanks again for your time and best regards,

Khaing
Competing Interests: No competing interests were disclosed. Close
Report a concern

Version 2

VERSION 2

PUBLISHED 10 Dec 2019

Revised

Views

Reviewer Report 12 Dec 2019

James F. Phillips, Heilbrunn Department of Population and Family Health, Mailman School of Public Health, Columbia University, New York City, NY, USA

Approved

https://doi.org/10.21956/gatesopenres.14246.r28354

I have no further comments. The authors have addressed ... Continue reading

CITE

Report a concern

Author Response 12 Dec 2019

Khaing Nwe Tin, Department of Public Health, Ministry of Health and Sports, Myanmar, Nay Pyi Taw, 15011, Myanmar

12 Dec 2019

Author Response

Dear Phillips,

On behalf of our team, I would like to thank you for your kind approval for our manuscript.

Best regards,
Khaing
(corresponding author)
Competing Interests: No competing interests were disclosed.
Dear Phillips,

On behalf of our team, I would like to thank you for your kind approval for our manuscript.

Best regards,
Khaing
(corresponding author)
Dear Phillips,

On behalf of our team, I would like to thank you for your kind approval for our manuscript.

Best regards,
Khaing
(corresponding author)
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 12 Dec 2019

Khaing Nwe Tin, Department of Public Health, Ministry of Health and Sports, Myanmar, Nay Pyi Taw, 15011, Myanmar

12 Dec 2019

Author Response

Dear Phillips,

On behalf of our team, I would like to thank you for your kind approval for our manuscript.

Best regards,
Khaing
(corresponding author)
Competing Interests: No competing interests were disclosed.
Dear Phillips,

On behalf of our team, I would like to thank you for your kind approval for our manuscript.

Best regards,
Khaing
(corresponding author)
Dear Phillips,

On behalf of our team, I would like to thank you for your kind approval for our manuscript.

Best regards,
Khaing
(corresponding author)
Competing Interests: No competing interests were disclosed. Close
Report a concern

Version 1

VERSION 1

PUBLISHED 18 Sep 2019

Views

Reviewer Report 30 Oct 2019

James F. Phillips, Heilbrunn Department of Population and Family Health, Mailman School of Public Health, Columbia University, New York City, NY, USA

Approved with Reservations

https://doi.org/10.21956/gatesopenres.14120.r28054

Overview
This manuscript aims to describe progress with family planning trends in Myanmar. Despite a long legacy of isolation of this country from externally supported programs and research, monitoring systems are in place that provide basic information on trends in contraceptive use. Four data sources are marshalled for this investigation, providing a basis for comparative analysis. A Demographic and Health Survey is available, permitting comparison of Myanmar with other countries. Service statistics data are adjusted with modeling tools to permit assessment of trends at the regional level. Results based on service statistics data contrast with survey results, in that levels of contraceptive use are consistently higher than DHS estimates. The analysis is mainly focused on comparison of data sources and results, with results that lead the authors to conclude that a tool known as “Track20 Family Planning Estimation Tool” is the method of choice.

This manuscript addresses a gap in the regional literature. While much is known about contraceptive use levels and trends in neighboring countries, Myanmar has had relatively little systematic attention. I found the rendition of the history of family planning program development in Myanmar to be helpful, as this information is somewhat difficult to locate elsewhere in the social and demographic literature. Also, the description of data limitations that appears in the Rationale section is useful background information.

Comments on the Methods section
This paper is more focused on the comparison of tools rather than the analysis of family planning, per se. A tool referred to as FPET and another tool LMIS are compared with DHS data, but the methods section is limited to a discussion of the logic of comparison rather than the logic that underlies each tool. The key objective concerns the need for a credible basis for assessing annual progress with family planning program functioning. It is critical to the paper for the reader to understand how results could be affected by methodological differences in the tools employed. Therefore, it is important for the text to clarify how these assessment tools work, either through citation of explanatory articles or through descriptive analysis of the methodologies that they employ. However, tools are cited in the Rationale section without citations or clarification of how they are similar and how they differ. As a consequence, the reader is left to wonder if the differences portrayed are actually significant. This rather simple addition to the paper would substantially strengthen the presentation. This reviewer recommends application of statistical procedures that compare cross-sectional time series data. By employing a method for statistical comparison, the conclusions and recommendations that are reached would have the credibility that is associated with the comparison of trends with confidence intervals¹.

Comparisons of method mix suggest that different tools produce similar use profiles but contrasting levels of use with health management information system (HMIS) data generating higher use estimates than corresponding DHS data. Under-estimation of the denominators for HMIS or over-estimation of continuity of use may explain these estimation differences. How such differences could arise is not adequately discussed.

The Recommendations section
The paper reads as if the different results that are observed are the end of the research process rather than an indication that further research is needed. Recommendations to strengthen the HMIS through reviewing the methodology is not likely to have much practical meaning. Nor are the other recommendations actionable. Exhortations to strengthen something in general does not convey practical guidance on what the program should actually do.

This reviewer has not had access to the data, but an illustrative example merits consideration by the authors, either by proposing some other adjustment procedure or by pursuing a version of what I recommend. Consider the following: At one point in the paper, the DHS is alluded to as the “gold standard” for mCPR estimation. The FPET and DHS estimates are similar (Figure 1), but HMIS data appear to be over-estimating the mCPR. If the DHS is considered the gold standard, then a plausible proposition is that its data can be used to develop areal adjustment factors. A regression analysis, with DHS data employed as the dependent variable, HMIS data as an independent variable, with direct and interaction effects for population characteristics and locality indicators could be used to develop regression weights that estimate the discrepancies associated with a set of indicators. The regression weights would define adjustment factors for correcting data to a DHS standard. The correspondence of FPET with the DHS invites an approach that would use the FPET as the “silver standard” for HMIS adjustment. While the paper might not take this up as a technical task, it should include discussion of methods of data adjustment that results appear to invite. This would be a more cogent set of recommendations than the text that now appears in the Recommendations section.

The figures
Figures 2 and 3 are hard to read and needlessly complex to interpret. There are many regions with pronounced variance between regions. Are there take home messages from these data that could be summarized more simply? For example, it is possible that the magnitude of discrepancies is associated with the level of the estimated mCPR. If HMIS level were on the Y axis and FPET were the X axis a simple regression line would simulate correspondence between the data sources, and the scatter around that line would show the pattern of discrepancy by level of mCPR.

The title of Figure 3 is confusing. I think that Figure 3 could be dropped.
Figure 4 is a largely empty depiction of a few data points. This could be replaced with a table.
Figure 5 is visually confusing. For some reason, data correspondence in Chin is very different from the other regions. Coloration might help, but tabular data might be more interpretable to a reader.

Minor comments and suggestions
The text of this paper begins with a reference to the maternal mortality rate, but the paper is on a different topic. This reviewer suggests that the authors open the paper with a statement that is a more direct reference to topic of the article and the lack of systematic attention to family planning research in Myanmar and the rationale for the paper. At some point in the discussion, reference to the relationship of family planning with the maternal mortality rate would be informative.

Is the work clearly and accurately presented and does it cite the current literature?

No
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

No
If applicable, is the statistical analysis and its interpretation appropriate?

Partly
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Partly

References

1. Chetty V: Pooling of Time Series and Cross Section Data. Econometrica. 1968; 36 (2). Publisher Full Text

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Demography and health systems research

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Author Response 10 Dec 2019

Khaing Nwe Tin, Department of Public Health, Ministry of Health and Sports, Myanmar, Nay Pyi Taw, 15011, Myanmar

10 Dec 2019

Author Response

Dear Phillips,

Thank you so much for your comments and suggestions and I would like to response as follows.

Comments on the Methods section

This paper is ... Continue reading Dear Phillips,

Thank you so much for your comments and suggestions and I would like to response as follows.

Comments on the Methods section

This paper is more focused on the comparison of tools rather than the analysis of family planning, per se. A tool referred to as FPET and another tool LMIS are compared with DHS data, but the methods section is limited to a discussion of the logic of comparison rather than the logic that underlies each tool. The key objective concerns the need for a credible basis for assessing annual progress with family planning program functioning. It is critical to the paper for the reader to understand how results could be affected by methodological differences in the tools employed. Therefore, it is important for the text to clarify how these assessment tools works, either through citation of explanatory articles or through descriptive analysis of the methodologies that they employ. However, tools are cited in the Rationale section without citations or clarification of how they are similar and how they differ. As a consequence, the reader is left to wonder if the differences portrayed are actually significant. This rather simple addition to the paper would substantially strengthen the presentation. This reviewer recommends application of statistical procedures that compare cross-sectional time series data. By employing a method for statistical comparison, the conclusions and recommendations that are reached would have the credibility that is associated with the comparison of trends with confidence intervals.

Thank you for your comments and allow me to respond as follows;

Actually, the main objective of this study is focused to compare the different data source for FP indicators to select the most reliable one for annual monitoring between the years of survey (DHS). As there is no other annual survey in Myanmar, the data from FPET is currently the most reliable source to monitor as it gives the same results as DHS. But, the other routine services like HMIS and LMIS are available, therefore, I would like to test the data whether service statistics data are reliable to use or not for annual monitoring (especially for State and Regional level) and to get the evidence which one is the most viable data sources (FPET/HMIS/LMIS) in Myanmar.
To be compared, four data sources (DHS, FPET, HMIS and LMIS) were used. (Previously I wrote as “compared” rather than “used” that mislead to readers). For annual progress of mCPR, services statistics data (HMIS and LMIS) were compared along with 95% CI of FPET for their accuracy (because FPET is most reliable source for the years between DHS) for three consecutive years. If the service statistics data points were fell within 95% CI of FPET, it was assumed as accurate and check how many years of consistency between them for each State/Regions (Figure 2 and 3).
For national comparison of mCPR between HMIS and FPET including 95%CI, three years data points were plotted as trend and DHS was put as one point in 2016 (Figure 3). It also shows the HMIS data were much over estimates of FPET and DHS.
When HMIS data were compared with FPET, the direct estimates of mCPR from HMIS were used. However, LMIS could not provide the mCPR to compare with FPET. LMIS can provide only commodity consumption data. Therefore, we have to use EMU (Estimated method use) excel tool which is developed by Track 20 team and it can provide EMU (Estimated method use-a proxy of mCPR) by converting commodity consumption data from LMIS.
The FPET and EMU tools are not comparable and EMU results can be used as input for FPET if we want to incorporate service statistics data into FPET in addition to survey data. But it is not the case of this study as Myanmar FPET estimates use only survey data (due to poor data validity of service statistics data). Here EMU tool was used to provide EMU (Estimated method use-a proxy of mCPR) by converting of commodity consumption data. The links of these tools are also added in revise version.
For comparing the pattern of method mix, FPET cannot provide that data, we used DHS data for only that year of survey (2016) and compared with HMIS data. So, actually it doesn’t compare the two tools (FPET and LMIS tool) and compare the service statistics data (HMIS and LMIS through EMU tool) with FPET for mCPR progress and HMIS with DHS for method mix.
To be more clear, I have tried to edit the methodology section as follows and added text are made as italic;

Methods
Data sources
In order to understand what data may best serve annual monitoring of the performance of the family planning program in Myanmar, both at the national and state/regional level, four data sources of modern contraceptive use:
< >Modelled mCPR estimates (and confidence intervals) from Track20’s FPET, based on nationally and state/regionally representative surveys for Myanmar: Myanmar DHS (2015-16), Multiple Indicator Cluster Survey (2010), Fertility and Reproductive Health Survey (1991, 1997, 2001 & 2007). (7). (While FPET tool can be incooperated with both survey and service statistics data, FPET run for Myanmar could not use the service statistics data due to data validity.)mCPR and Method-specific prevalence from the 2015-16 Myanmar DHS (6).mCPR estimates and contraceptive method prevalence from Myanmar’s HMIS system (8) based on local annual census conducted by midwives on contraceptive use among married women in their catchment areas; which is different from other routine HMIS indicators and collected once a year at the end of calendar year for the information of contraceptive use including type of methods used, and mCPR from Estimates of modern method use (EMU) tool based on contraceptive commodity consumption data from Myanmar’s LMIS system (9); were used to compare (Table 1). (While FPET can be incorporated service statistics data, it could not be put the direct estimates of mCPR, other commodity or visit data from service statistics. EMU excel tool can convert service statistics data to Estimated method use (which is a proxy for mCPR) to be can be used as input in the Family Planning Estimation Tool (FPET). EMU tool was developed by Track 20 team and available at its website http://www.track20.org/pages/our_work/innovative_tools/SS_to_EMU_tool.php. Here, EMU tool was used to provide EMU (Estimated method use-a proxy of mCPR) by converting of commodity consumption data.) strengthen of HMIS data through reviewing the methodology of data collection for family planning information with further primary research and validating the data quality;establish the Family Planning register/record and integrate into electronic HMIS/DHIS;strengthen the LMIS for reproductive health commodity in terms of data quality as well as area coverage.

Comments

This reviewer has not had access to the data, but an illustrative example merits consideration by the authors, either by proposing some other adjustment procedure or by pursuing a version of what I recommend. Consider the following: At one point in the paper, the DHS is alluded to as the “gold standard” for mCPR estimation. The FPET and DHS estimates are similar (Figure 1), but HMIS data appear to be over-estimating the mCPR. If the DHS is considered the gold standard, then a plausible proposition is that its data can be used to develop areal adjustment factors. A regression analysis, with DHS data employed as the dependent variable, HMIS data as an independent variable, with direct and interaction effects for population characteristics and locality indicators could be used to develop regression weights that estimate the discrepancies associated with a set of indicators. The regression weights would define adjustment factors for correcting data to a DHS standard. The correspondence of FPET with the DHS invites an approach that would use the FPET as the “silver standard” for HMIS adjustment. While the paper might not take this up as a technical task, it should include discussion of methods of data adjustment that results appear to invite. This would be a more cogent set of recommendations than the text that now appears in the Recommendations section.

Actually, FPET tool is based on Bayesian, hierarchical model that fits curves to historical data (all available survey and service statistics). The model fits a logistic growth curve to CPR data for all methods to determine the long term trend in contraceptive use and adds a time-series model with autocorrelation to capture country-specific deviations around the long term trend. Since FPET gives almost the same result as DHS, it is assumed as the most reliable data source for annual estimates so far to compare the service statistics data (HMIS and LMIS) in this study. I hope it has already incorporated the adjustment for comparison of HMIS data with DHS. Also, I will add about this in methodology section in my revise version.

Comments for the figures

Figures 2 and 3 are hard to read and needlessly complex to interpret. There are many regions with pronounced variance between regions. Are there take home messages from these data that could be summarized more simply? For example, it is possible that the magnitude of discrepancies is associated with the level of the estimated mCPR. If HMIS level were on the Y axis and FPET were the X axis a simple regression line would simulate correspondence between the data sources, and the scatter around that line would show the pattern of discrepancy by level of mCPR. The title of Figure 3 is confusing. I think that Figure 3 could be dropped.

In figure 2 and 3, it shows year of consistency for mCPR between the HMIS and FPET estimates including 95%CI for each 17 State/Regions for three consecutive year’s data. The bar shows the HMIS data and FPET data points including upper and lower border (95% CI) are shown on that bar for each three years for 17 State/Regions. We can see the consistency that whether HMIS data (top of the bar) are fell within these upper and lower borders of FPET. For me, it is not easy to draw the regression line for two data sources of three years data points for each 17 State/Regions in one picture. Also, I found that there is no relation between the consistency and magnitude of value. The key messages are there are only two States (Chin and Kayin) for consistency in all three years (Figure 2) and some are two years and some are only one year consistent. About half of State/Regions show no consistency at all (Figure 3) for three years. It means that direct estimates of mCPR from HMIS are not consistency with FPET and not accurate to be used for annual monitoring. I will change the title for Figure 3 as “Comparing the HMIS and FPET results of mCPR by regions of none of the year consistent estimates” to be more clear as previously mentioned that “ by regions of no consistent estimates”.

Comments: Figure 4 is a largely empty depiction of a few data points. This could be replaced with a table.

Thank you for your advice. Actually, I would like to show that the comparison of HMIS and FPET (with 95% CI) for national mCPR with trend (three years data points) and rate of progress. It highlights HMIS data were overestimates of FPET for three years (even over the upper border in 2016 and 2017) and DHS as well in 2016. So, I think it could not easy for readers to see these points from table.

Comments: Figure 5 is visually confusing. For some reason, data correspondence in Chin is very different from the other regions. Coloration might help, but tabular data might be more interpretable to a reader.

In Figure 5, it was aimed to show the comparison of different contraceptive method mix (Injection, pills, IUD, implants, condom and female sterilization) prevalence data of HMIS and DHIS (2016) as well as the comparison of national level and state/regional level (Chin and Kayin) where there were different patterns of national level. The key messages are the method mix pattern of HMIS and DHS are quite similar except female sterilization. And while injection was the highest proportion at national level (both HIS and DHS), high proportions of long term methods and female sterilization were found at Chin from DHS (quite different with HMIS) and pills use was the highest in Kayin in DHS. It means that HMIS data were quite similar with DHS for method mix at national and other State/regions except Chin and Kayin. So, I think it might be not easy for readers to interpret if data were shown table.

Minor comments and suggestions!
The text of this paper begins with a reference to the maternal mortality rate, but the paper is on a different topic. This reviewer suggests that the authors open the paper with a statement that is a more direct reference to topic of the article and the lack of systematic attention to family planning research in Myanmar and the rationale for the paper. At some point in the discussion, reference to the relationship of family planning with the maternal mortality rate would be informative.

Thank you for your suggestions. I edited the introduction a lot with starting of importance of family planning, about Myanmar FP commitment, Myanmar FP program implementation to achieve FP 2020 commitment and reduce MMR, then how importance of monitoring the progress and current data source. And some text “Also, lack of studies for systematic analysis of the family planning data source called to do this study to set the annual family planning monitoring system in Myanmar” has added as the last sentence in rationale.

In discussion part, I have added the following text at the beginning.

“Access to FP contributes up to a 44% reduction in maternal deaths and a 21% reduction of deaths in children under age 5 (Ref), therefore, Myanmar needs to endevaour strong efforts to increase access to family planning services to tackle the high MMR issue. In the meantime, setting the systematic approach for continuous monitoring is important to provide the valuable information to be considered for future strategies”

(Ref: Starbird E, Norton M, Marcus R Investing in Family Planning: Key to Achieving the Sustainable Development Goals. doi: 10.9745/GHSP-D-15-00374)
Also, some editions for another reviewer have been incorporated in my revised version.

Thanks again and best regards,
Khaing

Dear Phillips,

Thank you so much for your comments and suggestions and I would like to response as follows.

Comments on the Methods section

This paper is more focused on the comparison of tools rather than the analysis of family planning, per se. A tool referred to as FPET and another tool LMIS are compared with DHS data, but the methods section is limited to a discussion of the logic of comparison rather than the logic that underlies each tool. The key objective concerns the need for a credible basis for assessing annual progress with family planning program functioning. It is critical to the paper for the reader to understand how results could be affected by methodological differences in the tools employed. Therefore, it is important for the text to clarify how these assessment tools works, either through citation of explanatory articles or through descriptive analysis of the methodologies that they employ. However, tools are cited in the Rationale section without citations or clarification of how they are similar and how they differ. As a consequence, the reader is left to wonder if the differences portrayed are actually significant. This rather simple addition to the paper would substantially strengthen the presentation. This reviewer recommends application of statistical procedures that compare cross-sectional time series data. By employing a method for statistical comparison, the conclusions and recommendations that are reached would have the credibility that is associated with the comparison of trends with confidence intervals.

Thank you for your comments and allow me to respond as follows;

Actually, the main objective of this study is focused to compare the different data source for FP indicators to select the most reliable one for annual monitoring between the years of survey (DHS). As there is no other annual survey in Myanmar, the data from FPET is currently the most reliable source to monitor as it gives the same results as DHS. But, the other routine services like HMIS and LMIS are available, therefore, I would like to test the data whether service statistics data are reliable to use or not for annual monitoring (especially for State and Regional level) and to get the evidence which one is the most viable data sources (FPET/HMIS/LMIS) in Myanmar.
To be compared, four data sources (DHS, FPET, HMIS and LMIS) were used. (Previously I wrote as “compared” rather than “used” that mislead to readers). For annual progress of mCPR, services statistics data (HMIS and LMIS) were compared along with 95% CI of FPET for their accuracy (because FPET is most reliable source for the years between DHS) for three consecutive years. If the service statistics data points were fell within 95% CI of FPET, it was assumed as accurate and check how many years of consistency between them for each State/Regions (Figure 2 and 3).
For national comparison of mCPR between HMIS and FPET including 95%CI, three years data points were plotted as trend and DHS was put as one point in 2016 (Figure 3). It also shows the HMIS data were much over estimates of FPET and DHS.
When HMIS data were compared with FPET, the direct estimates of mCPR from HMIS were used. However, LMIS could not provide the mCPR to compare with FPET. LMIS can provide only commodity consumption data. Therefore, we have to use EMU (Estimated method use) excel tool which is developed by Track 20 team and it can provide EMU (Estimated method use-a proxy of mCPR) by converting commodity consumption data from LMIS.
The FPET and EMU tools are not comparable and EMU results can be used as input for FPET if we want to incorporate service statistics data into FPET in addition to survey data. But it is not the case of this study as Myanmar FPET estimates use only survey data (due to poor data validity of service statistics data). Here EMU tool was used to provide EMU (Estimated method use-a proxy of mCPR) by converting of commodity consumption data. The links of these tools are also added in revise version.
For comparing the pattern of method mix, FPET cannot provide that data, we used DHS data for only that year of survey (2016) and compared with HMIS data. So, actually it doesn’t compare the two tools (FPET and LMIS tool) and compare the service statistics data (HMIS and LMIS through EMU tool) with FPET for mCPR progress and HMIS with DHS for method mix.
To be more clear, I have tried to edit the methodology section as follows and added text are made as italic;

Methods
Data sources
In order to understand what data may best serve annual monitoring of the performance of the family planning program in Myanmar, both at the national and state/regional level, four data sources of modern contraceptive use:
< >Modelled mCPR estimates (and confidence intervals) from Track20’s FPET, based on nationally and state/regionally representative surveys for Myanmar: Myanmar DHS (2015-16), Multiple Indicator Cluster Survey (2010), Fertility and Reproductive Health Survey (1991, 1997, 2001 & 2007). (7). (While FPET tool can be incooperated with both survey and service statistics data, FPET run for Myanmar could not use the service statistics data due to data validity.)mCPR and Method-specific prevalence from the 2015-16 Myanmar DHS (6).mCPR estimates and contraceptive method prevalence from Myanmar’s HMIS system (8) based on local annual census conducted by midwives on contraceptive use among married women in their catchment areas; which is different from other routine HMIS indicators and collected once a year at the end of calendar year for the information of contraceptive use including type of methods used, and mCPR from Estimates of modern method use (EMU) tool based on contraceptive commodity consumption data from Myanmar’s LMIS system (9); were used to compare (Table 1). (While FPET can be incorporated service statistics data, it could not be put the direct estimates of mCPR, other commodity or visit data from service statistics. EMU excel tool can convert service statistics data to Estimated method use (which is a proxy for mCPR) to be can be used as input in the Family Planning Estimation Tool (FPET). EMU tool was developed by Track 20 team and available at its website http://www.track20.org/pages/our_work/innovative_tools/SS_to_EMU_tool.php. Here, EMU tool was used to provide EMU (Estimated method use-a proxy of mCPR) by converting of commodity consumption data.) strengthen of HMIS data through reviewing the methodology of data collection for family planning information with further primary research and validating the data quality;establish the Family Planning register/record and integrate into electronic HMIS/DHIS;strengthen the LMIS for reproductive health commodity in terms of data quality as well as area coverage.

Comments

This reviewer has not had access to the data, but an illustrative example merits consideration by the authors, either by proposing some other adjustment procedure or by pursuing a version of what I recommend. Consider the following: At one point in the paper, the DHS is alluded to as the “gold standard” for mCPR estimation. The FPET and DHS estimates are similar (Figure 1), but HMIS data appear to be over-estimating the mCPR. If the DHS is considered the gold standard, then a plausible proposition is that its data can be used to develop areal adjustment factors. A regression analysis, with DHS data employed as the dependent variable, HMIS data as an independent variable, with direct and interaction effects for population characteristics and locality indicators could be used to develop regression weights that estimate the discrepancies associated with a set of indicators. The regression weights would define adjustment factors for correcting data to a DHS standard. The correspondence of FPET with the DHS invites an approach that would use the FPET as the “silver standard” for HMIS adjustment. While the paper might not take this up as a technical task, it should include discussion of methods of data adjustment that results appear to invite. This would be a more cogent set of recommendations than the text that now appears in the Recommendations section.

Actually, FPET tool is based on Bayesian, hierarchical model that fits curves to historical data (all available survey and service statistics). The model fits a logistic growth curve to CPR data for all methods to determine the long term trend in contraceptive use and adds a time-series model with autocorrelation to capture country-specific deviations around the long term trend. Since FPET gives almost the same result as DHS, it is assumed as the most reliable data source for annual estimates so far to compare the service statistics data (HMIS and LMIS) in this study. I hope it has already incorporated the adjustment for comparison of HMIS data with DHS. Also, I will add about this in methodology section in my revise version.

Comments for the figures

Figures 2 and 3 are hard to read and needlessly complex to interpret. There are many regions with pronounced variance between regions. Are there take home messages from these data that could be summarized more simply? For example, it is possible that the magnitude of discrepancies is associated with the level of the estimated mCPR. If HMIS level were on the Y axis and FPET were the X axis a simple regression line would simulate correspondence between the data sources, and the scatter around that line would show the pattern of discrepancy by level of mCPR. The title of Figure 3 is confusing. I think that Figure 3 could be dropped.

In figure 2 and 3, it shows year of consistency for mCPR between the HMIS and FPET estimates including 95%CI for each 17 State/Regions for three consecutive year’s data. The bar shows the HMIS data and FPET data points including upper and lower border (95% CI) are shown on that bar for each three years for 17 State/Regions. We can see the consistency that whether HMIS data (top of the bar) are fell within these upper and lower borders of FPET. For me, it is not easy to draw the regression line for two data sources of three years data points for each 17 State/Regions in one picture. Also, I found that there is no relation between the consistency and magnitude of value. The key messages are there are only two States (Chin and Kayin) for consistency in all three years (Figure 2) and some are two years and some are only one year consistent. About half of State/Regions show no consistency at all (Figure 3) for three years. It means that direct estimates of mCPR from HMIS are not consistency with FPET and not accurate to be used for annual monitoring. I will change the title for Figure 3 as “Comparing the HMIS and FPET results of mCPR by regions of none of the year consistent estimates” to be more clear as previously mentioned that “ by regions of no consistent estimates”.

Comments: Figure 4 is a largely empty depiction of a few data points. This could be replaced with a table.

Thank you for your advice. Actually, I would like to show that the comparison of HMIS and FPET (with 95% CI) for national mCPR with trend (three years data points) and rate of progress. It highlights HMIS data were overestimates of FPET for three years (even over the upper border in 2016 and 2017) and DHS as well in 2016. So, I think it could not easy for readers to see these points from table.

Comments: Figure 5 is visually confusing. For some reason, data correspondence in Chin is very different from the other regions. Coloration might help, but tabular data might be more interpretable to a reader.

In Figure 5, it was aimed to show the comparison of different contraceptive method mix (Injection, pills, IUD, implants, condom and female sterilization) prevalence data of HMIS and DHIS (2016) as well as the comparison of national level and state/regional level (Chin and Kayin) where there were different patterns of national level. The key messages are the method mix pattern of HMIS and DHS are quite similar except female sterilization. And while injection was the highest proportion at national level (both HIS and DHS), high proportions of long term methods and female sterilization were found at Chin from DHS (quite different with HMIS) and pills use was the highest in Kayin in DHS. It means that HMIS data were quite similar with DHS for method mix at national and other State/regions except Chin and Kayin. So, I think it might be not easy for readers to interpret if data were shown table.

Minor comments and suggestions!
The text of this paper begins with a reference to the maternal mortality rate, but the paper is on a different topic. This reviewer suggests that the authors open the paper with a statement that is a more direct reference to topic of the article and the lack of systematic attention to family planning research in Myanmar and the rationale for the paper. At some point in the discussion, reference to the relationship of family planning with the maternal mortality rate would be informative.

Thank you for your suggestions. I edited the introduction a lot with starting of importance of family planning, about Myanmar FP commitment, Myanmar FP program implementation to achieve FP 2020 commitment and reduce MMR, then how importance of monitoring the progress and current data source. And some text “Also, lack of studies for systematic analysis of the family planning data source called to do this study to set the annual family planning monitoring system in Myanmar” has added as the last sentence in rationale.

In discussion part, I have added the following text at the beginning.

“Access to FP contributes up to a 44% reduction in maternal deaths and a 21% reduction of deaths in children under age 5 (Ref), therefore, Myanmar needs to endevaour strong efforts to increase access to family planning services to tackle the high MMR issue. In the meantime, setting the systematic approach for continuous monitoring is important to provide the valuable information to be considered for future strategies”

(Ref: Starbird E, Norton M, Marcus R Investing in Family Planning: Key to Achieving the Sustainable Development Goals. doi: 10.9745/GHSP-D-15-00374)
Also, some editions for another reviewer have been incorporated in my revised version.

Thanks again and best regards,
Khaing

Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 10 Dec 2019

Khaing Nwe Tin, Department of Public Health, Ministry of Health and Sports, Myanmar, Nay Pyi Taw, 15011, Myanmar

10 Dec 2019

Author Response

Dear Phillips,

Thank you so much for your comments and suggestions and I would like to response as follows.

Comments on the Methods section

This paper is ... Continue reading Dear Phillips,

Thank you so much for your comments and suggestions and I would like to response as follows.

Comments on the Methods section

This paper is more focused on the comparison of tools rather than the analysis of family planning, per se. A tool referred to as FPET and another tool LMIS are compared with DHS data, but the methods section is limited to a discussion of the logic of comparison rather than the logic that underlies each tool. The key objective concerns the need for a credible basis for assessing annual progress with family planning program functioning. It is critical to the paper for the reader to understand how results could be affected by methodological differences in the tools employed. Therefore, it is important for the text to clarify how these assessment tools works, either through citation of explanatory articles or through descriptive analysis of the methodologies that they employ. However, tools are cited in the Rationale section without citations or clarification of how they are similar and how they differ. As a consequence, the reader is left to wonder if the differences portrayed are actually significant. This rather simple addition to the paper would substantially strengthen the presentation. This reviewer recommends application of statistical procedures that compare cross-sectional time series data. By employing a method for statistical comparison, the conclusions and recommendations that are reached would have the credibility that is associated with the comparison of trends with confidence intervals.

Thank you for your comments and allow me to respond as follows;

Actually, the main objective of this study is focused to compare the different data source for FP indicators to select the most reliable one for annual monitoring between the years of survey (DHS). As there is no other annual survey in Myanmar, the data from FPET is currently the most reliable source to monitor as it gives the same results as DHS. But, the other routine services like HMIS and LMIS are available, therefore, I would like to test the data whether service statistics data are reliable to use or not for annual monitoring (especially for State and Regional level) and to get the evidence which one is the most viable data sources (FPET/HMIS/LMIS) in Myanmar.
To be compared, four data sources (DHS, FPET, HMIS and LMIS) were used. (Previously I wrote as “compared” rather than “used” that mislead to readers). For annual progress of mCPR, services statistics data (HMIS and LMIS) were compared along with 95% CI of FPET for their accuracy (because FPET is most reliable source for the years between DHS) for three consecutive years. If the service statistics data points were fell within 95% CI of FPET, it was assumed as accurate and check how many years of consistency between them for each State/Regions (Figure 2 and 3).
For national comparison of mCPR between HMIS and FPET including 95%CI, three years data points were plotted as trend and DHS was put as one point in 2016 (Figure 3). It also shows the HMIS data were much over estimates of FPET and DHS.
When HMIS data were compared with FPET, the direct estimates of mCPR from HMIS were used. However, LMIS could not provide the mCPR to compare with FPET. LMIS can provide only commodity consumption data. Therefore, we have to use EMU (Estimated method use) excel tool which is developed by Track 20 team and it can provide EMU (Estimated method use-a proxy of mCPR) by converting commodity consumption data from LMIS.
The FPET and EMU tools are not comparable and EMU results can be used as input for FPET if we want to incorporate service statistics data into FPET in addition to survey data. But it is not the case of this study as Myanmar FPET estimates use only survey data (due to poor data validity of service statistics data). Here EMU tool was used to provide EMU (Estimated method use-a proxy of mCPR) by converting of commodity consumption data. The links of these tools are also added in revise version.
For comparing the pattern of method mix, FPET cannot provide that data, we used DHS data for only that year of survey (2016) and compared with HMIS data. So, actually it doesn’t compare the two tools (FPET and LMIS tool) and compare the service statistics data (HMIS and LMIS through EMU tool) with FPET for mCPR progress and HMIS with DHS for method mix.
To be more clear, I have tried to edit the methodology section as follows and added text are made as italic;

Methods
Data sources
In order to understand what data may best serve annual monitoring of the performance of the family planning program in Myanmar, both at the national and state/regional level, four data sources of modern contraceptive use:
< >Modelled mCPR estimates (and confidence intervals) from Track20’s FPET, based on nationally and state/regionally representative surveys for Myanmar: Myanmar DHS (2015-16), Multiple Indicator Cluster Survey (2010), Fertility and Reproductive Health Survey (1991, 1997, 2001 & 2007). (7). (While FPET tool can be incooperated with both survey and service statistics data, FPET run for Myanmar could not use the service statistics data due to data validity.)mCPR and Method-specific prevalence from the 2015-16 Myanmar DHS (6).mCPR estimates and contraceptive method prevalence from Myanmar’s HMIS system (8) based on local annual census conducted by midwives on contraceptive use among married women in their catchment areas; which is different from other routine HMIS indicators and collected once a year at the end of calendar year for the information of contraceptive use including type of methods used, and mCPR from Estimates of modern method use (EMU) tool based on contraceptive commodity consumption data from Myanmar’s LMIS system (9); were used to compare (Table 1). (While FPET can be incorporated service statistics data, it could not be put the direct estimates of mCPR, other commodity or visit data from service statistics. EMU excel tool can convert service statistics data to Estimated method use (which is a proxy for mCPR) to be can be used as input in the Family Planning Estimation Tool (FPET). EMU tool was developed by Track 20 team and available at its website http://www.track20.org/pages/our_work/innovative_tools/SS_to_EMU_tool.php. Here, EMU tool was used to provide EMU (Estimated method use-a proxy of mCPR) by converting of commodity consumption data.) strengthen of HMIS data through reviewing the methodology of data collection for family planning information with further primary research and validating the data quality;establish the Family Planning register/record and integrate into electronic HMIS/DHIS;strengthen the LMIS for reproductive health commodity in terms of data quality as well as area coverage.

Comments

This reviewer has not had access to the data, but an illustrative example merits consideration by the authors, either by proposing some other adjustment procedure or by pursuing a version of what I recommend. Consider the following: At one point in the paper, the DHS is alluded to as the “gold standard” for mCPR estimation. The FPET and DHS estimates are similar (Figure 1), but HMIS data appear to be over-estimating the mCPR. If the DHS is considered the gold standard, then a plausible proposition is that its data can be used to develop areal adjustment factors. A regression analysis, with DHS data employed as the dependent variable, HMIS data as an independent variable, with direct and interaction effects for population characteristics and locality indicators could be used to develop regression weights that estimate the discrepancies associated with a set of indicators. The regression weights would define adjustment factors for correcting data to a DHS standard. The correspondence of FPET with the DHS invites an approach that would use the FPET as the “silver standard” for HMIS adjustment. While the paper might not take this up as a technical task, it should include discussion of methods of data adjustment that results appear to invite. This would be a more cogent set of recommendations than the text that now appears in the Recommendations section.

Actually, FPET tool is based on Bayesian, hierarchical model that fits curves to historical data (all available survey and service statistics). The model fits a logistic growth curve to CPR data for all methods to determine the long term trend in contraceptive use and adds a time-series model with autocorrelation to capture country-specific deviations around the long term trend. Since FPET gives almost the same result as DHS, it is assumed as the most reliable data source for annual estimates so far to compare the service statistics data (HMIS and LMIS) in this study. I hope it has already incorporated the adjustment for comparison of HMIS data with DHS. Also, I will add about this in methodology section in my revise version.

Comments for the figures

Figures 2 and 3 are hard to read and needlessly complex to interpret. There are many regions with pronounced variance between regions. Are there take home messages from these data that could be summarized more simply? For example, it is possible that the magnitude of discrepancies is associated with the level of the estimated mCPR. If HMIS level were on the Y axis and FPET were the X axis a simple regression line would simulate correspondence between the data sources, and the scatter around that line would show the pattern of discrepancy by level of mCPR. The title of Figure 3 is confusing. I think that Figure 3 could be dropped.

In figure 2 and 3, it shows year of consistency for mCPR between the HMIS and FPET estimates including 95%CI for each 17 State/Regions for three consecutive year’s data. The bar shows the HMIS data and FPET data points including upper and lower border (95% CI) are shown on that bar for each three years for 17 State/Regions. We can see the consistency that whether HMIS data (top of the bar) are fell within these upper and lower borders of FPET. For me, it is not easy to draw the regression line for two data sources of three years data points for each 17 State/Regions in one picture. Also, I found that there is no relation between the consistency and magnitude of value. The key messages are there are only two States (Chin and Kayin) for consistency in all three years (Figure 2) and some are two years and some are only one year consistent. About half of State/Regions show no consistency at all (Figure 3) for three years. It means that direct estimates of mCPR from HMIS are not consistency with FPET and not accurate to be used for annual monitoring. I will change the title for Figure 3 as “Comparing the HMIS and FPET results of mCPR by regions of none of the year consistent estimates” to be more clear as previously mentioned that “ by regions of no consistent estimates”.

Comments: Figure 4 is a largely empty depiction of a few data points. This could be replaced with a table.

Thank you for your advice. Actually, I would like to show that the comparison of HMIS and FPET (with 95% CI) for national mCPR with trend (three years data points) and rate of progress. It highlights HMIS data were overestimates of FPET for three years (even over the upper border in 2016 and 2017) and DHS as well in 2016. So, I think it could not easy for readers to see these points from table.

Comments: Figure 5 is visually confusing. For some reason, data correspondence in Chin is very different from the other regions. Coloration might help, but tabular data might be more interpretable to a reader.

In Figure 5, it was aimed to show the comparison of different contraceptive method mix (Injection, pills, IUD, implants, condom and female sterilization) prevalence data of HMIS and DHIS (2016) as well as the comparison of national level and state/regional level (Chin and Kayin) where there were different patterns of national level. The key messages are the method mix pattern of HMIS and DHS are quite similar except female sterilization. And while injection was the highest proportion at national level (both HIS and DHS), high proportions of long term methods and female sterilization were found at Chin from DHS (quite different with HMIS) and pills use was the highest in Kayin in DHS. It means that HMIS data were quite similar with DHS for method mix at national and other State/regions except Chin and Kayin. So, I think it might be not easy for readers to interpret if data were shown table.

Minor comments and suggestions!
The text of this paper begins with a reference to the maternal mortality rate, but the paper is on a different topic. This reviewer suggests that the authors open the paper with a statement that is a more direct reference to topic of the article and the lack of systematic attention to family planning research in Myanmar and the rationale for the paper. At some point in the discussion, reference to the relationship of family planning with the maternal mortality rate would be informative.

Thank you for your suggestions. I edited the introduction a lot with starting of importance of family planning, about Myanmar FP commitment, Myanmar FP program implementation to achieve FP 2020 commitment and reduce MMR, then how importance of monitoring the progress and current data source. And some text “Also, lack of studies for systematic analysis of the family planning data source called to do this study to set the annual family planning monitoring system in Myanmar” has added as the last sentence in rationale.

In discussion part, I have added the following text at the beginning.

“Access to FP contributes up to a 44% reduction in maternal deaths and a 21% reduction of deaths in children under age 5 (Ref), therefore, Myanmar needs to endevaour strong efforts to increase access to family planning services to tackle the high MMR issue. In the meantime, setting the systematic approach for continuous monitoring is important to provide the valuable information to be considered for future strategies”

(Ref: Starbird E, Norton M, Marcus R Investing in Family Planning: Key to Achieving the Sustainable Development Goals. doi: 10.9745/GHSP-D-15-00374)
Also, some editions for another reviewer have been incorporated in my revised version.

Thanks again and best regards,
Khaing

Dear Phillips,

Thank you so much for your comments and suggestions and I would like to response as follows.

Comments on the Methods section

This paper is more focused on the comparison of tools rather than the analysis of family planning, per se. A tool referred to as FPET and another tool LMIS are compared with DHS data, but the methods section is limited to a discussion of the logic of comparison rather than the logic that underlies each tool. The key objective concerns the need for a credible basis for assessing annual progress with family planning program functioning. It is critical to the paper for the reader to understand how results could be affected by methodological differences in the tools employed. Therefore, it is important for the text to clarify how these assessment tools works, either through citation of explanatory articles or through descriptive analysis of the methodologies that they employ. However, tools are cited in the Rationale section without citations or clarification of how they are similar and how they differ. As a consequence, the reader is left to wonder if the differences portrayed are actually significant. This rather simple addition to the paper would substantially strengthen the presentation. This reviewer recommends application of statistical procedures that compare cross-sectional time series data. By employing a method for statistical comparison, the conclusions and recommendations that are reached would have the credibility that is associated with the comparison of trends with confidence intervals.

Thank you for your comments and allow me to respond as follows;

Actually, the main objective of this study is focused to compare the different data source for FP indicators to select the most reliable one for annual monitoring between the years of survey (DHS). As there is no other annual survey in Myanmar, the data from FPET is currently the most reliable source to monitor as it gives the same results as DHS. But, the other routine services like HMIS and LMIS are available, therefore, I would like to test the data whether service statistics data are reliable to use or not for annual monitoring (especially for State and Regional level) and to get the evidence which one is the most viable data sources (FPET/HMIS/LMIS) in Myanmar.
To be compared, four data sources (DHS, FPET, HMIS and LMIS) were used. (Previously I wrote as “compared” rather than “used” that mislead to readers). For annual progress of mCPR, services statistics data (HMIS and LMIS) were compared along with 95% CI of FPET for their accuracy (because FPET is most reliable source for the years between DHS) for three consecutive years. If the service statistics data points were fell within 95% CI of FPET, it was assumed as accurate and check how many years of consistency between them for each State/Regions (Figure 2 and 3).
For national comparison of mCPR between HMIS and FPET including 95%CI, three years data points were plotted as trend and DHS was put as one point in 2016 (Figure 3). It also shows the HMIS data were much over estimates of FPET and DHS.
When HMIS data were compared with FPET, the direct estimates of mCPR from HMIS were used. However, LMIS could not provide the mCPR to compare with FPET. LMIS can provide only commodity consumption data. Therefore, we have to use EMU (Estimated method use) excel tool which is developed by Track 20 team and it can provide EMU (Estimated method use-a proxy of mCPR) by converting commodity consumption data from LMIS.
The FPET and EMU tools are not comparable and EMU results can be used as input for FPET if we want to incorporate service statistics data into FPET in addition to survey data. But it is not the case of this study as Myanmar FPET estimates use only survey data (due to poor data validity of service statistics data). Here EMU tool was used to provide EMU (Estimated method use-a proxy of mCPR) by converting of commodity consumption data. The links of these tools are also added in revise version.
For comparing the pattern of method mix, FPET cannot provide that data, we used DHS data for only that year of survey (2016) and compared with HMIS data. So, actually it doesn’t compare the two tools (FPET and LMIS tool) and compare the service statistics data (HMIS and LMIS through EMU tool) with FPET for mCPR progress and HMIS with DHS for method mix.
To be more clear, I have tried to edit the methodology section as follows and added text are made as italic;

Methods
Data sources
In order to understand what data may best serve annual monitoring of the performance of the family planning program in Myanmar, both at the national and state/regional level, four data sources of modern contraceptive use:
< >Modelled mCPR estimates (and confidence intervals) from Track20’s FPET, based on nationally and state/regionally representative surveys for Myanmar: Myanmar DHS (2015-16), Multiple Indicator Cluster Survey (2010), Fertility and Reproductive Health Survey (1991, 1997, 2001 & 2007). (7). (While FPET tool can be incooperated with both survey and service statistics data, FPET run for Myanmar could not use the service statistics data due to data validity.)mCPR and Method-specific prevalence from the 2015-16 Myanmar DHS (6).mCPR estimates and contraceptive method prevalence from Myanmar’s HMIS system (8) based on local annual census conducted by midwives on contraceptive use among married women in their catchment areas; which is different from other routine HMIS indicators and collected once a year at the end of calendar year for the information of contraceptive use including type of methods used, and mCPR from Estimates of modern method use (EMU) tool based on contraceptive commodity consumption data from Myanmar’s LMIS system (9); were used to compare (Table 1). (While FPET can be incorporated service statistics data, it could not be put the direct estimates of mCPR, other commodity or visit data from service statistics. EMU excel tool can convert service statistics data to Estimated method use (which is a proxy for mCPR) to be can be used as input in the Family Planning Estimation Tool (FPET). EMU tool was developed by Track 20 team and available at its website http://www.track20.org/pages/our_work/innovative_tools/SS_to_EMU_tool.php. Here, EMU tool was used to provide EMU (Estimated method use-a proxy of mCPR) by converting of commodity consumption data.) strengthen of HMIS data through reviewing the methodology of data collection for family planning information with further primary research and validating the data quality;establish the Family Planning register/record and integrate into electronic HMIS/DHIS;strengthen the LMIS for reproductive health commodity in terms of data quality as well as area coverage.

Comments

This reviewer has not had access to the data, but an illustrative example merits consideration by the authors, either by proposing some other adjustment procedure or by pursuing a version of what I recommend. Consider the following: At one point in the paper, the DHS is alluded to as the “gold standard” for mCPR estimation. The FPET and DHS estimates are similar (Figure 1), but HMIS data appear to be over-estimating the mCPR. If the DHS is considered the gold standard, then a plausible proposition is that its data can be used to develop areal adjustment factors. A regression analysis, with DHS data employed as the dependent variable, HMIS data as an independent variable, with direct and interaction effects for population characteristics and locality indicators could be used to develop regression weights that estimate the discrepancies associated with a set of indicators. The regression weights would define adjustment factors for correcting data to a DHS standard. The correspondence of FPET with the DHS invites an approach that would use the FPET as the “silver standard” for HMIS adjustment. While the paper might not take this up as a technical task, it should include discussion of methods of data adjustment that results appear to invite. This would be a more cogent set of recommendations than the text that now appears in the Recommendations section.

Actually, FPET tool is based on Bayesian, hierarchical model that fits curves to historical data (all available survey and service statistics). The model fits a logistic growth curve to CPR data for all methods to determine the long term trend in contraceptive use and adds a time-series model with autocorrelation to capture country-specific deviations around the long term trend. Since FPET gives almost the same result as DHS, it is assumed as the most reliable data source for annual estimates so far to compare the service statistics data (HMIS and LMIS) in this study. I hope it has already incorporated the adjustment for comparison of HMIS data with DHS. Also, I will add about this in methodology section in my revise version.

Comments for the figures

Figures 2 and 3 are hard to read and needlessly complex to interpret. There are many regions with pronounced variance between regions. Are there take home messages from these data that could be summarized more simply? For example, it is possible that the magnitude of discrepancies is associated with the level of the estimated mCPR. If HMIS level were on the Y axis and FPET were the X axis a simple regression line would simulate correspondence between the data sources, and the scatter around that line would show the pattern of discrepancy by level of mCPR. The title of Figure 3 is confusing. I think that Figure 3 could be dropped.

In figure 2 and 3, it shows year of consistency for mCPR between the HMIS and FPET estimates including 95%CI for each 17 State/Regions for three consecutive year’s data. The bar shows the HMIS data and FPET data points including upper and lower border (95% CI) are shown on that bar for each three years for 17 State/Regions. We can see the consistency that whether HMIS data (top of the bar) are fell within these upper and lower borders of FPET. For me, it is not easy to draw the regression line for two data sources of three years data points for each 17 State/Regions in one picture. Also, I found that there is no relation between the consistency and magnitude of value. The key messages are there are only two States (Chin and Kayin) for consistency in all three years (Figure 2) and some are two years and some are only one year consistent. About half of State/Regions show no consistency at all (Figure 3) for three years. It means that direct estimates of mCPR from HMIS are not consistency with FPET and not accurate to be used for annual monitoring. I will change the title for Figure 3 as “Comparing the HMIS and FPET results of mCPR by regions of none of the year consistent estimates” to be more clear as previously mentioned that “ by regions of no consistent estimates”.

Comments: Figure 4 is a largely empty depiction of a few data points. This could be replaced with a table.

Thank you for your advice. Actually, I would like to show that the comparison of HMIS and FPET (with 95% CI) for national mCPR with trend (three years data points) and rate of progress. It highlights HMIS data were overestimates of FPET for three years (even over the upper border in 2016 and 2017) and DHS as well in 2016. So, I think it could not easy for readers to see these points from table.

Comments: Figure 5 is visually confusing. For some reason, data correspondence in Chin is very different from the other regions. Coloration might help, but tabular data might be more interpretable to a reader.

In Figure 5, it was aimed to show the comparison of different contraceptive method mix (Injection, pills, IUD, implants, condom and female sterilization) prevalence data of HMIS and DHIS (2016) as well as the comparison of national level and state/regional level (Chin and Kayin) where there were different patterns of national level. The key messages are the method mix pattern of HMIS and DHS are quite similar except female sterilization. And while injection was the highest proportion at national level (both HIS and DHS), high proportions of long term methods and female sterilization were found at Chin from DHS (quite different with HMIS) and pills use was the highest in Kayin in DHS. It means that HMIS data were quite similar with DHS for method mix at national and other State/regions except Chin and Kayin. So, I think it might be not easy for readers to interpret if data were shown table.

Minor comments and suggestions!
The text of this paper begins with a reference to the maternal mortality rate, but the paper is on a different topic. This reviewer suggests that the authors open the paper with a statement that is a more direct reference to topic of the article and the lack of systematic attention to family planning research in Myanmar and the rationale for the paper. At some point in the discussion, reference to the relationship of family planning with the maternal mortality rate would be informative.

Thank you for your suggestions. I edited the introduction a lot with starting of importance of family planning, about Myanmar FP commitment, Myanmar FP program implementation to achieve FP 2020 commitment and reduce MMR, then how importance of monitoring the progress and current data source. And some text “Also, lack of studies for systematic analysis of the family planning data source called to do this study to set the annual family planning monitoring system in Myanmar” has added as the last sentence in rationale.

In discussion part, I have added the following text at the beginning.

“Access to FP contributes up to a 44% reduction in maternal deaths and a 21% reduction of deaths in children under age 5 (Ref), therefore, Myanmar needs to endevaour strong efforts to increase access to family planning services to tackle the high MMR issue. In the meantime, setting the systematic approach for continuous monitoring is important to provide the valuable information to be considered for future strategies”

(Ref: Starbird E, Norton M, Marcus R Investing in Family Planning: Key to Achieving the Sustainable Development Goals. doi: 10.9745/GHSP-D-15-00374)
Also, some editions for another reviewer have been incorporated in my revised version.

Thanks again and best regards,
Khaing

Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 24 Oct 2019

Janine Barden O'Fallon, Carolina Population Center, Maternal & Child Health Department, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

Approved

https://doi.org/10.21956/gatesopenres.14120.r27972

I thank the authors for providing a proof-of-concept from Myanmar on the application of the Family Planning Estimation Tool (FPET) for assessing routine data on key family planning outcomes. The article focuses on the potential for annual assessment of the modern contraceptive prevalence rate (mCPR) and contraceptive method mix (alternately referred to as “method prevalence”), making the justifiable argument that routine monitoring of family planning outcomes is important to the management and success of national family planning programs.

While the authors find that FPET produces estimates close to DHS estimates (i.e., the gold standard), and that health management information system (HMIS) and logistics management information system (LMIS) estimates do not consistently do so, their recommendations revolve around the strengthening of the HMIS and LMIS. There are many good reasons for this, including that health information systems evolve and system improvements can be ongoing. This means that HMIS and LMIS can become more reliable as data sources over time. Assessing how and why data quality problems arise and implementing practices that can reduce the problems are a big step toward that end. Standardizing the processes for data collection, reporting, analysis and use; implementing feedback systems for course corrections; and building and maintaining the human resources and capacity for managing the systems are also necessary investments to achieve full system functionality.

My comments and suggestions for minor revisions are provided below.

Introduction:

It would be helpful to have contextual information about the development of the HMIS and LMIS in this section.
Additionally, some information on commodity consumption data may help readers better understand and interpret the findings.

Methods:

Please name or describe the data sources included in the FPET tool in Myanmar.
It should be made clear that the FPET and DHS are being used as the standards against which the HMIS and LMIS estimates are being compared. The methods section reads as though these four data sources are being compared against each other, which is not really the case.
The HMIS mCPR estimate is based on “local annual census conducted by midwives on contraceptive use among married women in their catchment area.” This seems to be an unusual source of data for an HMIS; can the authors provide more information on this?

Recommendations:

This section introduces some concepts that are not previously mentioned in the article (“the family planning register,” the “DHIS tool”), thus the basis for the recommendations may not be well-understood by most readers.

Discussion & Conclusion:

Please see the second sentence in the first paragraph of this section and revise for clarity.
A tie-in to other literature on this topic is missing; what has other research on this topic found and how does it fit with these results?
Do authors have any explanation about why HMIS mCPR estimates appear to be over-estimates? What specific “data quality and methodology” issues have been identified? What is being done to improve data quality?
Information on study limitations is missing.
“Data” is plural; the document needs editing to ensure proper noun-verb agreement.

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Monitoring and evaluation in global health programs; family planning

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Author Response 10 Dec 2019

Khaing Nwe Tin, Department of Public Health, Ministry of Health and Sports, Myanmar, Nay Pyi Taw, 15011, Myanmar

10 Dec 2019

Author Response
Dear Sir,

Thank you so much for your comments and these are very useful to revise my article. I agreed all of your points and would like to respond as follows;

Introduction: ... Continue reading
Dear Sir,

Thank you so much for your comments and these are very useful to revise my article. I agreed all of your points and would like to respond as follows;

Introduction:

It would be helpful to have contextual information about the development of the HMIS and LMIS in this section.

Additionally, some information on commodity consumption data may help readers better understand and interpret the findings.

I will add one paragraph in the last of introduction section regarding the current data source for FP monitoring in Myanmar and will also explain about the HMIS and LMIS including commodity consumption data as follows:

“In order to monitor the progress of family planning program, Myanmar uses the FP indicators (CPR, method mixed prevalence, unmet need for family planning and demand satisfaction) from the limited available sources mainly; first ever Demographic and Health Survey (DHS)(2015-16) and Health Management Information System (HMIS), as well as commodity consumption data from Logistic Management Information System (LMIS). HMIS was developed since 1995 as paper based and now transformed to electronic system of HMIS as District Health Information System (DHIS) 2. It is the health information system of Myanmar Ministry of Health and Sports to monitor the performance of all public health facilities while it accounts as local annual census for family planning data and covers both public and private. In Myanmar, LMIS for reproductive health commodity (RHCLS: Reproductive health commodity logistic system) is initiated as national program since 2016 in selected region and rolling out phase by phase. It provides the information related with stock status as well as commodity consumption data; commodity distributed to the clients, from different public health facilities. However, systematic tracking of annual family planning progress with valid data source has not yet been set yet.”

Methods:

Please name or describe the data sources included in the FPET tool in Myanmar.

I will insert the names of surveys that included in FPET for Myanmar as follows in data source part:

“Modelled mCPR estimates (and confidence intervals) from Track20’s FPET, based on nationally and state/regionally representative surveys (7): Myanmar DHS (2015-16), Multiple Indicator Cluster Survey (2010), Fertility and Reproductive Health Survey (1991, 1997, 2001 & 2007).

It should be made clear that the FPET and DHS are being used as the standards against which the HMIS and LMIS estimates are being compared. The methods section reads as though these four data sources are being compared against each other, which is not really the case.

Thanks for suggestion and I removed the clause of “four data sources were compared” and edited as follows:

“In addition to FPET, to identify the most valid data source for annual monitoring, service statistics data (HMIS and LMIS) of CPR were compared along with the confidence intervals from FPET for three consecutive years from 2015 to 2017.”

The HMIS mCPR estimate is based on “local annual census conducted by midwives on contraceptive use among married women in their catchment area.” This seems to be an unusual source of data for an HMIS; can the authors provide more information on this?

Yes, it is unusual data collection and I also added some information in introduction and added in data source as follows;

“mCPR estimates and contraceptive method prevalence from Myanmar’s HMIS system (8), based on local annual census conducted by midwives on contraceptive use among married women in their catchment areas; which is different from other routine HMIS indicators and collected once a year at the end of calendar year for the information of contraceptive use including type of methods used,”

Recommendations:

This section introduces some concepts that are not previously mentioned in the article (“the family planning register,” the “DHIS tool”), thus the basis for the recommendations may not be well-understood by most readers.

I have added the following paragraph in discussion to be more understandable that recommendation.

“While the methodology for data collection is quite unusual as local annual census, it is also needed to reconsider to be consistent with other HMIS indicators through the register/record for family planning information and integrated into electronic HMIS (DHIS 2) to get the valid data of family planning service visit and contraceptive users at least for public sectors.”

Discussion & Conclusion:

Please see the second sentence in the first paragraph of this section and revise for clarity.

I have revised as follows;

When considering the data source for the annual tracking of mCPR (either estimates from FPET or routine service statistics), the use of direct estimates of HMIS should be considered carefully as it is much higher than the DHS survey and not matched with FPET results except in Chin and Kayin States.

A tie-in to other literature on this topic is missing; what has other research on this topic found and how does it fit with these results?

I have added the findings of international literature as follows;

These findings are consistent with other international literature. According analysis of three service statistics data (number of: contraceptive commodities distributed to clients, family planning service visits, and current contraceptive users) from 22 countries in Africa and Asia, it was observed that none of these three data were accurate to be used when comparing with United Nations Population Division World Contraceptive Use annual mCPR estimates as the "gold standard."
Although service statistics data for family planning are insufficient accuracy to use stand-alone like survey data, data on contraceptive commodities distributed to clients, family planning service visits, and current users can be used accurately when combined with survey data in FPET tools to estimates the trends of annual mCPR in the years between surveys.

(Ref) Magnani RJ, Ross J, Williamson J, Weinberger M. Can family planning service statistics be used to track population-level outcomes?.Glob Health Sci Pract. 2018;6(1):93-102.https://doi.org/10.9745/GHSP-D-17-00341

Do authors have any explanation about why HMIS mCPR estimates appear to be over-estimates? What specific “data quality and methodology” issues have been identified? What is being done to improve data quality?

As there has not been explored systematically yet, I have not known the exact reasons as this is just secondary data analysis, so I put in the recommendation. But I would add as follows in discussion to be more clear to readers;

Currently, it has limited information from documented references about that why HMIS appear overestimates and grow 5% annually, however, to my working experiences, it is doubtful in data quality by midwives. According to the methodology, it is needed to ask all married women under their jurisdiction areas, it is difficult to cover especially in big city, per-urban as well as hard-to-reach areas. Therefore, it might be possible that it includes only the information of women those who got family planning services from them and probably higher prevalence than those who could not access the health services. However, these are just only the opinions of authors and need to explore systematically and should improve based on that findings.

Information on study limitations is missing.

Thank you for your good point to remind me. I missed it and now I will add as follows;

“This study is only the secondary data analysis, therefore, the reasons behind the inaccuracy of service statistics data could not explain. It covers only Myanmar data so detail results might be limitation to be applied for other countries context although it is consistent with findings of analysis in 22 countries which is service statistics data were not sufficient in accuracy to be used stand alone for annual tracking.”

“Data” is plural; the document needs editing to ensure proper noun-verb agreement.

Thank you for your suggestions and will edit accordingly.

Best regards,
Corresponding Author: Khaing Nwe Tin
Dear Sir,

Thank you so much for your comments and these are very useful to revise my article. I agreed all of your points and would like to respond as follows;

Introduction:

It would be helpful to have contextual information about the development of the HMIS and LMIS in this section.

Additionally, some information on commodity consumption data may help readers better understand and interpret the findings.

I will add one paragraph in the last of introduction section regarding the current data source for FP monitoring in Myanmar and will also explain about the HMIS and LMIS including commodity consumption data as follows:

“In order to monitor the progress of family planning program, Myanmar uses the FP indicators (CPR, method mixed prevalence, unmet need for family planning and demand satisfaction) from the limited available sources mainly; first ever Demographic and Health Survey (DHS)(2015-16) and Health Management Information System (HMIS), as well as commodity consumption data from Logistic Management Information System (LMIS). HMIS was developed since 1995 as paper based and now transformed to electronic system of HMIS as District Health Information System (DHIS) 2. It is the health information system of Myanmar Ministry of Health and Sports to monitor the performance of all public health facilities while it accounts as local annual census for family planning data and covers both public and private. In Myanmar, LMIS for reproductive health commodity (RHCLS: Reproductive health commodity logistic system) is initiated as national program since 2016 in selected region and rolling out phase by phase. It provides the information related with stock status as well as commodity consumption data; commodity distributed to the clients, from different public health facilities. However, systematic tracking of annual family planning progress with valid data source has not yet been set yet.”

Methods:

Please name or describe the data sources included in the FPET tool in Myanmar.

I will insert the names of surveys that included in FPET for Myanmar as follows in data source part:

“Modelled mCPR estimates (and confidence intervals) from Track20’s FPET, based on nationally and state/regionally representative surveys (7): Myanmar DHS (2015-16), Multiple Indicator Cluster Survey (2010), Fertility and Reproductive Health Survey (1991, 1997, 2001 & 2007).

It should be made clear that the FPET and DHS are being used as the standards against which the HMIS and LMIS estimates are being compared. The methods section reads as though these four data sources are being compared against each other, which is not really the case.

Thanks for suggestion and I removed the clause of “four data sources were compared” and edited as follows:

“In addition to FPET, to identify the most valid data source for annual monitoring, service statistics data (HMIS and LMIS) of CPR were compared along with the confidence intervals from FPET for three consecutive years from 2015 to 2017.”

The HMIS mCPR estimate is based on “local annual census conducted by midwives on contraceptive use among married women in their catchment area.” This seems to be an unusual source of data for an HMIS; can the authors provide more information on this?

Yes, it is unusual data collection and I also added some information in introduction and added in data source as follows;

“mCPR estimates and contraceptive method prevalence from Myanmar’s HMIS system (8), based on local annual census conducted by midwives on contraceptive use among married women in their catchment areas; which is different from other routine HMIS indicators and collected once a year at the end of calendar year for the information of contraceptive use including type of methods used,”

Recommendations:

This section introduces some concepts that are not previously mentioned in the article (“the family planning register,” the “DHIS tool”), thus the basis for the recommendations may not be well-understood by most readers.

I have added the following paragraph in discussion to be more understandable that recommendation.

“While the methodology for data collection is quite unusual as local annual census, it is also needed to reconsider to be consistent with other HMIS indicators through the register/record for family planning information and integrated into electronic HMIS (DHIS 2) to get the valid data of family planning service visit and contraceptive users at least for public sectors.”

Discussion & Conclusion:

Please see the second sentence in the first paragraph of this section and revise for clarity.

I have revised as follows;

When considering the data source for the annual tracking of mCPR (either estimates from FPET or routine service statistics), the use of direct estimates of HMIS should be considered carefully as it is much higher than the DHS survey and not matched with FPET results except in Chin and Kayin States.

A tie-in to other literature on this topic is missing; what has other research on this topic found and how does it fit with these results?

I have added the findings of international literature as follows;

These findings are consistent with other international literature. According analysis of three service statistics data (number of: contraceptive commodities distributed to clients, family planning service visits, and current contraceptive users) from 22 countries in Africa and Asia, it was observed that none of these three data were accurate to be used when comparing with United Nations Population Division World Contraceptive Use annual mCPR estimates as the "gold standard."
Although service statistics data for family planning are insufficient accuracy to use stand-alone like survey data, data on contraceptive commodities distributed to clients, family planning service visits, and current users can be used accurately when combined with survey data in FPET tools to estimates the trends of annual mCPR in the years between surveys.

(Ref) Magnani RJ, Ross J, Williamson J, Weinberger M. Can family planning service statistics be used to track population-level outcomes?.Glob Health Sci Pract. 2018;6(1):93-102.https://doi.org/10.9745/GHSP-D-17-00341

Do authors have any explanation about why HMIS mCPR estimates appear to be over-estimates? What specific “data quality and methodology” issues have been identified? What is being done to improve data quality?

As there has not been explored systematically yet, I have not known the exact reasons as this is just secondary data analysis, so I put in the recommendation. But I would add as follows in discussion to be more clear to readers;

Currently, it has limited information from documented references about that why HMIS appear overestimates and grow 5% annually, however, to my working experiences, it is doubtful in data quality by midwives. According to the methodology, it is needed to ask all married women under their jurisdiction areas, it is difficult to cover especially in big city, per-urban as well as hard-to-reach areas. Therefore, it might be possible that it includes only the information of women those who got family planning services from them and probably higher prevalence than those who could not access the health services. However, these are just only the opinions of authors and need to explore systematically and should improve based on that findings.

Information on study limitations is missing.

Thank you for your good point to remind me. I missed it and now I will add as follows;

“This study is only the secondary data analysis, therefore, the reasons behind the inaccuracy of service statistics data could not explain. It covers only Myanmar data so detail results might be limitation to be applied for other countries context although it is consistent with findings of analysis in 22 countries which is service statistics data were not sufficient in accuracy to be used stand alone for annual tracking.”

“Data” is plural; the document needs editing to ensure proper noun-verb agreement.

Thank you for your suggestions and will edit accordingly.

Best regards,
Corresponding Author: Khaing Nwe Tin
Competing Interests: There is no competing interests. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 10 Dec 2019

Khaing Nwe Tin, Department of Public Health, Ministry of Health and Sports, Myanmar, Nay Pyi Taw, 15011, Myanmar

10 Dec 2019

Author Response
Dear Sir,

Thank you so much for your comments and these are very useful to revise my article. I agreed all of your points and would like to respond as follows;

Introduction: ... Continue reading
Dear Sir,

Thank you so much for your comments and these are very useful to revise my article. I agreed all of your points and would like to respond as follows;

Introduction:

It would be helpful to have contextual information about the development of the HMIS and LMIS in this section.

Additionally, some information on commodity consumption data may help readers better understand and interpret the findings.

I will add one paragraph in the last of introduction section regarding the current data source for FP monitoring in Myanmar and will also explain about the HMIS and LMIS including commodity consumption data as follows:

“In order to monitor the progress of family planning program, Myanmar uses the FP indicators (CPR, method mixed prevalence, unmet need for family planning and demand satisfaction) from the limited available sources mainly; first ever Demographic and Health Survey (DHS)(2015-16) and Health Management Information System (HMIS), as well as commodity consumption data from Logistic Management Information System (LMIS). HMIS was developed since 1995 as paper based and now transformed to electronic system of HMIS as District Health Information System (DHIS) 2. It is the health information system of Myanmar Ministry of Health and Sports to monitor the performance of all public health facilities while it accounts as local annual census for family planning data and covers both public and private. In Myanmar, LMIS for reproductive health commodity (RHCLS: Reproductive health commodity logistic system) is initiated as national program since 2016 in selected region and rolling out phase by phase. It provides the information related with stock status as well as commodity consumption data; commodity distributed to the clients, from different public health facilities. However, systematic tracking of annual family planning progress with valid data source has not yet been set yet.”

Methods:

Please name or describe the data sources included in the FPET tool in Myanmar.

I will insert the names of surveys that included in FPET for Myanmar as follows in data source part:

“Modelled mCPR estimates (and confidence intervals) from Track20’s FPET, based on nationally and state/regionally representative surveys (7): Myanmar DHS (2015-16), Multiple Indicator Cluster Survey (2010), Fertility and Reproductive Health Survey (1991, 1997, 2001 & 2007).

It should be made clear that the FPET and DHS are being used as the standards against which the HMIS and LMIS estimates are being compared. The methods section reads as though these four data sources are being compared against each other, which is not really the case.

Thanks for suggestion and I removed the clause of “four data sources were compared” and edited as follows:

“In addition to FPET, to identify the most valid data source for annual monitoring, service statistics data (HMIS and LMIS) of CPR were compared along with the confidence intervals from FPET for three consecutive years from 2015 to 2017.”

The HMIS mCPR estimate is based on “local annual census conducted by midwives on contraceptive use among married women in their catchment area.” This seems to be an unusual source of data for an HMIS; can the authors provide more information on this?

Yes, it is unusual data collection and I also added some information in introduction and added in data source as follows;

“mCPR estimates and contraceptive method prevalence from Myanmar’s HMIS system (8), based on local annual census conducted by midwives on contraceptive use among married women in their catchment areas; which is different from other routine HMIS indicators and collected once a year at the end of calendar year for the information of contraceptive use including type of methods used,”

Recommendations:

This section introduces some concepts that are not previously mentioned in the article (“the family planning register,” the “DHIS tool”), thus the basis for the recommendations may not be well-understood by most readers.

I have added the following paragraph in discussion to be more understandable that recommendation.

“While the methodology for data collection is quite unusual as local annual census, it is also needed to reconsider to be consistent with other HMIS indicators through the register/record for family planning information and integrated into electronic HMIS (DHIS 2) to get the valid data of family planning service visit and contraceptive users at least for public sectors.”

Discussion & Conclusion:

Please see the second sentence in the first paragraph of this section and revise for clarity.

I have revised as follows;

When considering the data source for the annual tracking of mCPR (either estimates from FPET or routine service statistics), the use of direct estimates of HMIS should be considered carefully as it is much higher than the DHS survey and not matched with FPET results except in Chin and Kayin States.

A tie-in to other literature on this topic is missing; what has other research on this topic found and how does it fit with these results?

I have added the findings of international literature as follows;

These findings are consistent with other international literature. According analysis of three service statistics data (number of: contraceptive commodities distributed to clients, family planning service visits, and current contraceptive users) from 22 countries in Africa and Asia, it was observed that none of these three data were accurate to be used when comparing with United Nations Population Division World Contraceptive Use annual mCPR estimates as the "gold standard."
Although service statistics data for family planning are insufficient accuracy to use stand-alone like survey data, data on contraceptive commodities distributed to clients, family planning service visits, and current users can be used accurately when combined with survey data in FPET tools to estimates the trends of annual mCPR in the years between surveys.

(Ref) Magnani RJ, Ross J, Williamson J, Weinberger M. Can family planning service statistics be used to track population-level outcomes?.Glob Health Sci Pract. 2018;6(1):93-102.https://doi.org/10.9745/GHSP-D-17-00341

Do authors have any explanation about why HMIS mCPR estimates appear to be over-estimates? What specific “data quality and methodology” issues have been identified? What is being done to improve data quality?

As there has not been explored systematically yet, I have not known the exact reasons as this is just secondary data analysis, so I put in the recommendation. But I would add as follows in discussion to be more clear to readers;

Currently, it has limited information from documented references about that why HMIS appear overestimates and grow 5% annually, however, to my working experiences, it is doubtful in data quality by midwives. According to the methodology, it is needed to ask all married women under their jurisdiction areas, it is difficult to cover especially in big city, per-urban as well as hard-to-reach areas. Therefore, it might be possible that it includes only the information of women those who got family planning services from them and probably higher prevalence than those who could not access the health services. However, these are just only the opinions of authors and need to explore systematically and should improve based on that findings.

Information on study limitations is missing.

Thank you for your good point to remind me. I missed it and now I will add as follows;

“This study is only the secondary data analysis, therefore, the reasons behind the inaccuracy of service statistics data could not explain. It covers only Myanmar data so detail results might be limitation to be applied for other countries context although it is consistent with findings of analysis in 22 countries which is service statistics data were not sufficient in accuracy to be used stand alone for annual tracking.”

“Data” is plural; the document needs editing to ensure proper noun-verb agreement.

Thank you for your suggestions and will edit accordingly.

Best regards,
Corresponding Author: Khaing Nwe Tin
Dear Sir,

Thank you so much for your comments and these are very useful to revise my article. I agreed all of your points and would like to respond as follows;

Introduction:

It would be helpful to have contextual information about the development of the HMIS and LMIS in this section.

Additionally, some information on commodity consumption data may help readers better understand and interpret the findings.

I will add one paragraph in the last of introduction section regarding the current data source for FP monitoring in Myanmar and will also explain about the HMIS and LMIS including commodity consumption data as follows:

“In order to monitor the progress of family planning program, Myanmar uses the FP indicators (CPR, method mixed prevalence, unmet need for family planning and demand satisfaction) from the limited available sources mainly; first ever Demographic and Health Survey (DHS)(2015-16) and Health Management Information System (HMIS), as well as commodity consumption data from Logistic Management Information System (LMIS). HMIS was developed since 1995 as paper based and now transformed to electronic system of HMIS as District Health Information System (DHIS) 2. It is the health information system of Myanmar Ministry of Health and Sports to monitor the performance of all public health facilities while it accounts as local annual census for family planning data and covers both public and private. In Myanmar, LMIS for reproductive health commodity (RHCLS: Reproductive health commodity logistic system) is initiated as national program since 2016 in selected region and rolling out phase by phase. It provides the information related with stock status as well as commodity consumption data; commodity distributed to the clients, from different public health facilities. However, systematic tracking of annual family planning progress with valid data source has not yet been set yet.”

Methods:

Please name or describe the data sources included in the FPET tool in Myanmar.

I will insert the names of surveys that included in FPET for Myanmar as follows in data source part:

“Modelled mCPR estimates (and confidence intervals) from Track20’s FPET, based on nationally and state/regionally representative surveys (7): Myanmar DHS (2015-16), Multiple Indicator Cluster Survey (2010), Fertility and Reproductive Health Survey (1991, 1997, 2001 & 2007).

It should be made clear that the FPET and DHS are being used as the standards against which the HMIS and LMIS estimates are being compared. The methods section reads as though these four data sources are being compared against each other, which is not really the case.

Thanks for suggestion and I removed the clause of “four data sources were compared” and edited as follows:

“In addition to FPET, to identify the most valid data source for annual monitoring, service statistics data (HMIS and LMIS) of CPR were compared along with the confidence intervals from FPET for three consecutive years from 2015 to 2017.”

The HMIS mCPR estimate is based on “local annual census conducted by midwives on contraceptive use among married women in their catchment area.” This seems to be an unusual source of data for an HMIS; can the authors provide more information on this?

Yes, it is unusual data collection and I also added some information in introduction and added in data source as follows;

“mCPR estimates and contraceptive method prevalence from Myanmar’s HMIS system (8), based on local annual census conducted by midwives on contraceptive use among married women in their catchment areas; which is different from other routine HMIS indicators and collected once a year at the end of calendar year for the information of contraceptive use including type of methods used,”

Recommendations:

This section introduces some concepts that are not previously mentioned in the article (“the family planning register,” the “DHIS tool”), thus the basis for the recommendations may not be well-understood by most readers.

I have added the following paragraph in discussion to be more understandable that recommendation.

“While the methodology for data collection is quite unusual as local annual census, it is also needed to reconsider to be consistent with other HMIS indicators through the register/record for family planning information and integrated into electronic HMIS (DHIS 2) to get the valid data of family planning service visit and contraceptive users at least for public sectors.”

Discussion & Conclusion:

Please see the second sentence in the first paragraph of this section and revise for clarity.

I have revised as follows;

When considering the data source for the annual tracking of mCPR (either estimates from FPET or routine service statistics), the use of direct estimates of HMIS should be considered carefully as it is much higher than the DHS survey and not matched with FPET results except in Chin and Kayin States.

A tie-in to other literature on this topic is missing; what has other research on this topic found and how does it fit with these results?

I have added the findings of international literature as follows;

These findings are consistent with other international literature. According analysis of three service statistics data (number of: contraceptive commodities distributed to clients, family planning service visits, and current contraceptive users) from 22 countries in Africa and Asia, it was observed that none of these three data were accurate to be used when comparing with United Nations Population Division World Contraceptive Use annual mCPR estimates as the "gold standard."
Although service statistics data for family planning are insufficient accuracy to use stand-alone like survey data, data on contraceptive commodities distributed to clients, family planning service visits, and current users can be used accurately when combined with survey data in FPET tools to estimates the trends of annual mCPR in the years between surveys.

(Ref) Magnani RJ, Ross J, Williamson J, Weinberger M. Can family planning service statistics be used to track population-level outcomes?.Glob Health Sci Pract. 2018;6(1):93-102.https://doi.org/10.9745/GHSP-D-17-00341

Do authors have any explanation about why HMIS mCPR estimates appear to be over-estimates? What specific “data quality and methodology” issues have been identified? What is being done to improve data quality?

As there has not been explored systematically yet, I have not known the exact reasons as this is just secondary data analysis, so I put in the recommendation. But I would add as follows in discussion to be more clear to readers;

Currently, it has limited information from documented references about that why HMIS appear overestimates and grow 5% annually, however, to my working experiences, it is doubtful in data quality by midwives. According to the methodology, it is needed to ask all married women under their jurisdiction areas, it is difficult to cover especially in big city, per-urban as well as hard-to-reach areas. Therefore, it might be possible that it includes only the information of women those who got family planning services from them and probably higher prevalence than those who could not access the health services. However, these are just only the opinions of authors and need to explore systematically and should improve based on that findings.

Information on study limitations is missing.

Thank you for your good point to remind me. I missed it and now I will add as follows;

“This study is only the secondary data analysis, therefore, the reasons behind the inaccuracy of service statistics data could not explain. It covers only Myanmar data so detail results might be limitation to be applied for other countries context although it is consistent with findings of analysis in 22 countries which is service statistics data were not sufficient in accuracy to be used stand alone for annual tracking.”

“Data” is plural; the document needs editing to ensure proper noun-verb agreement.

Thank you for your suggestions and will edit accordingly.

Best regards,
Corresponding Author: Khaing Nwe Tin
Competing Interests: There is no competing interests. Close
Report a concern

Comments on this article Comments (0)

Version 3

VERSION 3 PUBLISHED 18 Sep 2019

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 3 (revision) 18 Dec 19	read
Version 2 (revision) 10 Dec 19		read
Version 1 18 Sep 19	read	read

Janine Barden O'Fallon, University of North Carolina at Chapel Hill, Chapel Hill, USA
James F. Phillips, Columbia University, New York City, USA

Comments on this article

All Comments(0)

Add a comment

Back to all reports

Reviewer Report

7 Views

18 Dec 2019 | for Version 3

Janine Barden O'Fallon, Carolina Population Center, Maternal & Child Health Department, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

7 Views Cite this report Responses(1)

Approved

Thank you for making these revisions. I had some difficulty understanding the added limitations section, but otherwise agree with your changes.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Monitoring and evaluation in global health programs; family planning

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Author Response

18 Dec 2019

Khaing Nwe Tin, Department of Public Health, Ministry of Health and Sports, Myanmar, Nay Pyi Taw, 15011, Myanmar

Dear Janine,

Thank you for your comments. Please allow me to respond for the limitation section. As this study is secondary data analysis, it could not explain the definite reasons why the HMIS data are inaccurate and much higher than FPET and DHS.

Another point is that although the finding in this study (service statistics data are inaccurate to use stand alone to monitor FP annually) is consistent with the analysis of 22 countries (Ref), this study was used only Myanmar data so the application of the findings might be limited to the other specific country context.

Ref: Magnani RJ, Ross J, Williamson J, et al.: Can Family Planning Service Statistics Be Used to Track Population-Level Outcomes? Glob Health Sci Pract. 2018; 6(1): 93–102. PubMed Abstract | Publisher Full Text | Free Full Text

Thanks again for your time and best regards,

Khaing

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

6 Views

12 Dec 2019 | for Version 2

James F. Phillips, Heilbrunn Department of Population and Family Health, Mailman School of Public Health, Columbia University, New York City, NY, USA

6 Views Cite this report Responses(1)

Approved

I have no further comments. The authors have addressed the concerns that I have raised in my review.

Competing Interests

No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Back to all reports

Reviewer Report

16 Views

30 Oct 2019 | for Version 1

James F. Phillips, Heilbrunn Department of Population and Family Health, Mailman School of Public Health, Columbia University, New York City, NY, USA

16 Views Cite this report Responses(1)

Approved With Reservations

Is the work clearly and accurately presented and does it cite the current literature?

No
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

No
If applicable, is the statistical analysis and its interpretation appropriate?

Partly
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Partly

References

1. Chetty V: Pooling of Time Series and Cross Section Data. Econometrica. 1968; 36 (2). Publisher Full Text

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Demography and health systems research

Respond to this report

Responses (1)

Author Response

10 Dec 2019

Khaing Nwe Tin, Department of Public Health, Ministry of Health and Sports, Myanmar, Nay Pyi Taw, 15011, Myanmar

Dear Phillips,

Thank you so much for your comments and suggestions and I would like to response as follows.

Comments on the Methods section

This paper is more focused on the comparison of tools rather than the analysis of family planning, per se. A tool referred to as FPET and another tool LMIS are compared with DHS data, but the methods section is limited to a discussion of the logic of comparison rather than the logic that underlies each tool. The key objective concerns the need for a credible basis for assessing annual progress with family planning program functioning. It is critical to the paper for the reader to understand how results could be affected by methodological differences in the tools employed. Therefore, it is important for the text to clarify how these assessment tools works, either through citation of explanatory articles or through descriptive analysis of the methodologies that they employ. However, tools are cited in the Rationale section without citations or clarification of how they are similar and how they differ. As a consequence, the reader is left to wonder if the differences portrayed are actually significant. This rather simple addition to the paper would substantially strengthen the presentation. This reviewer recommends application of statistical procedures that compare cross-sectional time series data. By employing a method for statistical comparison, the conclusions and recommendations that are reached would have the credibility that is associated with the comparison of trends with confidence intervals.

Thank you for your comments and allow me to respond as follows;

Actually, the main objective of this study is focused to compare the different data source for FP indicators to select the most reliable one for annual monitoring between the years of survey (DHS). As there is no other annual survey in Myanmar, the data from FPET is currently the most reliable source to monitor as it gives the same results as DHS. But, the other routine services like HMIS and LMIS are available, therefore, I would like to test the data whether service statistics data are reliable to use or not for annual monitoring (especially for State and Regional level) and to get the evidence which one is the most viable data sources (FPET/HMIS/LMIS) in Myanmar.
To be compared, four data sources (DHS, FPET, HMIS and LMIS) were used. (Previously I wrote as “compared” rather than “used” that mislead to readers). For annual progress of mCPR, services statistics data (HMIS and LMIS) were compared along with 95% CI of FPET for their accuracy (because FPET is most reliable source for the years between DHS) for three consecutive years. If the service statistics data points were fell within 95% CI of FPET, it was assumed as accurate and check how many years of consistency between them for each State/Regions (Figure 2 and 3).
For national comparison of mCPR between HMIS and FPET including 95%CI, three years data points were plotted as trend and DHS was put as one point in 2016 (Figure 3). It also shows the HMIS data were much over estimates of FPET and DHS.
When HMIS data were compared with FPET, the direct estimates of mCPR from HMIS were used. However, LMIS could not provide the mCPR to compare with FPET. LMIS can provide only commodity consumption data. Therefore, we have to use EMU (Estimated method use) excel tool which is developed by Track 20 team and it can provide EMU (Estimated method use-a proxy of mCPR) by converting commodity consumption data from LMIS.
The FPET and EMU tools are not comparable and EMU results can be used as input for FPET if we want to incorporate service statistics data into FPET in addition to survey data. But it is not the case of this study as Myanmar FPET estimates use only survey data (due to poor data validity of service statistics data). Here EMU tool was used to provide EMU (Estimated method use-a proxy of mCPR) by converting of commodity consumption data. The links of these tools are also added in revise version.
For comparing the pattern of method mix, FPET cannot provide that data, we used DHS data for only that year of survey (2016) and compared with HMIS data. So, actually it doesn’t compare the two tools (FPET and LMIS tool) and compare the service statistics data (HMIS and LMIS through EMU tool) with FPET for mCPR progress and HMIS with DHS for method mix.
To be more clear, I have tried to edit the methodology section as follows and added text are made as italic;

Methods
Data sources
In order to understand what data may best serve annual monitoring of the performance of the family planning program in Myanmar, both at the national and state/regional level, four data sources of modern contraceptive use:
< >Modelled mCPR estimates (and confidence intervals) from Track20’s FPET, based on nationally and state/regionally representative surveys for Myanmar: Myanmar DHS (2015-16), Multiple Indicator Cluster Survey (2010), Fertility and Reproductive Health Survey (1991, 1997, 2001 & 2007). (7). (While FPET tool can be incooperated with both survey and service statistics data, FPET run for Myanmar could not use the service statistics data due to data validity.)mCPR and Method-specific prevalence from the 2015-16 Myanmar DHS (6).mCPR estimates and contraceptive method prevalence from Myanmar’s HMIS system (8) based on local annual census conducted by midwives on contraceptive use among married women in their catchment areas; which is different from other routine HMIS indicators and collected once a year at the end of calendar year for the information of contraceptive use including type of methods used, and mCPR from Estimates of modern method use (EMU) tool based on contraceptive commodity consumption data from Myanmar’s LMIS system (9); were used to compare (Table 1). (While FPET can be incorporated service statistics data, it could not be put the direct estimates of mCPR, other commodity or visit data from service statistics. EMU excel tool can convert service statistics data to Estimated method use (which is a proxy for mCPR) to be can be used as input in the Family Planning Estimation Tool (FPET). EMU tool was developed by Track 20 team and available at its website http://www.track20.org/pages/our_work/innovative_tools/SS_to_EMU_tool.php. Here, EMU tool was used to provide EMU (Estimated method use-a proxy of mCPR) by converting of commodity consumption data.) strengthen of HMIS data through reviewing the methodology of data collection for family planning information with further primary research and validating the data quality;establish the Family Planning register/record and integrate into electronic HMIS/DHIS;strengthen the LMIS for reproductive health commodity in terms of data quality as well as area coverage.

Comments

This reviewer has not had access to the data, but an illustrative example merits consideration by the authors, either by proposing some other adjustment procedure or by pursuing a version of what I recommend. Consider the following: At one point in the paper, the DHS is alluded to as the “gold standard” for mCPR estimation. The FPET and DHS estimates are similar (Figure 1), but HMIS data appear to be over-estimating the mCPR. If the DHS is considered the gold standard, then a plausible proposition is that its data can be used to develop areal adjustment factors. A regression analysis, with DHS data employed as the dependent variable, HMIS data as an independent variable, with direct and interaction effects for population characteristics and locality indicators could be used to develop regression weights that estimate the discrepancies associated with a set of indicators. The regression weights would define adjustment factors for correcting data to a DHS standard. The correspondence of FPET with the DHS invites an approach that would use the FPET as the “silver standard” for HMIS adjustment. While the paper might not take this up as a technical task, it should include discussion of methods of data adjustment that results appear to invite. This would be a more cogent set of recommendations than the text that now appears in the Recommendations section.

Actually, FPET tool is based on Bayesian, hierarchical model that fits curves to historical data (all available survey and service statistics). The model fits a logistic growth curve to CPR data for all methods to determine the long term trend in contraceptive use and adds a time-series model with autocorrelation to capture country-specific deviations around the long term trend. Since FPET gives almost the same result as DHS, it is assumed as the most reliable data source for annual estimates so far to compare the service statistics data (HMIS and LMIS) in this study. I hope it has already incorporated the adjustment for comparison of HMIS data with DHS. Also, I will add about this in methodology section in my revise version.

Comments for the figures

Figures 2 and 3 are hard to read and needlessly complex to interpret. There are many regions with pronounced variance between regions. Are there take home messages from these data that could be summarized more simply? For example, it is possible that the magnitude of discrepancies is associated with the level of the estimated mCPR. If HMIS level were on the Y axis and FPET were the X axis a simple regression line would simulate correspondence between the data sources, and the scatter around that line would show the pattern of discrepancy by level of mCPR. The title of Figure 3 is confusing. I think that Figure 3 could be dropped.

In figure 2 and 3, it shows year of consistency for mCPR between the HMIS and FPET estimates including 95%CI for each 17 State/Regions for three consecutive year’s data. The bar shows the HMIS data and FPET data points including upper and lower border (95% CI) are shown on that bar for each three years for 17 State/Regions. We can see the consistency that whether HMIS data (top of the bar) are fell within these upper and lower borders of FPET. For me, it is not easy to draw the regression line for two data sources of three years data points for each 17 State/Regions in one picture. Also, I found that there is no relation between the consistency and magnitude of value. The key messages are there are only two States (Chin and Kayin) for consistency in all three years (Figure 2) and some are two years and some are only one year consistent. About half of State/Regions show no consistency at all (Figure 3) for three years. It means that direct estimates of mCPR from HMIS are not consistency with FPET and not accurate to be used for annual monitoring. I will change the title for Figure 3 as “Comparing the HMIS and FPET results of mCPR by regions of none of the year consistent estimates” to be more clear as previously mentioned that “ by regions of no consistent estimates”.

Comments: Figure 4 is a largely empty depiction of a few data points. This could be replaced with a table.

Thank you for your advice. Actually, I would like to show that the comparison of HMIS and FPET (with 95% CI) for national mCPR with trend (three years data points) and rate of progress. It highlights HMIS data were overestimates of FPET for three years (even over the upper border in 2016 and 2017) and DHS as well in 2016. So, I think it could not easy for readers to see these points from table.

Comments: Figure 5 is visually confusing. For some reason, data correspondence in Chin is very different from the other regions. Coloration might help, but tabular data might be more interpretable to a reader.

In Figure 5, it was aimed to show the comparison of different contraceptive method mix (Injection, pills, IUD, implants, condom and female sterilization) prevalence data of HMIS and DHIS (2016) as well as the comparison of national level and state/regional level (Chin and Kayin) where there were different patterns of national level. The key messages are the method mix pattern of HMIS and DHS are quite similar except female sterilization. And while injection was the highest proportion at national level (both HIS and DHS), high proportions of long term methods and female sterilization were found at Chin from DHS (quite different with HMIS) and pills use was the highest in Kayin in DHS. It means that HMIS data were quite similar with DHS for method mix at national and other State/regions except Chin and Kayin. So, I think it might be not easy for readers to interpret if data were shown table.

Minor comments and suggestions!
The text of this paper begins with a reference to the maternal mortality rate, but the paper is on a different topic. This reviewer suggests that the authors open the paper with a statement that is a more direct reference to topic of the article and the lack of systematic attention to family planning research in Myanmar and the rationale for the paper. At some point in the discussion, reference to the relationship of family planning with the maternal mortality rate would be informative.

Thank you for your suggestions. I edited the introduction a lot with starting of importance of family planning, about Myanmar FP commitment, Myanmar FP program implementation to achieve FP 2020 commitment and reduce MMR, then how importance of monitoring the progress and current data source. And some text “Also, lack of studies for systematic analysis of the family planning data source called to do this study to set the annual family planning monitoring system in Myanmar” has added as the last sentence in rationale.

In discussion part, I have added the following text at the beginning.

“Access to FP contributes up to a 44% reduction in maternal deaths and a 21% reduction of deaths in children under age 5 (Ref), therefore, Myanmar needs to endevaour strong efforts to increase access to family planning services to tackle the high MMR issue. In the meantime, setting the systematic approach for continuous monitoring is important to provide the valuable information to be considered for future strategies”

(Ref: Starbird E, Norton M, Marcus R Investing in Family Planning: Key to Achieving the Sustainable Development Goals. doi: 10.9745/GHSP-D-15-00374)
Also, some editions for another reviewer have been incorporated in my revised version.

Thanks again and best regards,
Khaing

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

19 Views

24 Oct 2019 | for Version 1

Janine Barden O'Fallon, Carolina Population Center, Maternal & Child Health Department, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

19 Views Cite this report Responses(1)

Approved

It would be helpful to have contextual information about the development of the HMIS and LMIS in this section.
Additionally, some information on commodity consumption data may help readers better understand and interpret the findings.

Methods:

Please name or describe the data sources included in the FPET tool in Myanmar.
It should be made clear that the FPET and DHS are being used as the standards against which the HMIS and LMIS estimates are being compared. The methods section reads as though these four data sources are being compared against each other, which is not really the case.
The HMIS mCPR estimate is based on “local annual census conducted by midwives on contraceptive use among married women in their catchment area.” This seems to be an unusual source of data for an HMIS; can the authors provide more information on this?

Recommendations:

This section introduces some concepts that are not previously mentioned in the article (“the family planning register,” the “DHIS tool”), thus the basis for the recommendations may not be well-understood by most readers.

Discussion & Conclusion:

Please see the second sentence in the first paragraph of this section and revise for clarity.
A tie-in to other literature on this topic is missing; what has other research on this topic found and how does it fit with these results?
Do authors have any explanation about why HMIS mCPR estimates appear to be over-estimates? What specific “data quality and methodology” issues have been identified? What is being done to improve data quality?
Information on study limitations is missing.
“Data” is plural; the document needs editing to ensure proper noun-verb agreement.

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Monitoring and evaluation in global health programs; family planning

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Author Response

10 Dec 2019

Khaing Nwe Tin, Department of Public Health, Ministry of Health and Sports, Myanmar, Nay Pyi Taw, 15011, Myanmar

Dear Sir,

Thank you so much for your comments and these are very useful to revise my article. I agreed all of your points and would like to respond as follows;

Introduction:

It would be helpful to have contextual information about the development of the HMIS and LMIS in this section.
Additionally, some information on commodity consumption data may help readers better understand and interpret the findings.

I will add one paragraph in the last of introduction section regarding the current data source for FP monitoring in Myanmar and will also explain about the HMIS and LMIS including commodity consumption data as follows:

“In order to monitor the progress of family planning program, Myanmar uses the FP indicators (CPR, method mixed prevalence, unmet need for family planning and demand satisfaction) from the limited available sources mainly; first ever Demographic and Health Survey (DHS)(2015-16) and Health Management Information System (HMIS), as well as commodity consumption data from Logistic Management Information System (LMIS). HMIS was developed since 1995 as paper based and now transformed to electronic system of HMIS as District Health Information System (DHIS) 2. It is the health information system of Myanmar Ministry of Health and Sports to monitor the performance of all public health facilities while it accounts as local annual census for family planning data and covers both public and private. In Myanmar, LMIS for reproductive health commodity (RHCLS: Reproductive health commodity logistic system) is initiated as national program since 2016 in selected region and rolling out phase by phase. It provides the information related with stock status as well as commodity consumption data; commodity distributed to the clients, from different public health facilities. However, systematic tracking of annual family planning progress with valid data source has not yet been set yet.”

Methods:

Please name or describe the data sources included in the FPET tool in Myanmar.

I will insert the names of surveys that included in FPET for Myanmar as follows in data source part:

“Modelled mCPR estimates (and confidence intervals) from Track20’s FPET, based on nationally and state/regionally representative surveys (7): Myanmar DHS (2015-16), Multiple Indicator Cluster Survey (2010), Fertility and Reproductive Health Survey (1991, 1997, 2001 & 2007).

It should be made clear that the FPET and DHS are being used as the standards against which the HMIS and LMIS estimates are being compared. The methods section reads as though these four data sources are being compared against each other, which is not really the case.

Thanks for suggestion and I removed the clause of “four data sources were compared” and edited as follows:

“In addition to FPET, to identify the most valid data source for annual monitoring, service statistics data (HMIS and LMIS) of CPR were compared along with the confidence intervals from FPET for three consecutive years from 2015 to 2017.”

The HMIS mCPR estimate is based on “local annual census conducted by midwives on contraceptive use among married women in their catchment area.” This seems to be an unusual source of data for an HMIS; can the authors provide more information on this?

Yes, it is unusual data collection and I also added some information in introduction and added in data source as follows;

“mCPR estimates and contraceptive method prevalence from Myanmar’s HMIS system (8), based on local annual census conducted by midwives on contraceptive use among married women in their catchment areas; which is different from other routine HMIS indicators and collected once a year at the end of calendar year for the information of contraceptive use including type of methods used,”

Recommendations:

This section introduces some concepts that are not previously mentioned in the article (“the family planning register,” the “DHIS tool”), thus the basis for the recommendations may not be well-understood by most readers.

I have added the following paragraph in discussion to be more understandable that recommendation.

“While the methodology for data collection is quite unusual as local annual census, it is also needed to reconsider to be consistent with other HMIS indicators through the register/record for family planning information and integrated into electronic HMIS (DHIS 2) to get the valid data of family planning service visit and contraceptive users at least for public sectors.”

Discussion & Conclusion:

Please see the second sentence in the first paragraph of this section and revise for clarity.

I have revised as follows;

When considering the data source for the annual tracking of mCPR (either estimates from FPET or routine service statistics), the use of direct estimates of HMIS should be considered carefully as it is much higher than the DHS survey and not matched with FPET results except in Chin and Kayin States.

A tie-in to other literature on this topic is missing; what has other research on this topic found and how does it fit with these results?

I have added the findings of international literature as follows;

These findings are consistent with other international literature. According analysis of three service statistics data (number of: contraceptive commodities distributed to clients, family planning service visits, and current contraceptive users) from 22 countries in Africa and Asia, it was observed that none of these three data were accurate to be used when comparing with United Nations Population Division World Contraceptive Use annual mCPR estimates as the "gold standard."
Although service statistics data for family planning are insufficient accuracy to use stand-alone like survey data, data on contraceptive commodities distributed to clients, family planning service visits, and current users can be used accurately when combined with survey data in FPET tools to estimates the trends of annual mCPR in the years between surveys.

(Ref) Magnani RJ, Ross J, Williamson J, Weinberger M. Can family planning service statistics be used to track population-level outcomes?.Glob Health Sci Pract. 2018;6(1):93-102.https://doi.org/10.9745/GHSP-D-17-00341

Do authors have any explanation about why HMIS mCPR estimates appear to be over-estimates? What specific “data quality and methodology” issues have been identified? What is being done to improve data quality?

As there has not been explored systematically yet, I have not known the exact reasons as this is just secondary data analysis, so I put in the recommendation. But I would add as follows in discussion to be more clear to readers;

Currently, it has limited information from documented references about that why HMIS appear overestimates and grow 5% annually, however, to my working experiences, it is doubtful in data quality by midwives. According to the methodology, it is needed to ask all married women under their jurisdiction areas, it is difficult to cover especially in big city, per-urban as well as hard-to-reach areas. Therefore, it might be possible that it includes only the information of women those who got family planning services from them and probably higher prevalence than those who could not access the health services. However, these are just only the opinions of authors and need to explore systematically and should improve based on that findings.

Information on study limitations is missing.

Thank you for your good point to remind me. I missed it and now I will add as follows;

“This study is only the secondary data analysis, therefore, the reasons behind the inaccuracy of service statistics data could not explain. It covers only Myanmar data so detail results might be limitation to be applied for other countries context although it is consistent with findings of analysis in 22 countries which is service statistics data were not sufficient in accuracy to be used stand alone for annual tracking.”

“Data” is plural; the document needs editing to ensure proper noun-verb agreement.

Thank you for your suggestions and will edit accordingly.

Best regards,
Corresponding Author: Khaing Nwe Tin

View more View less

Competing Interests

There is no competing interests.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. Ministry of Health and Sports: Five-Year Strategic Plan for Reproductive Health (2014-2018). Nay Pyi Taw, The Republic of the Union of Myanmar. 2013. Reference Source

[2] 2. Department of Public Health: Costed Implementation to Meet FP2020 Commitments Myanmar 2014. Nay Pyi Taw, The Republic of the Union of Myanmar. 2015. Reference Source

[3] 3. Global Health Observatory data repository | By category | Maternal mortality - Data by country. In: WHO. Accessed 12 Aug 2018. Reference Source

[4] 4. Department of Population, Ministry of Immigration and Population: The 2014 Myanmar Population and Housing Census. Department of Population, Ministry of Immigration and Population, Nay Pyi Taw. 2015. Reference Source

[5] 5. Ministry of Health and Sports: Myanmar National Health Plan (2017-2021). Nay Pyi Taw, The Republic of the Union of Myanmar. 2016. Reference Source

[6] 6. Department of Population and UNFPA Myanmar: Country Report on 2007 Fertility and Reproductive Health Survey. 2009. Reference Source

[7] 7. Ministry of Health and Sports: Myanmar Demographic and Health Survey 2015-16. Nay Pyi Taw, Myanmar ICF, Ministry of Health and Sports, Nay Pyi Taw, The Republic of the Union of Myanmar. 2017. Reference Source

[8] 8. Ministry of Health and Sports: Health Management Information System data dictionary. 2018.

[9] 9. Maternal and Reproductive Health: Reproductive Health Commodity Logistic System report. 2017.

[10] 10. Maternal and Reproductive Health: Family Planning Estimation Tool results. 2017.

[11] 11. Ministry of Health and Sports: Health Management Information System data. 2015–17.

[12] 12. Ministry of Health and Sports: Reproductive Health Commodity Logistic System data. 2017.

[13] 13. Starbird E, Norton M, Marcus R: Investing in Family Planning: Key to Achieving the Sustainable Development Goals. Glob Health Sci Pract. 2016; 4(2): 191–210. PubMed Abstract | Publisher Full Text | Free Full Text

[14] 14. Magnani RJ, Ross J, Williamson J, et al.: Can Family Planning Service Statistics Be Used to Track Population-Level Outcomes? Glob Health Sci Pract. 2018; 6(1): 93–102. PubMed Abstract | Publisher Full Text | Free Full Text

Setting the stage for strengthened annual monitoring of Family Planning program performance at the National and Subnational level in Myanmar

Abstract

Keywords

Revised Amendments from Version 2

Introduction

Rationale

Figure 1. Comparing National mCPR estimates of DHS, FPET, and HMIS.

Objectives

Methods

Data sources

Table 1. Data sources for comparison.

Ethics approval and consent to participate

Results

Comparing mCPR from HMIS and FPET

Figure 2. Comparing the HMIS and FPET results of mCPR by Regions of at least one year consistent estimates.

Figure 3. Comparing the HMIS and FPET results of mCPR by Regions of zero years consistent estimates.

Figure 4. Comparing National mCPR of HMIS and FEPT estimates.

Comparing the method prevalence/mix from HMIS and DHS

Figure 5. Comparing the modern contraceptive method mix.

Comparing the data of LMIS with HMIS data and FEPT results

Discussion and conclusion

Limitation of study

Recommendations

Data availability

Underlying data

Acknowledgments

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Are you a Gates-funded researcher?

Thank you!

Setting the stage for strengthened annual monitoring of Family Planning program performance at the National and Subnational level in Myanmar

Abstract

Keywords

Revised Amendments from Version 2

Introduction

Rationale

Figure 1. Comparing National mCPR estimates of DHS, FPET, and HMIS.

Objectives

Methods

Data sources

Table 1. Data sources for comparison.

Ethics approval and consent to participate

Results

Comparing mCPR from HMIS and FPET

Figure 2. Comparing the HMIS and FPET results of mCPR by Regions of at least one year consistent estimates.

Figure 3. Comparing the HMIS and FPET results of mCPR by Regions of zero years consistent estimates.

Figure 4. Comparing National mCPR of HMIS and FEPT estimates.

Comparing the method prevalence/mix from HMIS and DHS

Figure 5. Comparing the modern contraceptive method mix.

Comparing the data of LMIS with HMIS data and FEPT results

Discussion and conclusion

Limitation of study

Recommendations

Data availability

Underlying data

Acknowledgments

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Competing Interests Policy

Stay Updated

Are you a Gates-funded researcher?

Thank you!