Estimated Modern Use: Employing A Service Statistics-Based Indicator to Monitor Family Planning Programs

Except for a few countries that collect true FP Users data, which tracks individual users over time, FP routine service statistics data in most countries do not represent actual FP use, but rather, a count of FP services provided, or FP commodities distributed. To account for this, some adjustments are made to the raw data provided on FP Commodities or FP Visits to convert them to estimated FP Users to produce the numerator for the EMU. The adjustments made to convert the raw service statistics into FP Users considers 3 groups of FP Users in any given year:

Current Users

To estimate the number of current FP Users in the year a service was provided, we vary our assumptions by method type.

For short-term methods, those methods that are only effective for a short duration and require refills or revisits within the same year to continue to protect against pregnancy, we use the globally accepted CYP factors to convert services or products into users. For example, four visits for DMPA-IM, or 4 injectables dispensed, is equal to one CYP and one estimated modern user.

Using condom distribution numbers from HMIS is particularly challenging. It can be difficult to determine the number of condoms used specifically to prevent pregnancy because they may be distributed for multiple purposes and through non-FP programs and are often left for clients to take freely instead of distributed in a carefully documented manner. Because of this, in countries where condoms are not a common method of family planning, they may be excluded from EMU calculations.

Short-Term Methods

Where:

$m_{\mathit{STM}}$ is the type of short-term method,

s is the type of service statistic (commodities to clients, commodities to facilities, visits),

t is year,

$U_{m_{\mathit{STM}},s,t}$ is the number of users of a short-term method, $m_{\mathit{STM}}$ , of service statistics type s, in year t,

$C_{m_{\mathit{STM}},s,t}$ is the number of commodities or visits of the short-term method, $m_{\mathit{STM}}$ , of service statistics type s, in year t

${\mathit{CYP}}_{m_{\mathit{STM}},s}$ is the couple years protection for the short-term method, $m_{\mathit{STM}}$ , and service statistics type, s

For long acting reversible methods, which are provided in a given year but can continue to provide protection in subsequent years, we estimate that each service or product (IUD/Implant) inserted is equivalent to one user, with some small adjustment to account for first-year discontinuation (for example, CYP continuation curves estimate there is a 92% continuation in the first year for Copper T IUDs). More discussion on how the calculation accounts for future years is included in the next section on “Continuing Users.”

Long-Term Method, Current Users

Where:

$m_{\mathit{LTM}}$ is the type of long-term method,

s is the type of service statistic (commodities to clients, commodities to facilities, visits),

t is year,

$U_{m_{\mathit{LTM}},s,t}^{\mathit{Current}}$ is the number of current users (who received their method in year t) of a long-term method, $m_{\mathit{LTM}}$ , of service statistics type s, in year t,

$C_{m_{\mathit{LTM}},s,t}$ is the number of commodities or visits of the long-term method, $m_{\mathit{LTM}}$ , of service statistics type s, in year t

${\mathit{Con}}_{m_{\mathit{LTM}},1}$ is the continuation rate of method $m_{\mathit{LTM}}$ in year 1

For Permanent Methods, which are provided in a given year but continue to provide protection for the duration of the user’s reproductive life, we estimate that each service is equal to one user in that year. More discussion on how the calculation accounts for future years is included in the next section on “Continuing Users”.

Continuing Users

A critical difference between CYP and EMU is the distribution of contraceptive use over time. CYPs apply the full impact of long acting and permanent methods (LAPMs) to the year in which the method was provided. When counted in this way, an implant, for example, would show benefit in the year it was inserted, but no impact in future years, even though it continues to provide contraceptive protection for several years into the future. A direct conversion of CYPs to FP Users would result in an overestimation of users in the year of service provision and can skew trends in users when rapid scale-up or declines in service provision occur. In the EMU calculation, a “carryover” approach is used, in which that impact is distributed across the years a method would be in use, based on the standard continuation rates used to develop the CYP factors ⁷ ( Figure 1).

Figure 1. Long acting reversible contraception continuation curves.

Long-term Method, Continuing Users

$m_{\mathit{LTM}}$ is the type of long-term method,

s is the type of service statistic (commodities to clients, commodities to facilities, visits),

t is year,

$t_0$ is the first year of data collection,

$U_{m_{\mathit{LTM}},s,t}^{\mathit{Continuing}}$ is the number of continuing users (who received their method during data collection, prior to year t, and are still expected to be users in year t) of a long-term method, $m_{\mathit{LTM}}$ , of service statistics type s, in year t,

$C_{m_{\mathit{LTM}},s,t-x}$ is the number of commodities or visits of the long-term method, $m_{\mathit{LTM}}$ , of service statistics type s, x years prior to year t

${\mathit{Con}}_{m_{\mathit{LTM}},x+1}$ is the continuation rate of method $m_{\mathit{LTM}}$ in x+1 years of use

In the example illustrated in Figure 2, 100 of each LAPM were provided in 2020. Based on those services in 2020, we estimate 670 users in 2020. Each year some of those users are assumed to continue using their method until it is removed, no longer effective, or until they age out of “childbearing age”. By 2030, we assume all of those users who initiated in 2020 are no longer using that method, or in the case of sterilization, no longer of childbearing age. As a specific example, if a Copper-T 10-year IUD was inserted in 2020, all 4.6 CYPs would be counted in that year. However, because EMU estimates users in a given year, if 100 Cooper-T are inserted in 2020, the SS-to-EMU tool assumes that 92 will be used for a year, 77 for two years, 65 for three years and so on, following the continuation curve which underlies the CYP calculation. This serves to smooth out trends in long acting contraceptive use, as there may be fluctuations in the acceptance of LAPMs by clients which are unlikely to result in immediate fluctuations in use as women continue to use these methods over several years.

Figure 2. Estimated users for long acting and permanent methods from one year of services.

One limitation of the current approach is that a single global continuation curve is used for each method, which does not take into account the country-specific variability in method continuation indicated by DHS data. Limitations to a country-specific approach include lack of discontinuation data for many countries and timeliness of data for newly popular methods when using a historical reproductive calendar for calculating discontinuation rates. The continuation rates from CYP estimates are calculated using results from multiple studies. Cohort studies were prioritized for the CYP analysis, to reduce recall bias from the retrospective calendars in DHS. However, there were not enough cohort studies to rely solely on this approach, so the estimates were created using a combination of cohort data and secondary analysis of DHS data.

Track20 tested using regional discontinuation rates, but the variation within regions (among countries with discontinuation data) were the same in scale as those using the global rates, meaning that they did not improve on the calculation. Future testing could include different ways to reflect country variation.

Historic Users

As previously discussed, LAPM services require special consideration when estimating users because their use/impact can continue for up to 10 years [ 4] after a service is provided. Routine service statistics data are often limited in terms of the time trends available due to system changes and lack of electronification of past service data. As a result, it is common that countries have less than 10 years of service statistics data available in their current system, meaning that there are women using LAPMs that were provided before the first year of available data, but about whom we have no information. Disregarding these users runs the risk of substantially underestimating current levels of contraceptive use, especially in countries where a greater proportion of women use LAPMs, as it would assume that no service provision occurred before the first year of available data.

To estimate “historic users” (those still using a method provided before the first year of available data), we make some assumptions about LAPM distribution before the first year of available data. The calculation of historic users is based upon the maturity of the FP program in the first year of data collection for each LAPM- was provision of this method already at scale, still scaling up, or just beginning the year data collection began? If provision of the method started in the first year of data collection, we assume there are no historic users. If the method being provided was at scale in the first year of data collection, we assume that the number of insertions in previous years was similar to the number of insertions in the first year of data collection. If the program was scaling up distribution of this method, we assume the number of historical users falls between the two other scenarios. For example, if a data system began recording methods distributed in 2017, but the country began inserting Implanon in 2013, we expect that some women received implants in 2016 or earlier and are still implant users in 2017 and beyond ( Figure 3). Based on the scenario identified and the first year of data available, the EMU calculation estimates the number of women who received LAPMs before the first year of data and then applies the same continuation trends used to estimate continuing users. This allows the EMU to better approximate actual contraceptive use, integrating LAPM users from prior years into the years for which data is available.

Figure 3. Estimated implanon users, by year of use and year of insertion.

Long-term Method, Historic Users

$m_{\mathit{LTM}}$ is the type of long-term method,

s is the type of service statistic (commodities to clients, commodities to facilities, visits),

t is year,

$t_0$ is the first year of data collection,

$U_{m_{\mathit{LTM}},s,t}^{\mathit{Historic}}$ is the number of estimated historic users (who received their method before data collection started and are still expected to be users in year t) of a long-term method, $m_{\mathit{LTM}}$ , of service statistics type s, in year t,

$C_{m_{\mathit{LTM}},s,t_0}$ is the number of commodities or visits of the long-term method, $m_{\mathit{LTM}}$ , of service statistics type s, in the first year of data collection, $t_0$

$L_{m_{\mathit{LTM}}}^{\mathit{max}}$ is the maximum number of years of continuation for long-term method, $m_{\mathit{LTM}}$ ,

${\mathit{Con}}_{m_{\mathit{LTM}},x+2}$ is the continuation rate of method $m_{\mathit{LTM}}$ in x+2 years of use

The extent to which these estimates impact a countries EMU depends on the country’s method mix, number of years they have data in their HMIS, and the extent to which their family planning program has changed or prioritized different methods in the recent past. A country that primarily relies on short acting methods will see little impact in their current number of users based on these estimations. Similarly, a country that has had a robust implant program for more than five years will not see an impact on estimates of current users, since the available data covers the period of effectiveness prior to the current year. These calculations will have the largest impact in countries that are scaling up long acting or permanent methods and have been doing so for only a few years. Importantly, as countries accumulate more years of data, these calculations have no impact on current data.

While historic users will always be an estimate, the benefits of this approach outweigh the risk of ignoring prior service provision, and those risks are minimized by our approach in the following ways:

Adding all three types of long-term method users together results in the estimated number of long-term methods users in a year.

Long-term Methods, Total Users

Where:

$U_{m_{\mathit{LTM}},s,t}$ is the total number of users of long-term method, $m_{\mathit{LTM}}$ , of service statistics type s, in year t,

$U_{m_{\mathit{LTM}},s,t}^{\mathit{Current}}$ is the number of current users (who received their method in year t) of a long-term method, $m_{\mathit{LTM}}$ , of service statistics type s, in year t,

$U_{m_{\mathit{LTM}},s,t}^{\mathit{Continuing}}$ is the number of continuing users (who received their method during data collection, prior to year t, and are still expected to be users in year t) of a long-term method, $m_{\mathit{LTM}}$ , of service statistics type s, in year t,

$U_{m_{\mathit{LTM}},s,t}^{\mathit{Historic}}$ is the number of estimated historic users (who received their method before data collection started and are still expected to be users in year t) of a long-term method, $m_{\mathit{LTM}}$ , of service statistics type s, in year t.

Combining short-term and long-term users together leads to the total number of estimated modern users.

Estimated Modern Users

Where:

s is the type of service statistic (commodities to clients, commodities to facilities, visits),

t is year,

$U_{m_{{\mathit{STM}}_i},s,t}$ is the number of users of a short-term method, $m_{{\mathit{STM}}_i}$ , of service statistics type s, in year t,

$U_{m_{{\mathit{LTM}}_i},s,t}$ is the number of users of a long-term method, ${m_{\mathit{LTM}}}_i$ , of service statistics type s, in year t,

$M_{\mathit{STM}}$ is the total number of short-term methods,

$M_{\mathit{LTM}}$ is the total number of long-term methods,

${\mathit{EMU}}_{s,t}$ is the number of estimated modern users captured by service statistics type s, in year t

At this point, the EMU calculation represents estimated modern users whose family planning services are recorded in a country’s HMIS. This version of the indicator is used by countries to monitor their programs and track trends in contraceptive use at the national and subnational levels. In most countries, for the EMU to be used as a data source to model contraceptive prevalence, there is one additional adjustment that needs to be made - accounting for family planning services that are not included in a country’s HMIS.

The three groups - current users, continuing users, and historic users - combine to form an “unadjusted EMU”. It is unadjusted because it does not consider women who may have received contraceptive methods outside of the facilities and/or sectors that report into the national HMIS. The final adjustment integrated into the EMU calculation is intended to make the EMU more representative of the full FP market. In most countries, parts of the private sector report data into HMIS, but it is very rare that the full private sector reports. Efforts have been made to better integrate data from private sector providers, including NGOs providing family planning services and private hospitals and clinics, but few countries have pharmacies or other non-medical sources reporting into the system. This impacts methods differently, for example in many countries, shops are key sources of pills and condoms, and they rarely report into HMIS. The EMU calculation uses a method-specific adjustment factor, applied to the service statistics data, to account for provision in sectors not represented in the available data. This adjustment factor is based on data from DHS (where available) concerning the source of methods for current users of contraception, along with information estimating how much private sector provision is captured in their data.

Multiplying unadjusted EMU (by method) by the method-specific private sector adjustment factors results in the adjusted EMU. All method-specific EMUs are aggregated to form the total EMU. Dividing the adjusted EMU by women of reproductive age gives us the EMU value that can be used in FPET to inform annual family planning estimates.

These transformations are all intended to help the EMU serve as a proxy for contraceptive coverage accounting for the historic provision of LAPMs prior to available data, the continued impact of LAPMs over years following the initial provision, and the provision of services in sectors not represented in HMIS. This positions the EMU to be different, yet complimentary, to CYP, with the EMU being better positioned for active monitoring.

A variety of data inputs are used for producing the EMU, see Box 1. Some data are entered by the user, and other data are pre-populated in the tool from trusted data sources.

EMU is calculated using any of the four types of family planning routine data typically collected by programs: commodities distributed to clients, commodities distributed to facilities, family planning visits, and family planning users. Service statistics are entered into SS-to-EMU by the user and when possible, users are encouraged to enter more than one type of service statistic to allow for comparison during the data review phase.

The SS-to-EMU Tool makes the previously discussed adjustments to the service statistics entered to convert them to FP Users for the EMU calculation.

Population estimates for all women ages 15 – 49, by year, are pre-loaded into the SS-to-EMU Tool from the UN Population Division. ⁸ These estimates serve as the denominator for the EMU calculation. Population estimates for all women of reproductive age are preferred as the denominator (as compared to married women of reproductive age) as aggregated service statistics data generally does not contain information about the marital status of the clients who receive FP Services [ 5]. If the user has a preferred data source for population estimates, such as a census, the user may use their own source and replace the pre-populated data.

Data on the distribution of modern users of each method by the source from which they received that method is pulled from DHS to inform the EMU private sector adjustment discussed previously, providing insight on the public sector market share.

Reporting rates are entered into SS-to-EMU by the user. The reporting rate provides an indication of the share of facilities represented in the HMIS data. Reporting rates are a standard measure in many countries’ HMIS, though they vary extensively in what they represent. Generally, reporting rates refer to the percentage of facilities that submit monthly or annual reporting forms (reporting routine service statistics data) into the national HMIS. Some systems differentiate between “on time” reporting rates (the % of facilities submitting forms by the monthly reporting deadline) and “overall” reporting rates (the % of facilities that have submitted forms at all by the time data is being extracted from the system). Reporting rates recorded in HMIS may not be specific to FP data but can refer to reporting for a broader health area. This depends on whether there is a distinct reporting form for submitting FP-related data or whether FP is included in a broader maternal and reproductive health form or if all health areas are reported up from the facility in a single form. For the purpose of the SS to EMU, FP-specific reporting rates are preferred.

Data from nationally representative surveys including DHS, MICS, Performance Monitoring for Action (PMA), and other national surveys provide method prevalence and method mix data for benchmarking. SS-to-EMU also uses mCP estimates from FPET for benchmarking.

Because routine service statistics are based on facilities reporting into HMIS, data completeness, an important element of data quality, is dependent on high reporting rates. A reporting rate of 100% would indicate that all services delivered by the public sector are captured in the HMIS data. The Track20 project typically recommends a reporting rate of 80% or higher for service statistics to be included in the EMU trend to ensure data are representative and largely complete. The SS-to-EMU Tool does not adjust for low reporting rates as it does for missing private sector data. Instead, low reporting rates will result in the exclusion of EMUs in that year from trend data. However, no matter the reporting rate, working through the data review and assessment processes can be an important and routine step incorporated into a country’s family planning monitoring system.

Users are encouraged to enter as many types of service statistics into the SS-to-EMU tool as are available in their system. This allows the user to examine the relationship between different types of service statistics, which can provide important internal benchmarking. The SS-to-EMU Tool displays the inputted data in ways that allow the user to easily identify outlier data points, facilitating follow-up and potential data corrections, if necessary.

Ideally, trends in the different data types should be similar, as should the distribution of methods. There are some typical relationships between data sources that users can look for. For example, LMIS data on commodities sent to facilities should typically be higher than data on commodities given to clients because not all methods sent to facilities are used. Commodities to clients and visits data for single dose methods (IUDs, DMPA-IM, implants) will be similar because one commodity unit is distributed per visit. Sterilization is more commonly recorded in visits and users, but less likely to be reported in commodities data, resulting in a potential gap when only looking at one type of data.

The tool helps walk through these different relationships and uses clear visualizations that can help users see how their data relate to each other and if the relationships make sense.

The SS-to-EMU tool is unique in that it includes routine service statistics alongside survey data and mCP from FPET global projections, so tool users can benchmark their EMU trends (derived from their service statistics) against trusted external data.

For years with survey data available, users can compare rates and absolute numbers. For absolute numbers, estimated users by method can be compared to the number of users calculated from survey method prevalence multiplied by the population of women of reproductive age. Although one would not expect estimated users from surveys and service statistics to be equal (due to methodology and adjustments), how they compare can provide some indication of whether the data is generally consistent with external sources or can point to potential issues with the data inputs. For example, one would expect that unadjusted estimated users (before one accounts for the private sector) would generally be lower than survey-based estimates of users and that the private sector adjustment would bring the estimate of users closer to the survey-based estimates. There is a specific graph included in the data review that shows the impact of the private sector adjustment factor in comparison to survey-based estimates and allows the user to revisit the adjustment factors if the impact of those adjustments appears to over- or under-estimate use. If the estimates of users are substantially higher than survey-based estimates, especially for a specific method, that may point to potential issues with double counting or quality issues in HMIS. Finally, one would also expect that estimated users is likely to diverge from survey data the farther out one is from a survey. When comparing service statistics from 2023 against a survey conducted in 2018, one would expect some variation, especially as program priorities and the scale of method use may have changed.

While it is expected that EMU and surveys will trend together, there are some specific FP methods that can cause particular challenges for estimating use, generally related to how data is recorded and gaps between distribution and consumption. LAM (Lactation Amenorrhea Method) is not always recorded in the HMIS, or if it is recorded, it is in reference to LAM counseling as opposed to measured use. The tool allows for the exclusion of condoms because of the various motivations for condom use and in accurate recording in HMIS. Overall, understanding how and why different methods may result in an EMU that varies from survey data can help the user identify potential data quality issues, make decisions about how to use the data, and determine which data type is providing the most accurate picture of use.

In addition to doing comparisons in the year of a survey, users also look at annual rates of growth between surveys and the corresponding trends to EMU. Major differences in rates of change can highlight potential data quality issues.

Aligned with its original intent, the EMU indicator has supported annual progress reporting for the FP2020 initiative, and its follow-on FP2030, as well as the Ouagadougou Partnership, through its use as an input into FPET and the resulting estimates of key family planning indicators (mCP, Unmet Need, and Demand Satisfied). EMU allows for the introduction of service statistics into family planning modeling, particularly beneficial in years without surveys. It provides better representation of recent changes in family planning programs, impacting estimates and narrowing confidence intervals. Since 2014, 26 countries have used EMU to inform their annual FPET estimates in at least one year of reporting.

Because EMU is a standard measure, it has been adopted by various donors and partners as part of their annual monitoring process. EMU is a required core indicator on the USAID MOMENTUM suite of grants which implements activities across 38 countries. EMU has also been adopted by the Countdown to 2030 global movement for women’s, children’s, and adolescents’ health as its approach to measure coverage for family planning across 81 focus countries.

In addition, because the EMU can be calculated at sub-national levels, it can be particularly useful for tracking progress in specific sub-national geographies where different investments are made and would provide governments, projects, and donors with a common indicator to use across national and subnational investments.

The SS-to-EMU tool is used to calculate EMU independently in over 30 countries as part of routine annual family planning monitoring and EMU is now auto-calculated monthly within 13 [ 6] country FP HMIS environments through Track20’s FP DataPro. FP DataPro is an FP and MNH environment available for download from the DHIS2 App Hub that includes EMU calculation and aspects of the SS-to-EMU data quality review, along with complex data analytics and visualizations to support active monitoring. ¹⁰

Regular utilization of the data quality review through the SS-to-EMU tool facilitates the identification of outliers and other data anomalies and supports quick follow-up with facilities to confirm entries and make corrections if necessary, improving data quality over time.

Because EMU is a service-based indicator and can be calculated annually, quarterly, or monthly, it is responsive to immediate changes in service volume. This means that the EMU will reflect trend shifts due to program scale-ups, service disruptions, and other changes long before a survey would, providing more immediate feedback. EMU proved especially helpful during the COVID-19 pandemic as countries strove to understand whether COVID-19 impacted FP uptake and use. Subnational EMU calculations provide additional depth. This has allowed countries to make more responsive and targeted programmatic changes between surveys.