The National Composite Index for Family Planning (NCIFP): Results and Methodological Issues

Global and regional results

In 2017, the average total score was 63.9, about two-thirds of the maximum of 100.0. This varied from 57.3 to 76.6 across the five dimensions of effort. The 2017 scores by dimension and region are displayed in Figure 1. Anglophone sub-Saharan Africa (SSAF-A) scored the highest across all dimensions. Rankings for the other regions varied, except that the Eastern Europe and Central Asia (EECA) region fell well below the others on all dimensions except equity. In every region except EECA, Strategy was the highest scoring dimension.

Figure 1. 2017 NCIFP by Region and Dimension (weighted).

Trends

How much did the scores improve between 2014 and 2017? Because of methodological changes between the two rounds, the trends can be assessed only for score items that were worded the same in both surveys. These are the “FPE” items that were carried over from the long-term series conducted approximately every five years starting in the 1987s. By dimension these are:

Strategy:

Data:

Quality:

Equity:

(The fifth dimension, Accountability, was created with entirely new questions.)

Overall, these scores improved over time, as they did in the inter-survey intervals in previous FPE surveys. Figure 2 presents the 2014 and 2017 results for the 13 items above. Overall, the scores ranged in the mid-50s, with the highest for Strategy in 2017 at 57.9 and the lowest two for Data and Equity at 51.8 and 52.0 in 2014.

Figure 2. Comparison of 2014 and 2017 Scores for FPE Items, by Dimension.

Against those levels, the increases ranged from 1.3 to 3.2 across the four dimensions, or roughly 2.2%, 5.0%, 6.1%, and 2.6% for Strategy, Data, Quality and Equity, respectively. The score change in Data between 2014 and 2017 suggests an increased emphasis on program monitoring through data. Nevertheless, the 2017 level is just above half of programs with adequate record keeping and data use.

Patterns have remained largely the same with Strategy being the highest scoring dimension by a small margin in both years. These results are shown by individual scores in Figure 4 and Figure 5, first for levels and second for changes; note that under Equity access to the seven contraceptive methods is combined for long term versus short-term methods.

Country variations

Score variations by country and dimension are shown in Figure 3, arranged by the total score within each region in 2017. There is a substantial country variation in scores within each region^e. EECA has the widest range of total scores, with Tajikistan as the highest score (87.9) and Romania as the lowest (30.0). Countries in the Middle East and North Africa (MENA) have the smallest range of total scores, from 59.5 for Morocco to 74.7 for Jordan. Among all regions, Rwanda has the highest score (91.6) and Romania the lowest.

Figure 3. 2017 NCIFP Dimension Scores by Country and Region.

The relative bar lengths across the five dimensions within the shrinking total lengths are fairly consistent in the two African regions. That is, the relative space given to each color does not change much as the total bar shortens. That suggests that where total effort is less the relative stress allocated to each dimension remains about even.

The pattern of fairly even shares of attention appears also in MENA and Asia, but less so in Latin America and the Caribbean (LAC) and EECA.

Consistent with the previous figures, this again shows that for most countries, the highest dimension score is for Strategy, and the lowest is for Accountability.

Patterns for the 35 scores by region

A decided commonality exists across regions in the uneven profile of effort across the 35 scores. The regional lines, to a remarkable degree, move up and down together. The agreements emerge despite the large disparities in regional characteristics and programs. Each regional line comes from the average across its member countries, so the disparities in both country and regional patterns are over-ridden by the nature of each score. For example, it is much easier in most countries, under Strategy, to create a set of objectives (at the left) than to obtain high level support (at the right). Under Equity, it is simpler to declare an anti-discrimination policy than to deploy a CBD program. Therefore, the highs and lows in Figure 4 (total line) appear to reflect the essential “ease” by which countries can score on each kind of effort. The countries clearly exert stronger efforts on some of the 35 items than others, but they face dissimilar barriers across the items.

Figure 4. Individual NCIFP Scores According to Region (weighted), 2017.

For four of the regions (SSAF-F, Asia, LAC and EECA), the highest scoring item was a measure of Quality: “Are FP Standard Operating Procedures in line with WHO and used for determining areas of need for quality FP improvement?”. In SSAF-A, the highest scoring item was a measure of Strategy: “Does the national family planning action plan include objectives to reach the poorest and most vulnerable groups with quality FP information and services?”. MENA’s highest scoring item was also related to Strategy: “Does the national FP action plan include defined objectives over a 5 to 10-year period, including quantitative targets?”. There was less consistency in terms of the lowest scores by region. Lowest scores per region were as follows:

The lowest scoring item for SSAF-A and SSAF-F fell within the Accountability dimension. For Asia and EECA, the lowest scoring item was in the Quality dimension, and for LAC and MENA, the lowest scoring item fell within the Equity dimension. None of the lowest scoring items fell within the Strategy dimension.

Gains over time occurred in nearly all of the 35 items, as shown in Figure 5, based on the median scores across countries (those with data in both years). Among these countries, the largest gain was for the item “Does the National Family Planning Action Plan include a mechanism and funding to support meaningful participation of diverse stakeholders?” (30 points). For three items, the difference between 2014 and 2017 was negative: “Extent to which import laws and regulations facilitate the importation of contraceptive supplies or extent to which contraceptives are manufactured locally” (−0.3 points), “Extent to which the entire population has ready and easy access to IUD removal” (−1.3 points) and “Extent to which areas of the country not easily serviced by clinics or other service points are covered by CBD programs for distribution of contraceptives (especially rural areas)” (−5.4 points). Half of the 35 items had an unweighted median point increase of 10 or more points. Every item in the Accountability dimension had a median point increase of at least 15 points.

Figure 5. Gains in the Scores between 2014 and 2017.

Relationship of contraceptive use to the NCIFP scores

To examine the relationship between contraceptive use and NCIFP, we had to combine the smaller regional groupings (i.e., SSAF-A, SSAF-A, Asia, LAC, MENA, EECA) into larger regional groupings (sub-Saharan Africa [SSA] and non-sub-Saharan Africa [non-SSA]) to allow for more data points per regional grouping. Higher modern contraceptive use was correlated with the NCIPF scores in both the SSA and non-SSA regional groupings (Figure 6), but the relationship was much stronger for SSA, which is of special interest as the region with the lowest contraceptive use, the highest fertility rates, and the highest levels of infant and child mortality. In SSA, a ten-point increase in the total score is accompanied by a 6-point increase in mCPR, but by a mere 1-point increase in Non-SSA. The average mCPR^f across SSA countries is 23.4%, with a minimum of 4.4% (South Sudan) and a maximum of 51.3% (Eswatini). Among Non-SSA countries the average mCPR is higher by half, at 34.2%, with a minimum of 14.4% (Afghanistan) and a maximum of 58.9% (Colombia).

Figure 6. Total 2017 NCIFP Score and mCPR, by regional groupings.

A similar analysis was done to explore the relationship of the total fertility rate (TFR) to the NCIFP, which found a much weaker correlation than for the mCPR. That is not entirely surprising as the TFR responds to several determinants, including the use of traditional methods, and is not as directly impacted by programmatic efforts as mCPR.

Correlations by the five NCIPF dimensions give further evidence of closer relationships to contraceptive use in the SSA region (Table 2). By the total score, the correlation is r=0.53 in SSA and only r=0.11 in Non-SSA. There is little variation across dimensions, with SSA Data dimension having the highest coefficient (0.55) and Quality the lowest (0.41).

In Non-SSA countries, the mCPR is very weakly correlated with the dimension scores. The Equity dimension had the highest correlation coefficient (0.26) and Strategy the lowest (0.02).

Relation of contraceptive use to access

Among the 35 scores, two are related to access, one for long-term and one for short-term methods. The items were scored from 1 to 10, where 10 meant full access. They read as “Extent to which the entire population has ready access to long-acting and permanent methods (LAPMs)” and “Extent to which the entire population has ready access to short-term methods (STMs).” Scores for access to STMs averaged 74.4 across all countries, reflecting easy access to some of these methods, particularly condoms. Scores for access to LAPMs, however, averaged only 44.2 across all countries. These methods, including sterilization, IUDs and implants are entirely clinical, which tends to constrain use in many countries in SSA and elsewhere. The seven regions differed little around the averages.

Interestingly, the correlation between access and use for LAPMs was a high (r=0.58) but that for STMs was a low (r=0.15).^g Perhaps one reason for the disparity is that use of the long-term methods, all requiring clinical visits and procedures, requires more effort by the family planning programs and makes for a closer connection, though other dynamics are likely involved.

While it is encouraging that access to LAPMs is improving, and that access as measured by the NCIFP appears to be correlated with use, it is also important to note access to implant removal was one of the lowest-scoring items in the quality dimension (50.5 in 2017) and the item on access to IUD removal declined 1.3 percentage points from 2014 to 2017. Access to removal is a critical component in the right to choose when to have a child, so it is imperative that programs ensure that access to removals keeps pace with access to long-acting methods.

Relation of use by youth to selected NCIFP indicators

The use of modern methods by youth can be related to two items in the Equity dimension that concern policies and programs affecting young people. These pertain to the extent to which service providers do not discriminate against youth, and the extent to which policies are in place to prevent discrimination towards youth. These scores were paired to DHS data on the percent of sexually active youth (aged 15–24 years) currently using a modern method^h for the 17 countries with data from both sources. No association between the youth equity score and mCPR among sexually active youth (ages 15–24 years) was found. These results reflect the presence of other determinants and the disparate set of countries for which data were available. Freedom from prejudices eases one barrier to contraceptive use for youth, but other constraints are also important.

Yes/no responses versus the 1 to 10 scale

In 2014, the score for most questions was simply the percent of respondents who said yes. The yes/no questions did not allow for gradations in the response; additionally, for some questions a clear cut ‘yes’ or ‘no’ answer was problematic because the question asked about multiple issues.

In 2017, to allow for nuances in the rating, respondents choosing "yes" for an item were asked to rate it on a scale from 1 to 10, where 1 indicated the effort was non-existent, equivalent to having responded “no”. Then 2 represented very weak effort and 10 an extremely strong effort.

That permitted a comparison of the two types of ratings: those based only on yes/no replies and those based on the ratings. For the 20 items for which both ratings were used (Table 1 in the Extended data), the scores were considerably higher when based on the yes/no replies than when based on the scaled ones (Figure 7). The total score was higher by 22.8 points (71.0 versus 48.2). The differences for the dimensions were not greatly different from each other; in order starting from the Strategy dimension they were 25.5. 25.1, 22.0, 19.5 and 20.9.

Figure 7. Comparison of Dimension Scores Based on Yes/No Responses and 1-10 Scale Responses.

Overall, scores based on the 1–10 scale responses tend to be more moderate in range, and more closely grouped across regions. Table 3 provides an overview of the score ranges by response type.

Looking at total scores averaged across all countries, the minimum score for all yes/no items and 1–10 scale items was for “Are violations reviewed on a regular basis?” (41.4 points and 30.7 points, respectively). The maximum score for all yes/no items was for the item “Are FP operating procedures in line with WHO and used for determining areas of need for quality FP improvement?” (93.4), but for the 1–10 items, it was “Extent to which the entire population has ready access to STMs” (74.4 points). The range between the highest-scoring and lowest-scoring yes/no item was 52.0 points, but the range between the highest-scoring and lowest-scoring 1–10 scale item was 43.7 points. Based on these total scores, we see that the 1–10 scale items tend to be more moderate in range than the yes/no items.

For differences across regions (Table 4), the item with the largest yes/no score difference was “Are indicators for quality of care collected and used for private sector family planning services?” EECA scored lowest across regions (36.7) and SSAF-F scored the highest (78.9), making the regional range 42.2 points. For the 1-10 items, “Extent to which the entire population has ready and easy access to IUD removal” had the widest range across regions (28.3 points), with SSAF-A scoring the lowest (47.9) and EECA scoring the highest (76.2).

The item with the smallest yes/no score difference across regions was “Are FP operating procedures in line with WHO and used for determining areas of need for quality FP improvement?”, with a difference of only 12.9 points between EECA (84.4) and SSAF-F (97.3). The regional range even smaller among the 1–10 items with a range of only 4.6 for “Are there mechanisms in place to monitor whether or not access to voluntary, non-discriminatory FP information and services is being achieved”, with SSAF-F scoring lowest (44.8) and MENA scoring highest (49.4).

Based on the greatest and smallest range between regions, we notice that scores are much more closely grouped across regions for the 1–10 items compared with the yes/no items.

Analysis of response rates

To ensure that all responses were informed opinions rather than guesses, respondents were instructed to leave a question blank if they did not know the answer. Non-response therefore serves is an indicator of which components respondents knew less about or which were unclear in the questionnaire wording. That affects the non-response to both yes/no replies and especially to the 1–10 ratings since most respondents choosing a “No” response did not give a 1–10 response.

The response rate refers to the percentage of items in the questionnaire to which respondent gave a reply, not leaving the item blank. This varied both by question and by country. The average response rate across all items and countries was 89.0%, ranging from 61.1% to 99.9% across the 83 countries and from 63.6% to 97.9% across the 35 items.

By question type the average response rate was 91.2% across all yes/no items and 87.8% across all 1–10 scale response items. The ten items with the lowest response rates and the percentage failing to receive a 1–10 rating after a “No” reply are listed in Table 3 of the Extended data. Four of the five Accountability items appear in this list, suggesting that the respondents know the least about this component of the family planning program or that the items are the most difficult for respondents to understand. Table 3 of the Extended data also shows the ten items with the highest response rates. These were dominated by the scaled scores (nine of 10 items). Four of the items with the highest response rates were related to access or data to monitor access.

Response rates by region and country

For each country, the average response rate across all 35 questions was calculated and averaged by region. Around the mean rate of 89.5% the regions ranged within a narrow span of 84.3% (MENA) to 90.9% (SSAF-A). The top four regions were clustered at 90%. (Table 5)

By country, the range was much greater, from the high 90s down to the low 70s. The ten highest and ten lowest scoring countries are in Table 4 of the Extended data, which shows the lack of any regional pattern in the response rates. The high and low countries each fall into a wide variety of regions, so no clear pattern emerges for the determinants of response rates, which differ most by the yes/no versus the scaled replies.

Improvement of response rates between the 2014 and 2017 rounds

The response rates improved between the two rounds of the NCIFP, from 86% to 94% (for countries with data in both rounds).

Every dimension showed a gain; the largest was for Accountability, which moved up from a very low score in 2014 (Figure 8). In a sense it had the greatest “room” to grow. Otherwise, the 2017 levels all fell in the 90s.

Figure 8. 2014 and 2017 NCIFP Response Rates for each Dimension.

Response rate gains by both domain and region

The regions differed considerably in the degree of gain across the five dimensions. Figure 9 shows the difference between the median scores registered in 2014 and 2017 (for countries with data in both rounds). SSAF-F experienced the greatest gain in every dimension, and SSAF-A was second in three dimensions as well as in the overall total). The rank order for the other regions was generally consistent, especially marked in the Equity dimension. The MENA region’s scores changed the least, and lost ground in two of the dimensions. The Accountability gains were clearly the greatest, starting from the lowest 2014 levels as noted above.

Figure 9. Median Difference in Response Rates from 2014 to 2017.

These patterns are suggestive for the policy and programmatic improvements that occurred differentially in the various regions and in the five dimensions. The changes cannot be decomposed by actual improvements and methodological factors, but they point in a positive direction. By country, differential response rates may not be very telling concerning the programmatic efforts as they may partly reflect the selection of respondents and the follow-up efforts of the consultants.

Accountability dimension

The decomposition question is especially germane to the Accountability dimension, since its average score rose by 20 points. This may be due to actual improvements in some of its five indices. It may also reflect the higher response rates in 2017, which were possibly selective by respondent type, introducing better levels of knowledge that favored higher scores. Another consideration is the change in the questionnaire, asking for a 1–10 rating in 2017 in addition to the yes/no rating; however as noted above, the Results section above is based on analyses using just the 2014 approach, for comparability across rounds.

To explore reasons for the large increase in part of the Accountability score, we looked at the scores (yes, no, or no responses, adding to 100%) across all replies in Table 6 for each of the five Accountability items (again only for countries that completed the NCIFP in both 2014 and 2017; also, Table 6 pertains only to the yes/no responses in since the 1–10 scale response was not an option for any of the Accountability items in 2014).

Frequency of “Yes” replies rose markedly in 2017, in every item, gaining 29, 20, 16, 30, and 29 percentage points in order across the five indices, and by 25 percentage points overall. These increases came partly at the expense of the frequency of “No” replies, which declined by 17, 5, 4, 13, 12, percentage points respectively, and by 10 percentage points overall. They also came partly at the expense of the “no response” replies, which declined by 12, 15, 12, 17, and 18 percentage points in order across the five items, and by 15 percentage points overall. These shifts clearly suggest real gains in effort, and the possibility that the large increase in replies were selective in giving higher ratings, possibility due to a firmer grasp of the nature of the program and of the indices during the three years.

That raises the question of the kinds of training provided in that period and the stress given to all dimensions, but in particular to the Accountability dimension, which was scored lowest in 2014. Actions taken at and after the 2012 London Summit on Family Planning have been reviewed by Hardee¹¹.

Limitations

The results in this report are limited in several respects, partly for the methodological issues discussed. The time trends shown are only for countries participating in both rounds, and the sets of respondents between rounds in each country varied to some extent. The degrees of respondent biases in the scoring of program features are unknown; the use of four types of respondents in each country acted as a partial protection against them, as did the consistency checks in the centralized data processing. Finally, the scores rely on the judgments of program observers with a necessary element of subjectivity; that is the price paid for obtaining scores for most developing countries on a standard questionnaire, at a single point in time.