Gates Open ResGates Open Research2572-4754F1000 Research LimitedLondon, UK10.12688/gatesopenres.13145.1Research ArticleArticlesHow are countries planning for costs of nutrition data and information systems?
[version 1; peer review: 2 approved]
ManoratReneeConceptualizationFunding AcquisitionMethodologyProject AdministrationSupervision1RanaYashodharaProject AdministrationSupervisionVisualizationWriting – Original Draft PreparationWriting – Review & Editinghttps://orcid.org/0000-0003-1551-8278a2BorcesKyleData CurationFormal AnalysisMethodologyVisualization2BeckerLauraData CurationFormal AnalysisMethodology3FloryAugustinConceptualizationMethodologyWriting – Review & Editing2
Independent Researcher, Washington, District of Columbia, USA
Results for Development, Washington, District of Columbia, 20036, USA
Independent Researcher, Seltjarnarnes, Icelandyrana@r4d.org
Background: The first Global Nutrition Report in 2014 called for a “data revolution” in nutrition, so that countries have the latest data to set priorities and monitor progress. Integral to this revolution is understanding how countries are investing in the data, systems and capacity required to support decision-making around nutrition, i.e. their nutrition data and information system (NDIS).
Methods: For this reason, our team conducted a desk review of national nutrition plans for 58 Scaling Up Nutrition (SUN) countries to better understand how countries are planning for and estimating the costs of their NDIS.
Results: We found that of the SUN national nutrition plans that are publicly accessible, not all are costed and less than half of these have explicit data and monitoring and evaluation (M&E) sections. Of the 19 national plans that had costed data and M&E sections, our initial estimates show costs for data systems ranged from 0.1%–12.8% of total plan costs with limited information on data system components.
Conclusions: There is an imminent need for more comprehensive and strategic approaches – including the planning for and financing of – NDIS in countries.
Nutrition information systemNutrition data and information systemGates FoundationINV-007332This work was supported by the Gates Foundation [OPP1174256].Introduction
With preparations underway for the Nutrition for Growth (N4G) Summit, the global nutrition community is examining its prior commitments, progress made to date, and outlining the path forward for achieving the World Health Assembly (WHA) global nutrition targets by 2025. There is increasing recognition that the way forward will require higher quality and timely data and investments in all elements of the nutrition data value chain (
Figure 1). Not surprisingly, the 2018 Global Nutrition Report called for increased prioritization and investment in nutrition data recognizing that progress is not possible if we cannot identify where action is most needed (
Development Initiatives, 2018).
Nutrition Data Value Chain.
Adapted from
Piwoz
et al., 2019.
Despite this recognition, there is limited guidance for countries regarding how to invest in their NDIS. SUN’s checklist for the development of national nutrition plans includes a monitoring and evaluation framework. More specifically, the framework recommends a multisectoral nutrition information platform to support analysis, knowledge management, learning and communication. This checklist, however, does not provide further guidance regarding how countries can begin to plan, cost and implement a NDIS (
Scaling Up Nutrition, 2016). UNICEF with support from the WHO-UNICEF
Technical Expert Advisory group on nutrition Monitoring (TEAM) is currently developing a report to meet country demand for more support in planning and implementation of nutrition information systems (
WHO, 2015).
As country-level NDIS is critical for countries to assess progress towards achieving targets, it is important to understand how countries are planning for and costing NDIS. An improved understanding of countries’ approach can help in the identification of best practices and key gaps that may complement the work being supported by WHO-UNICEF TEAM. Towards this end, our team conducted a review of national nutrition plans for 58 SUN countries to better understand how countries are planning for and estimating the costs of their NDIS (
Manorat
et al., 2019).
Methods
For this study, the team accessed the most current national nutrition plans that were publicly available or made available to the team by the SUN Secretariat. The plans that were consulted for this analysis are noted in
Extended data, Appendix 3 (
Manorat
et al., 2020). To ensure our team used a consistent approach to review the national nutrition plans, obtained either online or through the SUN Secretariat, we used a framework developed by
DataDENT (
Figure 2) that outlines the major cost components needed for establishing and maintaining NDIS. The framework was refined with select country participants from the National Information Platforms for Nutrition (NIPN) Global Gathering in May 2019 and in consultation with stakeholders from Nutrition International (NI), Maximising the Quality of Scaling Up Nutrition Plus (MQSUN+), and the Bill & Melinda Gates Foundation. Please refer to
Extended data, Appendix 1 (
Manorat
et al., 2020) for the full list of stakeholders consulted.
DataDENT framework on the major costs behind nutrition data systems.
Briefly, the framework consists of three main components followed by cross-cutting components — (i)
Periodic data collection includes data collection activities conducted on a periodic basis, primarily large-scale household surveys conducted annually or every 3–5 years; (ii)
Administrative/routine systems involves the development, implementation, and data quality assurance of routine management information systems (MIS); (iii)
Information synthesis and decision-making pertains to the collation, analysis, visualization, and dissemination of nutrition data. In addition to these three main components, we also recognize that there can be a fourth “ad hoc” category that straddles between periodic and routine data collection. This category can include nutrition assessments during emergencies, for instance.
The cross-cutting categories include
planning, design, and coordination which involves the development of overarching plans for NDIS or monitoring frameworks.
Human resources and capacity building category includes costs associated with the salaries, training and/or sensitizing people who maintain data systems, analyze data or use the information.
Equipment and infrastructure describes the infrastructure, supply, and transportation costs for maintaining the information system. Finally, the
measurement innovations category involves any new tools or processes for collecting, monitoring, and evaluating nutrition data. In
Extended data, Appendix 2 (
Manorat
et al., 2020), we have elaborated on these categories and sub-categories along with providing examples drawn from our review.
Examination and analysis of national nutrition plans
Using this framework, a team of two reviewers examined national nutrition plans for 58 SUN countries to understand how they are currently costing for NDIS activities. A total of 31 countries had costed plans that were accessible for our review. Of these costed plans, we found 22 plans that costed for NDIS activities, but two of these plans had insufficient costing details i.e. monitoring and evaluation costs were bucketed with other objectives and so disaggregation was not possible and one plan had outlier cost estimates. In
Figure 3, we briefly summarize the process followed for the review.
Breakdown of the review process.
We conducted an in-depth analysis of 19 national nutrition plans where we extracted the relevant line items that pertained to monitoring, evaluation and data, categorized them according to the framework and then summed up the costs to create an overall cost estimate for NDIS activities. Please refer to the
Underlying data (
Manorat
et al., 2020), which provides these details. Given that the desk review is subjective by nature, we took measures to improve confidence in our findings. We developed a codebook prior to conducting the analysis and had two reviewers check each other’s coding. A group of three experts, each with more than 10 years of experience on nutrition and nutrition data-related issues, weighed in cases where it was unclear how best to code cost buckets against the NDIS framework. The team also validated the approach through consultations with SUN stakeholders in Uganda and Vietnam and technical assistance providers from Kenya.
Two key limitations are important to note. First, this research does not intend to provide a comprehensive reflection of a country’s investments in NDIS, but rather focuses on how countries are costing for NDIS specifically in their national nutrition plans. Second, our analysis is limited to the level of reporting available in each plan. For instance, if a country costed for implementing national nutrition surveys or conducting data quality audits of its routine health information system, but then reported them in the plan under one line item for “developing the nutrition information system” then we would not be able to disaggregate this data.
ResultsThe status quo: Inclusion of NDIS costs in national nutrition plans
We found that approximately one-third of SUN countries have costed plans with data and M&E sections. Of these plans, we found that data systems costs ranged from 0.1% to 12.8% of total budget costs (
Figure 4). Whereas some countries integrated data-related activities across objectives (e.g., Cote d’Ivoire), others had specific line items for nutrition M&E activities (e.g., Uganda).
NDIS cost estimates out of total plan costs.
In general, we found that costed plans included limited information on costs of nutrition data and M&E systems beyond a single or limited number of high-level item lines. As can be seen in
Figure 5, we have included the
broad nutrition data category in several countries since we could not further disaggregate the budget lines. As an example, Burkina Faso’s national nutrition plan included a budget line that notes “the nutrition monitoring and evaluation system is improved”. Additionally, Myanmar’s national plan has one costed line item with a description for “monitoring and evaluation”. In both cases, these are very broad activities that could relate to several components of the NDIS framework.
Annual NDIS cost estimates by framework components.
Among the plans we reviewed, we found that the most commonly costed components were periodic data collection, administrative/routine systems and capacity building. On the other hand, there was limited mention of other components critical to the development and maintenance of data systems. Only a few countries included costs for information synthesis and decision-making such as the costs for annual review meetings in Zimbabwe or developing new scorecards and dashboards to visualize nutrition data in Kenya. Countries such as Senegal elaborated more on periodic or administrative/routine data collection activities and had limited mention of line items on the latter half of the nutrition value chain (see
Figure 1) that are as critical to ensure the effective use of data to support decision-making. It may be the case that costs associated with the synthesis and analysis of critical nutrition data such as from the Demographic and Health Surveys (DHS) is often paid for by donors and hence not budgeted by countries or that the lack of specificity about assumptions for costing may mean that information analysis and syntheses costs could have been included within the periodic and administrative costs buckets.
Likewise, cross-cutting categories such as planning, design and coordination, measurement innovations and equipment/infrastructure had the least amount of costing information available. Only one plan included measurement innovations, and a few plans budgeted for equipment and supplies. In addition, only four countries had budgeted for resource tracking which is a critical activity that countries need to assess and secure funding towards the implementation of their nutrition national plans.
Positive outlier: Kenya’s National Nutrition Plan
While our review revealed several gaps, we also found a few countries with more detailed cost estimates for NDIS in their national nutrition plans. Of the costed plans reviewed, Kenya’s national nutrition plan stood out as an exemplar for its detailed budget lines for its monitoring and evaluation framework.
i Nearly all components of the framework (except for equipment and supplies and ad hoc category) was specifically captured in the costed plan (Please see
Table 1). The plan covers the full nutrition data value chain, including data use activities and also includes cost estimates for M&E related activities outside of the health sector, such as education and social protection. In sum, the plan allocates approximately 6% of the total estimated resource need for nutrition towards data and information systems.
Kenya national nutrition action plan.
Nutrition data component
Total cost of
M&E section
of plan
Examples of line items included
1. Periodic data collection
27%
Major nutrition-focused surveys including SMART, MIYCN KAP, and
other coverage surveys
2. Administrative/routine
systems
23%
Activities focused on strengthening the routine HMIS systems, and
integration of data systems for nutrition services delivered through
HIV and TB programs
3. Information synthesis &
decision making
28%
Development of nutrition dashboards, scorecards, or other electronic
data visualization tools, as well as utilization of nutrition information to
inform program quality improvement
4. Ad hoc data collection
0%
5. Planning
9%
Reviewing and updating the Kenya M&E framework and to support
the development and progress of other multi-year plans
6. Measurement innovations
2%
Investments in emerging technologies for nutrition assessment and
diagnostics for HIV/TB patients
7. Human resources/capacity
building
11%
Develop capacity for use of mHealth systems at the community level
8. Equipment/infrastructure
0%
Discussion
Nutrition data and information systems play a critical role in ensuring valid, reliable, and timely nutrition data are available, accessible, and used by key nutrition stakeholders to inform decision-making. However, our review revealed that there is a strong need for tools to support countries efforts in planning and costing NDIS. Countries need to develop comprehensive nutrition data plans to ensure prioritization, coherence and coordination of NDIS investments with nutrition relevant sectors, taking a long-term perspective. These data plans should be ideally developed with investment cases so that the latter can function as a guide to secure needed funding.
For the development of NDIS plans, countries may benefit from technical and financial support. For instance, countries may need support for developing the plan, costing it, ensuring relevant activities throughout the data value chain are considered. Financial support could be catalytic for fiscally constrained governments because strengthening NDIS could cost a substantial share of the total cost of the national nutrition plan (e.g. 6% in the case of Kenya) and generally tends to be underprioritized. For these reasons, key events like the N4G Summit provide an opportunity for the global community to make commitments to nutrition data systems and thereby support countries in their respective journeys.
Given that this is an area of limited research, we also wanted to share some reflections and questions that came up during our review, which may warrant further research and discussions-
What would be an appropriate cost benchmark for national nutrition plans data related activities?
How to ensure that NDIS is being systematically incorporated in relevant sectoral plans and budgets? What guidance could be provided to countries on how best to integrate NDIS within existing systems set up by relevant Ministries and departments such as the Statistical office, Health Ministry, Agriculture Ministry among others?
As service delivery activities are increasingly being managed at the local level, information needs are getting localized as well as financial flows. When developing and costing NDIS plans, what should be the cost-sharing between the national and sub-national levels?
How to integrate the costing and budgeting of NDIS within existing annual planning and budgeting processes?
How can donors better align with country governments when thinking about financing sources for nutrition plans especially for NDIS costs?
How should countries cost for key surveys such as the DHS and Multiple Indicator Cluster Surveys (MICS) that provide critical nutrition data but are costed and financed through other sectors or sources? More broadly, given that nutrition relies on data from broader systems, how should the NDIS approach the costing of these components?
It is often argued that when it comes to allocation of scarce resources, it is hard to prioritize data systems over programmatic implementation. What advocacy efforts can help make the case that data funding can support countries to better plan, target and deliver their nutrition programs, and thus make their resources go further?
Data availabilityUnderlying data
DANS: How are countries planning for costs of nutrition data and information systems?
https://doi.org/10.17026/dans-xap-jq2v (
Manorat
et al., 2020).
This project contains the file ‘N4G_Plans_Analysis_to share’. (This file includes the full list of national nutrition plans that were included in the analysis as well as individual country level data for the 19 countries that had costed nutrition plans in addition to costed NDIS related activities. For each country, the document includes a summary of the total plan costs, breakdown costs of NDIS activities, and percent of budg
et allocated towards nutrition data.)
Extended data
DANS: How are countries planning for costs of nutrition data and information systems?
https://doi.org/10.17026/dans-xap-jq2v (
Manorat
et al., 2020).
Appendix 1. List of Stakeholders consulted (PDF).
Appendix 2. Brief description of framework components with select examples from national nutrition plans (PDF).
Appendix 3. List of national nutrition plans reviewed (PDF).
Data are available under the terms of the
Creative Commons Zero “No rights reserved” data waiver (CC0 1.0 Public domain dedication).
Notes
iPlease note that we reviewed the draft version of the 2018–2022 Kenya Nutrition Action Plan (KNAP).
Acknowledgements
In addition to those consulted for this review, the authors would like to gratefully acknowledge comments on this work from Ellen Piwoz and Rahul Rawat (Bill and Melinda Gates Foundation) and Rebecca Heidkamp and Tricia Aung (Johns Hopkins Bloomberg School of Public Health). We would also like to thank Debora Di Dio and also Scaling Up Nutrition (SUN) Movement Secretariat for providing us information on the latest national nutrition plans for the SUN member countries.
Please note that the team has shared a condensed version of the findings of this research in their blog titled “
How can we better support countries to build and maintain nutrition data and information systems?”
Development Initiatives:
2018 Global Nutrition Report: Shining a Light to Spur Action on Nutrition. Global Nutrition Report. 2018.
Reference SourceManoratRBeckerLBorcesK:
How can we better support countries to build and maintain nutrition data and information systems?.DataDENT Blog.2019.
Reference SourceManoratRRanaYBorcesK:
How are countries planning for costs of nutrition data and information systems?[..Ods]. Data Archiving and Networked Services (DANS).2020.
10.17026/DANS-XAP-JQ2VPiwozERawatRFracassiP:
Strengthening the Nutrition Data Value Chain for Accountability and Action. 2019;33:6.
Reference SourceScaling Up Nutrition:
Checklist on the critieria and characteristics of “good” national nutrition plans. Scaling Up Nutrition. 2016.
Reference SourceWHO:
WHO | WHO/UNICEF Technical expert advisory group on nutrition monitoring (TEAM).WHO; World Health Organization.2015.
Reference Source10.21956/gatesopenres.14336.r29875Reviewer response for version 1DiazTheresa1Refereehttps://orcid.org/0000-0001-5063-8078
Department of Maternal, Newborn, Child and Adolescent Health and Ageing, World Health Organization, Geneva, Switzerland
Competing interests: No competing interests were disclosed.
This is a well written comprehensive review of investments in data systems and capacity to use these data based on national nutrition plans. The authors clearly explain the nutrition data value chain and follow a framework that outlines the major components needed for establishing and maintaining nutrition data information systems. They also clearly explain the search criteria and number of documents that contain costed plans.
They also highlight the one costed plan, Kenya, that contained a budget for the different components a full nutrition nutritional data information system. But it should be noted that Kenya is one of the premier countries in the health data collaborative that worked hard to have a one monitoring and evaluation plan for the country aligning donors, partners and health programs. Details can be found here
https://www.healthdatacollaborative.org/where-we-work/kenya/. This may be in part why the nutrition data information plan is so well documented.
The major limitation of this study is that resource allocation and budgeting for national health information systems are normally part of national monitoring and evaluation plans. Although specific health programs should allocate portions of their budget toward data collection, analysis interpretation, it is difficult to determine if the funding in the nutrition plan aligns or adds to the national monitoring and evaluation plans and budget.
The questions at the end of the article address some of these issues, specifically how NDIS is being systematically incorporated into in relevant sectorial plans and existing data systems and how countries cost for key surveys and how financing of these may be from donors and other sectors.
The question on “What would be an appropriate cost benchmark for nutrition plans?” really needs to address what the gaps are in the overall monitoring and evaluation plans and budget for the country. Would it not be better for national population based surveys, health information management systems basic infrastructure, capacity building etc. to be funded by the monitoring and evaluation budget and that specific health areas focus more on information synthesis and decision making?
If there is any way for the authors can also conduct a search of the monitoring and evaluation plans and review their costing it would greatly contribute to this analysis and help better assess if the nutrition data information costs in the nutrition plans are well aligned.
Is the work clearly and accurately presented and does it cite the current literature?
Yes
If applicable, is the statistical analysis and its interpretation appropriate?
Not applicable
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Is the study design appropriate and is the work technically sound?
Partly
Are the conclusions drawn adequately supported by the results?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Yes
Reviewer Expertise:
I am the Unit Chief for Epidemiology and Monitoring and Evaluation for the Maternal Newborn Child and Adolescent Health and Ageing Department at WHO. I have over 30 years experience in surveillance and monitoring and evaluation including use of routine health information systems. I am not an expert in nutrition.
I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
10.21956/gatesopenres.14336.r28925Reviewer response for version 1SiekmansKendra1Refereehttps://orcid.org/0000-0002-3116-2278
Nutrition International, Guelph, Canada
Competing interests: I work as a Technical Assistance provider for Nutrition International's TAN project, supporting the SUN Movement Secretariat in monitoring & evaluation activities at the global level.
Consistent with the high quality and clarity of their previous work, the Results for Development team have conducted a useful and methodical analysis of the nutrition data and information system components included in SUN Movement member country national nutrition plans. SUN countries are committed to creating enabling environments for ending malnutrition in all its forms, including efforts to bring together stakeholders from multiple sectors and help each of those sectors recognize their role to play. National nutrition plans are an important tool to identify priority actions and who is responsible for delivering them, as well as how progress will be measured. As described by Renee Manorat and colleagues, at least 58 of the current 61 SUN countries have developed national nutrition plans and many of these are considered multi-sectoral, i.e. they represent the commitments to action by multiple government sectors toward the common goal of improving nutrition.
To better understand what nutrition data and information system components were included in these plans and how much these actions were expected to cost, Renee Manorat and colleagues conducted an in-depth review of 19 national nutrition plans which were publicly available and fully costed. Using a conceptual framework that describes the main components of nutrition data and information systems, the authors categorized the monitoring, evaluation and data activities in each country plan by these components, along with the cost associated with each activity. This allowed them to observe the range in types of nutrition data and information system activities identified in the plans as well as calculate the sum of their expected cost by each core and cross-cutting component.
The results reveal that these 19 SUN countries vary widely in terms of the number of nutrition data and information system-related activities included in their national nutrition plan, the level of detail in which they are described and the costs associated with them. In fact, while the authors provide very nice data visualizations comparing the cost estimates, it is difficult to do much with these estimates, given the uncertainty underlying them. Put another way, the results show that few countries have identified (or included in their national nutrition plan) the specific activities needed to monitor and evaluate the results of their efforts to improve nutrition. This is, in itself, a very important and sobering finding.
The authors of this study acknowledge this limitation in their approach - the summary of activities listed in the national nutrition plan does not fully represent the country's investments in nutrition data and information systems. However, a more extensive discussion on the possible reasons for this under-representation of nutrition data-related activities in these plans would be helpful to inform future technical and financial support.
One important factor that I recommend the authors consider further in this work is the political economy of nutrition, as described by Yarlini Balarajan and Michael Reich (2016)
1. For example, their description of the "institutional homelessness" of nutrition within government structures is an important factor influencing how effectively nutrition actors engage with other sectors in the planning and budgeting process. It would be interesting to explore this further with the data from this review, examining whether the position of nutrition within the government structure (e.g. supra-sectoral level such as the Prime Minister's Office vs. line ministry level such as within the Ministry of Health) influences the scale and types of nutrition data and information system actions outlined in the national nutrition plan.
Another important factor to explore further is the tension between working with existing data and information systems vs. aspirational goals of identifying the data needed and seeking to find ways to collect them. The finding that only one country plan included measurement innovations suggests there is little vision for how data and information systems can be strengthened and expanded to better support nutrition policy and programme decision-making. DataDENT and National Information Platforms for Nutrition (NIPN) have provided leadership in this regard by working with countries to first identify the policy questions they have and then looking at what data are needed to answer them. What learning has come from those initiatives that could help put the findings of this review in context and further inform the recommendations made to strengthen these plans?
The results also highlight the complex journey that SUN member countries find themselves on, with few exemplars to follow in terms of successful national multisectoral nutrition approaches. While the development and ratification of a national multi-sectoral nutrition plan is an achievement in itself, there is a need to continue learning and strengthening the content of these plans with each revision. Six of the plans reviewed in this study ended in the year 2018 or earlier and may have been the very first national nutrition plan developed in that country. It would be interesting to see whether countries with newer or updated plans were more likely to include costed nutrition data and information system activities.
I recommend that in the Discussion section, the authors consider placing their results in the context of other research that has been done recently. For example, MQSUN+’s review of the quality of SUN national nutrition plans
2 and SPRING’s Pathways to Better Nutrition case studies from Nepal and Uganda
3 are two relevant reviews. I also wonder whether the discussion could include some additional reflection on the challenges experienced to date by SUN countries in the process of costing their nutrition plans, given that so many country plans were not fully costed and that even those with costed plans often did not have their nutrition data-related activities disaggregated to a level that made it possible to categorize them.
Balarajan and Reich (2016) argue that the nutrition community's limited ability to clearly demonstrate results continues to hamper its efforts to rise on the political agenda. They suggest that greater attention to and investment in rigorous monitoring and evaluation as well as routine information systems is required to demonstrate clear progress and develop a compelling narrative that makes championing nutrition a more attractive option for politicians. The results of this study provide a clear wake-up call to global and country-level nutrition actors alike - increased technical and financial investment is needed to support countries in identifying and budgeting for the nutrition data and information systems actions that will enable them to track progress, adjust their programmes where needed, and clearly demonstrate progress towards their goals.
Is the work clearly and accurately presented and does it cite the current literature?
Partly
If applicable, is the statistical analysis and its interpretation appropriate?
Yes
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Is the study design appropriate and is the work technically sound?
Yes
Are the conclusions drawn adequately supported by the results?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Yes
Reviewer Expertise:
Multi-sectoral nutrition programme monitoring & evaluation in low- and middle-income countries; SUN Movement; infant and young child feeding; micronutrient deficiencies
I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
References:
Political economy challenges in nutrition.
Globalization and Health.2016;12(1) :
10.1186/s12992-016-0204-610.1186/s12992-016-0204-6Review of SUN Country National Nutrition Plans.
MQSUN.2020;
Reference source:
Recommendations for multi-sector nutrition planning: Cross-context lessons from Nepal and Uganda.
Reference source