<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.2 20190208//EN" "http://jats.nlm.nih.gov/publishing/1.2/JATS-journalpublishing1.dtd"><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="letter" dtd-version="1.2" xml:lang="en">
    <front>
        <journal-meta>
            <journal-id journal-id-type="pmc">Gates Open Res</journal-id>
            <journal-title-group>
                <journal-title>Gates Open Research</journal-title>
            </journal-title-group>
            <issn pub-type="epub">2572-4754</issn>
            <publisher>
                <publisher-name>F1000 Research Limited</publisher-name>
                <publisher-loc>London, UK</publisher-loc>
            </publisher>
        </journal-meta>
        <article-meta>
            <article-id pub-id-type="doi">10.12688/gatesopenres.13319.1</article-id>
            <article-categories>
                <subj-group subj-group-type="heading">
                    <subject>Open Letter</subject>
                </subj-group>
                <subj-group>
                    <subject>Articles</subject>
                </subj-group>
            </article-categories>
            <title-group>
                <article-title>Developing effective data visualization tools for nutrition: reflections on the design of a Nutrition Scorecard in Nigeria</article-title>
                <fn-group content-type="pub-status">
                    <fn>
                        <p>[version 1; peer review: 2 approved]</p>
                    </fn>
                </fn-group>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author" corresp="yes">
                    <name>
                        <surname>Rana</surname>
                        <given-names>Yashodhara</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Conceptualization</role>
                    <role content-type="http://credit.niso.org/">Funding Acquisition</role>
                    <role content-type="http://credit.niso.org/">Methodology</role>
                    <role content-type="http://credit.niso.org/">Project Administration</role>
                    <role content-type="http://credit.niso.org/">Supervision</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Original Draft Preparation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <uri content-type="orcid">https://orcid.org/0000-0003-1551-8278</uri>
                    <xref ref-type="corresp" rid="c1">a</xref>
                    <xref ref-type="aff" rid="a1">1</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Dongo</surname>
                        <given-names>Gianni</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Conceptualization</role>
                    <role content-type="http://credit.niso.org/">Data Curation</role>
                    <role content-type="http://credit.niso.org/">Formal Analysis</role>
                    <role content-type="http://credit.niso.org/">Methodology</role>
                    <role content-type="http://credit.niso.org/">Project Administration</role>
                    <role content-type="http://credit.niso.org/">Supervision</role>
                    <role content-type="http://credit.niso.org/">Validation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <uri content-type="orcid">https://orcid.org/0000-0003-4201-5246</uri>
                    <xref ref-type="aff" rid="a2">2</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Snead</surname>
                        <given-names>Caroline</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Data Curation</role>
                    <role content-type="http://credit.niso.org/">Formal Analysis</role>
                    <role content-type="http://credit.niso.org/">Validation</role>
                    <role content-type="http://credit.niso.org/">Visualization</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Original Draft Preparation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <xref ref-type="aff" rid="a1">1</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Agi</surname>
                        <given-names>Grace</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Data Curation</role>
                    <role content-type="http://credit.niso.org/">Formal Analysis</role>
                    <role content-type="http://credit.niso.org/">Methodology</role>
                    <role content-type="http://credit.niso.org/">Project Administration</role>
                    <role content-type="http://credit.niso.org/">Supervision</role>
                    <role content-type="http://credit.niso.org/">Validation</role>
                    <role content-type="http://credit.niso.org/">Visualization</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <uri content-type="orcid">https://orcid.org/0000-0002-7961-2020</uri>
                    <xref ref-type="aff" rid="a2">2</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Sadik</surname>
                        <given-names>Oluwagbenga</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Data Curation</role>
                    <role content-type="http://credit.niso.org/">Formal Analysis</role>
                    <role content-type="http://credit.niso.org/">Project Administration</role>
                    <role content-type="http://credit.niso.org/">Supervision</role>
                    <role content-type="http://credit.niso.org/">Validation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <uri content-type="orcid">https://orcid.org/0000-0001-9758-6455</uri>
                    <xref ref-type="aff" rid="a2">2</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Heidkamp</surname>
                        <given-names>Rebecca</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <uri content-type="orcid">https://orcid.org/0000-0003-3146-1320</uri>
                    <xref ref-type="aff" rid="a3">3</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Abdulwahab</surname>
                        <given-names>Ahmad</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Conceptualization</role>
                    <role content-type="http://credit.niso.org/">Funding Acquisition</role>
                    <role content-type="http://credit.niso.org/">Methodology</role>
                    <role content-type="http://credit.niso.org/">Supervision</role>
                    <role content-type="http://credit.niso.org/">Visualization</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <xref ref-type="aff" rid="a2">2</xref>
                </contrib>
                <aff id="a1">
                    <label>1</label>Results for Development, 1111 19th Street NW, Suite 700, Washington, D.C., 20036, USA</aff>
                <aff id="a2">
                    <label>2</label>Nigeria Governors&#x2019; Forum, 51 Lake Chad Crescent, Maitama, Abuja, Nigeria</aff>
                <aff id="a3">
                    <label>3</label>Institute of International Programs, John Hopkins Bloomberg School of Public Health, 615 N. Wolfe Street, Baltimore, Maryland, 21205, USA</aff>
            </contrib-group>
            <author-notes>
                <corresp id="c1">
                    <label>a</label>
                    <email xlink:href="mailto:yrana@r4d.org">yrana@r4d.org</email>
                </corresp>
                <fn fn-type="conflict">
                    <p>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>5</day>
                <month>7</month>
                <year>2021</year>
            </pub-date>
            <pub-date pub-type="collection">
                <year>2021</year>
            </pub-date>
            <volume>5</volume>
            <elocation-id>98</elocation-id>
            <history>
                <date date-type="accepted">
                    <day>21</day>
                    <month>6</month>
                    <year>2021</year>
                </date>
            </history>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2021 Rana Y et al.</copyright-statement>
                <copyright-year>2021</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <self-uri content-type="pdf" xlink:href="https://gatesopenresearch.org/articles/5-98/pdf"/>
            <abstract>
                <p>There has been a growing number of nutrition data visualization tools (DVTs) to monitor progress towards targets and encourage action. However, there are few documented examples of how to go about designing effective DVTs for nutrition-related audiences. In this Open Letter, we summarize reflections from collaborative efforts between the Nigeria Governors&#x2019; Forum (NGF) and the Data for Decisions to Expand Nutrition Transformation project (DataDENT) in 2019&#x2013;2021 to design a sub-national nutrition scorecard that aims to hold Nigeria&#x2019;s 36 Governors accountable to nutrition commitments. Our reflections add to an emerging body of work advocating for DVT design processes to develop a specific theory of change for how the DVT will influence target groups and achieve aims. Once the target audience is identified, it is important to create a strong engagement strategy to ensure that the DVT promotes constructive action. We also highlight the importance of identifying actionable indicators through participatory processes. We hope that these insights about collaborative DVT design can be applied by countries and institutions who want to develop similar tools to advance the nutrition agenda in their context.</p>
            </abstract>
            <kwd-group kwd-group-type="author">
                <kwd>Nutrition data visualization tool</kwd>
                <kwd>data-driven nutrition accountability</kwd>
                <kwd>data-driven nutrition advocacy</kwd>
            </kwd-group>
            <funding-group>
                <award-group id="fund-1" xlink:href="http://dx.doi.org/10.13039/100000865">
                    <funding-source>Gates Foundation</funding-source>
                    <award-id>OPP1174256</award-id>
                </award-group>
                <funding-statement>This work was supported by the Gates Foundation [OPP1174256]. This article was led by the Nigeria Governors&#x2019; Forum and Results for Development as part of the Data for Decisions to Expand Nutrition Transformation (DataDENT) initiative. DataDENT is a four-year initiative (2017-2021) that aims to transform the availability and use of nutrition data by addressing gaps in nutrition measurement and advocating for stronger nutrition data systems. DataDENT is funded by the Gates Foundation [OPP1174256], and is implemented by three institutions: Institute for International Programs (IIP) at the Johns Hopkins Bloomberg School of Public Health, the International Food Policy Research Institute (IFPRI), and Results for Development (R4D).</funding-statement>
                <funding-statement>
                    <italic>The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.</italic>
                </funding-statement>
            </funding-group>
        </article-meta>
    </front>
    <body>
        <sec sec-type="intro">
            <title>Introduction</title>
            <p>
                <ext-link ext-link-type="uri" xlink:href="https://globalnutritionreport.org/reports/2014-global-nutrition-report/">The 2014 Global Nutrition Report</ext-link> made a strong call for a nutrition data revolution so that governments can better utilize data to set their agenda, monitor progress, and ensure accountability for nutrition programs. This requires strengthening all links of the nutrition data value chain (DVC), including development of effective approaches for analysis and translation of key data for use by decision makers (
                <xref ref-type="fig" rid="f1">Figure 1</xref>)
                <sup>
                    <xref ref-type="bibr" rid="ref-1">1</xref>
                </sup>. In the last few years, the nutrition community has seen a proliferation of data visualization tools (DVTs) targeting audiences at global
                <sup>
                    <xref ref-type="bibr" rid="ref-2">2</xref>
                </sup> and national levels
                <sup>
                    <xref ref-type="bibr" rid="ref-3">3</xref>,
                    <xref ref-type="bibr" rid="ref-4">4</xref>
                </sup>. However, many of these tools have not been designed with a clear user audience in mind nor an understanding of how the tool will be practically used as part of decision-making processes. There are few documented examples of how to go about designing effective DVTs for nutrition-related audiences.</p>
            <fig fig-type="figure" id="f1" orientation="portrait" position="float">
                <label>Figure 1. </label>
                <caption>
                    <title>Nutrition data value chain. </title>
                    <p>Adapted from Piwoz 
                        <italic toggle="yes">et al.</italic>, 2019
                        <sup>
                            <xref ref-type="bibr" rid="ref-5">5</xref>
                        </sup>.</p>
                </caption>
                <graphic orientation="portrait" position="float" xlink:href="https://gatesopenresearch-files.f1000.com/manuscripts/14562/16e0e9f4-75f7-44c9-80e1-b65b9064d8d3_figure1.gif"/>
            </fig>
            <p>In this article, we summarize reflections from collaborative efforts between the Nigeria Governors&#x2019; Forum (NGF) and the Data for Decisions to Expand Nutrition Transformation project (DataDENT) in 2019&#x2013;2021 to design a sub-national nutrition scorecard that aims to hold Nigeria&#x2019;s 36 Governors accountable to nutrition commitments. We first outline the purpose of the tool, then briefly describe the process we followed to develop it, and finally share reflections from the design process (summarized in 
                <xref ref-type="other" rid="B1">Box 1</xref>). Our aim is to share insights about collaborative DVT design that can be applied by countries and institutions who want to develop similar tools to advance the nutrition agenda in their context.</p>
            <p>
                <boxed-text content-type="website" id="B1" orientation="portrait" position="float">
                    <caption>
                        <title>Box 1. Key reflections</title>
                    </caption>
                    <list list-type="bullet">
                        <list-item>
                            <label>1. </label>
                            <p>Start with a clear theory of change including an explicit understanding of who the target user group is and how the tool will influence their actions,</p>
                        </list-item>
                        <list-item>
                            <label>2. </label>
                            <p>The purpose and audience dictate key features of the data visualization tool (DVT),</p>
                        </list-item>
                        <list-item>
                            <label>3. </label>
                            <p>Directly engage target users in selecting or &#x201c;co-creating&#x201d; indicators as the process of indicator specification itself can influence policy design and implementation,</p>
                        </list-item>
                        <list-item>
                            <label>4. </label>
                            <p>Select indicators that reflect progress in both the presence and quality dimensions of the commitments,</p>
                        </list-item>
                        <list-item>
                            <label>5. </label>
                            <p>Consider both interim and aspirational indicators, when desired data are not available at design stage but feasible to collect over time, </p>
                        </list-item>
                        <list-item>
                            <label>6. </label>
                            <p>Employ strategies to ensure data quality and build data ownership,</p>
                        </list-item>
                        <list-item>
                            <label>7. </label>
                            <p>Ensure visualization features respond to the target audiences&#x2019; needs.</p>
                        </list-item>
                        <list-item>
                            <label>8. </label>
                            <p>Create a strong engagement strategy with the target audience to ensure that the DVT promotes constructive action.</p>
                        </list-item>
                    </list>
                </boxed-text>
</p>
        </sec>
        <sec>
            <title>Purpose of NGF&#x2019;s Nutrition Scorecard</title>
            <p>Improved nutrition is a key ingredient for the development of Nigeria&#x2019;s rich human capital potential. Improvements in maternal and early childhood nutrition could yield $10 in economic returns for evey dollar invested through improved productivity and reduced healthcare costs over the lifespan
                <sup>
                    <xref ref-type="bibr" rid="ref-6">6</xref>
                </sup>. Nigeria is projected to have the world&#x2019;s second largest working age population at the end of this century
                <sup>
                    <xref ref-type="bibr" rid="ref-7">7</xref>
                </sup>. However, at present, Nigeria has the second highest burden of stunted children in the world; nearly half of all deaths in Nigerian children under 5 years can be attributed to malnutrition
                <sup>
                    <xref ref-type="bibr" rid="ref-8">8</xref>
                </sup>. Nigeria&#x2019;s 36 state Governors are well-positioned to advance the nutrition agenda by authorizing and supporting increased nutrition investment and providing oversight to nutrition activities. However, the Governors are faced with many competing priorities and demands for the available resources within their scope of influence. It is in this regard that the NGF Nutrition Scorecard was developed as an advocacy, accountability and monitoring tool for the nutrition commitments targeted towards achieving better nutrition outcomes. Launched in 2021, the scorecard is expected to be shared with the Governors at least semi-annually.</p>
            <p>The 
                <ext-link ext-link-type="uri" xlink:href="http://ngf.org.ng/">Nigeria Governors&#x2019; Forum (NGF)</ext-link> is the non-partisan association of the 36 elected state Governors of Nigeria. Policy advisors who staff the NGF Secretariat provide administrative and technical support aligned with the forum&#x2019;s vision to promote good governance and sustainable development. The NGF has a strong track record of using data to support decision-making and advocacy among Nigeria&#x2019;s Governors. For instance, starting in 2009, the NGF designed and implemented state level scorecards to engage Governors around polio eradication efforts and to promote the establishment of state-supported health insurance schemes and release of basic health care provision funds
                <sup>
                    <xref ref-type="bibr" rid="ref-9">9</xref>
                </sup>. Given its successes deploying scorecards to promote Governors&#x2019; engagement on other issues, the NGF Secretariat decided to develop a similar tool for nutrition. The vision was for the scorecard to serve as an advocacy tool that will facilitate political commitment and investment in nutrition, promote Governor-level accountability and ultimately lead to improved nutrition outcomes across the states.	</p>
            <p>
                <ext-link ext-link-type="uri" xlink:href="http://www.datadent.org/">DataDENT</ext-link>, an initiative funded by the Bill &amp; Melinda Gates Foundation and focused on strengthening global and national DVC for nutrition, partnered with the NGF for the scorecard design process, building from experience landscaping users and producers of nutrition DVTs reaching global and Indian audiences, respectively
                <sup>
                    <xref ref-type="bibr" rid="ref-2">2</xref>,
                    <xref ref-type="bibr" rid="ref-10">10</xref>
                </sup>.</p>
        </sec>
        <sec>
            <title>Process for Nutrition Scorecard development</title>
            <p>In 
                <xref ref-type="fig" rid="f2">Figure 2</xref>, we briefly describe the five key phases of the scorecard development process.</p>
            <fig fig-type="figure" id="f2" orientation="portrait" position="float">
                <label>Figure 2. </label>
                <caption>
                    <title>Nutrition Scorecard development process.</title>
                    <p>SCFN=State Committees on Food and Nutrition; DHIS-2=District Health Information System 2.</p>
                </caption>
                <graphic orientation="portrait" position="float" xlink:href="https://gatesopenresearch-files.f1000.com/manuscripts/14562/16e0e9f4-75f7-44c9-80e1-b65b9064d8d3_figure2.gif"/>
            </fig>
            <p>The 
                <bold>first phase (Sep 2019 - Oct 2019) focused on conceptualization</bold> where the design team
                <sup>
                    <xref ref-type="other" rid="FN1">a</xref>
                </sup>, led by the health team of the NGF Secretariat, agreed on the overall theory of change for the nutrition scorecard. To support these efforts, the team systematically identified and actively engaged state and federal levels stakeholders who are key players positioned to drive state-level nutrition action. These included:</p>
            <p>&#x2022;&#x00a0;&#x00a0;
                <bold>Governors and deputy Governors</bold>: The design team engaged specific Governors who were known to champion nutrition and sought feedback from them about their experience with NGF scorecards on other thematic areas. Engaging early on with members of the primary user group helped give the project traction. </p>
            <p>&#x2022;&#x00a0;&#x00a0;
                <bold>Federal and state-level multi-sector nutrition coordination bodies</bold>: In Nigeria, the National and State Committees on Food and Nutrition (SCFN) established by the Nigeria&#x2019;s National Policy on Food and Nutrition, as well as the Federal Ministry of Finance, Budget, and National Planning are the primary stakeholders who coordinate cross-sector nutrition implementation in their jurisdictions.</p>
            <p>&#x2022;&#x00a0;&#x00a0;
                <bold>Development partners</bold>: Through the Nigeria Nutrition Partners Forum, the design team engaged leading development partners working in nutrition including UNICEF, Alive &amp; Thrive, Save the Children, World Bank Accelerating Nutrition Results in Nigeria (ANRiN) project, and Civil Society &#x2013; Scaling Up Nutrition in Nigeria (CS-SUNN). The early involvement of these influential stakeholders helped steady the project course and provided a broad range of technical resources available to the team during indicator development.</p>
            <p>The 
                <bold>second phase (Nov 2019 - Feb 2020) focused on development of key scorecard components including indicators</bold> for the core nutrition commitments that the NGF had already identified as most directly shaping the enabling environment for nutrition in Nigeria (
                <xref ref-type="fig" rid="f3">Figure 3</xref>). The design team especially consulted Nigeria-based stakeholders (i.e., both state representatives and development partners) to unpack key challenges that states were facing in making progress on the four commitment areas that could be converted into indicators for inclusion in the scorecard. The team considered indicators that were meaningful, easily understood, time-bound, measurable, and regularly available.</p>
            <fig fig-type="figure" id="f3" orientation="portrait" position="float">
                <label>Figure 3. </label>
                <caption>
                    <title>Nigeria Governors&#x2019; Forum commitments in nutrition.</title>
                </caption>
                <graphic orientation="portrait" position="float" xlink:href="https://gatesopenresearch-files.f1000.com/manuscripts/14562/16e0e9f4-75f7-44c9-80e1-b65b9064d8d3_figure3.gif"/>
            </fig>
            <p>The 
                <bold>third phase (end of Feb 2020 - Apr 2020) focused on refinement of the scorecard components</bold>. A key activity was a participatory workshop with 30 federal and state-level government stakeholders and development partners. The workshop aims included 1) to present a revised draft of scorecard indicators based on earlier series of bilateral meetings with stakeholders, 2) to validate the draft indicators and arrive at a final set of indicators for the scorecard, 3) to familiarize a wider net of state actors with content of the scorecard for better ownership and seamless data sourcing/refreshing once the scorecard is launched, and 4) to identify additional uses for the scorecard. Post workshop, the design team revised indicators and followed up with select participants to ensure the revisions reflected the workshop inputs. 
                <xref ref-type="table" rid="T1">Table 1</xref> includes the final list of indicators included in scorecard&#x2019;s first round of data collection.</p>
            <table-wrap id="T1" orientation="portrait" position="anchor">
                <label>Table 1. </label>
                <caption>
                    <title>Nigeria Governors&#x2019; Forum nutrition indicators, scoring &amp; validation methods, and inclusion in final scorecard for presentation to Governors.</title>
                    <p>SCFN=State Committees on Food and Nutrition; MDA=Ministries, Departments, and Agencies; MSPAN=multisectoral strategic plan of action on nutrition.</p>
                </caption>
                <table content-type="article-table" frame="hsides">
                    <thead>
                        <tr>
                            <th align="left" colspan="1" rowspan="1">Commitment</th>
                            <th align="left" colspan="1" rowspan="1">Indicator</th>
                            <th align="left" colspan="1" rowspan="1">Scoring Method</th>
                            <th align="left" colspan="1" rowspan="1">Validation Method</th>
                            <th align="left" colspan="1" rowspan="1">Included in
                                <break/>Scorecard?</th>
                        </tr>
                    </thead>
                    <tbody>
                        <tr>
                            <td align="left" colspan="1" rowspan="5">
                                <bold>To set up/revitalize SCFN</bold>
                            </td>
                            <td align="left" colspan="1" rowspan="1">SCFN held meeting in previous quarter (Jan - Mar 2020)</td>
                            <td align="left" colspan="1" rowspan="1">Yes &#x2013; green
                                <break/>No &#x2013; red</td>
                            <td align="left" colspan="1" rowspan="1">SCFN last meeting minutes</td>
                            <td align="center" colspan="1" rowspan="1">
                                <styled-content style="#00B050" style-type="color">
                                    <bold>YES</bold>
                                </styled-content>
                            </td>
                        </tr>
                        <tr>
                            <td align="left" colspan="1" rowspan="1">Chairperson of SCFN chaired the previous quarter meeting</td>
                            <td align="left" colspan="1" rowspan="1">Yes &#x2013; green
                                <break/>No &#x2013; red</td>
                            <td align="left" colspan="1" rowspan="1">SCFN last meeting minutes</td>
                            <td align="center" colspan="1" rowspan="1">
                                <styled-content style="#00B050" style-type="color">
                                    <bold>YES</bold>
                                </styled-content>
                            </td>
                        </tr>
                        <tr>
                            <td align="left" colspan="1" rowspan="1">Representatives from 5 nutrition relevant MDAs attended
                                <break/>previous quarter meeting</td>
                            <td align="left" colspan="1" rowspan="1">Yes &#x2013; green
                                <break/>No &#x2013; red (if any MDA is left out, the
                                <break/>entire indicator is red)</td>
                            <td align="left" colspan="1" rowspan="1">SCFN last meeting minutes</td>
                            <td align="center" colspan="1" rowspan="1">
                                <styled-content style="#FF0000" style-type="color">
                                    <bold>NO</bold>
                                </styled-content>
                            </td>
                        </tr>
                        <tr>
                            <td align="left" colspan="1" rowspan="1">SCFN has a costed workplan for the current year</td>
                            <td align="left" colspan="1" rowspan="1">Yes &#x2013; green
                                <break/>No &#x2013; red</td>
                            <td align="left" colspan="1" rowspan="1">Copy of costed workplan</td>
                            <td align="center" colspan="1" rowspan="1">
                                <styled-content style="#FF0000" style-type="color">
                                    <bold>NO</bold>
                                </styled-content>
                            </td>
                        </tr>
                        <tr>
                            <td align="left" colspan="1" rowspan="1">At least 75% necessary funding for workplan released
                                <break/>quarterly</td>
                            <td align="left" colspan="1" rowspan="1">Yes &#x2013; green
                                <break/>No &#x2013; red</td>
                            <td align="left" colspan="1" rowspan="1">Email verification from
                                <break/>SCFN/state nutrition officer</td>
                            <td align="center" colspan="1" rowspan="1">
                                <styled-content style="#00B050" style-type="color">
                                    <bold>YES</bold>
                                </styled-content>
                            </td>
                        </tr>
                        <tr>
                            <td align="left" colspan="1" rowspan="3">
                                <bold>To establish MSPANs</bold>
                            </td>
                            <td align="left" colspan="1" rowspan="1">MSPAN developed and costed; includes component
                                <break/>activities from 2 nutrition-relevant MDAs</td>
                            <td align="left" colspan="1" rowspan="1">Yes &#x2013; green (must be costed AND
                                <break/>have activities from at least 2 MDAs)
                                <break/>No &#x2013; red</td>
                            <td align="left" colspan="1" rowspan="1">Copy of MSPAN</td>
                            <td align="center" colspan="1" rowspan="1">
                                <styled-content style="#00B050" style-type="color">
                                    <bold>YES</bold>
                                </styled-content>
                            </td>
                        </tr>
                        <tr>
                            <td align="left" colspan="1" rowspan="1">Approved MSPAN</td>
                            <td align="left" colspan="1" rowspan="1">Yes &#x2013; green
                                <break/>No &#x2013; red</td>
                            <td align="left" colspan="1" rowspan="1">Copy of MSPAN</td>
                            <td align="center" colspan="1" rowspan="1">
                                <styled-content style="#00B050" style-type="color">
                                    <bold>YES</bold>
                                </styled-content>
                            </td>
                        </tr>
                        <tr>
                            <td align="left" colspan="1" rowspan="1">MSPAN progress reviewed annually</td>
                            <td align="left" colspan="1" rowspan="1">Yes &#x2013; green
                                <break/>No &#x2013; red</td>
                            <td align="left" colspan="1" rowspan="1">Copy of meeting minutes
                                <break/>from review meeting</td>
                            <td align="center" colspan="1" rowspan="1">
                                <styled-content style="#FF0000" style-type="color">
                                    <bold>NO</bold>
                                </styled-content>
                            </td>
                        </tr>
                        <tr>
                            <td align="left" colspan="1" rowspan="1">
                                <bold>To increase budgetary</bold>
                                <break/>
                                <bold> spending on nutrition</bold>
                                <break/>
                                <bold>interventions</bold>
                            </td>
                            <td align="left" colspan="1" rowspan="1">State list of prioritized nutrition programs (from the MSPAN,
                                <break/>if available) for budget tracking shared</td>
                            <td align="left" colspan="1" rowspan="1">Yes &#x2013; green
                                <break/>No &#x2013; red</td>
                            <td align="left" colspan="1" rowspan="1">Copy of prioritized list</td>
                            <td align="center" colspan="1" rowspan="1">
                                <styled-content style="#FF0000" style-type="color">
                                    <bold>NO</bold>
                                </styled-content>
                            </td>
                        </tr>
                        <tr>
                            <td align="left" colspan="1" rowspan="5">
                                <bold>To promote maternity</bold>
                                <break/>
                                <bold>protections for civil</bold>
                                <break/>
                                <bold>servants</bold>
                            </td>
                            <td align="left" colspan="1" rowspan="1">6-month maternity leave exists</td>
                            <td align="left" colspan="1" rowspan="1">Yes &#x2013; green
                                <break/>No &#x2013; red</td>
                            <td align="left" colspan="1" rowspan="1">Copy of circular</td>
                            <td align="center" colspan="1" rowspan="1">
                                <styled-content style="#00B050" style-type="color">
                                    <bold>YES</bold>
                                </styled-content>
                            </td>
                        </tr>
                        <tr>
                            <td align="left" colspan="1" rowspan="1">6-month maternity leave comes with full pay</td>
                            <td align="left" colspan="1" rowspan="1">Yes &#x2013; green
                                <break/>No &#x2013; red</td>
                            <td align="left" colspan="1" rowspan="1">Copy of circular</td>
                            <td align="center" colspan="1" rowspan="1">
                                <styled-content style="#00B050" style-type="color">
                                    <bold>YES</bold>
                                </styled-content>
                            </td>
                        </tr>
                        <tr>
                            <td align="left" colspan="1" rowspan="1">Registered creche within the State Secretariat</td>
                            <td align="left" colspan="1" rowspan="1">Yes &#x2013; green
                                <break/>No &#x2013; red</td>
                            <td align="left" colspan="1" rowspan="1">No validation currently
                                <break/>required</td>
                            <td align="center" colspan="1" rowspan="1">
                                <styled-content style="#00B050" style-type="color">
                                    <bold>YES</bold>
                                </styled-content>
                            </td>
                        </tr>
                        <tr>
                            <td align="left" colspan="1" rowspan="1">State Secretariat creche is functional (ratio of 5 children to 1
                                <break/>nanny, handwashing station)</td>
                            <td align="left" colspan="1" rowspan="1">Yes &#x2013; green (both the ratio of children
                                <break/>to nannies and the handwashing
                                <break/>station must be present)
                                <break/>No &#x2013; red</td>
                            <td align="left" colspan="1" rowspan="1">No validation currently
                                <break/>required</td>
                            <td align="center" colspan="1" rowspan="1">
                                <styled-content style="#00B050" style-type="color">
                                    <bold>YES</bold>
                                </styled-content>
                            </td>
                        </tr>
                        <tr>
                            <td align="left" colspan="1" rowspan="1">State Secretariat creche has a breastfeeding/
                                <break/>complementary feeding corner</td>
                            <td align="left" colspan="1" rowspan="1">Yes &#x2013; green
                                <break/>No &#x2013; red</td>
                            <td align="left" colspan="1" rowspan="1">No validation currently
                                <break/>required</td>
                            <td align="center" colspan="1" rowspan="1">
                                <styled-content style="#FF0000" style-type="color">
                                    <bold>NO</bold>
                                </styled-content>
                            </td>
                        </tr>
                    </tbody>
                </table>
            </table-wrap>
            <p>The 
                <bold>fourth phase (May 2020 - October 2020) focused on collecting and validating state-level data to populate the scorecard.</bold> The SCFN were identified as the most appropriate source for reporting on the scorecard indicators. The design team adapted to the coronavirus disease 2019 (COVID-19) pandemic lockdowns and conducted a virtual workshop with the chairpersons and secretaries of the SCFN. Aims included 1) understanding the challenges and opportunities that the SCFN Chairs foresee with establishing a functional SCFN in their respective states and solicit suggestions in response to these challenges; 2) creating a framework for how SCFN develop their annual workplans; and 3) familiarizing and generating buy-in for the NGF nutrition scorecard and to capture inputs on the scorecard data collection instrument. All 36 states submitted some data within five weeks of the workshop. The design team reviewed submissions and engaged each SCFN to validate results and follow-up on missing data. Prior to the presentation to the Governors, the design team also conducted a 1.5 hour virtual meeting with all state representatives and partners to 1) provide an overview of the findings and key takeaways from the first round of data population of the scorecard for feedback, and 2) explain how the NGF Secretariat intends to use the scorecard with the Governors and their teams.</p>
            <p>
                <bold>The fifth and final phase (end of October 2020 &#x2013; April 2021) focused on designing outcome indicators and launching the scorecard.</bold> The NGF sought to include a nutrition coverage or outcome indicator on the scorecard, in addition to the enabling environment indicators, to sensitize Governors to the nutrition situation in their respective states and motivate action. The Secretariat is rightly committed to using administrative data from the 
                <ext-link ext-link-type="uri" xlink:href="https://dhis2.org/">District Health Information System 2 (DHIS-2)</ext-link> to populate this indicator on the scorecard, given that it is updated more frequently than survey estimates and use of this data is necessary to prompt improvements in data quality. After a review of available indicators and data quality constraints, which included consultations with nutrition measurement experts and the DHIS-2 team in Nigeria, the design team selected severe acute malnutrition (SAM) treatment coverage as the nutrition coverage indicator to include on the scorecard (
                <xref ref-type="table" rid="T2">Table 2</xref>). </p>
            <table-wrap id="T2" orientation="portrait" position="anchor">
                <label>Table 2. </label>
                <caption>
                    <title>Coverage Indicator Included on the Nigeria Governors&#x2019; Forum Nutrition Scorecard.</title>
                    <p>SAM=severe acute malnutrition; U5=under 5 years.</p>
                </caption>
                <table content-type="article-table" frame="hsides">
                    <thead>
                        <tr>
                            <th align="left" colspan="1" rowspan="1">Indicator</th>
                            <th align="left" colspan="1" rowspan="1">Definition</th>
                            <th align="left" colspan="1" rowspan="1">Source</th>
                            <th align="left" colspan="1" rowspan="1">Threshold</th>
                        </tr>
                    </thead>
                    <tbody>
                        <tr>
                            <td align="left" colspan="1" rowspan="1">
                                <bold>U5 SAM Coverage</bold>
                            </td>
                            <td align="left" colspan="1" rowspan="1">% of children under five with
                                <break/>SAM placed on treatment</td>
                            <td align="left" colspan="1" rowspan="1">District Health Information
                                <break/>System 2 (Jan-Mar 2020)</td>
                            <td align="left" colspan="1" rowspan="1">75%</td>
                        </tr>
                    </tbody>
                </table>
            </table-wrap>
            <p>The scorecard was presented to the Governors at the meeting of the NGF in April 2021. The Governors acknowledged the importance of nutrition and the need to urgently address poor nutrition outcomes in the country. They agreed to follow up with their respective Commissioners of Health to galvanize appropriate actions to address identified gaps. Highlights of the scorecard were also shared with the Vice President of the Federal Republic of Nigeria, who serves as the chairman of the National Council on Nutrition. The NGF will continue to update the scorecard quarterly and present it to the Governors every six months.</p>
        </sec>
        <sec>
            <title>Key reflections</title>
            <p>In the following section, we present key reflections based on the design process described above.</p>
            <sec>
                <title>Start with a clear theory of change including an explicit understanding of who the target user group is and how the tool will influence their actions</title>
                <p>The &#x201c;build-it-and-they-will-come&#x201d; approach is insufficient. To ensure the use of data for decision-making, DVTs need an intentional theory of change about a) the target audience, b) clear decisions or behavior that the DVT is trying to influence, and c) a pathway of how the data/actions lead to change. Given this learning from the team&#x2019;s landscape review of global nutrition DVTs
                    <sup>
                        <xref ref-type="bibr" rid="ref-2">2</xref>
                    </sup>, the design team had several rounds of discussions to specify the primary scorecard users: the Governors and their support staff at the onset of phase I. The team envisioned that the Governors would review the updated scorecard during a brief slot in the regular NGF quarterly meeting agenda. The guiding theory of change was that by comparing progress across states, the scorecard would motivate the Governors to maintain a good ranking by taking actions towards the commitments.</p>
            </sec>
            <sec>
                <title>The purpose and audience dictate key features of the DVT</title>
                <p>For the NGF use case, the purpose and audience dictated two key choices: the type of DVT and the types of indicators. The design team chose a scorecard over other types of DVTs (e.g. dashboard; profile) because it was familiar to the Governors and would best facilitate cross-state comparisons of performance. The design team kept the scorecard focused on the four commitments that the Governors have the power to influence and that also have a conceptual linkage to improved nutrition outcomes. The commitments are focused on creating &#x201c;enabling environments&#x201d; for other stakeholders to implement effective nutrition programs. In turn, the indicators tracked under each commitment were selected to be &#x201c;actionable&#x201d; by the Governors. Rather than tracking quarterly changes in implementation inputs or coverage of specific nutrition interventions (which would be actionable indicators for other implementation-focused audiences), the indicators for the NGF audience are tied to policy, coordination, and budget.</p>
            </sec>
            <sec>
                <title>Directly engage target users in selecting or &#x201c;co-creating&#x201d; indicators as the process of indicator specification itself can influence policy design and implementation</title>
                <p>The choice of indicators plays a powerful role in determining how people conceptualize a policy or program, what gets assessed, and consequently elevated for attention
                    <sup>
                        <xref ref-type="bibr" rid="ref-11">11</xref>
                    </sup>. While the commitment areas had been identified before DVT design phase I, the indicators to monitor those commitments had not. By engaging state-level representatives and development partners in indicator selection discussions, current barriers to progress on these commitments were identified. For example, discussion of the proposed indicator for &#x201c;quarterly meetings of the SCFN with the Chairperson in attendance&#x201d; revealed that these meetings were not happening and needed Governor attention. In addition, as indicators can influence user priorities, it was important to engage other coalitions involved with related issues to ensure the DVT indicators complement rather than compete with their ongoing efforts. Here, it is important to note that, in the case of the NGF Nutrition Scorecard, the design team developed unique fit-to-purpose indicator definitions, but it is preferable, whenever feasible, to align DVT indicators with established national and/or global standard definitions. </p>
            </sec>
            <sec>
                <title>Select indicators that reflect progress in both the presence and quality dimensions of the commitments</title>
                <p>Indicators can measure whether a critical component of the enabling environment exists, (e.g. whether a multi stakeholder platform exists; whether a legislation has been passed) without reflecting the quality or effectiveness of their implementation. Such indicators provide policymakers and advocates with limited information and, in some cases, a false notion that progress is being made
                    <sup>
                        <xref ref-type="bibr" rid="ref-12">12</xref>
                    </sup>. For this reason, the design team selected indicators that reflected progress in both the presence and quality dimensions of commitments. For example, states progress on the state level multisectoral nutrition plan commitment required that the plans were both multisectoral (i.e. included the participation of at least two nutrition related government ministries and agencies) and costed. An indicator under the SCFN commitment specified the quarterly release of at least 75% of necessary funding for the SCFN workplan because state-level stakeholders identified lack of funding as a key barrier to SCFN functioning. </p>
            </sec>
            <sec>
                <title>Consider both interim and aspirational indicators, when desired data are not available at design stage but feasible to collect over time</title>
                <p>There was no existing system to collect data for the proposed indicators for the four commitments, but it was feasible for the SCFN, with some initial support, to start to report on three of them. However, in the case of the nutrition spending commitment, most states did not have nutrition budget lines at the time of scorecard launch that could facilitate reporting. Many states did not even have costed multisectoral plans that identify which programs were relevant to track for nutrition spending. Therefore, for this commitment, the design team identified an interim indicator which assessed whether states had taken the first step of identifying which programs to track for nutrition spending. The design team then began engaging state actors and partners regarding how to develop the necessary state-level finance tracking systems to collect data on possible aspirational indicators (i.e. total nutrition spending per child under five) and drew attention to the need to collect this data. </p>
            </sec>
            <sec>
                <title>Employ strategies to ensure data quality and build data ownership</title>
                <p>Since the scorecard relied on self-reported data from the SCFN, the design team took several measures to ensure accurate reporting including hosting a virtual workshop with the chairpersons and secretaries of the SCFN to identify potential challenges with filling out the data collection questionnaire and ways to strengthen reporting capacity. After the workshop, the NGF team started a WhatsApp group with the SCFN representatives to facilitate reporting reminders, answer questions, and disseminate relevant information and events. For the first round, the NGF team was able to solicit responses from all 36 states within five weeks of the questionnaire being launched. The NGF team also asked for documentation to validate the SCFN self-report whenever feasible and appropriate (
                    <xref ref-type="table" rid="T1">Table 1</xref>). For example, those reporting that the SCFN met in the last quarter (Jan - March 2020) were also asked to share meeting minutes to confirm who specifically attended the meeting (scorecard indicator required that the chairperson of the SCFN attend the meeting). Such validation was essential to ensure accuracy and consistency of data across states. The SCFN were able to review their scorecard values before the presentation to Governors. </p>
            </sec>
            <sec>
                <title>Visualization features should respond to the target audiences&#x2019; needs</title>
                <p>It is important to tailor the visualization format to the needs of the intended audience. For the NGF Nutrition Scorecard, the design team gathered feedback on the Governors&#x2019; experience with other NGF scorecards. Feedback included that Governors do not have a lot of time to review the DVT in detail and prefer a simple visualization format that helps them assess their state&#x2019;s progress over time compared to other states. The design team condensed the scorecard in a single page with fewer key indicators and, similar to earlier scorecards, used basic red vs. green color coding to show whether the indicator had been achieved. In addition, the design team also included a coverage indicator &#x2013; SAM treatment coverage - in the scorecard and presented survey data on stunting and exclusive breastfeeding separately so that Governors had an eye on the overall state of nutrition in their respective states while assessing their progress on the enabling environment commitments. In future quarterly updates, cells will also include arrows to indicate direction of progress since the last presentation (
                    <xref ref-type="fig" rid="f4">Figure 4</xref>). A second expanded version of the scorecard with the full set of indicators was developed for the SCFN to meet their internal tracking needs (
                    <xref ref-type="fig" rid="f5">Figure 5</xref>).</p>
                <fig fig-type="figure" id="f4" orientation="portrait" position="float">
                    <label>Figure 4. </label>
                    <caption>
                        <title>Final visualization format for the Nigeria Governors&#x2019; Forum nutrition scorecard.</title>
                        <p>Green=State met commitment/target; Red=State did not meet commitment/target; MSPAN=multisectoral strategic plan of action on nutrition; DHIS-2=District Health Information System 2.</p>
                    </caption>
                    <graphic orientation="portrait" position="float" xlink:href="https://gatesopenresearch-files.f1000.com/manuscripts/14562/16e0e9f4-75f7-44c9-80e1-b65b9064d8d3_figure4.gif"/>
                </fig>
                <fig fig-type="figure" id="f5" orientation="portrait" position="float">
                    <label>Figure 5. </label>
                    <caption>
                        <title>Full scorecard for State Committee on Food and Nutrition (SCFN)internal tracking purposes.</title>
                        <p>Green=State met commitment/target; Red=State did not meet commitment/target.</p>
                    </caption>
                    <graphic orientation="portrait" position="float" xlink:href="https://gatesopenresearch-files.f1000.com/manuscripts/14562/16e0e9f4-75f7-44c9-80e1-b65b9064d8d3_figure5.gif"/>
                </fig>
            </sec>
            <sec>
                <title>Create a strong engagement strategy with the target audience to ensure that the DVT promotes constructive action</title>
                <p>In addition to the quarterly NGF meetings, the design team worked closely with nutrition stakeholders to develop guidelines that clearly lay out the actions required at state level to achieve each commitment and a roster of federal and state-level partners active in each commitment area who can provide Governors and the SCFN with technical assistance. The NGF Secretariat can use these resources to productively engage with Governors&#x2019; offices and SCFN in follow up to quarterly scorecard presentations. </p>
            </sec>
        </sec>
        <sec sec-type="discussion">
            <title>Discussion</title>
            <p>To our knowledge, this is one of the first studies to share reflections stemming from the design phase of a DVT that aims to influence nutrition-related decision making in low- or middle-income countries. Our reflections add to an emerging body of work advocating for DVT design processes to develop a specific theory of change for how the DVT will influence target groups and achieve aims
                <sup>
                    <xref ref-type="bibr" rid="ref-2">2</xref>,
                    <xref ref-type="bibr" rid="ref-3">3</xref>,
                    <xref ref-type="bibr" rid="ref-13">13</xref>
                </sup>. This is in contrast to the &#x201c;if we build it, they will come&#x201d; approach that is often used to justify DVT development. We hope it also provides practical insights to DVT producers about the importance of identifying actionable indicators through participatory processes.</p>
            <p>The influence of the NGF nutrition scorecard is yet to be seen. In the next phase of work, the NGF and DataDENT design team will engage in rapid learning cycles to get Governor feedback and further adapt the scorecard. We plan to evaluate the impact of the scorecard and share findings with the global community.</p>
        </sec>
        <sec>
            <title>Data availability</title>
            <p>No data are associated with this article.</p>
        </sec>
    </body>
    <back>
        <ack>
            <title>Acknowledgements</title>
            <p>The authors gratefully acknowledge comments on this manuscript from Tricia Aung (Johns Hopkins Bloomberg School of Public Health), Muchiri Nyaggah (Local Development Research Institute) and Augustin Flory (Results for Development).</p>
        </ack>
        <fn-group>
            <fn id="FN1">
                <p>
                    <sup>a</sup>The scorecard design team was led by the health team at the NGF Secretariat and supported by the DataDENT project</p>
            </fn>
        </fn-group>
        <ref-list>
            <ref id="ref-1">
                <label>1</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Heidkamp</surname>
                            <given-names>R</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Aung</surname>
                            <given-names>T</given-names>
                        </name>
</person-group>:
                    <article-title>Has the revolution arrived? Why 2020 is a crucial year for nutrition data.</article-title>
                    <source>

                        <italic toggle="yes">Global Nutrition Report blog.</italic>
</source>
                    <year>2020</year>.
                    <ext-link ext-link-type="uri" xlink:href="https://globalnutritionreport.org/blog/has-revolution-arrived-why-2020-crucial-year-nutrition-data/">Reference Source</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-2">
                <label>2</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Manorat</surname>
                            <given-names>R</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Becker</surname>
                            <given-names>L</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Flory</surname>
                            <given-names>A</given-names>
                        </name>
</person-group>:
                    <article-title>Global Data Visualization Tools to Empower Decision- Making in Nutrition.</article-title>
                    <source>

                        <italic toggle="yes">Sight Life Mag.</italic>
</source>
                    <year>2019</year>;<volume>33</volume>(<issue>1</issue>):<fpage>108</fpage>&#x2013;<lpage>114</lpage>.
                    <ext-link ext-link-type="uri" xlink:href="https://sightandlife.org/wp-content/uploads/2019/08/SightandLifeMagazine_2019_Data_in_Nutrition_GlobalDataVisualizationTools.pdf">Reference Source</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-3">
                <label>3</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Van Wyk</surname>
                            <given-names>M</given-names>
                        </name>
</person-group>:
                    <article-title>Developing a dashboard of indicators.</article-title>
                    <source>

                        <italic toggle="yes">SUN.</italic>
</source>
                    <year>2014</year>.
                    <ext-link ext-link-type="uri" xlink:href="https://scalingupnutrition.org/news/developing-a-dashboard-of-indicators/">Reference Source</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-4">
                <label>4</label>
                <mixed-citation publication-type="journal">
                    <collab>POSHAN</collab>:
                    <article-title>Nutrition Data Resources.</article-title>POSHAN Led By IFRPI,<year>2020</year>.
                    <ext-link ext-link-type="uri" xlink:href="https://poshan.ifpri.info/nutrition-data-resources/">Reference Source</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-5">
                <label>5</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Piwoz</surname>
                            <given-names>E</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Rawat</surname>
                            <given-names>R</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Fracassi</surname>
                            <given-names>P</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Strengthening the Nutrition Data Value Chain for Accountability and Action.</article-title>
                    <source>

                        <italic toggle="yes">Sight and Life.</italic>
</source>
                    <year>2019</year>;<volume>33</volume>(<issue>1</issue>):<fpage>38</fpage>&#x2013;<lpage>43</lpage>.
                    <ext-link ext-link-type="uri" xlink:href="https://www.anh-academy.org/sites/default/files/SightandLifeMagazine_2019_Data_in_Nutrition_StrengtheningtheDataValueChain-AccountabilityandAction.pdf">Reference Source</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-6">
                <label>6</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Shekar</surname>
                            <given-names>M</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Kakietek</surname>
                            <given-names>J</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Dayton Eberwein</surname>
                            <given-names>J</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>An Investment Framework for Nutrition: Reaching the Global Targets for Stunting, Anemia, Breastfeeding, and Wasting.</article-title>The World Bank,<year>2017</year>.
                    <pub-id pub-id-type="doi">10.1596/978-1-4648-1010-7</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-7">
                <label>7</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Vollset</surname>
                            <given-names>SE</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Goren</surname>
                            <given-names>E</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Yuan</surname>
                            <given-names>CW</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Fertility, mortality, migration, and population scenarios for 195 countries and territories from 2017 to 2100: a forecasting analysis for the Global Burden of Disease Study.</article-title>
                    <source>

                        <italic toggle="yes">Lancet.</italic>
</source>
                    <year>2020</year>;<volume>396</volume>(<issue>10258</issue>):<fpage>1285</fpage>&#x2013;<lpage>1306</lpage>.
                    <pub-id pub-id-type="pmid">32679112</pub-id>
                    <pub-id pub-id-type="doi">10.1016/S0140-6736(20)30677-2</pub-id>
                    <pub-id pub-id-type="pmcid">7561721</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-8">
                <label>8</label>
                <mixed-citation publication-type="journal">
                    <collab>UNICEF</collab>:
                    <article-title>Nutrition | UNICEF Nigeria</article-title>. UNICEF,<year>2018</year>.
                    <ext-link ext-link-type="uri" xlink:href="https://www.unicef.org/nigeria/nutrition">Reference Source</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-9">
                <label>9</label>
                <mixed-citation publication-type="journal">
                    <collab>Nigeria Governors&#x2019; Forum</collab>:
                    <article-title>Achievements</article-title>. Nigeria Governors&#x2019; Forum,<year>2016</year>.
                    <ext-link ext-link-type="uri" xlink:href="https://www.nggovernorsforum.org/index.php/the-ngf/about-the-ngf/achievements">Reference Source</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-10">
                <label>10</label>
                <mixed-citation publication-type="journal">
                    <collab>IFPRI &amp; R4D</collab>:
                    <article-title>Landscaping of data visualization tools for nutrition - India</article-title>.<year>2020</year>.
                    <ext-link ext-link-type="uri" xlink:href="https://datadent.org/landscaping-of-data-visualization-tools-for-nutrition-india/">Reference Source</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-11">
                <label>11</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Borgn&#x00e4;s,</surname>
                            <given-names>K</given-names>
                        </name>
</person-group>:
                    <article-title>The Policy Influence of Sustainability Indicators: Examining Use and Influence of Indicators in German Sustainability Policy Making.</article-title>
                    <source>

                        <italic toggle="yes">German Politics.</italic>
</source>
                    <year>2016</year>;<volume>25</volume>(<issue>4</issue>):<fpage>480</fpage>&#x2013;<lpage>499</lpage>.
                    <pub-id pub-id-type="doi">10.1080/09644008.2016.1193160</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-12">
                <label>12</label>
                <mixed-citation publication-type="journal">
                    <collab>Feed the Future Enabling Environment for Food Security Project</collab>:
                    <article-title>Measuring the Enabling Environment: What Works, and What&#x2019;s Missing?</article-title>
                    <year> 2020</year>.
                    <ext-link ext-link-type="uri" xlink:href="https://www.agrilinks.org/post/measuring-enabling-environment-what-works-and-whats-missing">Reference Source</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-13">
                <label>13</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Monsivais</surname>
                            <given-names>P</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Francis</surname>
                            <given-names>O</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Lovelace</surname>
                            <given-names>R</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Data visualisation to support obesity policy: case studies of data tools for planning and transport policy in the UK.</article-title>
                    <source>

                        <italic toggle="yes">Int J Obes (Lond).</italic>
</source>
                    <year>2018</year>;<volume>42</volume>(<issue>12</issue>):<fpage>1977</fpage>&#x2013;<lpage>1986</lpage>.
                    <pub-id pub-id-type="pmid">30470805</pub-id>
                    <pub-id pub-id-type="doi">10.1038/s41366-018-0243-6</pub-id>
                    <pub-id pub-id-type="pmcid">6291420</pub-id>
                </mixed-citation>
            </ref>
        </ref-list>
    </back>
    <sub-article article-type="reviewer-report" id="report31300">
        <front-stub>
            <article-id pub-id-type="doi">10.21956/gatesopenres.14562.r31300</article-id>
            <title-group>
                <article-title>Reviewer response for version 1</article-title>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author">
                    <name>
                        <surname>Siekmans</surname>
                        <given-names>Kendra</given-names>
                    </name>
                    <xref ref-type="aff" rid="r31300a1">1</xref>
                    <role>Referee</role>
                    <uri content-type="orcid">https://orcid.org/0000-0002-3116-2278</uri>
                </contrib>
                <aff id="r31300a1">
                    <label>1</label>Independent Consultant, Public Health Nutrition, Guelph, ON, Canada</aff>
            </contrib-group>
            <author-notes>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>22</day>
                <month>11</month>
                <year>2021</year>
            </pub-date>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2021 Siekmans K</copyright-statement>
                <copyright-year>2021</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <related-article ext-link-type="doi" id="relatedArticleReport31300" related-article-type="peer-reviewed-article" xlink:href="10.12688/gatesopenres.13319.1"/>
            <custom-meta-group>
                <custom-meta>
                    <meta-name>recommendation</meta-name>
                    <meta-value>approve</meta-value>
                </custom-meta>
            </custom-meta-group>
        </front-stub>
        <body>
            <p>Yashodhara Rana and colleagues provide a useful overview of their experience in designing a scorecard targeted to Nigeria's State Governors and State Committees on Food and Nutrition. They describe the purpose of the scorecard, the underlying theory of change informing its design, and the steps taken in the design process. The article ends with a list of reflections on their learning through the process, providing helpful guidance to those who may want to replicate such a process. While no information is provided about follow-up actions taken by the scorecard's target audience since the launch, the authors note that there are plans in place for learning processes and an evaluation of the scorecard's impact in the future. Thus, although the article is useful as an example of what a user-designed process looks like, there is no evidence yet of whether the theory of change will hold true and if/how the scorecard will contribute to increased investment and action for nutrition.</p>
            <p> </p>
            <p> Based on recent reviews of country experiences with nutrition technical assistance (see references 1 to 5), there is a need for context-specific tools to assist national and sub-national government stakeholders in monitoring progress for nutrition. It is commendable that the process described here was led by a national&#x00a0;design team that continues to be involved and has a defined role in preparing and disseminating the scorecard to the Governors on a quarterly basis. The ongoing engagement with State Committees on Food and Nutrition and plans for providing technical assistance are expected to be important for building state-level capacity and supporting the system changes needed.</p>
            <p>Does the article adequately reference differing views and opinions?</p>
            <p>Yes</p>
            <p>Are all factual statements correct, and are statements and arguments made adequately supported by citations?</p>
            <p>Yes</p>
            <p>Is the Open Letter written in accessible language?</p>
            <p>Yes</p>
            <p>Where applicable, are recommendations and next steps explained clearly for others to follow?</p>
            <p>Yes</p>
            <p>Is the rationale for the Open Letter provided in sufficient detail?</p>
            <p>Yes</p>
            <p>Reviewer Expertise:</p>
            <p>nutrition programme monitoring and evaluation, information systems for nutrition, enabling environment for nutrition, nutrition technical assistance</p>
            <p>I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.</p>
        </body>
        <back>
            <ref-list>
                <title>References</title>
                <ref id="rep-ref-31300-1">
                    <label>1</label>
                    <mixed-citation>
                        <article-title>Measuring Progress: Summary of Progress Assessments of Eight Technical Assistance Assignments from 2017-2019 by NTEAM&#x2019;s TAN Project</article-title>.
                        <source>
                            <italic>Nutrition International</italic>
                        </source>.<year>2020</year>;
                        <ext-link ext-link-type="uri" xlink:href="https://www.nutritionintl.org/wp-content/uploads/2020/12/MEASUR1-1.pdf">Reference source</ext-link>
                    </mixed-citation>
                </ref>
                <ref id="rep-ref-31300-2">
                    <label>2</label>
                    <mixed-citation>
                        <person-group person-group-type="author"/>:
                        <article-title>External Progress Assessment of Technical Assistance Delivered under Nutrition International&#x2019;s TAN Project</article-title>.
                        <source>
                            <italic>Nutrition International</italic>
                        </source>.<year>2021</year>;
                        <ext-link ext-link-type="uri" xlink:href="https://www.nutritionintl.org/learning-resource/external-progress-assessment-technical-assistance-delivered-nutrition-internationals-tan-project/">Reference source</ext-link>
                    </mixed-citation>
                </ref>
                <ref id="rep-ref-31300-3">
                    <label>3</label>
                    <mixed-citation>
                        <article-title>Providing technical assistance for SUN countries: Documenting the added value of Nutrition International and MQSUN technical assistance for Nutrition in eight SUN countries</article-title>.
                        <source>
                            <italic>ENN</italic>
                        </source>.<year>2020</year>;
                        <ext-link ext-link-type="uri" xlink:href="https://mqsunplus.path.org/resources/providing-technical-assistance-for-sun-countries-documenting-the-added-value-of-nutrition-international-and-mqsun-technical-assistance-for-nutrition-in-eight-sun-countries/">Reference source</ext-link>
                    </mixed-citation>
                </ref>
                <ref id="rep-ref-31300-4">
                    <label>4</label>
                    <mixed-citation>
                        <person-group person-group-type="author"/>:
                        <article-title>Designing Effective Multisectoral Nutrition Collaboration: Insights and Lessons Learned from the National Information Platforms for Nutrition International Partnerships</article-title>.
                        <source>
                            <italic>National Information Platforms for Nutrition</italic>
                        </source>.
                        <ext-link ext-link-type="uri" xlink:href="https://www.nipn-nutrition-platforms.org/IMG/pdf/nipn_lessons_learnt_-_web.pdf">Reference source</ext-link>
                    </mixed-citation>
                </ref>
                <ref id="rep-ref-31300-5">
                    <label>5</label>
                    <mixed-citation>
                        <article-title>Effectiveness of Technical Assistance: Synthesis Report on Key Findings from Six Case Studies on the Effectiveness of Technical Assistance Provided by the TAN MQSUN Facility 2017-2020</article-title>.
                        <source>
                            <italic>TASC</italic>
                        </source>.<year>2021</year>;</mixed-citation>
                </ref>
            </ref-list>
        </back>
    </sub-article>
    <sub-article article-type="reviewer-report" id="report30870">
        <front-stub>
            <article-id pub-id-type="doi">10.21956/gatesopenres.14562.r30870</article-id>
            <title-group>
                <article-title>Reviewer response for version 1</article-title>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author">
                    <name>
                        <surname>Renshaw</surname>
                        <given-names>Melanie</given-names>
                    </name>
                    <xref ref-type="aff" rid="r30870a1">1</xref>
                    <role>Referee</role>
                </contrib>
                <aff id="r30870a1">
                    <label>1</label>African Leaders Malaria Alliance (ALMA), Nairobi, Kenya</aff>
            </contrib-group>
            <author-notes>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>19</day>
                <month>7</month>
                <year>2021</year>
            </pub-date>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2021 Renshaw M</copyright-statement>
                <copyright-year>2021</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <related-article ext-link-type="doi" id="relatedArticleReport30870" related-article-type="peer-reviewed-article" xlink:href="10.12688/gatesopenres.13319.1"/>
            <custom-meta-group>
                <custom-meta>
                    <meta-name>recommendation</meta-name>
                    <meta-value>approve</meta-value>
                </custom-meta>
            </custom-meta-group>
        </front-stub>
        <body>
            <p>The report is clear and well written. I agree with the need for simplicity but would include a 'yellow' for some progress to encourage the decision makers that there has been progress. It is not clear how actions will be captured and progress against the actions tracked, so perhaps more details on this. ALMA has created a scorecard hub which documents scorecard best practices and includes 'how to' guidelines,as well as including country uploaded scorecards - this could perhaps be considered as a reference? (
                <ext-link ext-link-type="uri" xlink:href="https://scorecardhub.org/">https://scorecardhub.org/</ext-link>). Finally, it would be good to understand the multisectoral dimension including how data across sectors will be systematically collected to allow automatic data population over time.</p>
            <p>Does the article adequately reference differing views and opinions?</p>
            <p>Yes</p>
            <p>Are all factual statements correct, and are statements and arguments made adequately supported by citations?</p>
            <p>Yes</p>
            <p>Is the Open Letter written in accessible language?</p>
            <p>Yes</p>
            <p>Where applicable, are recommendations and next steps explained clearly for others to follow?</p>
            <p>Yes</p>
            <p>Is the rationale for the Open Letter provided in sufficient detail?</p>
            <p>Yes</p>
            <p>Reviewer Expertise:</p>
            <p>ALMA has supported over 40 countries in the roll out of country scorecards and accountability tools including for malaria, RMNCAH, nutrition, NTDs and community scorecards</p>
            <p>I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.</p>
        </body>
    </sub-article>
</article>
