<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.2 20190208//EN" "http://jats.nlm.nih.gov/publishing/1.2/JATS-journalpublishing1.dtd"><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="data-paper" dtd-version="1.2" xml:lang="en">
    <front>
        <journal-meta>
            <journal-id journal-id-type="pmc">Gates Open Res</journal-id>
            <journal-title-group>
                <journal-title>Gates Open Research</journal-title>
            </journal-title-group>
            <issn pub-type="epub">2572-4754</issn>
            <publisher>
                <publisher-name>F1000 Research Limited</publisher-name>
                <publisher-loc>London, UK</publisher-loc>
            </publisher>
        </journal-meta>
        <article-meta>
            <article-id pub-id-type="doi">10.12688/gatesopenres.12812.1</article-id>
            <article-categories>
                <subj-group subj-group-type="heading">
                    <subject>Data Note</subject>
                </subj-group>
                <subj-group>
                    <subject>Articles</subject>
                </subj-group>
            </article-categories>
            <title-group>
                <article-title>A de-identified database of 11,979 verbal autopsy open-ended responses</article-title>
                <fn-group content-type="pub-status">
                    <fn>
                        <p>[version 1; peer review: 2 approved]</p>
                    </fn>
                </fn-group>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author" corresp="yes">
                    <name>
                        <surname>Flaxman</surname>
                        <given-names>Abraham D.</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Conceptualization</role>
                    <role content-type="http://credit.niso.org/">Data Curation</role>
                    <role content-type="http://credit.niso.org/">Investigation</role>
                    <role content-type="http://credit.niso.org/">Methodology</role>
                    <role content-type="http://credit.niso.org/">Validation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Original Draft Preparation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <uri content-type="orcid">https://orcid.org/0000-0001-6033-4713</uri>
                    <xref ref-type="corresp" rid="c1">a</xref>
                    <xref ref-type="aff" rid="a1">1</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Harman</surname>
                        <given-names>Lisa</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Data Curation</role>
                    <role content-type="http://credit.niso.org/">Investigation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <xref ref-type="aff" rid="a1">1</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Joseph</surname>
                        <given-names>Jonathan</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Data Curation</role>
                    <role content-type="http://credit.niso.org/">Investigation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <xref ref-type="aff" rid="a1">1</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Brown</surname>
                        <given-names>Jonathan</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Data Curation</role>
                    <role content-type="http://credit.niso.org/">Project Administration</role>
                    <role content-type="http://credit.niso.org/">Supervision</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <xref ref-type="aff" rid="a1">1</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Murray</surname>
                        <given-names>Christopher J.L.</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Conceptualization</role>
                    <role content-type="http://credit.niso.org/">Funding Acquisition</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <xref ref-type="aff" rid="a1">1</xref>
                </contrib>
                <aff id="a1">
                    <label>1</label>Institute for Health Metrics and Evaluation, University of Washington, Seattle, WA, 98121, USA</aff>
            </contrib-group>
            <author-notes>
                <corresp id="c1">
                    <label>a</label>
                    <email xlink:href="mailto:abie@uw.edu">abie@uw.edu</email>
                </corresp>
                <fn fn-type="conflict">
                    <p>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>17</day>
                <month>4</month>
                <year>2018</year>
            </pub-date>
            <pub-date pub-type="collection">
                <year>2018</year>
            </pub-date>
            <volume>2</volume>
            <elocation-id>18</elocation-id>
            <history>
                <date date-type="accepted">
                    <day>10</day>
                    <month>4</month>
                    <year>2018</year>
                </date>
            </history>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2018 Flaxman AD et al.</copyright-statement>
                <copyright-year>2018</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <self-uri content-type="pdf" xlink:href="https://gatesopenresearch.org/articles/2-18/pdf"/>
            <abstract>
                <p>As part of the Gates Grand Challenge 13, the Population Health Metrics Research Consortium (PHMRC) collected data to enable the development and validation of methods that measure cause-specific mortality in populations with incomplete or inadequate cause of death coding.</p>
                <p>This work yielded 11,979 verbal autopsy interviews (VAIs). In each, a field interviewer spoke with an individual familiar with the deceased and their final illness, and used a semi-structured questionnaire to collect information about the symptoms of the deceased in their final illness. The VAI collected demographic characteristics, possible risk factors (such as tobacco use), and other potentially contributing characteristics. It also included the open-ended question, 
                    <italic toggle="yes">&#x201c;Could you please summarize, or tell us in your own words, any additional information about the illness and/or death of your loved one?&#x201d;</italic> (open narrative).</p>
                <p>The VAI data were released in a de-identified format in September 2013 through the Global Health Data Exchange, in files that contain verbal autopsies that were collected at six sites in four countries (India, Mexico, Tanzania, and the Philippines).</p>
                <p>Due to research interest, we have now created redacted versions of the open narratives from the open-ended question of the questionnaire. We hope that this database will be the source of innovations that increase our knowledge about the causes of ill health and, through this knowledge, produce improvements in health for individuals and populations.</p>
            </abstract>
            <kwd-group kwd-group-type="author">
                <kwd>Verbal autopsy</kwd>
                <kwd>cause of death</kwd>
                <kwd>natural language processing</kwd>
                <kwd>open data</kwd>
            </kwd-group>
            <funding-group>
                <award-group id="fund-1" xlink:href="http://dx.doi.org/10.13039/100000865">
                    <funding-source>Gates Foundation</funding-source>
                    <award-id>OPP37883</award-id>
                </award-group>
                <funding-statement>This work was funded by a grant from the Gates Foundation through the Grand Challenges in Global Health initiative [OPP37883]. The funders had no role in study design, data collection and analysis, interpretation of data, decision to publish, or preparation of the manuscript. The corresponding author had full access to all data analyzed and had final responsibility for the decision to submit this original research paper for publication.</funding-statement>
            </funding-group>
        </article-meta>
    </front>
    <body>
        <sec sec-type="intro">
            <title>Introduction</title>
            <p>Population health information that is both accurate and comprehensive can aid program implementation, monitoring, and evaluation, resource allocation and planning. However, there are currently large gaps in the technologies and measurement methods that are available to generate this information, and this makes it difficult to address health inequities through effective policy
                <sup>
                    <xref ref-type="bibr" rid="ref-1">1</xref>
                </sup>.</p>
            <p>The Population Health Metrics Research Consortium (PHMRC) conducted data collection to enable the development and validation of methods that measure cause-specific mortality in populations with incomplete or inadequate cause of death coding. This work produced around 12,000 verbal autopsy interviews (VAIs), in which a relative or someone familiar with the final illness of the deceased, provides information about the signs symptoms of the final illness, as well as demographic characteristics, and information on risk factor exposures (such as tobacco use), and other potentially relevant characteristics
                <sup>
                    <xref ref-type="bibr" rid="ref-2">2</xref>
                </sup>.</p>
            <p>The VAI data were released in a de-identified format in September 2013, through the Global Health Data Exchange, in files that contain verbal autopsies from six sites in four countries (India, Mexico, Tanzania, and the Philippines) using a standardized VA questionnaire developed by the PHMRC. The data is organized into three parts corresponding to the questionnaire modules for each age group: neonate, child, and adult. Each VAI in the database is matched with a &#x201c;gold standard&#x201d; diagnoses of underlying causes of death, typically identified from medical records, and using stringent diagnostic criteria (such as laboratory, pathology, or medical imaging findings.)
                <sup>
                    <xref ref-type="bibr" rid="ref-3">3</xref>
                </sup>
            </p>
            <p>One portion of a VAI is the &#x201c;open narrative,&#x201d; where the respondent has the opportunity to tell, in their own words, what happened during the illness that led to the death being investigated. This was collected as a final question in the PHMRC survey, after the structured interview, when the respondent was asked, 
                <italic toggle="yes">&#x201c;Could you please summarize, or tell us in your own words, any additional information about the illness and/or death of your loved one?&#x201d;</italic> The full response to this question was transcribed and translated into English, and the 2013 data release included counts of stemmed keywords as variables in the final dataset, to allow researchers access to this rich source of unstructured data, while also removing any potentially personally identifiable information (PII) in that portion of the interview.</p>
            <p>Due to research interest, we have now created redacted versions of 11,979 open narratives to allow researchers the opportunity to learn even more about how deaths are described. We hope that this database will be the source of innovations that increase our knowledge about the causes of ill health and through knowledge produce improvements in health for individuals and populations.</p>
        </sec>
        <sec sec-type="methods">
            <title>Methods</title>
            <p>The process of collecting the VAIs has been described in detail previously
                <sup>
                    <xref ref-type="bibr" rid="ref-1">1</xref>
                </sup>. In this article, we provide a detailed account of the protocol used to redact personal information from the open-ended question, and therefore allow the release of the full text of the open narrative collected in the VAIs.</p>
            <p>Study participants provided their consent to participate with the knowledge that &#x201c;reports of the data &#x2026; will not identify any individual person.&#x201d; We chose also to redact the names of specific health facilities to avoid the risk of identifying individual health service providers indirectly, through their association with individual facilities. To retain the most information possible for future research, we replaced PII with &#x201c;tags&#x201d; that denote what sort of information has been redacted.</p>
            <p>An example makes this clear: a typical text was redacted to read, &#x201c;vaginal bleeding and delay to receive care at [HOSPITAL] was the main cause of death. he said that his wife arrive at the hospital at 8pm and didn&#x2019;t receive any care until 8am.&#x201d; Instead of including the name of the specific hospital, we redacted it to [HOSPITAL]. The tags used to replace PII are [HOSPITAL], [DOCTOR], [PATIENT], [PLACE], [PERSON], and [YEAR].</p>
            <p>We initially planned to redact dates entirely but chose to redact only the year, to make it easier for future researchers to measure the time between events. To allow for different years, we used the tag [YEAR + n]. An example is &#x201c;last november of [YEAR]-the deceased got stroke left side of his body. was hospitalized due to high blood pressure last year. january this year was his last hospitalization that leads to death. jan. 26, [YEAR+1]. experienced fast breathing, that&#x2019;s why he was brought to the hospital (provincial hospital). with oxygen and ngt; got fever and cough; in coma. jan. 31, [YEAR+1]. was tried to revive around 11:00 pm to 12 midnight, but was not able to revive him. around 3:00 am (at dawn), he died.&#x201d;</p>
            <p>When a response referred to multiple different specific hospitals, we redacted the hospitals to [HOSPITAL] and [HOSPITAL2]. Subsequent distinct hospitals in same passage were redacted to [HOSPITAL3], [HOSPITAL4], etc.</p>
            <p>We included all VAIs for which there was an open-response string available to redact, even when the response was devoid of information.</p>
            <p>We implemented the redaction process in a spreadsheet using Excel 2010, redacted manually by a single data analyst (LH), who read each open-response and replaced each piece of PII with the appropriate tag.</p>
            <sec>
                <title>Redaction rules, with some examples and counterexamples</title>
                <list list-type="bullet">
                    <list-item>
                        <label>1. </label>
                        <p>Specific patient becomes [PATIENT]</p>
                        <p>Example: John Smith was taken to &#x2026; -&gt; [PATIENT] was taken to</p>
                        <p>Counterexamples (no redaction for the following): She was taken to -&gt; She was taken to (no change)</p>
                        <p>My uncle was taken to -&gt; My uncle was taken to (no change)</p>
                        <p>The patient was taken to -&gt; The patient was taken to (no change)</p>
                    </list-item>
                    <list-item>
                        <label>2. </label>
                        <p>Specific health facility becomes [HOSPITAL],</p>
                    </list-item>
                    <list-item>
                        <label>3. </label>
                        <p>Specific doctor becomes [DOCTOR]</p>
                    </list-item>
                    <list-item>
                        <label>4. </label>
                        <p>Specific place that is not a health facility becomes [PLACE]</p>
                    </list-item>
                    <list-item>
                        <label>5. </label>
                        <p>Specific person that is not doctor or patient becomes [PERSON]</p>
                    </list-item>
                </list>
            </sec>
            <sec>
                <title>Iterative development of redaction rules</title>
                <p>We originally planned to redact date (including day, month, and year) to [DATE], but to maintain time sequence, we changed this to not redact entire date where month and/or day show time progression. Only redact [YEAR] to keep the reference to time elapsed. See example below where specific dates show progression of time.</p>
            </sec>
            <sec>
                <title>Examples of [YEAR] redactions</title>
                <p>jan. 12, [YEAR]. she was bumped by a motorcycle which seened like it had no lights. the deceased had a little drink at that time and her sense of hearing was poor. she was going to cross the street when that happened. she was brought to the hospital but she was unconscious. her breathing was controlled by a pump. the accident happened at around 6 pm jan. 13, [YEAR]. at around 6 am we found out she's dead because the cardiac monitor showed a straight line.</p>
                <p>If progression of time spans over years, [YEAR+n] is used. See example below, where passage refers to following year:</p>
                <p>last november of [YEAR]-the deceased got stroke left side of his body. was hospitalized due to high blood pressure last year. january this year was his last hospitalization that leads to death. jan. 26, [YEAR+1]. experienced fast breathing, that&#x2019;s why he was brought to the hospital (provincial hospital). with oxygen and ngt; got fever and cough; in coma. jan. 31, [YEAR+1]. was tried to revive around 11:00 pm to 12 midnight, but was not able to revive him. around 3:00 am (at dawn), he died.</p>
                <p>Where a passage refers to two different hospitals, hospitals are redacted to [HOSPITAL] and [HOSPITAL2]. Subsequent hospitals in same passage would be [HOSPITAL3, [HOSPITAL4], etc:</p>
                <p>may 16, [YEAR]. he got accident. was brought immediately to [HOSPITAL] then referred directly to [HOSPITAL2], there his wound was stitched. his head was the affected part. was referred to [HOSPITAL3]. was ct scanned in [HOSPITAL4], there was a break on his forehead. was operated after 2 days. after operation he got fever. the deceased also had cough. as per respondent, it was not just the accident alone who led the deceased to death. there was also a complication of his kidney disease. long before (respondent was not able to remember the exact date), the deceased experienced inability to walk but it was not consulted to the doctor for the deceased doesn't want to. they only went to a traditional healer for treatment. the deceased can't walk for about 7 months but then later on he was able to walk again. after he was also hospitalized at [HOSPITAL5], it was known that he have kidney disease.</p>
            </sec>
            <sec>
                <title>Additional clarifications</title>
                <p>Midwife names were redacted to [DOCTOR].</p>
            </sec>
            <sec>
                <title>Dataset validation</title>
                <p>We reviewed progress weekly and discussed emerging challenges as they arose. For example, we determined that the original plan of redacting dates entirely to [DATE] seemed to be obscuring valuable information about the time between symptoms. One week later, we determined that our first attempt at a remedy, to include [DATE+days] was to labor intensive, and would prevent redaction from completing within our budget. Our next remedy worked, and that is how we developed the [YEAR+n] approach described above. When redaction was completed, we reviewed a simple random sample of redacted texts and confirmed that all were devoid of PII.</p>
            </sec>
        </sec>
        <sec>
            <title>Ethics approval</title>
            <p>This study was approved by the Human Subjects Division of the University of Washington (application number 34413). Ethical approval sought for the VAIs is stated in 
                <xref ref-type="bibr" rid="ref-1">1</xref>. All data were collected with informed verbal consent from participants before beginning the interview.</p>
        </sec>
        <sec>
            <title>Data availability</title>
            <p>Data underlying the study are available on OSF: 
                <ext-link ext-link-type="uri" xlink:href="http://doi.org/10.17605/OSF.IO/XUK5Q">http://doi.org/10.17605/OSF.IO/XUK5Q</ext-link>
                <sup>
                    <xref ref-type="bibr" rid="ref-4">4</xref>
                </sup>
            </p>
            <p>Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).</p>
        </sec>
    </body>
    <back>
        <ack>
            <title>Acknowledgements</title>
            <p>The authors would like to thank Scott Lee for the data request that first demonstrated demand for making the data public.</p>
        </ack>
        <ref-list>
            <ref id="ref-1">
                <label>1</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Mikkelsen</surname>
                            <given-names>L</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Phillips</surname>
                            <given-names>DE</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>AbouZahr</surname>
                            <given-names>C</given-names>
                        </name>
						
                        <etal/>
					</person-group>:
                    <article-title>A global assessment of civil registration and vital statistics systems: monitoring data quality and progress.</article-title>
                    <source>
						
                        <italic toggle="yes">Lancet.</italic>
					</source>
                    <year>2015</year>;<volume>386</volume>(<issue>10001</issue>):<fpage>1395</fpage>&#x2013;<lpage>406</lpage>.
                    <pub-id pub-id-type="pmid">25971218</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-2">
                <label>2</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Murray</surname>
                            <given-names>CJ</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Lopez</surname>
                            <given-names>AD</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Black</surname>
                            <given-names>R</given-names>
                        </name>
						
                        <etal/>
					</person-group>:
                    <article-title>Population Health Metrics Research Consortium gold standard verbal autopsy validation study: design, implementation, and development of analysis datasets.</article-title>
                    <source>
						
                        <italic toggle="yes">Popul Health Metr.</italic>
					</source>BioMed Central;<year>2011</year>;<volume>9</volume>:<fpage>27</fpage>.
                    <pub-id pub-id-type="pmid">21816095</pub-id>
                    <pub-id pub-id-type="doi">10.1186/1478-7954-9-27</pub-id>
                    <pub-id pub-id-type="pmcid">3160920</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-3">
                <label>3</label>
                <mixed-citation publication-type="journal">
                    <article-title>Population health metrics research consortium gold standard verbal autopsy data 2005&#x2013;2011</article-title>. Population Health Metrics Research Consortium (PHMRC);<year>2013</year>.
                    <ext-link ext-link-type="uri" xlink:href="http://ghdx.healthdata.org/record/population-health-metrics-research-consortium-gold-standard-verbal-autopsy-data-2005-2011">Reference Source</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-4">
                <label>4</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Flaxman</surname>
                            <given-names>AD</given-names>
                        </name>
					</person-group>:
                    <article-title>&#x201c;A de-Identified Database of 11,979 Verbal Autopsy Open-Ended Responses.&#x201d;</article-title>
                    <source>
                        <italic toggle="yes">Open Science Framework.</italic>
                    </source>
                    <year>2018</year>.
                    <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.17605/OSF.IO/XUK5Q">Data Source</ext-link>
                </mixed-citation>
            </ref>
        </ref-list>
    </back>
    <sub-article article-type="reviewer-report" id="report26471">
        <front-stub>
            <article-id pub-id-type="doi">10.21956/gatesopenres.13879.r26471</article-id>
            <title-group>
                <article-title>Reviewer response for version 1</article-title>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author">
                    <name>
                        <surname>Hart</surname>
                        <given-names>John</given-names>
                    </name>
                    <xref ref-type="aff" rid="r26471a1">1</xref>
                    <xref ref-type="aff" rid="r26471a2">2</xref>
                    <role>Referee</role>
                    <uri content-type="orcid">https://orcid.org/0000-0002-9637-7974</uri>
                </contrib>
                <aff id="r26471a1">
                    <label>1</label>Melbourne School of Population and Global Health, The University of Melbourne, Melbourne, Vic, Australia</aff>
                <aff id="r26471a2">
                    <label>2</label>Department of Clinical Research, London School of Hygiene and Tropical Medicine, London, UK</aff>
            </contrib-group>
            <author-notes>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>14</day>
                <month>6</month>
                <year>2018</year>
            </pub-date>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2018 Hart J</copyright-statement>
                <copyright-year>2018</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <related-article ext-link-type="doi" id="relatedArticleReport26471" related-article-type="peer-reviewed-article" xlink:href="10.12688/gatesopenres.12812.1"/>
            <custom-meta-group>
                <custom-meta>
                    <meta-name>recommendation</meta-name>
                    <meta-value>approve</meta-value>
                </custom-meta>
            </custom-meta-group>
        </front-stub>
        <body>
            <p>The authors present the release of narrative free text from verbal autopsy (VA) interview data collected as part of the PHMRC gold standard VA validation study. The methods describe the rules and processes for redacting personally identifiable information.</p>
            <p> The release of the data is welcome as including the open narrative in automated VA analysis programs can significantly alter output diagnoses. Better understanding of the pros and cons of using free text, and the appropriate weighting for different sources of free text, is likely to be of further interest to researchers as their methods and technology develop. To illustrate, simple text mining for key words such as "malaria" could correctly provide positive evidence towards a diagnosis if the free text read "She suffered from malaria..." but incorrectly if the free text read "Her malaria tests at the health centre were negative".</p>
            <p> </p>
            <p> Research may use the narrative free text alone but its use in VA analysis algorithms is likely to complement answers to the structured interview component of VA. This dataset includes the&#x00a0;gold standard diagnosis and narrative free text&#x00a0;but no link to the structured answers. The authors may wish to comment on this.</p>
            <p> </p>
            <p> The methods are well described, with appropriate&#x00a0;examples,&#x00a0;and the released dataset clear to understand whilst minimising risk of individual identification.</p>
            <p> </p>
            <p> Typos</p>
            <p> Page 3: "Each VAI in the database is matched with a &#x201c;gold standard&#x201d; diagnoses of underlying causes of death" rather "Each VAI in the database is matched with a &#x201c;gold standard&#x201d; diagnosis of underlying cause of death".</p>
            <p>Are sufficient details of methods and materials provided to allow replication by others?</p>
            <p>Yes</p>
            <p>Is the rationale for creating the dataset(s) clearly described?</p>
            <p>Yes</p>
            <p>Are the datasets clearly presented in a useable and accessible format?</p>
            <p>Yes</p>
            <p>Are the protocols appropriate and is the work technically sound?</p>
            <p>Yes</p>
            <p>Reviewer Expertise:</p>
            <p>NA</p>
            <p>I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.</p>
        </body>
    </sub-article>
    <sub-article article-type="reviewer-report" id="report26486">
        <front-stub>
            <article-id pub-id-type="doi">10.21956/gatesopenres.13879.r26486</article-id>
            <title-group>
                <article-title>Reviewer response for version 1</article-title>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author">
                    <name>
                        <surname>Garenne</surname>
                        <given-names>Michel</given-names>
                    </name>
                    <xref ref-type="aff" rid="r26486a1">1</xref>
                    <role>Referee</role>
                    <uri content-type="orcid">https://orcid.org/0000-0001-6073-7803</uri>
                </contrib>
                <aff id="r26486a1">
                    <label>1</label>MRC/Wits Rural Public Health&#x00a0;and&#x00a0;Health Transitions Research Unit, School of Public Health, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa</aff>
            </contrib-group>
            <author-notes>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>7</day>
                <month>6</month>
                <year>2018</year>
            </pub-date>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2018 Garenne M</copyright-statement>
                <copyright-year>2018</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <related-article ext-link-type="doi" id="relatedArticleReport26486" related-article-type="peer-reviewed-article" xlink:href="10.12688/gatesopenres.12812.1"/>
            <custom-meta-group>
                <custom-meta>
                    <meta-name>recommendation</meta-name>
                    <meta-value>approve</meta-value>
                </custom-meta>
            </custom-meta-group>
        </front-stub>
        <body>
            <p>This note presents a new database of Verbal Autopsy (VA) narratives gathered by the Population Health Metrics project. The database is placed in open access and is easily accessible; it ensures the confidentiality of the persons who answered the questionnaire.</p>
            <p> </p>
            <p> This database has a rich potential for further research. Firstly, it can be used for analyzing what caught people&#x2019;s eye: signs, symptoms and circumstances that people tend report in specific circumstances; for analyzing what was ignored or remained unreported although most likely present according to the medical diagnosis; and for determining who knows about the precise cause of death learned from the medical authorities. This information could be used for better understanding VA narratives. Secondly, it could be used for further refining the VA diagnoses, although this was already done in parts by detecting key words in earlier studies. Thirdly, the database can be used for a full scale textual analysis, including style of reporting, keywords, underlying emotions, selectivity, etc. Lastly, relating the full scale textual analysis with the medical diagnosis and with the answers to the VA structured questionnaire could be most useful for further research.</p>
            <p> </p>
            <p> This database is therefore most welcome, and likely to become a source of numerous research exercises.</p>
            <p> &#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;</p>
            <p> 
                <underline>Typos:</underline> 
                <list list-type="order">
                    <list-item>
                        <p>Page 4: &#x201c;
                            <italic>she was bumped by a motorcycle which seened like it had no lights&#x201d;.&#x00a0;</italic>Rather &#x201c;seemed&#x201d;</p>
                    </list-item>
                    <list-item>
                        <p>In the data file, the codebook for Study Site is duplicated.</p>
                    </list-item>
                    <list-item>
                        <p>There are many typos in the narratives (example: diabtes for diabetes, etc.). Could this be cleaned? This would help for searching key words and performing textual analysis.&#x00a0;</p>
                    </list-item>
                </list>
            </p>
            <p>Are sufficient details of methods and materials provided to allow replication by others?</p>
            <p>Yes</p>
            <p>Is the rationale for creating the dataset(s) clearly described?</p>
            <p>Yes</p>
            <p>Are the datasets clearly presented in a useable and accessible format?</p>
            <p>Yes</p>
            <p>Are the protocols appropriate and is the work technically sound?</p>
            <p>Yes</p>
            <p>Reviewer Expertise:</p>
            <p>NA</p>
            <p>I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.</p>
        </body>
    </sub-article>
</article>
