How modelling can help steer the course set by the World Health Organization 2021-2030 roadmap on neglected tropical diseases

The World Health Organization recently launched its 2021-2030 roadmap, Ending the Neglect to Attain the Sustainable Development Goals , an updated call to arms to end the suffering caused by neglected tropical diseases. Modelling and quantitative analyses played a significant role in forming these latest goals. In this collection, we discuss the insights, the resulting recommendations and identified challenges of public health modelling for 13 of the target diseases: Chagas disease, dengue, gambiense human African trypanosomiasis (gHAT), lymphatic filariasis (LF), onchocerciasis, rabies, scabies, schistosomiasis, soil-transmitted helminthiases (STH), Taenia solium taeniasis/ cysticercosis, trachoma, visceral leishmaniasis (VL) and yaws. This piece reflects the three cross-cutting themes identified across the collection, regarding the contribution that modelling can make to timelines, programme design, drug development and clinical trials.

Neglected tropical diseases continue to affect over one billion people 16 as the result of the considerable inequalities in global healthcare systems that fail to support those most in need 17 . The burden of NTDs falls largely on the poorest communities, resulting in an unrelenting cycle of poverty that is driven by negative social, health and economic impacts of infection on individuals and families, augmenting existing social divides. For infections with a substantial zoonotic component, morbidity and mortality among livestock also affect people's livelihood with economic impacts that transcend medical implications. Notable progress to reduce the burden of NTDs has been made as a result of the commitments made in 2012 through the WHO 2020 NTD Roadmap 18 and the London Declaration on NTDs 19 . As a result, 500 million people no longer require interventions against several NTDs and 40 countries, territories and areas have eliminated at least one disease 1 . These wins are the outcome of concerted and consolidated efforts from endemic communities and invaluable volunteers, governments, donor agencies and the pharmaceutical industry. Despite such early gains, reaching the endgame presents some of the greatest challenges -namely sustaining those early gains whilst identifying and averting small numbers of sparsely distributed cases. The 2030 roadmap is shaped around three pillars that aim to support global efforts to maintain the gains, address the challenges and ultimately combat NTDs 1 : 1. Accelerating programmatic action. 2. Intensifying cross-cutting approaches and 3. Shifting operating models and culture to facilitate in country ownership.
The use of mathematical and statistical modelling in NTD research and policy has until recently, and with a few exceptions (e.g., onchocerciasis 20 ), lagged behind other groups of infectious diseases that receive more focus and funding (often, diseases that impact wealthier individuals and nations, or those perceived to potentially impact these). However, this is changing with the advent of groups like the NTD Modelling Consortium 21 , who have developed the Policy-Relevant Items for Reporting Models in Epidemiology of Neglected Tropical Diseases (PRIME-NTD) principles, as a guide to communicate the quality and relevance of modelling to stakeholders 20 . This has added clout to the call for modelling in the policy arena as well as setting a high bar of best practice for the wider modelling community. Having now gained significant traction, the use of modelling in NTD policy has contributed to new intervention tools 22 , vector control strategies 23-26 , shaped policy responding to COVID-19-related programme disruptions [27][28][29][30][31][32][33][34][35] and has aided in the development of WHO guidelines 36,37 . For this positive relationship to continue, it is imperative to invest in a mutual understanding through ongoing conversation between policy-makers and modellers, to determine what kind of questions are the "right" questions, how to interpret uncertainty and what the models can and cannot be used for.
This piece introduces a collection of papers borne of a meeting in Geneva, in April 2019 attended, among others, by the NTD Modelling Consortium and convened by the WHO: Achieving NTD control, Elimination and Eradication Targets Post-2020; Modelling Perspectives & Priorities 2 . As new management targets and strategies took shape, the meeting provided policy makers and modellers the space to ask and answer specific questions regarding the proposed 2030 goals and the intended strategies to achieve them. Although the roadmap covers a range of diseases with diverse epidemiologies and differing management recommendations, the priority questions identified by modelers and stakeholders during the 2019 meeting and echoed by the authors of the technical commentaries shared three similar themes that should be considered in NTD modelling moving forward: timelines, programme design, and clinical study design.

Timelines
Goals are only worth setting in the context of time. It is therefore not surprising that many of the technical commentaries in this collection identified timelines as a priority issue. The public health and economic benefits of reaching goals are innumerable but can only be achieved by the target year through appropriate mobilisation of diverse resources. Modelling in the forms of past inference and forward projections can align many moving parts (for example epidemiological, demographic, and social considerations) to inform our understanding of the reasons why programmes succeed or fail 38,39 . Forecasts have  played a crucial role in understanding whether the 2020 40 and  associated collection 41,42 , 2025 43 and 2030 3-15,34 goals can be reached under current strategies with the caveat that long-term predictions naturally become more uncertain.
In some instances, whether a goal can or will be met on time is relatively easy to ascertain -for example it is a resounding no for leprosy and rabies, which are hindered by passive case control, long quiescent incubation periods, and inadequate investment in interventions 15,44 . Alternatively, the goals for schistosomiasis 11 , STH 8 , and onchocerciasis 13 seem achievable in some or most settings, depending on localised parameters like baseline prevalence, and already experienced duration of and adherence to mass drug administration (MDA) programmes. In the case of T. solium, a lack of internationally agreed goals for elimination or control curtails the ability to effectively model timelines; for example, the 2021-2030 NTD roadmap proposes the overall milestone of achieving "intensified control in hyperendemic areas", without agreeing on technical definitions for T. solium endemicity levels, or defining measurable criteria for attaining "intensified" control 14 .

Programme design
The diseases considered by the London Declaration and WHO roadmaps are at differing stages in their trajectories. Whilst some are on the cusp of achieving their goals, others face political and epidemiological barriers to progress. Both scenarios raise several priority questions regarding programme design, where 'programme' can mean intervention or surveillance. In addition to determining success or failure within the defined intervention time frames, modelling has provided insights into key factors of operational design like the treatment coverage necessary to reach goals in a given setting. Where it may not be possible, models can be used to test the efficacy of separate and combined chemotherapeutic 37 and non-pharmaceutical interventions 23,45,46 , including combined interventions that target multi-host systems for zoonotic NTDs 14 . Additionally, deciding the optimal timing 47 or frequency 48,49 of treatment, and knowing who to treat 50,51 are essential to the success of all interventions. Of course, the intervention strategies most likely to lead to achievement of the goals may not be sustainable in terms of cost to individuals, governments, or donors. By partnering highly detailed transmission models with cost-effectiveness analysis, modelling can also contribute to tailored insights regarding the affordability and benefits versus costs of interventions 52-62 . Models can also be used to explore integration between NTD programmes, or to understand the potential cross-utility of existing NTD programmes on other helminth species, such as exploring the additional benefit of national schistosomiasis control programmes using praziquantel on T. solium prevalence in co-endemic areas 14 . Understanding this cross-utility is vital to intensifying cross-cutting approachesone of the three core pillars of the roadmap, that differentiates the framework from its predecessor.
These are all very practical features of intervention programmes that can in principle be planned for, but underlying features of target populations and human nature can undermine this. Survey data in recent years have made it evident that whilst the aim may be to deliver treatment at a certain geographical and therapeutic coverage, it is not analogous with consumption, as treatment is systematically not ingested by some 63,64 , or is not disseminated to the full intended group, reducing the true coverage. There are a variety of reasons for this 65,66 , but it is likely that similar mechanisms impact participation in surveillance, therefore biasing the estimates of prevalence, particularly when treatment and surveillance are co-occurring (e.g., gHAT 9,67 , rabies 15,68 ). Modelling shows that the impact of this variable effective coverage depends on the pathogen in question and transmission intensity 64,69-71 but it undoubtedly has an impact on reaching public health goals 72,73 , and on the reliability of the projected intervention intensity needed to reach them. It has also been suggested (in the context of VL though applicable beyond), that modelling results -and therefore policy based on them -may be erroneous without better capturing socio-economic and human behaviour risk factors, including feedback loops of behaviour change as a result of perceived risk 74 . This also highlights the need for ongoing surveillance and the use of modelling throughout to provide real-time insight into post-intervention population-level infection dynamics.
Once a strategy has been deemed effective and prevalence targets are attained, it is likely that these interventions either transition, such as going from MDA to identified case management, or they stop all together. Establishing robust surveillance strategies at this point is vital, but obviously not everyone can be regularly sampled and not every incident infection case will be detected. Stochastic events like reinfection and reintroduction are risks that can drive resurgence. Modelling can support the identification of the optimal surveillance strategy and determine which prevalence or intensity indicators need to be monitored to ensure the desired public health goal 75-77 , although challenges remain in developing long-term strategies 78 . Modelling can make useful contributions in developing sustainable, effective interventions and surveillance strategies and should therefore be included in any programmatic design from the start. As embodied by the 2021-2030 NTD roadmap, impactful interventions cannot be achieved by working in silos, but instead require continuous communication between all parties of an interdisciplinary team.

Drug development and clinical study design
Though modelling is increasingly used in public health decision making, the use of modelling to direct clinical trial design and drug development is not so common, and even less so for NTDs. Chemotherapeutic interventions are the cornerstone of large-scale intervention strategies to reach goals like elimination as a public health programme or elimination of transmission. Though treatment options are limited for the likes of STH, LF and trachoma, they are themselves considered sufficiently effective at reducing prevalence and transmission. There is therefore a cautiously sanguine view that consistent application and uptake is enough to reach target public health goals 1,4,5 . However, confidence in the existing treatment for onchocerciasis (targeted for elimination of transmission) is less apparent because the standard ivermectin dose only kills skin dwelling transmission stages with sub-optimal efficacy against adult stages. Modelling suggests this will not be sufficient to reach public health goals 13,79 , pressing the need for novel chemotherapeutics. However, financial returns on investments into NTDs are limited and therefore largely unappealing, particularly because of the heavy reliance by endemic nations on donations from pharmaceutical producers. Increased use of mathematical modelling could reduce the financial waste associated with the drug-development-to-distribution-pipeline 79 . If we consider this pipeline in three parts; pre-clinical, clinical trial and distribution, it is clear that modelling can provide valuable insight at each stage. Onchocerciasis and LF have recently benefited from pharmacokinetic-pharmacodynamics modelling, translating pre-clinical non-human experimental results into quantitative insights relevant to human treatment 80 . Clinical trial simulations are designed to include all aspects of a clinical trial protocol including (but not limited to) recruitment criteria, drug properties/effectiveness and follow-up times 81 , providing valuable guidance that translates into more effective, efficient, cost-efficient and robust clinical trials. In addition to providing insight into the optimal distribution of new drugs 82 , rethinking the distribution of existing drugs to achieve public health targets can also be guided by modelling 37,48 .

Challenges
Modelling has certainly addressed many of the key questions asked of modellers at the 2019 meeting 2 . However, crossdisease challenges remain 83 . The most common of these, highlighted by all groups involved in the meeting report 2 and this collection, is undoubtedly a lack of data or poor data quality. This could be because certain parameters simply cannot be measured; because of vast heterogeneity or because they have yet to be collected 84 . A previous collection details the data needs to improve modelling 51,84-94 , across the NTDs, so great detail will not be provided here. However, for example, VL has a highly variable incubation period, unknown duration of asymptomatic infection and estimates for the duration of lasting immunity are ill-defined 6,85,95 , introducing uncertainty into the temporal dynamics underlying any projections. Chagas disease, gHAT and leprosy also suffer from r elatively long, but indeterminate incubation periods 9,12,21 impacting case detection and adding greater uncertainty in epidemiological estimates fitted to by models 85,96 . Asymptomatic or pre-symptomatic infection is common of many NTDs and presents a significant challenge to their management. For example, asymptomatic VL infections cannot be treated, whereas it is possible to treat asymptomatic gHAT but only if it is able to be detected. Identifying their respective proportions in an infected population, particularly in the absence of high surveillance coverage, means accounting for this group using roundabout methods and proxy diagnostics 6,9 .
Many diagnostics are indirect, proxy measures of case detection, often with less than perfect sensitivity or specificity 97,98 , and have a direct effect on perceived prevalence and individual burdens of infection 99,100 . Given that models are only as good as the data to which they are fitted, this has a significant impact on the utility of model results. For example, in the instances of STH and intestinal schistosomiasis (Schistosoma mansoni), WHO targets are given in terms of eggs per gram of faecal matter as detected with the Kato-Katz method, which notoriously suffers from poor sensitivity, particularly for low intensity infections 101 , invariably underestimating prevalence. Where a multi-host system is present for zoonotic NTDs, though it is possible to measure infection through direct observation of parasite stages in the animal host(s) 14 , via necropsy or other methods 102 , it is likely that this approach is inappropriate for monitoring and evaluating the likes of T. solium control programmes, due to the large animal sample sizes required to detect a statistically meaningful impact on transmission, especially in low prevalence settings 14 . Molecular xenomonitoring (testing vectors for the parasite instead of human hosts) for LF and onchocerciasis has shown promise 103 but operational research gaps remain, impacting large-scale utilisation 104 . Reconciling these different streams of imperfect diagnostic data will be key to their utility in modelling and indeed to reaching and sustaining public health goals.
The operational units over which epidemiological data are collected, and projections made are also often over somewhat arbitrary administrative borders that infectious diseases do not adhere to. For rabies, non-spatial models are inadequate for capturing the low-endemicity incidence rates 15 such that more data-intensive modelling approaches are required. In addition to questionable detection success, VL surveillance has operated over geographical units that are too large to evaluate the success of control methods 6 , despite modelling showing that transmission is highly localised over smaller spatial scales (i.e. 85% of inferred transmission distances ≤300m) 105 .
Similarly for onchocerciasis, modelling shows that the rate at which interventions can be scaled down depend strongly on the spatial units of assessment 13,106 . Clustering of T. solium porcine cysticercosis around human taeniasis carriers, particularly evident in South American communities, demonstrates the need for spatially explicit models in certain settings 14,107 , such as the recently developed CystiAgent model for Peru 108 , capable of testing spatially structured interventions. From this it is evident that whilst spatial heterogeneity requires nuanced model structure, the leading challenge here is the paucity of data at the spatial level necessary to parameterise the models for spatially relevant insights. This will become ever more important as all NTDs move towards low-prevalence and spatially-heterogenous incidence patterns.
The assumptions made to overcome these uncertainties often differ across models -which then produce differing results. This is somewhat overcome by the practice of model comparison 109,110 , which highlights important biological and population processes that impact epidemiological trajectories, Understanding where these differences in modelling results come from and what these differences can tell us is critical to the interpretation of modelling results. This waves a clear flag for collaborative opportunities between modellers, field epidemiologists and clinicians, to generate the necessary data to inform model parameters, or provide setting-specific insight, improving projections and the cross-discipline understanding of model results. Indeed, the optimal working relationship is a synergistic pathway, where the model's needs drive data collection, the data shapes further model iterations, and these then inform policy and the outcomes at the programmatic and clinical level 51,83-94 . Improving communication between these groups is critical to achieving the desired public health gains 20 .
The integration of modelling across public health hierarchy is also crucial but is an ongoing challenge. Whilst leading global health bodies like the WHO use modelling results to generate broad guidance at the international level, the truth is that this one-size-fits-all approach is unlikely to sufficiently describe intervention needs in every setting, such that local decision makers may be unsure why interventions -as advised by modelling -have not reached public health targets, when to stop MDA 79 , or why resurgence occurs. This is not necessarily a failure of the modelling process, but of the framework in which modelling results are used and the way in which model results are accessed. One way to overcome this, and indeed many of the above challenges, is to provide in-country/ local decisions makers with modelling tools, making clear what data are needed to provide location-specific insights. Such tools do exist and efforts to extend the availability of these are ongoing where research and public health funding allow; for gHAT across the Democratic Republic of Congo, modelling has been used to identify target health zones, accompanied by an interactive visual tool 111 . A tool for LF is also available 112 , however the authors highlight the advantages and disadvantages associated with such tools. For example, the level of expertise needed to harness the tool and interpret the results will depend on the level of automation 113 -which itself creates further trade-offs between usability and correct model specification, with fewer parameters available for change within an interface, limiting calibration to local settings. There is also an increased risk of incorrect interpretation including poor understanding of uncertainty and where it comes from, which could lead to reduced trust in the results and modelling methods. It is therefore imperative to balance the availability of tools at local scales with the expertise to use the models correctly.

Conclusion
The increased use of mathematical and statistical modelling over the last decade has helped move the field of NTDs into a more quantitative space, providing the link between epidemiological concepts and observed reality. For modelling to continue to fill this role and influence decision-making, ongoing conversations and engagement between all parties will be paramount. These will, in turn, overcome the continuous challenges of data quality and access, and the consequent model assumptions required. As programme and disease management move towards a country-ownership framework under the new roadmap, it will be key that modelling follows suit, overcoming systematic notions of knowledge ownership and challenging associated power dynamics [114][115][116] . In this way, future modelling will work to support this new NTD landscape.

Data availability
No data are associated with this article. Organization in the 2021-2030 roadmap for NTDs. The article provides a good overview of the series which in turn provides welcome visibility into how mathematical models informed the development of the 2030 roadmap.The article makes the case for use of modeling in regards to informing: 1) Programmatic timelines, 2) Program design and 3) Drug development.
With regard to the first 2 items (timelines and program design), the case is clearly made for the potential of modeling to inform these, at least at a global level. What is less clear is how modeling can inform decision making at a national level to answer similar questions. This would require the use of an interactive tool that can be used by NTD program managers and their partners to input their parameters (e.g. disease agent, vector species, baseline prevalence, different intervention details, etc With regards to drug development I am not aware of the need for new drugs for LF or trachoma (the references provided did not collaborate and these are not listed as challenges in the Roadmap).
As a reviewer I was asked to comment on whether the article adequately referenced different views and opinions, to which I respond, only partially. The article is authored by modellers who have a much clearer insight into the pro modelling perspective. There are still many in the NTD community who remain skeptical -due to one of the challenges that is clearly presented in this article, that is the reliability of the data on which the models are built. It would be helpful to have all the assumptions, parameters, and data sources used in the models published online in one easy-to-access place. This would enable more informed discussions to take place, across a wider group of participants, as existing evidence is weighed in making policy decisions. It would also be helpful to have a summary from this of the biggest evidence gaps -to drive research and documentation of programmatic results.

Does the article adequately reference differing views and opinions? Partly
Are all factual statements correct, and are statements and arguments made adequately supported by citations?
A similar point has been raised by reviewer 1. A new final paragraph of the Challenges section has been added. In short, the translation of international-level policy to locallevel implementation does mean that local-level specific details are often overlooked, and general policy recommendations fail to meet all conditions of that setting. Whilst providing tools locally is an obvious path to improve national (or smaller) level control, it comes with a trade-off. In making the models more user-friendly, some of the complexities that allow for fine scale tuning of certain parameters may have to become fixed, such that the insight is less specific. There is also the matter of interpretation and understandings the limitations of the models. As such, local expertise is still necessary even in the hands of decision-makers.
With regards to drug development I am not aware of the need for new drugs for LF or trachoma (the references provided did not collaborate and these are not listed as challenges in the Roadmap).
This misunderstanding was a result of poor sentence structure. We have rectified this by rewriting the opening to the Drug development and clinical study design section.

As a reviewer I was asked to comment on whether the article adequately referenced different views and opinions, to
which I respond, only partially. The article is authored by modellers who have a much clearer insight into the pro modelling perspective. There are still many in the NTD community who remain skeptical -due to one of the challenges that is clearly presented in this article, that is the reliability of the data on which the models are built. It would be helpful to have all the assumptions, parameters, and data sources used in the models published online in one easy-to-access place. This would enable more informed discussions to take place, across a wider group of participants, as existing evidence is weighed in making policy decisions. It would also be helpful to have a summary from this of the biggest evidence gaps -to drive research and documentation of programmatic results.
It is out with the scope of this article to provide a thorough overview of the data needs to improve modelling insights. There is however a PLoS NTD collection that was cited in the main text. To make this clearer, text has been added text to the Challenges section, directing the reader to this in-depth collection. With regards to the assumptions, parameters and data sources; these are provided in every publication that presents modelling results, and the code for each model is public. These are not included here because there are no models specifically presented here. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Angus McLure
National Centre for Epidemiology and Population Health, Australian National University, Acton,

ACT, Australia
This open letter summarises the contributions of (infectious) disease modelling to inform elimination efforts for Neglected Tropical Diseases (NTDs), highlighting key areas of contributions (timelines/forecasts/predictions, Programme/intervention/surveillance design, and drug development) and naming some of the key challenges. To be frank, it appears to me the kind of letter that doesn't fit very well into the peer-review paradigm as the goal is primarily to introduce a collection of papers. However, I have tried to answer the peer-review questions as posed and provide constructive feedback where I can. I should note that my relevant area of knowledge is mostly around LF and infectious disease modelling more broadly, so my review is skewed towards topics most relevant to LF.
Regarding the correctness of all statements, to my knowledge, nearly all the statements are correct. However, I would like to highlight one statement which seemed at odds with the literature. The line comes in paragraph 10: 'To reach goals like elimination as a public health Having said all this, I think the 'challenges' section of the paper covers the major challenges reasonably enough. This section can perhaps be improved by giving more examples of how these challenges translate into things that models cannot or have not yet been able to do; currently the section feels a little too modelling-centric rather than programme-centric. An example of something models have not been able to do (that I am aware of) is around critical prevalence thresholds for LF/oncho elimination. Various models investigating these have found that the critical threshold varies substantially based on setting-specific assumptions. However, knowing that the threshold will be different in different settings doesn't really help you pick a target threshold that's appropriate for your setting. And since there haven't been resources in place to fit models to every setting, we have ended up with the very crude 1/2% prevalence thresholds which will be unnecessarily low in some settings and too high in others (e.g. places with high and highly heterogenous biting rates). This to me is an example not of the failure of modellers, but a limitation of modelling as an approach to help inform specific interventions/programmes. You might argue that the 1%/2% thresholds are targets not necessarily meant to be identified with critical transmission tipping points. However my reading of the non-modelling literature and conversations with non-modellers suggests that many people believe that the 1%/2% target thresholds entail the interruption of transmission and that many believe that the universal application of these thresholds have a rigorous evidence base. Moreover, discussion of resurgence often blame failure to achieve the threshold rather than suggesting that the threshold might be wrong for the setting. Again this perhaps points to a failure of communication rather than a failure of the modelling itself, but as this letter is about how modelling informs practice, this example (or better example along a similar line) may be relevant for inclusion (if they can manage to express it more succinctly than I have!). This is an interesting point and, if we have distilled it correctly, we completely agree. What you describe is perhaps not so much a failure of modelling, or something models cannot do, but is a symptom of the framework in which we apply mathematical modelling to NTDs. Large-scale international-level policy and guidance is the outcome of mathematical models that are indeed fit to data from specific locations, likely using more than one model, however it is unlikely that this insight will adequately describe all settings. So, there is a disparity across the public health hierarchy from international-level modelling use to a local level. The point you raise about nonmodellers then using certain values as wrote is, as you rightfully state, an outcome of communication and understanding. This is a nuanced point connected to local access to, and use of, models. Reviewer two has raised the point that a discussion should be had in this letter around the topic of making models available as tools. These two comments are not mutually exclusive. We have therefore addressed them simultaneously in an new final paragraph of the Challenges section. This has been rectified, thank you.

Competing Interests:
No competing interests were disclosed.