Gates Open Res Gates Open Research 2572-4754 F1000 Research Limited London, UK 10.12688/gatesopenres.14768.1 Data Note Articles Worldwide Index of Serotype-Specific Pneumococcal Antibody Responses (WISSPAR): A curated database of clinical trial data

[version 1; peer review: 3 approved]

 Perniciaro Stephanie Data Curation Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0002-3146-2182 1 Cooper-Wooton Dominic Data Curation Software 1 Knoll Maria Resources Supervision Writing – Review & Editing https://orcid.org/0000-0001-9041-2671 2 Weinberger Daniel Conceptualization Data Curation Formal Analysis Funding Acquisition Investigation Methodology Project Administration Resources Software Supervision Validation Visualization Writing – Review & Editing https://orcid.org/0000-0003-1178-8086 a 1 Epidemiology of Microbial Diseases, Yale University, New Haven, Connecticut, USA International Vaccine Access Center, Department of International Health, Johns Hopkins University, Baltimore, Maryland, USA daniel.weinberger@yale.edu

Competing interests: DMW has received consulting fees from Pfizer, Merck, Affinivax, Matrivax, and GSK for work unrelated to this manuscript and is principal investigator on grants from Pfizer and Merck to Yale University for work unrelated to this manuscript. SP has received consulting fees from Global Diagnostic Solutions, Inventprise, and Vaxcyte and is principal investigator on a grant from Merck to Yale University. MDK has received consulting fees from Merck for work unrelated to this manuscript and receives funding from Pfizer and Merck contracts with the Johns Hopkins University for work unrelated to this manuscript. DC-W has no competing interests to disclose.

 13 7 2023 2023 7 109 9 6 2023 Copyright: © 2023 Perniciaro S et al. 2023 This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

The Worldwide Index of Serotype Specific Pneumococcal Antibody Responses (WISSPAR; https://wisspar.com), is a centralized, online platform housing data on immunogenicity from clinical trials of pneumococcal vaccines. The data on WISSPAR are primarily curated from outcomes tables from clinical trials and are made available in a searchable format that can be readily used for downstream analyses. The WISSPAR database includes trials covering numerous vaccine products, manufacturers, dosing schedules, age groups, immunocompromised groups, and geographic regions. Customizable data visualization tools are embedded within the site, or the data can be exported for further analyses. Users can also browse summary information about the clinical trials and their results. WISSPAR provides a platform for analysts and policy makers to efficiently gather, compare, and collate clinical trial data about pneumococcal vaccines.

 clinical trials pneumococcal disease pneumococcal conjugate vaccines immunogenicity Streptococcus pneumoniae Gates Foundation INV-007834 This work was supported by the Gates Foundation [INV-007834]. Introduction

Pneumococcal disease, caused by Streptococcus pneumoniae, includes common, noninvasive infections such as otitis media, and severe, life-threatening infections including pneumonia, sepsis and meningitis Pneumococcal disease causes hundreds of thousands of deaths per year worldwide, most of these occurring in children, older adults, and people with immunocompromising conditions. The burden of pneumococcal disease is greatest in low- and middle- income countries, but there are high burdens of disease in infants and older adults in both high- and low-income settings 1, 2 .

Pneumococcal conjugate vaccines (PCVs) have been shown to have high efficacy against disease and mortality 3 . Vaccines targeting different serotypes and used with various dosing schedules have been employed in a wide array of populations globally 4, 5 . With several new pneumococcal vaccines recently licensed and in development, there is a need to compare PCVs and ensure maintenance of vaccine effectiveness 6– 9 . New PCVs are typically licensed based on comparisons of immunogenicity with existing vaccines. Therefore, there is a need to be able to readily compare the immunogenicity of different PCVs across studies. These immunogenicity data are used by regulators and by national immunization technical advisory groups (NITAGs) to determine which vaccine products, dosing schedules, and age groups will be included in immunization policy recommendations for their respective populations.

Immunogenicity data for pneumococcal vaccine trials should be compared and interpreted with caution, and the web interface has prominent statements about the caveats of the data. Data on immunogenicity alone cannot be used to infer differences in effectiveness between vaccines. These data need to be combined with information on the protective concentration of antibodies required to protect against each serotype in different populations for meaningful comparisons 10 . Despite these cautions, pulling immunogenicity data together into a single location will make this necessary process of comparison easier and less labor-intensive.

Review and synthesis of the data to inform these policy decisions can be labor intensive and cumbersome. Streamlining this process and making immunogenicity data available in a format that can be readily searched and extracted has value for policy makers and for researchers interested in vaccine efficacy, effectiveness, impact, and optimization. WISSPAR is a resource that provides an interactive platform to summarize and compare the immunogenicity data from clinical trials of pneumococcal vaccines.

 Methods Data sources

Data currently displayed on WISSPAR are from the Outcome Measures subsection of the Results section on clinicaltrials.gov. Several other datasets and types are available, including adverse events reporting, subject demographics, retention patterns. There are currently data from more than 57 trials available to compare, though these are updated frequently. If users need data from a trial that is not yet included on WISSPAR, users are encouraged to contact the authors with requests.

 Outcome data

The current version of WISSPAR is focused on two measures of serotype-specific immunogenicity: geometric mean concentrations of immunoglobulin G levels measured by enzyme-linked immunosorbent assay (ELISA) or Electrochemiluminescence (ECL) assay and geometric mean titers measured with opsonophagocytic assays (OPA). These two measurements are commonly taken from blood serum of clinical trial participants at specified time intervals following doses of pneumococcal vaccines (e.g. 1 month following a dose received between 11–15 months of age), and provide a largely standardized basis for comparison, though differences between assays can cause variation. These data are extracted from outcomes tables from clinical trials, and can be filtered, visualized, or exported according to the specifications of the user, including selecting trials of certain vaccine products, manufacturers, dosing schedules, geographic regions, immunocompromised populations, or age groups.

 Dataset validation

Data are manually validated for completeness, clarity and formatting from the customized WISSPAR backend. Data from clinical trials are imported exactly as they are recorded in clinicaltrials.gov, so if data are incorrectly entered into clinicaltrials.gov, the accuracy of WISSPAR could be affected.

 Dashboard contents

Public-facing WISSPAR content is organized into Data Dashboards, Analysis Tools, and Resources ( Table 1). The Data Dashboards provide customizable visualizations of the serotype-specific immunogenicity data in the Graphical Overview ( Figure 1), and summary information from the included trials in the Clinical Trials Overview. The analysis tools allow users to efficiently pinpoint trials of interest in the Look up a trial tool, and export custom .csv files for downstream analyses using the Immunogenicity Data Export tool. There are community resources available, including a blog to keep users abreast of updates and demonstrate WISSPAR capabilities example analyses. An example of how WISSPAR can be used to make comparisons of serotype-specific differences in immunogenicity between three vaccine products is published on the WISSPAR blog 11 .

Descriptions of page content for the Worldwide Index of Serotype-Specific Antibody Responses, WISSPAR.

WISSPAR has a suite of tools to summarize, compare, and export immunogenicity data from clinical trials of pneumococcal vaccines.

WISSPAR Website Pages Content Summary
PUBLIC DATA DASHBOARDS
Graphical View An interactive Shiny app that allows users to visualize measurements of serotype-specific immunogenicity and export images. Available variables include:   •  Vaccine   •  Serotypes   •  Age category   •  Schedule   •  Doses received and timing   •  Trial phase   •  Sponsor Users can visualize measurements of IgG Geometric Mean Concentrations (GMCs), Opsonophagocytic Activity assays (OPAs), and GMC and OPA ratios.
Clinical Trials Overview Filterable listing of clinical trials included on WISSPAR, allowing users to search for particular clinical trials or criteria of interest, including:   •  Vaccines   •  Schedule   •  Manufacturers   •  Age Category   •  Ethnicity   •  Continent   •  Immunocompromised Groups   •  Gender
ANALYSIS TOOLS
Look up a trial Quick-search function that allows users to access complete trial information by entering the NCT ID from clinicaltrials.gov
Immunogenicity Data Export Highly customizable data export tool where users can create customized datasets for downstream analyses. Variables include:
Filters Fields
  •  Income group   •  Trial phase   •  Vaccine   •  Age category   •  Immunocompromised group   •  Responsible party   •  Clinical trial data   •  Study eligibility   •  Study location   •  Outcome Measures   •  Outcome Overview
RESOURCES
Blog Link to WISSPAR blog posts, which includes updates from the team, examples of how to use WISSPAR to compare trial data, and more.
Videos Video walk-through of WISSPAR features and functions
Data visualization app on the Worldwide Index of Serotype-Specific Antibody Responses, WISSPAR.

WISSPAR’s data visualization app is based in R Shiny and allows users to generate customizable, downloadable figures showing immunogenicity measurements stratified by serotype, vaccine product, vaccine manufacturer, trial sponsor, and vaccination schedule.

 Data hosting and database structure

The WISSPAR platform consists of a few different services hosted on Digital Ocean ( Figure 2). The backend database is PostgreSQL (V13) with connection pools set up for security and guaranteed availability; there are also automated backups for recovery. The web application is written in Golang (1.16) for the backend connecting to the database and integrating with the clinicaltrials.gov website. On the front end of the web application, the javascript framework is SvelteJS to allow for quick component loading times and better code organization over traditional javascript. We used GitLab for our software version control, when a change is made in the repository on GitLab it is detected by our Digital Ocean repository and automatically deployed. We also have fail safes built in so that if a code build fails to deploy it will revert to the last working version without any downtime to the web application. Interactive plots, with filters and selectors, was created in RShiny and deployed through the shiny.io server.

Organizational Schematic of the Worldwide Index of Serotype-Specific Antibody Responses, WISSPAR.

The data import process from clinicaltrials.gov and the sources of data and the data types used in WISSPAR are described here.

 Use of data

All data on WISSPAR are publicly available or previously published.

 Data availability

Zenodo: weinbergerlab/WISSPAR: v1_1_1, https://doi.org/10.5281/zenodo.7055186 12 .

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

 Acknowledgments

We thank representatives from the CDC, Merck, Pfizer, and the Gates Foundation for critical feedback about the development of the dashboard.

 Wahl B O’Brien KL Greenbaum A : Burden of Streptococcus pneumoniae and Haemophilus influenzae type b disease in children in the era of conjugate vaccines: global, regional, and national estimates for 2000-15. Lancet Glob Health. 2018;6(7):e744–e757. 29903376 10.1016/S2214-109X(18)30247-X 6005122 Global Burden of Disease Results: Institute for Health Metrics and Evaluation.[cited 2023 May 30]. Reference Source Lucero MG Dulalia VE Nillos LT : Pneumococcal conjugate vaccines for preventing vaccine‐type invasive pneumococcal disease and X‐ray defined pneumonia in children less than two years of age. Cochrane Database Syst Rev. 2009;2009(4):CD004977. 19821336 10.1002/14651858.CD004977.pub2 6464899 Vaccination schedule for Pneumococcal disease. [cited 2023 Apr 26]. Reference Source Pneumococcal Conjugate Vaccine: ViewHub. [cited 2023 May 30]. Reference Source Datta A Kapre K Andi-Lolo I : Multi-valent pneumococcal conjugate vaccine for global health: From problem to platform to production. Hum Vaccin Immunother. 2022;18(4):2117949. 36239596 10.1080/21645515.2022.2117949 9721430 Fairman J Agarwal P Barbanel S : Non-clinical immunological comparison of a Next-Generation 24-valent pneumococcal conjugate vaccine (VAX-24) using site-specific carrier protein conjugation to the current standard of care (PCV13 and PPV23). Vaccine. 2021;39(23):3197–3206. 33965258 10.1016/j.vaccine.2021.03.070 Chichili GR Smulders R Santos V : Phase 1/2 study of a novel 24-valent pneumococcal vaccine in healthy adults aged 18 to 64 years and in older adults aged 65 to 85 years. Vaccine. 2022;40(31):4190–4198. 35690500 10.1016/j.vaccine.2022.05.079 Platt H Omole T Cardona J : Safety, tolerability, and immunogenicity of a 21-valent pneumococcal conjugate vaccine, V116, in healthy adults: phase 1/2, randomised, double-blind, active comparator-controlled, multicentre, US-based trial. Lancet Infect Dis. 2023;23(2):233–246. 36116461 10.1016/S1473-3099(22)00526-6 Andrews NJ Waight PA Burbidge P : Serotype-specific effectiveness and correlates of protection for the 13-valent pneumococcal conjugate vaccine: a postlicensure indirect cohort study. Lancet Infect Dis. 2014;14(9):839–46. 25042756 10.1016/S1473-3099(14)70822-9 Perniciaro S : Changes in Immunogenicity Across Three Generations of Pneumococcal Conjugate Vaccines: PCV7-PCV13-PCV20 – WISSPAR. 2023; [cited 2023 May 30]. Reference Source Dominic Cooper-Wootton: weinbergerlab/WISSPAR: v1_1_1 (v1_1_1). Zenodo.[Dataset],2022. http://www.doi.org/10.5281/zenodo.7055186 10.21956/gatesopenres.16094.r34343 Reviewer response for version 1 Avci Fikri Y 1 2 Referee https://orcid.org/0000-0002-1902-7532 School of Medicine Department of Biochemistry, Emory University, Atlanta, Georgia, USA Emory Vaccine Center, Emory University, Atlanta, Georgia, USA

Competing interests: No competing interests were disclosed.

 15 8 2023 Copyright: © 2023 Avci FY 2023 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve

WISSPAR is a much-needed platform that allows quick, comprehensive, and guided access to antibody titers and functional efficacy (OPA) data from worldwide pneumococcal vaccine immunization clinical trials. 

The authors cite a WISSPAR blog for an example of how WISSPAR can be used and list in Table 1 a video walk-through of WISSPAR, which are highly useful to enable its widespread use. It may be beneficial to add here a step-by-step tutorial for the cited example in a simple protocol format (eg Nat Protoc). Also, in the platform, displaying premade graphs for two or three major clinical trials may give novice users a seed to modulate themselves. CDC's IPD epidemiology platform and WISSPAR complement and appeal to similar users. To increase WISSPAR's visibility, CDC, WHO, and similar platforms can cite it. Finally, allowing user interphase may improve the platform and curate the clinical data. Overall, WISSPAR is a well-implemented important resource.

Are sufficient details of methods and materials provided to allow replication by others?

Yes

Is the rationale for creating the dataset(s) clearly described?

Yes

Are the datasets clearly presented in a useable and accessible format?

Yes

Are the protocols appropriate and is the work technically sound?

Yes

Reviewer Expertise:

Immunology, Infectious Diseases, Glycobiology, Vaccine Design and Development

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

 10.21956/gatesopenres.16094.r34280 Reviewer response for version 1 Muñoz-Almagro Carmen 1 2 3 Referee RDI Microbiology Department, Institut de Recerca Sant Joan de Déu, Hospital Sant Joan de Déu, Barcelona, Catalonia, Spain CIBER of Epidemiology and Public Health (CIBERESP), Madrid, Spain Medicine Department, Universitat Internacional de Catalunya, Barcelona, Catalonia, Spain

Competing interests: I have received fees as speaker from GSK, Pfizer and Sanofi-Pasteur.

 8 8 2023 Copyright: © 2023 Muñoz-Almagro C 2023 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve

The Worldwide Index of Serotype Specific Pneumococcal Antibody Responses (WISSPAR) is a helpful and comprehensive online tool including immunogenicity data from clinical trials of pneumococcal vaccines (PCVs) according to numerous vaccine products, manufacturers, dosing schedules, age groups, immunocompromised groups, and geographic regions.

The platform is user-friendly and without a doubt will be a very useful tool for researchers, regulators and technical advisory groups that need to compare the immunogenicity of different PCVs across studies. At present data of WISSPAR come from more than 57 trials but  WISSPAR curators update the database frequently. In addition, if users need data from a trial that is not yet included on WISSPAR, they are encouraged to contact the authors with requests. Measures of serotype-specific immunogenicity include IgG Geometric Mean Concentrations (GMCs), Opsonophagocytic Activity assays (OPAs), and GMC and OPA ratio.

My only minor concern is about data validation: data from clinical trials are imported exactly as they are recorded in clinicaltrials.gov, so if data are incorrectly entered into clinicaltrials.gov, the accuracy of WISSPAR could be affected. My suggestion would be including an alert if they or the scientific community detect any data that might reasonably considered to be erroneous. On the other hand, it could be interesting to provide a brief audit on the methodological quality of data reported in the clinical trials included in the tool, using widely accepted review checklists such as those of CONSORT reporting guidelines.

Are sufficient details of methods and materials provided to allow replication by others?

Yes

Is the rationale for creating the dataset(s) clearly described?

Yes

Are the datasets clearly presented in a useable and accessible format?

Yes

Are the protocols appropriate and is the work technically sound?

Yes

Reviewer Expertise:

Pediatric Infectious Diseases; molecular epidemiology;  invasive pneumococcal disease

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

 10.21956/gatesopenres.16094.r34277 Reviewer response for version 1 Ercoli Giuseppe 1 Referee Centre for Inflammation and Tissue Repair, UCL Respiratory, University College London, London, UK

Competing interests: No competing interests were disclosed.

 2 8 2023 Copyright: © 2023 Ercoli G 2023 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve

The WISSPAR database presented by Perniciaro et al., provides an extremely valuable tool to keep track of clinical trials data on pneumococcal vaccines. The tool allows any user to easily access the clinical trial data and compare different datasets. Having a tool that allows to compare PCVs and assess vaccine effectiveness would greatly help vaccine research. The new resource is especially useful for vaccines in early development stages or discovery phase, providing access and facilitating the comparison to previously published data. The rationale for creating the dataset is clearly described by the authors and both the methodology and data sources are reported in details.

Overall the database presented is certainly suitable for indexing, my only comment would be about future updates. It would be interesting to know if the authors have already planned how future updates of the system will be released. If new datasets will become available, will they be immediately included? What's the planned frequency of the updates? Will the frequency of the system updates be time-dependent or data-dependent? Is there any plan to add any extra function?

Are sufficient details of methods and materials provided to allow replication by others?

Yes

Is the rationale for creating the dataset(s) clearly described?

Yes

Are the datasets clearly presented in a useable and accessible format?

Yes

Are the protocols appropriate and is the work technically sound?

Yes

Reviewer Expertise:

Vaccine development / infectious diseases

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.