Better pandemic preparedness does correlate with lower COVID-19 mortality

Research note

An apparent paradox of the coronavirus disease 2019 (COVID-19) pandemic has been that many countries with the strongest public health systems have reported the highest numbers of COVID-19 cases and deaths. Early in the pandemic it became clear that national-level assessments using preparedness measures such as the World Health Organization (WHO) Joint External Evaluation (JEE) or the Global Health Security (GHS) Index showed poor correlation with countries’ reported COVID-19 mortality, which was much better predicted by simple measures such as the percentage of the population at 65 years and above^1,2. Analyses of US states similarly showed poor correlation with the National Health Security Preparedness Index and the reported rates of COVID-19 mortality in the first 6 months of the pandemic³. A more recent analysis showed that countries with higher scores on the GHS Index and the International Health Regulations (IHR) self-assessments did have fewer reported COVID-19 deaths, but only for the first 8 weeks⁴. A comparison of 12 pandemic preparedness indices with cumulative infection-fatality ratios showed that the age profile of the country, but not preparedness index scores predicted infection-fatality ratios⁵.

Consistent and standardized reporting has been a global challenge. Two main problems confound the use of reported COVID-19 mortality in these analyses – younger ages and more under-reporting are both more common in low-income countries with limited testing. Although age is the strongest determinant of COVID-19 mortality, many countries do not report the ages of COVID-19 deaths, and analyses to date are surprisingly inconsistent in attempts to age-standardize. Because poor countries tend to have both lower preparedness scores and younger populations low preparedness tends to correlate with low COVID-19 mortality without age standardization. Second, those countries with the strongest and most transparent public health surveillance systems are likely to detect and report the most COVID-19 deaths. Under-reporting of COVID-19 mortality by factors of 50- or 100-fold is common among low-income countries with weaker testing and public health surveillance^6–8.

Recently several groups including the Institute for Health Metrics and Evaluation (IHME) have produced estimates of COVID-19 mortality using total excess mortality and modeling approaches to adjust for low testing and under-reporting^6,8,9. In combination with indirect age-standardization it is now possible to directly compare relative COVID-19 mortality across countries after adjusting for a country’s specific age distribution and its under-reporting of COVID-19 deaths. Although the lack of age specific excess mortality estimates precludes the direct comparison of age standardized excess mortality rates, the comparative mortality ratio, a form of indirect age standardization, can be computed by comparing total excess mortality estimates with an expected COVID-19 mortality based on a global age pattern of COVID-19 mortality rate and age specific population from countries.

The comparative mortality ratio is calculated using the following formula:

Where CMR is the comparative mortality ratio; c refers to country; g refers to global; i refers to age, with maximum at A; p is population (person years of exposure); and µ is the mortality rate.

A different picture emerges once countries are ranked using comparative mortality ratios, rather than excess mortality estimates¹⁰. Many high-income countries including the United States, Italy, Germany, Great Britain, Spain, and France had more than 150,000 estimated excess deaths through the first 2 years of the pandemic. Even after adjusting for their large population sizes, their excess mortality rates still rank among the top half of all countries. However, once the older age structure of the population is taken into consideration, the ranking of comparative mortality ratios among these nations falls to the bottom third of all nations. On the other hand, many low- and middle-income countries with lower excess death counts and excess mortality rate exhibit much higher comparative mortality ratios, indicating worse performance during the pandemic, as shown in Figure 1.

Figure 1. Global picture of comparative coronavirus disease 2019 (COVID-19) mortality for countries and subnational regions.

Comparative mortality ratios adjust for different age distributions using indirect age standardization and under-reporting by estimations using total excess mortality.

The correlation between the GHS Index preparedness score and this age- and underreporting-adjusted mortality is shown in Figure 2. The figure and correlation analysis were done using Excel version 2204. Countries with higher preparedness scores have significantly lower relative COVID-19 mortality as estimated by the comparative mortality ratio (r(192) = -.32, p<.001).

Figure 2. Coronavirus disease 2019 (COVID-19) mortality by preparedness score for 194 countries.

Preparedness scoring is by the Global Health Security Index in 2021². COVID-19 mortality is adjusted for age by indirect standardization and for under-reporting by estimations using total excess mortality⁶. The correlation is statistically significant (r(192) = -.32, p<.001).

Demonstrating that better preparedness did correlate with better control of COVID-19 is important as countries and institutions begin to turn from acute efforts to control the pandemic to longer term plans for preventing and controlling the next one. Although it is not surprising that countries with the highest scores in areas such as laboratory testing, disease surveillance, and transparency in reporting would also be the most likely to detect and report COVID-19 deaths, it has been surprisingly difficult to adjust for this bias in measuring COVID-19 mortality. The adjusted measure not only confirms the expected relationship to preparedness but should allow for future comparisons of countries with similar preparedness scores and much different adjusted COVID-19 mortality. This relationship allows for better understanding of the impact of specific public health mandates, health system capacities, vaccines, or therapeutics. Better preparedness, as measured by the GHS Index, the JEEs, and similar indices, before a crisis such as COVID-19 offers countries the best protection against excess pandemic mortality.

Data availability