April 01, 2020
Stats with bad numbers
Modern societies rely on the knowledge and wisdom of experts, but beware when these experts go outside their reservation and dabble in probability and statistics. Doctors and journalists are not trained properly in those disciplines. Some economists are, but not all, and many forget the fundamental rule of data analysis: try to gain a deep understanding of the data before you apply your statistical power tools. The recorded Covid-19 statistics, such as infection rates and the number of reported fatalities, are so out of whack with what we already know that it is folly to draw charts based on those numbers, let alone to try and predict how many days country X is behind country Y. This is statistical illiteracy of the highest order.
We would like to draw readers’ attention to a study looking at real-time forecasts during the 2009 influenza epidemic. Those were all over the place or, as the authors more kindly put it, exhibited substantial heterogeneity. Future studies will show the same about this crisis. This particular study used as a metric the case fatality rate, which is not expressed as a number but as a function, a so-called conditional probability. Fatality rates are statistically very complicated to assess because you have to make estimates about measurement errors, like the number of deaths that were not recorded, and deaths that were recorded but wrongly attributed to the virus. Both categories might be substantial. The gap in the number of funerals in Wuhan and Bergamo, for example, between the recent period and the same period in 2019, suggests that the number of epidemic victims is a multiple of the recorded numbers.
This morning we read in Die Welt an article citing a German virologist who says there is no scientific evidence of using face masks. This is another case of an expert making a lazy and potentially dangerous statistical assessment. If you look at the countries that have managed to achieve some stability, such as China, Japan, Singapore or South Korea, face masks are in wide use unlike in Europe or the US. This is not factoid to be dismissed with the usual throwaway remark that there is no scientific evidence.
Our concluding thought is that it is more often useful to think of risk in terms of insurance pricing rather than in terms of numeric probabilities. If you are dealing with radical uncertainty, one that has no known probability distribution, you should ask yourself what price you would be willing to pay to prevent a catastrophic outcome. When evidence of the virus became widely know in January, the western world could have opted for a number of targeted measures: a rule against large gatherings, which later became the source of contamination in Bergamo and western Germany; an immediate stop of flights from and towards affected areas; production of N95 face masks and protective gear for doctors and nurses. The costs would have been very small compared to the cost of the measures that were needed subsequently. If you see risk as a number, and conclude that it is very small, you become complacent. If instead you think of it in terms of the price of insurance against a non-calculable catastrophic risk, you might be the very opposite.