The perils of premature evaluation: reassessing the application of Benford’s Law to the USA’s COVID-19 data

Ravi Dutta-Powell

doi:https://doi.org/10.59139/stattrans-2025-024

The perils of premature evaluation: reassessing the application of Benford’s Law to the USA’s COVID-19 data

Ravi Dutta-Powell Independent researcher. L12 309 Kent St Sydney NSW 2000, Australia ORCID:https://orcid.org/0000-0003-4197-3477 Statistics in Transition new series, vol. 26, 2025, 3, pages: 1-13 Published online: 5 September 2025 https://doi.org/10.59139/stattrans-2025-024 Citation: Dutta-Powell R., 2025. The perils of premature evaluation: reassessing the application of Benford’s Law to the USA’s COVID-19 data. Statistics in Transition new series, 26(3), pp. 1-13 https://doi.org/10.59139/stattrans-2025-025

11 Views 8 Downloads

ARTICLE

(English) PDF

ABSTRACT

This paper reviews earlier applications of Benford’s Law to the COVID-19 data in the United States that claimed these data’s non-conformity with Benford's Law, and uses later and more granular data to demonstrate that this was likely due to the earlier data being unsuitable for such applications. It also demonstrates that the same dataset, analyzed in different ways, can show vastly different levels of conformity with Benford’s Law. Specifically, most US states show high degrees of conformity for the COVID-19 cases and cumulative deaths when the Robust Order of Magnitude (ROM) is over 3 and data at the county level is used to analyze state outcomes. Conversely, when the county data is aggregated to the state level and analyzed (i.e. case totals for all counties are summed to create a single state figure for each day of the pandemic), every state shows non-conformity. Only new deaths showed the reverse pattern - this is likely because new deaths at the county level do not span sufficient orders of magnitude, and aggregation to the state level overcomes this. This suggests that some instances of non-conformity with Benford’s Law in the literature may be caused by its applications to inappropriate datasets or methodological issues.

KEYWORDS

Benford’s Law, COVID-19 data

REFERENCES

Ausloos, M., Ficcadenti, V., Dhesi, G. and Shakeel, M., (2021). Benford’s laws tests on S&P500 daily closing values and the corresponding daily log-returns both point to huge non-conformity. Physica A: Statistical Mechanics and Its Applications, 574, 125969. https://doi.org/10.1016/j.physa.2021.125969.

Balashov, V. S., Yan, Y. and Zhu, X., (2021). Using the Newcomb–Benford law to study the association between a country’s COVID-19 reporting accuracy and its development. Scientific Reports, 11(1), 22914. https://doi.org/10.1038/s41598-021-02367-z.

Benford, F. R., (1938). The Law of Anomalous Numbers. Proceedings of the American Philosophical Society, 78(4), pp. 551–572.

Campanelli, L., (2023). Breaking Benford’s law: a statistical analysis of COVID-19 data using the Euclidean distance statistic. Statistics in Transition new series, 24(2), pp. 201–215. https://doi.org/10.59170/stattrans-2023-028.

Campolieti, M., (2022). COVID-19 deaths in the USA: Benford’s law and underreporting. Journal of Public Health, 44(2), e268–e271. https://doi.org/10.1093/pubmed/fdab161.

Cerqueti, R., Lupi, C., (2023). Severe testing of Benford’s law. Test, 32(2), pp. 677–694. https://doi.org/10.1007/.

Cerqueti, R., Provenzano, D., (2023). Benford’s Law for economic data reliability: The case of tourism flows in Sicily. Chaos, Solitons & Fractals, 173, 113635. https://doi.org/10.1016/j.chaos.2023.113635.

Drake, P. D., Nigrini, M. J., (2000). Computer assisted analytical procedures using Benford’s Law. Journal of Accounting Education, 18(2), pp. 127–146. https://doi.org/doi:10.1016/s0748-5751(00)00008-7.

Eutsler, J., Kathleen Harris, M., Tyler Williams, L. and Cornejo, O. E., (2023). Accounting for partisanship and politicization: Employing Benford’s Law to examine misreporting of COVID-19 infection cases and deaths in the United States. Accounting, Organizations and Society, 108, 101455. https://doi.org/10.1016/j.aos.2023.101455.

Farhadi, N., (2021). Can we rely on COVID-19 data? An assessment of data from over 200 countries worldwide. Science Progress, 104(2), 00368504211021232. https://doi.org/10.1177/00368504211021232.

Farhadi, N., Lahooti, H., (2021). Are COVID-19 Data Reliable? A Quantitative Analysis of Pandemic Data from 182 Countries. COVID, 1(1), Article 1. https://doi.org/10.3390/covid1010013.

Farhadi, N., Lahooti, H., (2022a). Forensic Analysis of COVID-19 Data from 198nCountries Two Years after the Pandemic Outbreak. COVID, 2(4), Article 4. https://doi.org/10.3390/covid2040034.

Farhadi, N., Lahooti, H., (2022b). Reply to Morillas-Jurado et al. Benford Law to Monitor COVID-19 Registration Data. Comment on “Farhadi, N.; Lahooti, H. Forensic Analysis of COVID-19 Data from 198 Countries Two Years after the Pandemic Outbreak. COVID 2022, 2, pp. 472–484”. COVID, 2(7), Article 7. https://doi.org/10.3390/covid2070070.

Fewster, R. M., (2009). A Simple Explanation of Benford’s Law. The American Statistician, 63(1), pp. 26–32. https://doi.org/10.1198/tast.2009.0005.

Goodman, W., (2016). The promises and pitfalls of Benford’s law. Significance, 13(3), pp. 38–41. https://doi.org/10.1111/j.1740-9713.2016.00919.x.

Goodman, W., (2023). Applying and Testing Benford’s Law Are Not the Same. Spanish Journal of Statistics, 5(1), pp. 43–53. https://doi.org/10.37830/SJS.2023.1.03.

Idrovo, A. J., Fernández-Nino, J. A., Bojórquez-Chapela, I. and Moreno-Montoya, J., (2011). Performance of public health surveillance systems during the influenza A(H1N1) pandemic in the Americas: Testing a new method based on Benford’s Law. Epidemiology and Infection, 139(12), pp. 1827–1834. https://doi.org/10.1017/S095026881100015X.

Isea, R., (2020). How Valid are the Reported Cases of People Infected with Covid-19 in the World? International Journal of Coronaviruses, 1, pp. 53–56. https://doi.org/10.14302/issn.2692-1537.ijcv-20-3376.

Kilani, A., (2021). Authoritarian regimes’ propensity to manipulate Covid-19 data: A statistical analysis using Benford’s Law. Commonwealth & Comparative Politics, 59(3), pp. 319–333. https://doi.org/10.1080/14662043.2021.1916207.

Koch, C., Okamura, K., (2020). Benford’s Law and COVID-19 reporting. Economics Letters, 196, 109573. https://doi.org/10.1016/j.econlet.2020.109573.

Kolias, P., (2022). Applying Benford’s law to COVID-19 data: The case of the European Union. Journal of Public Health, 44(2), e221–e226. https://doi.org/10.1093/pubmed/fdac005.

Kossovsky, A. E., (2021). On the Mistaken Use of the Chi-Square Test in Benford’s Law. Stats, 4(2), Article 2. https://doi.org/10.3390/stats4020027.

McHugh, M. L. (2013). The Chi-square test of independence. Biochemia Medica, pp. 143–149. https://doi.org/10.11613/BM.2013.018.

Neumayer, E., Plümper, T. (2022). Does ‘Data fudging’ explain the autocratic advantage? Evidence from the gap between Official Covid-19 mortality and excess mortality. SSM – Population Health, 19, 101247. https://doi.org/10.1016/j.ssmph.2022.101247.

Nigrini, M. J., (2012). Benford’s Law: Applications for forensic accounting, auditing, and fraud detection (Vol. 586). John Wiley & Sons.

Rocha Filho, T. M., Mendes, J. F. F., Lucio, M. L. and Moret, M. A., (2023). COVID-19 data, mitigation policies and Newcomb–Benford law. Chaos, Solitons & Fractals, 174. http://arxiv.org/abs/2208.11226.

Sambridge, M., Jackson, A., (2020). National COVID numbers—Benford’s law looks for errors. Nature, 581(7809), pp. 384–385.

World Health Organisation, (2024). COVID-19 deaths reported (2024 global). WHO COVID-19 Dashboard. https://data.who.int/dashboards/covid19/circulation.