Transforming cases by report date to cases by infection date
To estimate Rt, analysts need to know the number of cases by infection date. However, for most COVID-19 cases, the precise date of infection is unknown. Instead, we know the number of cases by the date that a positive COVID-19 test was reported. For example, if you have a positive COVID-19 test today, you were likely infected with SARS-CoV-2 before today.
Calculating Rt directly from cases by report date would cause two problems. First, the estimates would be delayed, since reported cases are the result of infections that originated in the past. Second, the estimates would fail to account for the uncertainty of when those infections actually occurred. To transform cases by report date to cases by infection date, one must account for various forms of delay between infection and report date (e.g., generation time, incubation period, and reporting delay).
Making these transformations from scratch would be time-consuming, but preexisting software packages make the solution easier. For example, our team used a publicly available R package, EpiNow2, which includes built-in distributions for different types of delay and ready-made functions to make the necessary adjustments to the data the user provides.
Generally speaking, it is not feasible to compute real-time Rt estimates on a personal computer. The complex methods necessary to transform cases by report date to cases by infection date demand too much computing power. This challenge is compounded when Rt estimates are needed for numerous geographic areas.
Our team used cloud computing to access the necessary computing power on demand. Cloud computing is powerful and flexible, but it adds a few hurdles of its own. If confidential data are used to generate Rt estimates, security measures and clearance may be necessary to upload the data to the cloud. There are also fees associated with cloud computing. Finally, more development time is needed to get things up and running.
Incomplete observation and sampling bias
The Centers for Disease Control and Prevention estimates that 1 in every 4.6 SARS-CoV-2 infections was reported in 2020. As testing availability and behavior evolve, the number and type of SARS-CoV-2 infections which are reported may also change over time. This can generate bias, which means that the reported cases may not represent the true population of people with COVID-19. For example, if it becomes easier to get a COVID-19 test, more people may get tested, which could result in an increase in reported cases without an increase in the true number of people with COVID-19.
Unfortunately, no easy solution exists for this problem. Sentinel surveillance and mass testing programs can provide additional insight, but these programs are difficult to implement and bring their own set of challenges. Other teams have adjusted reported case counts by testing volume, but this approach does not account for all the reasons testing behavior may change. Ultimately, the most important response to this challenge is to ensure that all stakeholders making decisions based on Rt estimates are aware of this limitation.
Many observers argue that some decision makers have misinterpreted Rt estimates during the pandemic. Analysts can help mitigate this by presenting Rt estimates in a way that helps users interpret them with caution. Our team found two steps especially crucial. First, we emphasized the uncertainty inherent in Rt estimates by showing a range of potential values (an interval estimate) instead of one value (a point estimate). This is particularly important when generating estimates for areas with few reported cases (e.g., an individual county) where the estimate may be highly uncertain. Figure 1 illustrates this dynamic.
Figure 1: Comparison of Point and Interval Estimates of Rt
Figure 1: This comparison is based on publicly available reported COVID-19 case data from a North Carolina county from April-October 2020. Decision makers presented with the point estimate only might conclude that Rt is fairly stable around 1. However, Rt was beginning to increase at this time as part of the autumn surge in COVID-19.
Second, we presented Rt estimates alongside other important indicators. Rt is a useful metric, but it does not convey the entirety of the pandemic’s trajectory. For example, according to Covid Act Now, on January 29, 2021, California had the third lowest Rt (0.78) among 53 US states and territories. However, on the same date, California had among the highest daily new cases per 100k people and percentage of ICU capacity used. Clearly, despite the low Rt, COVID-19 transmission was still a critical concern in the state at the time.