Package {tbl.now}

Type:	Package
Title:	Tidy Data and Workflow Layer for Epidemic Nowcasting
Version:	1.0.0
Description:	Defines tidy data structures and package-agnostic workflows for epidemiological nowcasting. The 'tbl_now' class records event, report, and revision dates alongside strata, covariates, censoring, and reporting-delay metadata while remaining compatible with 'dplyr'. Tools support validation, manipulation, diagnostics, visualization, format conversion, retrospective evaluation, and multiple modelling engines. The 'tbl_nowcast' class standardizes probabilistic predictions for plotting, scoring, comparison, and ensembling.
License:	MIT + file LICENSE
Encoding:	UTF-8
LazyData:	true
LazyDataCompression:	xz
Depends:	R (≥ 4.2.0)
Suggests:	almanac, baselinenowcast (≥ 0.2.1), diseasenowcasting (≥ 2.4.0), data.table, epidist (≥ 0.4.1), EpiNow2 (≥ 1.9.0), epinowcast (≥ 0.7.0), knitr, modifiedmk (≥ 1.6.0), NobBS (≥ 1.1.1), patchwork, plotly, rmarkdown, scoringutils (≥ 2.0.0), surveillance (≥ 1.26.1), testthat (≥ 3.0.0), tsibble, withr
Additional_repositories:	https://epinowcast.r-universe.dev, https://davisvaughan.r-universe.dev, https://rodrigozepeda.r-universe.dev
Config/testthat/edition:	3
Config/testthat/parallel:	false
Imports:	cli, generics, grid, dplyr, ggplot2, lifecycle, lubridate, methods, pillar, rlang, S7, scales, stats, tibble, tidyr, tidyselect, utils
URL:	https://rodrigozepeda.github.io/tbl.now/, https://github.com/RodrigoZepeda/tbl.now
BugReports:	https://github.com/RodrigoZepeda/tbl.now/issues
VignetteBuilder:	knitr
Config/Needs/website:	rmarkdown
Config/roxygen2/version:	8.1.0
NeedsCompilation:	no
Packaged:	2026-09-11 10:38:28 UTC; rodzepeda
Author:	Rodrigo Zepeda-Tello [aut, cre], Rami Yaari [aut], Matteo Perini [aut], Teresa Yamana [ctb], Jeffrey Shaman [ctb], Columbia University in the City of New York [cph, fnd]
Maintainer:	Rodrigo Zepeda-Tello <rzepeda17@gmail.com>
Repository:	CRAN
Date/Publication:	2026-09-21 21:10:02 UTC

tbl.now: Tidy Data and Workflow Layer for Epidemic Nowcasting

Description

Defines tidy data structures and package-agnostic workflows for epidemiological nowcasting. The 'tbl_now' class records event, report, and revision dates alongside strata, covariates, censoring, and reporting-delay metadata while remaining compatible with 'dplyr'. Tools support validation, manipulation, diagnostics, visualization, format conversion, retrospective evaluation, and multiple modelling engines. The 'tbl_nowcast' class standardizes probabilistic predictions for plotting, scoring, comparison, and ensembling.

Details

Surveillance data arrives late. A case that happened on Monday may not reach the system until Thursday, so the most recent counts always look lower than they will turn out to be. Nowcasting corrects that: it estimates how many cases have already happened but have not been reported yet.

tbl.now is the tidy scaffolding around that problem. You declare which columns hold the event date, the report date and anything else that matters once, and everything else – describing, diagnosing, plotting, fitting, scoring – follows from that declaration.

The workflow

1. Declare. tbl_now() turns a data.frame into a tbl_now, or as_tbl_now() converts an object from another nowcasting package. The result is still a tibble, so dplyr keeps working.

2. Describe. summary() says what is in the data; autoplot() draws it.

3. Diagnose. diagnose() says what is wrong with it, and points at the statistical tests worth running – diagnose_drift() for delays that are getting longer, diagnose_batches() for backlog releases, diagnostic_plot() for the reporting process as a picture.

4. Reshape. to_count(), complete_zeroes(), aggregate_time_units(), align_weeks() and censor_reports() put the data on the grid a model needs.

5. Fit. run_nowcast() takes the data and an engine() – one interface over epinowcast, baselinenowcast, NobBS, EpiNow2, surveillance and diseasenowcasting. Write your own with nowcast_fit() and nowcast_tidy().

6. Check. score_nowcast() and nowcast_backtest() say whether the nowcast was any good; nowcast_ensemble() combines several.

vignette("tbl.now") walks this end to end, from a raw line list to a scored nowcast, and marks the point where you choose between a modelling package's native fitting API and the common cross-engine one. The package website carries longer articles on the class itself, the modelling packages, batch reporting, ensembles and writing your own backend.

Datasets

Seven surveillance datasets ship with the package for experimenting: denguedat, mpoxdat, flusight, covid_colombia, covid_us, sari_bh and hai_bucaramanga – the last deliberately messy, for the diagnostics.

Author(s)

Maintainer: Rodrigo Zepeda-Tello rzepeda17@gmail.com (ORCID)

Authors:

• Rodrigo Zepeda-Tello rzepeda17@gmail.com (ORCID)

• Rami Yaari (ORCID)

• Matteo Perini (ORCID)

Other contributors:

• Teresa Yamana (ORCID) [contributor]

• Jeffrey Shaman (ORCID) [contributor]

• Columbia University in the City of New York [copyright holder, funder]

See Also

Useful links:

• https://rodrigozepeda.github.io/tbl.now/

• https://github.com/RodrigoZepeda/tbl.now

• Report bugs at https://github.com/RodrigoZepeda/tbl.now/issues

Set, change and remove the attributes of a tbl_now

Description

A tbl_now() remembers which of its columns are the event date, the report date, the strata, and so on. These functions edit that memory after the object has been built – useful when new columns appear part-way through a pipeline, or when you want to nowcast the same data broken down a different way.

There are three verbs, and they differ only in what they do to what is already recorded:

• ⁠add_*()⁠ keeps what is there and adds to it. add_strata(x, age_group) on an object already stratified by gender leaves you stratified by both.

• ⁠change_*()⁠ replaces it. change_strata(x, age_group) on the same object leaves you stratified by age group only.

• ⁠remove_*()⁠ takes it away. remove_all_strata(x) forgets every stratum; remove_strata(x, gender) forgets just that one.

Nothing here touches the data itself. Renaming an attribute does not rename, create or delete a column – it only changes which existing column the object treats as playing that role.

Usage

change_now(x, now = NULL, verbose = TRUE)

update_now(x, verbose = TRUE)

change_event_date(x, event_date)

change_report_date(x, report_date)

change_case_count(x, case_count)

change_is_censored_report(x, is_censored_report)

remove_is_censored_report(x)

add_is_censored_report(x, is_censored_report)

change_strata(x, ..., warn_now = TRUE, warn_non_uniqueness = TRUE)

remove_strata(x, ...)

add_strata(x, ...)

remove_all_strata(x)

change_covariates(x, ..., warn_now = TRUE, warn_non_uniqueness = TRUE)

remove_covariates(x, ...)

add_covariates(x, ...)

remove_all_covariates(x)

replace_temporal_effects(x, t_effects)

remove_temporal_effects(x)

change_is_censored_revision(x, is_censored_revision)

add_is_censored_revision(x, is_censored_revision)

remove_is_censored_revision(x)

add_revision_date(
  x,
  revision_date,
  revision_type = NULL,
  revision_units = "auto",
  revision_levels = NULL
)

change_revision_date(
  x,
  revision_date,
  revision_type = NULL,
  revision_units = "auto",
  revision_levels = NULL
)

remove_revision_date(x)

Arguments

Details

Columns are chosen with tidy-select, so a bare column name works, and so do the helpers dplyr::starts_with(), dplyr::all_of() and dplyr::where(). See dplyr::select() for the full set.

update_now() deserves a note of its own. The now of a nowcast is the day you are standing on, and it does not move when you filter the data – an object filtered down to 1992 still believes now is 2010, which is usually not what you want. update_now() resets it to the latest date actually present:

data(denguedat)
ndata <- tbl_now(denguedat,
  event_date = onset_week, report_date = report_week, verbose = FALSE
)

# `now` is in 2010, because the data runs that far.
get_now(ndata)
#> [1] "2010-12-20"

# Filtering the data does not move it ...
ndata_1992 <- ndata |>
  dplyr::filter(
    onset_week <= as.Date("1992/01/01") & report_week <= as.Date("1992/01/01")
  )
get_now(ndata_1992)
#> [1] "2010-12-20"

# ... but `update_now()` does.
get_now(update_now(ndata_1992))
#> [1] "1991-12-30"

Value

A tbl_now object with the attribute updated. The data are returned unchanged; only what the object records about itself differs.

The revision process, the optional third date

add_revision_date(), change_revision_date() and remove_revision_date() set the third date a surveillance record can carry: after the event happened and after it was reported, somebody decided whether it was real. For influenza that is the laboratory result – and it can come back negative, in which case the case is retracted rather than confirmed.

Attaching one is the only verb on this page that changes more than a name:

• now moves. A revision is an observation, so the as-of moment becomes the latest of the report and revision dates. Revision refuses an object whose now falls before a revision that has already happened.

• Two columns appear. .revision_num is the date on the same numeric anchor as .event_num/.report_num; .revision_delay is the time from report to resolution. Both are protected, like .delay.

• Counting gains a dimension. to_count() groups by the revision date and outcome as well, so a confirmed and a retracted case on the same ⁠(event, report)⁠ pair stay separate rather than being summed together.

• The timeline is checked. ⁠event_date <= report_date <= revision_date⁠; rows that break it are warned about, not silently accepted.

A date on its own cannot say whether the test came back positive or negative, so leaving revision_type out gives every dated row NA and warns.

Two optional pieces travel with the third date. revision_levels is a named dictionary translating the labels in your data into the four values revision_type may hold – c(confirmado = "confirmed", ...) – so the recoding happens once rather than in every script. And add_is_censored_revision() names a logical column marking rows whose revision delay is a bound rather than a measurement, the revision-axis twin of add_is_censored_report(); censor_revision_delays_above() sets it for you.

change_now() is revision-aware in both directions. Moving now forward does nothing to the data; moving it backwards, which is how a backtest asks what was known at an earlier date, returns every revision dated after that moment to "pending" and masks its date. A resolution that has not happened yet is not a resolution.

See Also

tbl_now() for setting these when the object is first built; the getters for reading them back; tbl_now_attributes() to list them all at once; add_temporal_effects() and temporal_effects() for calendar structure; update() for appending new rows rather than editing attributes.

Examples

data(denguedat)
# These verbs only touch what the object records about itself, never the
# rows, so a couple of years stands in for the full twenty-year series.
recent <- denguedat[denguedat$onset_week >= as.Date("2009-01-01"), ]
ndata <- tbl_now(recent,
  event_date = onset_week,
  report_date = report_week,
  strata = gender,
  verbose = FALSE
)

## ---- Strata: add, change, remove ------------------------------------

ndata$age_group <- sample(c("<18", "18-60", "60+"), nrow(ndata), replace = TRUE)

## `add_strata()` keeps gender and adds age group.
get_strata(add_strata(ndata, age_group))

## `change_strata()` replaces gender with age group.
get_strata(change_strata(ndata, age_group))

## `remove_strata()` drops one; `remove_all_strata()` drops the lot.
get_strata(remove_strata(ndata, gender))
get_strata(remove_all_strata(add_strata(ndata, age_group)))

## ---- Covariates behave the same way ---------------------------------

# Covariates influence the nowcast but are not of interest in themselves.
# They should be values known at the fitted object's `now`; do not let
# future realized covariates leak into a backtest snapshot.
ndata$temperature <- rnorm(nrow(ndata), 25, 4)
ndata$humidity <- rbeta(nrow(ndata), 0.6, 0.4)
ndata <- ndata |> add_covariates(temperature, humidity)
get_covariates(ndata)

ndata |>
  remove_covariates(humidity) |>
  get_covariates()

## ---- Pointing an attribute at a different column ---------------------

## Suppose onset was recorded a day late and you correct it. `change_event_date()`
# tells the object to use the corrected column instead.
ndata$corrected_onset <- ndata$onset_week - lubridate::days(1)
ndata <- ndata |> change_event_date(corrected_onset)
get_event_date(ndata)

## ---- The censoring indicators ----------------------------------------

## TRUE means the report date is only an upper bound (e.g. a backlog dump).
ndata$is_censored_report <- FALSE
ndata <- ndata |> add_is_censored_report(is_censored_report)
get_is_censored_report(ndata)
ndata <- remove_is_censored_report(ndata)

## ---- `now` -----------------------------------------------------------

# Set it by hand ...
get_now(change_now(ndata, now = as.Date("2011-01-01")))

# ... or snap it back to the latest date actually observed.
get_now(update_now(ndata))

## ---- Count data: which column holds the counts ------------------------

counts <- to_count(ndata, to = "count-incidence")
counts |>
  dplyr::mutate(inflated = round(1.15 * n)) |>
  change_case_count(inflated) |>
  get_case_count()

## ---- The revision process, the optional third date -----------------

data(covid_us)
covid <- covid_us |>
  dplyr::filter(onset_dt >= as.Date("2020-12-01")) |>
  tbl_now(
    event_date = onset_dt, report_date = pos_spec_dt,
    case_count = n, data_type = "count-incidence",
    verbose = FALSE, warn_non_uniqueness = FALSE
  )

## Onset -> positive specimen -> registration at CDC. A date alone cannot say
# how the case resolved, so this warns until an outcome column is supplied.
covid <- suppressWarnings(add_revision_date(covid, cdc_report_dt))
get_revision_date(covid)

## CDC's own labels are not this package's four, which is what
# `revision_levels` translates.
covid <- change_revision_date(covid, cdc_report_dt,
  revision_type = current_status,
  revision_levels = c(
    "Laboratory-confirmed case" = "confirmed", "Probable Case" = "pending"
  )
)
table(covid[[get_revision_type(covid)]])
get_revision_levels(covid)

## A revision delay you refuse to believe is a bound, not a measurement.
covid <- censor_revision_delays_above(covid, 45, verbose = FALSE)
get_is_censored_revision(covid)

## Dropping the third date leaves an ordinary two-date object.
has_revision(remove_revision_date(covid))

## ---- Temporal effects --------------------------------------------------

# Recorded lazily: `replace_*` swaps the specification, `remove_*` forgets it.
ndata <- ndata |>
  add_temporal_effects(
    t_effects = temporal_effects(week_of_year = TRUE, month_of_year = TRUE)
  )
get_temporal_effects(ndata)

ndata |>
  replace_temporal_effects(t_effects = temporal_effects(seasons = 52)) |>
  get_temporal_effects()

ndata |>
  remove_temporal_effects() |>
  get_temporal_effects()

Attach calendar effects to a tbl_now, and turn them into columns

Description

These are the second and third steps of using calendar structure in a nowcast. First you write down which patterns you want with temporal_effects(); then:

• add_temporal_effects() records that request on the object. Nothing is computed and no columns appear – the specification is stored lazily, so it survives filtering and joining without going stale.

• compute_temporal_effects() materialises it, building one column per effect from the object's dates.

The split matters because the columns depend on the data. If you computed them first and then filtered, or changed the event-date column, the columns would silently describe the wrong rows. Recording the request and computing it at the end avoids that.

Call add_temporal_effects() more than once to accumulate several specifications on the same object; compute_temporal_effects() builds all of them.

Usage

add_temporal_effects(x, t_effects = NULL, overwrite = FALSE, ...)

## S3 method for class 'data.frame'
add_temporal_effects(
  x,
  t_effects = NULL,
  overwrite = FALSE,
  ...,
  date_col = NULL,
  numeric_col = NULL,
  name_prefix = paste0(".", date_col),
  weekend_days = c("Sat", "Sun"),
  units = "days"
)

## S3 method for class 'tbl_now'
add_temporal_effects(
  x,
  t_effects = NULL,
  overwrite = FALSE,
  ...,
  date_type = "event_date",
  weekend_days = c("Sat", "Sun")
)

compute_temporal_effects(x, overwrite = FALSE)

Arguments

Value

add_temporal_effects() returns the object with the specification recorded. For a tbl_now no columns are added; for a plain data.frame, which has nowhere to record a specification, the columns are computed immediately.

compute_temporal_effects() returns a tbl_now with one column per effect appended. The specification is kept, so it still prints; the names of the columns just created are available from get_temporal_effect_cols().

See Also

temporal_effects() to build the specification, and for what each effect means; replace_temporal_effects() and remove_temporal_effects() to swap or drop it; get_temporal_effects() and get_temporal_effect_cols() to read back the request and the columns; calendar_effect_plots to see the patterns; add() for the other attribute setters.

Examples

data(denguedat)
dengue <- tbl_now(denguedat,
  event_date = "onset_week",
  report_date = "report_week",
  strata = "gender",
  verbose = FALSE
)

# Step 1-2: say you want a week-of-year effect, and record it.
dengue <- dengue |>
  add_temporal_effects(t_effects = temporal_effects(week_of_year = TRUE))

# The request is stored, but no column has been built yet.
get_temporal_effects(dengue)
get_temporal_effect_cols(dengue)

# Step 3: materialise it.
computed <- compute_temporal_effects(dengue)
get_temporal_effect_cols(computed)
head(computed[[get_temporal_effect_cols(computed)[1]]])

# Specifications accumulate, so you can add a second pattern ...
both <- dengue |>
  add_temporal_effects(t_effects = temporal_effects(month_of_year = TRUE))
get_temporal_effect_cols(compute_temporal_effects(both))

# ... swap the whole specification for another ...
dengue |>
  replace_temporal_effects(t_effects = temporal_effects(seasons = 52)) |>
  get_temporal_effects()

# ... or forget it entirely.
dengue |>
  remove_temporal_effects() |>
  get_temporal_effects()

Coarsen a tbl_now onto a bigger time unit

Description

Daily surveillance data is often too sparse to nowcast: most (event date, report date) cells hold a zero or a one, and the delay distribution is mostly noise. The usual fix is to work in weeks instead. aggregate_time_units() does that in one call – it moves every date onto the coarser grid, adds the counts up, and returns a tbl_now that knows it is now weekly, so .delay, the converters and the models all count in weeks from then on.

Usage

aggregate_time_units(
  x,
  to = "weeks",
  axes = "all",
  label = c("start", "end"),
  align_on_day = 7,
  type = "epi",
  verbose = TRUE
)

Arguments

Details

Each date is replaced by the start of the period it falls in: the Sunday that begins its epidemiological week, the first of its month, the first of January of its year (label = "end" names the period by its last day instead). Weeks go through the same epi/ISO machinery align_weeks() uses, so type and align_on_day mean exactly what they mean there.

What happens to the rows depends on the data type:

• linelist – one row is still one case; only the dates move.

• count-incidence – rows that land in the same (event, report) cell are summed.

• count-cumulative – cumulative totals are not additive, so the series is de-accumulated to increments first, aggregated, and accumulated again on the new grid.

Two consequences worth knowing:

• Weeks do not nest inside months. Aggregating daily data to weeks and then to months is not the same as going straight to months: the second pass sees only the week's label, so a week beginning 31 December lands in December even though most of its cases happened in January. Aggregate once, to the unit you actually want.

• An NA count means not yet observed, so a period containing one has an unknown total, not a total that quietly leaves it out. Use complete_zeroes() first if the NAs are really zeroes.

Value

A tbl_now on the coarser grid, with event_units, report_units and revision_units updated for the axes that were aggregated, and now moved onto the new grid.

What happens to the temporal effects

Any temporal-effect columns that were materialised by compute_temporal_effects() are dropped, because a day-of-week term computed on daily dates is meaningless once those dates are weeks.

The lazy temporal_effects() specification moves onto the new grid with everything else, and the effects the new grid cannot express are dropped from it rather than silently rebuilt on dates that cannot carry them:

• day_of_week, weekend, day_of_month, holiday_lags and weekend_lags are properties of a day, so they survive only to = "days".

• week_of_year survives "weeks"; month_of_year survives "months".

• seasons are rescaled. A Fourier period is a length, not a position in the calendar, so it converts: a 365-day season becomes a 52.14-week one, and ⁠seasons = 52, season_length = 7⁠ becomes seasons = 52 in weeks. A period that ends up two units or shorter is dropped – it is at or below the new grid's Nyquist limit, so it can no longer be told from a constant or from a longer wave.

• holidays are kept. On a coarser grid the holiday column stops being a 0/1 indicator and becomes the share of the period's days that the calendar marks – 1/7 for a week containing Christmas Day – which is the same number as the indicator when the period is one day long.

A specification with nothing left is removed. verbose = TRUE says what was dropped and what was rescaled.

Aggregating one axis only

axes exists because the two axes do not always move together: a system may record the day a specimen was taken but only publish weekly report batches. Two things follow:

• tbl_now() requires the report axis to be at least as coarse as the event axis, so aggregating only the event axis of a daily object is refused. Aggregate the report axis too, or use axes = "all".

• A week named by the day it starts sits before every date inside it, so coarsening a later axis alone – the report against a daily event, or the revision against a daily report – with label = "start" produces negative delays, and validate_tbl_now() warns. Use label = "end": a report that arrived somewhere in week W is known by the end of W, which is the honest bound. When the axes move together the labelling cancels out and either choice gives the same delays.

See Also

align_weeks(), which snaps weekly dates to a common weekday without changing the units or the counts; to_count() to change data type without touching the dates; complete_zeroes() for the cells the coarser grid still leaves empty; tbl_now()'s units argument to declare the units up front; temporal_effects() and compute_temporal_effects() for the specification this coarsens.

Examples

# A sparse daily line list: one or two cases a day.
df <- data.frame(
  onset = as.Date("2024-01-01") + c(0, 1, 3, 8, 9, 15),
  reported = as.Date("2024-01-01") + c(2, 2, 5, 9, 12, 16),
  sex = c("F", "M", "F", "M", "F", "M")
)
daily <- tbl_now(df,
  event_date = onset, report_date = reported, strata = sex,
  data_type = "linelist", units = "days", verbose = FALSE
)
get_event_units(daily)

# The same cases, on a weekly grid: every date moves to the Sunday that
# starts its epidemiological week, and the delays are whole weeks.
weekly <- aggregate_time_units(daily, to = "weeks", verbose = FALSE)
get_event_units(weekly)
weekly[[get_event_date(weekly)]]
weekly$.delay

# Count data is added up rather than merely relabelled.
counts <- to_count(daily, to = "count-incidence")
sum(counts$n)
sum(aggregate_time_units(counts, to = "months", verbose = FALSE)$n)

# A weekly grid cannot carry a day-of-week effect, so it is dropped from the
# specification; the 365-day season is rescaled to 52.14 weeks instead.
spec <- daily |>
  add_temporal_effects(
    temporal_effects(day_of_week = TRUE, seasons = c(7, 365))
  )
get_temporal_effects(aggregate_time_units(spec, to = "weeks", verbose = FALSE))

Put weekly data on a common weekday

Description

Weekly surveillance data is rarely as tidy as it looks. The same series may be stamped with a Wednesday one year and a Thursday the next, or event dates may fall on a Sunday while reports fall on a Saturday. When that happens the delay between the two stops being a whole number of weeks – you get delays of 2.86 weeks – and most nowcasting models, which count in whole periods, either refuse the data or quietly round it.

align_weeks() snaps every date to the same weekday, so week differences come out as integers. week_2_date() solves the neighbouring problem: you have epiweek (or ISO week) numbers rather than dates, and need real dates to build a tbl_now from.

Usage

align_weeks(.data, align_on_day = 7, type = "epi", ...)

## S3 method for class 'data.frame'
align_weeks(
  .data,
  align_on_day = 7,
  type = "epi",
  ...,
  date_col,
  new_date_col = NULL
)

## S3 method for class 'tbl_now'
align_weeks(.data, align_on_day = 7, type = "epi", ...)

week_2_date(
  .data,
  week_col,
  year_col,
  align_on_day = 7,
  week_fun = lubridate::epiweek,
  year_fun = lubridate::epiyear,
  date_col_name = "date"
)

Arguments

Details

Applied to a data.frame, align_weeks() adds an aligned copy of the column you name. Applied to a tbl_now, it aligns the event and report dates together and recomputes the delay, so the object stays coherent.

Epi weeks and ISO weeks disagree about where a year starts, so type picks which convention to use: "epi" uses lubridate::epiweek() / lubridate::epiyear(), "iso" uses lubridate::isoweek() / lubridate::isoyear().

Value

align_weeks() returns its input with an aligned date column added (data.frame method), or a tbl_now whose dates have been aligned and whose .delay has been recomputed.

week_2_date() returns the input data.frame with a new date column appended.

Note

Useful whenever week boundaries differ between systems or between years, which is the normal state of affairs for epiweek and ISO week data.

See Also

tbl_now(), whose align_weeks = TRUE argument does this at construction time; is_weekday(), which numbers weekdays the same way; complete_zeroes() for filling the weeks where nothing was reported; temporal_effects() for using week-of-year as a model term.

Examples

## ---- Plain data frames ------------------------------------------------

# Three dates falling on different weekdays.
df <- data.frame(date = as.Date(c("2022-11-02", "2022-11-07", "2022-11-13")))
weekdays(df$date)

# Snap them all back to the Sunday that starts their week.
aligned <- align_weeks(df, date_col = date)
aligned
weekdays(aligned$date_aligned)

# Or to Tuesday. Weekday numbers are ISO: 1 = Monday, so Tuesday is 2.
align_weeks(df, date_col = date, align_on_day = 2)

## ---- A tbl_now: making the delays whole numbers -------------------------

data(flusight)

# One state is enough to see the problem.
texas <- flusight[flusight$location_name == "Texas", ]
flutbl <- tbl_now(texas,
  event_date = "target_end_date",
  report_date = "as_of", case_count = "observation",
  strata = "location_name", verbose = FALSE
)

# `as_of` is sometimes a Saturday and sometimes a Wednesday, so some delays
# land between whole weeks.
mean(flutbl$.delay != round(flutbl$.delay))

# After aligning, every delay is a whole number of weeks.
flutbl <- align_weeks(flutbl)
mean(flutbl$.delay != round(flutbl$.delay))

## ---- Week numbers instead of dates --------------------------------------

# Data reported as "week 1 of 2024" and so on, with no usable date column.
df <- data.frame(
  epidemiological_week = 1:5,
  epidemiological_year = rep(2024, 5)
)

## week_2_date() turns those into the Sunday that starts each epiweek.
week_2_date(df,
  week_col = epidemiological_week,
  year_col = epidemiological_year
)

Transform an object into a tbl_now

Description

Convert a supported object into a tbl_now. For a plain data.frame / data.table (or an existing tbl_now) you supply the event_date and report_date columns yourself. For objects produced by other packages the conversion is delegated to the matching ⁠tbl_now_from_*()⁠ converter, which already knows how to map that format – so those methods do not take event_date / report_date.

Usage

as_tbl_now(object, ...)

## S3 method for class 'tbl_now'
as_tbl_now(object, event_date, report_date, ...)

## S3 method for class 'data.frame'
as_tbl_now(object, event_date, report_date, ...)

## S3 method for class 'enw_preprocess_data'
as_tbl_now(object, ...)

## S3 method for class 'reporting_triangle'
as_tbl_now(object, ...)

## S3 method for class 'epidist_linelist_data'
as_tbl_now(object, ...)

## S3 method for class 'epidist_aggregate_data'
as_tbl_now(object, ...)

## S3 method for class 'tbl_ts'
as_tbl_now(object, report_date, event_date = NULL, ...)

## S3 method for class 'data.table'
as_tbl_now(object, event_date, report_date, ...)

## S3 method for class 'tbl_now_epinow2_snapshots'
as_tbl_now(object, ...)

## S3 method for class 'tbl_now_triangle_list'
as_tbl_now(object, ...)

## S3 method for class 'tbl_now_surveillance_list'
as_tbl_now(object, ...)

Arguments

Details

Package-specific inputs forward to a dedicated converter. See that converter for the extra arguments it accepts (e.g. strata, max_delay, format, delays_unit, ...), for which columns are carried over, and for the transformation notes / round-trip caveats in its Round-trip section:

• enw_preprocess_data (or a fitted epinowcast object) -> tbl_now_from_epinowcast()

• reporting_triangle (baselinenowcast) -> tbl_now_from_baselinenowcast()

• epidist_linelist_data / epidist_aggregate_data (epidist) -> tbl_now_from_epidist()

• tbl_ts (tsibble) -> tbl_now_from_tsibble()

• data.table -> tbl_now_from_data_table()

Anything passed through ... is forwarded to the underlying converter (and on to tbl_now()), so options such as event_units, now or verbose can be supplied here too.

Value

A tbl_now object.

See Also

tbl_now() to build one from scratch; the converters this dispatches to – tbl_now_from_epinowcast(), tbl_now_from_baselinenowcast(), tbl_now_from_epidist(), tbl_now_from_tsibble(), tbl_now_from_data_table() – and the ⁠tbl_now_to_*()⁠ functions that go the other way. The One dataset, many nowcasts article shows the round trip against each modelling package.

Examples

## For a plain data.frame this is a synonym for tbl_now(): you name the
# columns yourself.
data(denguedat)
as_tbl_now(denguedat, event_date = "onset_week", report_date = "report_week")

# For an object built by another nowcasting package you often do not name them,
# because that format already fixes which column is the event date and which
# is the report date. Here we send a tbl_now out to tsibble and bring it back.
if (requireNamespace("tsibble", quietly = TRUE)) {
  ndata <- tbl_now(denguedat,
    event_date = onset_week, report_date = report_week, verbose = FALSE
  )
  ts <- suppressWarnings(tbl_now_to_tsibble(ndata, verbose = FALSE))

  # Bare names and strings both work.
  as_tbl_now(ts, event_date = onset_week, report_date = report_week)
}

Coerce a tbl_now to a tibble or a data frame

Description

tibble::as_tibble() and as.data.frame() methods for tbl_now. They drop the tbl_now class and metadata, returning a plain tibble / data.frame.

Set compute_temporal_effects = TRUE to materialise the lazy temporal_effects() specification with compute_temporal_effects() first, so the holiday / Fourier / calendar columns are present in the result. The input tbl_now is left unchanged (the spec is materialised on a copy).

The default is FALSE on purpose: dplyr uses these coercions internally as cheap declassers (for example inside group_by() and the data mask behind mutate() / filter() / slice()). Materialising there would both break the lazy temporal_effects contract and recurse, so materialising is strictly opt-in. To get a modelling-ready frame with the effects computed, either pass compute_temporal_effects = TRUE here or call compute_temporal_effects() on the tbl_now beforehand.

Usage

## S3 method for class 'tbl_now'
as_tibble(x, ..., compute_temporal_effects = FALSE)

## S3 method for class 'grouped_tbl_now'
as_tibble(x, ..., compute_temporal_effects = FALSE)

## S3 method for class 'tbl_now'
as.data.frame(x, ..., compute_temporal_effects = FALSE)

## S3 method for class 'grouped_tbl_now'
as.data.frame(x, ..., compute_temporal_effects = FALSE)

Arguments

Value

A tibble (for as_tibble()) or a data.frame (for as.data.frame()); it carries the temporal-effect columns when compute_temporal_effects = TRUE.

See Also

compute_temporal_effects(), temporal_effects()

Examples

data(denguedat)
df_now <- tbl_now(denguedat,
  event_date  = onset_week,
  report_date = report_week,
  t_effects   = temporal_effects(week_of_year = TRUE),
  verbose     = FALSE
)

## Plain coercion leaves the spec lazy (no temporal-effect columns):
".event_week_of_year" %in% names(tibble::as_tibble(df_now)) # FALSE

# Opt in to materialise the holiday / Fourier / calendar columns:
as_tibble(df_now, compute_temporal_effects = TRUE)
as.data.frame(df_now, compute_temporal_effects = TRUE)

Coerce a tbl_nowcast into a tibble

Description

Turns a fitted nowcast into an ordinary tibble you can plot, join or write out: one row per event date and quantile level by default, or one row per posterior draw with type = "draws".

Not every engine keeps draws. When the backend returned only summarised quantiles, type = "draws" has nothing to give you.

Usage

## S3 method for class 'tbl_nowcast'
as_tibble(x, ..., type = c("quantiles", "draws"))

Arguments

Details

Registered in .onLoad() rather than assigned with ⁠S7::method(as_tibble, tbl_nowcast) <- ⁠, which is what the neighbouring print() and as.data.frame() methods use. Assigning an S7 method onto an imported generic copies that generic into this namespace, and tibble::as_tibble() carries rownames = pkgconfig::get_config(...) as a default argument – so the copy makes ⁠R CMD check⁠ report pkgconfig as an undeclared :: import of a package this one never uses. Plain S3 registration dispatches on class(x), which for an S7 object is "tbl.now::tbl_nowcast", and copies nothing.

The registered class string cannot be spelled as a method name – there is no writable as_tibble.tbl.now::tbl_nowcast – but the function itself is still named for the method it implements, so that ⁠R CMD check⁠ can resolve this topic's usage section back to an object that exists.

Value

A tibble. For type = "quantiles", one row per event date, stratum and quantile level, with the quantile level in .quantile_level and the predicted count in .value. For type = "draws", one row per draw.

See Also

tidy() for the same content with the engine's metadata attached; autoplot() to plot it; score_nowcast() to score it against observed counts.

Examples

predictions <- data.frame(
  onset_week = as.Date("2020-01-05"),
  .quantile_level = c(0.5, 0.9), .value = c(10, 14)
)
nc <- tbl_nowcast(predictions = predictions, method = "toy", event_date = "onset_week")
tibble::as_tibble(nc)

Base R operations on a tbl_now

Description

A tbl_now() is a tibble with extra attributes recording which column is the event date, which is the report date, and so on. These methods make sure those attributes survive ordinary base-R manipulation, so that x[1:10, ], names(x) <- ... and x$new <- ... give you back a tbl_now rather than a bare data frame.

You never call them directly – they are what makes the operators work.

Usage

## S3 method for class 'tbl_now'
x[...]

## S3 method for class 'grouped_tbl_now'
x[...]

## S3 replacement method for class 'tbl_now'
names(x) <- value

## S3 replacement method for class 'grouped_tbl_now'
names(x) <- value

## S3 replacement method for class 'tbl_now'
x$name <- value

## S3 replacement method for class 'grouped_tbl_now'
x$name <- value

Arguments

Details

When an operation leaves the object unable to describe a nowcast – because it dropped or renamed the event-date column, say – the class cannot honestly be kept. In that case the result is demoted to a plain data frame (with a warning), rather than pretending to still be a tbl_now. Attributes that are still meaningful are preserved on the way down.

The same applies to dplyr verbs, through dplyr_reconstruct().

Value

A tbl_now object, or a plain data frame when the operation inrevised the class.

See Also

tbl_now() for the attributes being preserved; tbl_now_attributes() to check what survived; as_tibble() to drop the class on purpose; validate_tbl_now() to confirm the result is still well formed.

Examples

data(denguedat)
dengue <- tbl_now(denguedat,
  event_date = onset_week, report_date = report_week,
  strata = gender, verbose = FALSE
)

# Subsetting rows keeps the class and everything it knows.
small <- dengue[1:10, ]
class(small)[1]
get_event_date(small)

# So does adding a column with `$<-`.
dengue$season <- ifelse(
  lubridate::month(dengue$onset_week) %in% 6:11, "wet", "dry"
)
class(dengue)[1]

# And renaming an unimportant column with `names<-`.
renamed <- dengue
names(renamed)[names(renamed) == "season"] <- "period"
get_event_date(renamed)

# But dropping the event date leaves nothing a nowcast could use, so the
# object is demoted to a plain tibble instead of lying about itself.
demoted <- suppressWarnings(dengue[, c("report_week", "gender")])
class(demoted)[1]

Diagnostic autoplot for a tbl_now

Description

Produces a multi-panel diagnostic overview of a tbl_now using ggplot2::ggplot() and patchwork. The gallery is a matrix with one column per process: the case counts on the left, the reporting delay next to them, and — when the object declares a revision axis — the revision process on the right. Each row asks the same question of every process, so an object with two dates comes out two columns wide and one with three dates three columns wide. You choose which panels to draw with the panels argument, and the number of columns follows: a selection covering one family only (panels = "calendar") is one column.

A row a process cannot answer — weekly data has no day-of-week panel — leaves that cell empty rather than closing the gap, so the columns keep their meaning all the way down.

Case-count panels

• "delay_distribution" — a (case-count weighted) histogram of the reporting delay (.delay). For count-cumulative data this panel instead shows the cumulative growth by delay: boxplots (on a log scale, with a dashed reference at 1) of the ratio of each event date's cumulative count at a delay to its cumulative count at the previous delay. A ratio above 1 is an upward revision, below 1 a downward one, and the boxes converge to 1 as reporting completes.

• "epidemic" — the latest reported case counts per event_date, with a dashed vertical line marking where the data become incomplete (less than level of the delay distribution has arrived). Holidays from the attached temporal_effects() spec are marked with dots.

• "calendar_weekday", "calendar_week", "calendar_month" — boxplots of the normalized case effect (each event date's cases divided by the overall mean, so 1 is average) by day of week, epidemiological week, or month.

• "calendar_holiday" — the same normalized boxplots by day type, titled "Weekend and/or holiday effects" because that is what the day types are. The categories follow the attached temporal_effects() spec: a holiday calendar and a weekend effect together give Weekday / Weekend / Holiday, a calendar alone gives Non-holiday / Holiday, and a weekend effect alone gives Weekday / Weekend. A holiday falling on a weekend counts as a holiday.

• "calendar_holiday_lag" — the same normalized boxplots by position relative to the nearest holiday, as asked for by holiday_lags (see temporal_effects()): "2 before", "1 before", "Holiday", "1 after", ..., plus "Other" for every other day as the reference. It shows exactly the days the ..._holiday_lag_k / ..._holiday_lead_k columns flag — weekends and other holidays are skipped when counting working days — so you can see whether the lags you asked for are the ones that matter.

• "seasonality" — a cycles periodogram of the incidence series whose dominant peak suggests a Fourier season length for temporal_effects().

Reporting-delay panels (to inspect delay effects)

• "delay_weekday", "delay_week", "delay_month" — boxplots of the normalized mean reporting delay (each event date's mean delay divided by the overall mean delay, so 1 is average) by day of week, epidemiological week, or month; these reveal whether the delay itself has a calendar pattern. Normalizing keeps them on the same scale as the case-count calendar panels and makes them comparable across strata.

• "delay_holiday", "delay_holiday_lag" — the reporting-delay twins of the two holiday panels above: the normalized mean delay by day type and by position relative to the nearest holiday. These are often the more telling pair — a holiday usually does not change how many cases occur, but it very much changes how long they take to be reported.

• "delay_seasonality" — a cycles periodogram of the mean-delay series, whose peak marks a cycle in the reporting delay (e.g. a weekly reporting rhythm).

Revision-process panels (only when the object declares a revision axis)

• "revision_distribution" — the reporting-delay histogram's twin on the revision axis: a (case-count weighted) histogram of .revision_delay, the time from a report to its resolution. A case still "pending" has no resolution and so no revision delay, and does not appear.

• "revision_weekday", "revision_week", "revision_month", "revision_holiday", "revision_holiday_lag", "revision_seasonality" — the same calendar and periodogram questions asked of the dates resolutions arrived on.

Every panel is colour-coded by the process it describes — red for the reporting-delay panels, green for the case-count (epidemic) ones — and says which one it is in its subtitle, so a single panel still reads on its own.

Which panels are available depends on the object. The calendar/delay panels follow the event unit: daily data offers day-of-week and week-of-year panels, weekly data week-of-year, monthly data month-of-year. The four holiday panels describe the attached temporal_effects() spec, so they appear only when there is one to describe: "calendar_holiday" / "delay_holiday" need a holidays calendar or a weekend effect, and the two lag panels additionally need a non-zero holiday_lags. Requesting a holiday panel without the matching effect warns and skips it. The spec is read directly, so you do not need to call compute_temporal_effects() first.

The delay panels are computed on the complete portion of the series (event dates on or before the incompleteness line) so the recent reporting truncation does not bias them.

Usage

## S3 method for class 'tbl_now'
autoplot(
  object,
  ...,
  panels = "all",
  by_strata = FALSE,
  strata = NULL,
  measure = c("percent", "normalized"),
  by_revision_type = FALSE,
  level = 0.95,
  plotly = FALSE,
  size = 1,
  linewidth = 1,
  palette = .tbl_now_palette(),
  delay_distribution_xlim = NULL,
  event_date_xlim = NULL,
  calendar_effect_xlim = NULL,
  seasonality_xlim = NULL
)

Arguments

Value

A patchwork object combining the selected panels, one column per process, or — when a single panel is selected — that panel as a ggplot2 object.

See Also

diagnostic_plot() for the companion gallery, which looks at the reporting process – when reports arrived and whether any of it is artificial – rather than at the case counts; the panels here as standalone functions: plot_observed_cases(), plot_delay_distribution(), plot_cycles(), calendar_effect_plots and plot_delay_drift(); summary() and diagnose() for the same information as tables. The Diagnosing a tbl_now article reads the panels one at a time.

Examples

data(denguedat)
# A few recent months keep the example fast; the panels look the same on
# twenty years of data, they just take longer to draw.
recent <- denguedat[denguedat$onset_week >= as.Date("2010-11-01"), ]
dengue <- tbl_now(recent,
  event_date = "onset_week",
  report_date = "report_week", strata = "gender", verbose = FALSE
)
autoplot(dengue)

# A single panel comes back as a plain ggplot, so it can be selected, split
# by stratum and then treated like any other plot. `panels` takes any of the
# names listed above; `by_strata = TRUE` works on the whole gallery too, but
# one panel keeps the example quick.
autoplot(dengue, panels = "delay_week", by_strata = TRUE)

# Panel-specific arguments compose with it: zoom the delay panel to 0-10 weeks
autoplot(dengue, panels = "delay_distribution", delay_distribution_xlim = c(0, 10))

Plot a nowcast

Description

Draws a fan chart of a tbl_nowcast: the counts reported so far as grey columns, one shaded band per central prediction interval over them, and the median as a line, so that the size of the correction the model is applying is visible as the gap between the bars and the fan.

Usage

## S3 method for class 'tbl_nowcast'
autoplot(object, ..., levels = NULL,
  show_reported = TRUE, colour = NULL, linewidth = 1, date_lim = NULL,
  ylim = NULL, palette = .tbl_now_palette())

Arguments

Details

A named function registered in .onLoad(), like the neighbouring tidy() and as_tibble() methods and for the same two reasons.

⁠S7::method(autoplot, tbl_nowcast) <- ⁠ would be shorter, but ⁠method<-⁠ is a replacement function, so R rewrites the call as an assignment back to autoplot and leaves a copy of ggplot2's generic in this namespace. That copy is what ⁠R CMD check⁠ sees when it decides autoplot is a generic this package owns and exports, which in turn makes every ⁠autoplot.*⁠ function here look like an S3 method that was never registered. Plain registration copies nothing.

The function is also named for the method it implements, rather than being assigned anonymously into the generic, so that ⁠R CMD check⁠ can resolve this topic's usage section back to an object that exists.

Value

A ggplot object.

See Also

run_nowcast(), nowcast_ensemble()

Examples

predictions <- tidyr::expand_grid(
  onset_week = as.Date("2020-01-05") + seq(0, 28, by = 7),
  .quantile_level = c(0.05, 0.25, 0.5, 0.75, 0.95)
)
predictions$.value <- 10 + 30 * predictions$.quantile_level
nc <- tbl_nowcast(predictions = predictions, method = "toy", event_date = "onset_week")

autoplot(nc)

# Zoom on to the corrected weeks without dropping the rows that build the fan.
autoplot(nc, date_lim = c(as.Date("2020-01-19"), as.Date("2020-02-02")))

Calendar effects on the case counts or on the reporting delay

Description

One panel of autoplot(), drawn on its own. Each function shows the same boxplots the corresponding autoplot() panel does, for one calendar grouping:

• plot_day_of_week_effects() — by day of week (daily data only).

• plot_week_of_year_effects() — by epidemiological week.

• plot_month_of_year_effects() — by month (monthly data only).

• plot_holiday_effects() — by day type (Weekday / Weekend / Holiday, following the attached temporal_effects() spec).

• plot_weekend_effects() — the same panel, on an object that may not carry a spec yet: it attaches temporal_effects(weekend = TRUE) when there is no weekend effect already, so the weekend boxes appear without a separate add_temporal_effects() call. The spec goes on a copy — your object is not modified — and a calendar already attached still contributes its Holiday box. Daily data only, since a weekend is a property of the day.

• plot_holiday_lag_effects() — by position relative to the nearest holiday ("1 before", "Holiday", "1 after", ..., plus "Other").

type picks which process to describe: "epidemic" (green — how the cases vary by calendar group), "report" (red — how the reporting does), or "revision" (ochre — how resolved cases arrive on revision dates).

The three day-type / holiday-lag functions have no measure argument: they are always normalized. Their categories are not equal-sized parts of a calendar block — the weekend is two days in seven — so a percentage share would mostly restate the calendar rather than the data ("29% of the cases at the weekend" is average, not low). The day-of-week, week-of-year and month-of-year functions keep both measures.

Use these when you want one effect, in its own figure, at its own size; use autoplot() when you want the diagnostic grid in one call. Everything else is the same: autoplot(x, panels = "calendar_weekday") and plot_day_of_week_effects(x) return the identical plot.

Usage

plot_day_of_week_effects(
  x,
  type = c("epidemic", "report", "revision"),
  measure = c("percent", "normalized"),
  ...
)

plot_week_of_year_effects(
  x,
  type = c("epidemic", "report", "revision"),
  measure = c("percent", "normalized"),
  ...
)

plot_month_of_year_effects(
  x,
  type = c("epidemic", "report", "revision"),
  measure = c("percent", "normalized"),
  ...
)

plot_holiday_effects(x, type = c("epidemic", "report", "revision"), ...)

plot_weekend_effects(
  x,
  type = c("epidemic", "report", "revision"),
  weekend_days = c("Sat", "Sun"),
  ...
)

plot_holiday_lag_effects(x, type = c("epidemic", "report", "revision"), ...)

Arguments

Value

A ggplot2 object (or a plotly widget when plotly = TRUE).

See Also

autoplot.tbl_now(), plot_cycles(), plot_delay_distribution(), plot_observed_cases(); temporal_effects() and add_temporal_effects() for the specification the day-type and holiday-lag panels describe.

Examples

data(denguedat)
# First few years only, to keep the example quick; the full data works the same.
dengue_now <- tbl_now(denguedat[1:2500, ], onset_week, report_week, verbose = FALSE)

# How the cases vary by epidemiological week
plot_week_of_year_effects(dengue_now)

# The weekend on its own, on daily data, with no spec to attach first
days <- seq(as.Date("2021-01-01"), as.Date("2021-06-30"), by = "day")
daily_now <- tbl_now(
  data.frame(event_date = days, report_date = days + 1),
  event_date, report_date, verbose = FALSE
)
plot_weekend_effects(daily_now)

# By month, on monthly-unit data. `type` picks the process and `measure`
# picks the scale; both compose, and work the same way on every calendar
# function here. (The day-type and holiday-lag panels are always
# normalized, so they take `type` but not `measure`.)
monthly_now <- tbl_now(
  data.frame(
    event_date  = seq(as.Date("2018-01-01"), as.Date("2021-12-01"), by = "month"),
    report_date = seq(as.Date("2018-02-01"), as.Date("2022-01-01"), by = "month")
  ),
  event_date, report_date,
  event_units = "months", report_units = "months", verbose = FALSE
)
plot_month_of_year_effects(monthly_now, type = "report", measure = "normalized")

if (requireNamespace("almanac", quietly = TRUE)){

  ## By day type (weekday / weekend / holiday), once a holiday calendar is attached
  holiday_now <- dengue_now |>
   add_temporal_effects(temporal_effects(weekend = TRUE, holidays = almanac::cal_us_federal()))
  plot_holiday_effects(holiday_now)

  # By position relative to the nearest holiday
  holiday_lag_now <- dengue_now |>
    add_temporal_effects(temporal_effects(holidays = almanac::cal_us_federal(), holiday_lags = 2))
  plot_holiday_lag_effects(holiday_lag_now)

}

Record a report or a delay as a bound rather than a fact

Description

Surveillance data is full of dates that are not really dates. A case with onset in March turns up in December; a report date is missing altogether; a system codes "never reported" as 2222-02-22. Deleting those records throws away real cases, and believing them drags the estimated delay distribution to the right until the nowcast thinks reporting is far slower than it is.

These functions censor instead: the object keeps the case, but records its delay as "at least this long" rather than "exactly this long".

There are two axes to censor and three ways to say which rows, so there are six verbs. On the reporting axis (event date to report date, the is_censored_report flag):

• censor_reports() – rows matching a condition get a replacement report date (the missing ones become now, say) and the flag.

• censor_reporting_delays() – the same, said as a delay rather than a date; with no replacement it only sets the flag.

• censor_reporting_delays_above() – considers as censored every delay longer than max_delay, in the object's event units, and leaves every other row alone. This is the one you want when the threshold is the rule: "anything that took more than 60 days is a lower bound, not a measurement".

On the revision axis (report date to resolution, the is_censored_revision flag), the same three:

• censor_revisions() – rows matching a condition get a replacement revision date and the flag.

• censor_revision_delays() – the same, said as a delay from the report.

• censor_revision_delays_above() – considers as censored every revision delay longer than max_delay, in the object's revision units: a laboratory result that took months is a case you have stopped believing the turnaround of.

Usage

censor_reporting_delays_above(x, max_delay, verbose = TRUE)

censor_reports(x, condition, to_report = get_now(x), verbose = TRUE)

censor_reporting_delays(x, condition, to_delay = NULL, verbose = TRUE)

censor_revisions(x, condition, to_revision = get_now(x), verbose = TRUE)

censor_revision_delays(x, condition, to_delay = NULL, verbose = TRUE)

censor_revision_delays_above(x, max_delay, verbose = TRUE)

Arguments

Details

The reporting delay is read from the generated .delay column (report date minus event date, in the object's event units); the revision delay from .revision_delay (revision date minus report date, in revision units). Existing censoring flags are merged rather than overwritten, so a delay that was already censored stays censored, and the flag column is created (as .is_censored_report / .is_censored_revision) when the object has none.

The threshold functions keep the case and its date. Nothing is deleted and no outcome is rewritten: the flag says the delay is a bound rather than a measurement, and it is up to the model to use that. A case that was confirmed after 200 days is still a confirmed case, and get_latest_revised_cases() still counts it.

condition is evaluated inside the data, like a dplyr::filter() expression, so it can name any column – including the generated .delay. Rows where it comes out NA are not censored: a condition that cannot be evaluated is not a condition that was met.

The four verbs that take a condition can move a date, and then they rebuild the object: the generated numeric and delay columns are recomputed, and now moves forward when a replacement lands after it, never backwards – now is where you are standing, not the last date in the data. Nothing stops a replacement from landing before the date it is measured from; that is a negative delay, and validate_tbl_now() says so.

Any temporal-effect column materialised on the report date (⁠.report_*⁠, from compute_temporal_effects()) is dropped when the report date moves, because it describes a date that has just changed; run compute_temporal_effects() again to rebuild it. The ⁠.event_*⁠ ones are kept – the event dates never move – and neither is touched by the revision verbs.

Value

A tbl_now with that axis's censoring column updated, creating it when absent (.is_censored_report or .is_censored_revision), and with the dates replaced where a replacement was asked for. The three reporting verbs touch is_censored_report and the report date; the three revision verbs touch is_censored_revision and the revision date. Neither rewrites revision_type, and nothing is ever deleted.

Pending cases are skipped

"pending" means reported and still waiting, so a pending case has no revision date – that is the whole difference between it and a resolution that was never recorded. Writing a date onto one would assert a resolution that never happened, and make the case look resolved to everything counting arrivals on the revision axis.

So censor_revisions() and censor_revision_delays() skip pending rows when they would write a date, and say how many they skipped. To censor a case that really was resolved but whose date is missing, make sure its revision_type says so first. Flagging without a replacement is not affected: no date is written, so there is nothing to contradict.

See Also

add_is_censored_report() and change_is_censored_report() to set the flag by hand, and add_is_censored_revision() for the revision axis; diagnose_revision_delay() and plot_delay_distribution() to find the threshold worth using; diagnose_truncation() for the delays that are missing rather than long; complete_zeroes() for the opposite problem.

Examples

# Four cases, one of which took 300 days to be reported.
df <- data.frame(
  onset = as.Date("2020-01-01") + c(0, 0, 1, 2),
  reported = as.Date("2020-01-01") + c(1, 5, 2, 300)
)
tn <- tbl_now(df,
  event_date = onset, report_date = reported,
  data_type = "linelist", verbose = FALSE
)
tn$.delay

# Anything slower than 60 days is recorded as a lower bound, not a fact.
censored <- censor_reporting_delays_above(tn, max_delay = 60)
censored[[get_is_censored_report(censored)]]

# The same rule written by hand, and capped at 60 days as well, so the
# 300-day outlier stops dominating the delay distribution.
capped <- censor_reporting_delays(tn, .delay > 60, to_delay = 60, verbose = FALSE)
capped$.delay

## ---- Reports that never arrived ---------------------------------------

# A missing report date, and a system that codes "never" as a date in 2222.
messy <- data.frame(
  onset = as.Date("2020-01-01") + 0:3,
  reported = as.Date(c("2020-01-03", NA, "2222-02-22", "2020-01-06"))
)
messy_now <- suppressWarnings(tbl_now(messy,
  event_date = onset, report_date = reported,
  data_type = "linelist", units = "days", verbose = FALSE,
  now = as.Date("2020-01-10")
))

# Both are "not reported yet", so both become `now` and are flagged censored.
# Wrapped in suppressWarnings because the object keeps warning about the
# dates being fixed.
fixed <- suppressWarnings(censor_reports(messy_now,
  is.na(reported) | reported > as.Date("2100-01-01"),
  verbose = FALSE
))
fixed[[get_report_date(fixed)]]
fixed[[get_is_censored_report(fixed)]]

## ---- The revision counterpart ----------------------------------------

cases <- data.frame(
  onset = as.Date("2021-01-04") + 0:4,
  visit = as.Date("2021-01-05") + 0:4,
  result = as.Date("2021-01-05") + 0:4 + c(1, 2, 1, 90, 2),
  outcome = rep("confirmed", 5)
)
flu <- tbl_now(cases,
  event_date = onset, report_date = visit,
  revision_date = result, revision_type = outcome,
  data_type = "linelist", verbose = FALSE
)

# That one is flagged; all five stay confirmed, and the date is kept.
flagged <- censor_revision_delays_above(flu, 30, verbose = FALSE)
flagged[[get_is_censored_revision(flagged)]]
table(flagged[["outcome"]])

# The condition form: cap that turnaround at 30 days from the report, which
# moves the revision date to match.
capped_lab <- censor_revision_delays(flu, .revision_delay > 30,
  to_delay = 30, verbose = FALSE
)
capped_lab$.revision_delay

## A pending case has no resolution date, so there is nothing to censor --
# it is skipped rather than given a date it never had.
waiting <- flu
waiting[["outcome"]][2] <- "pending"
waiting[["result"]][2] <- as.Date(NA)
waiting <- change_revision_date(waiting, "result", "outcome")
out <- censor_revisions(waiting, is.na(result), verbose = FALSE)
out[["result"]][2] # still NA

Fill in the days when nothing was reported

Description

Surveillance data records what happened, not what didn't. If no dengue case with onset on 3 January was reported on 5 January, there is simply no row for that combination – which is not the same as a row saying zero, even though it means the same thing.

Most nowcasting models need the difference spelled out. They work on a complete rectangle of (event date x report date) cells, and a missing cell is ambiguous: it could be a genuine zero, or a delay so long the report has not arrived yet. complete_zeroes() writes the genuine zeros in explicitly, for every stratum, leaving only the not-yet-reported cells absent.

Usage

complete_zeroes(x, max_delay = NULL, until = NULL)

Arguments

Details

Zeros are only filled where a report could have arrived: cells with a report date on or before the event date's now, and within max_delay. Filling beyond that would invent observations from the future.

Rows with a missing date

A row whose event or report date is NA has no cell on the rectangle, so it takes no part in the grid: the bounds (max_delay, the first and last event date, the last report date) are all computed ignoring it. It is still a case, though, so it is carried through unchanged rather than dropped – use censor_reports() to give it a bound, or dplyr::filter() to remove it, if you would rather it were on the grid or gone. Only an object in which every row is missing one of the two dates is refused, because then there is no grid to complete at all.

Value

A tbl_now object with the same columns as x, plus the rows that were implicitly zero, carrying 0 in the case_count column. Explicit missing counts in the input remain NA; only cells created by complete_zeroes() are filled. The data type is preserved, as are any computed temporal-effect columns (recomputed over the new rows).

See Also

to_count() for the data shapes this operates on; censor_reporting_delays_above() for the opposite problem, delays that are too long; diagnose_missing() and diagnose_truncation() to find the gaps first; plot_reporting_triangle() to see the rectangle being filled.

Examples

ndata <- dplyr::tibble(
  event = rep(c(
    as.Date("2020/01/01"), as.Date("2020/01/01"),
    as.Date("2020/01/02"), as.Date("2020/01/04"),
    as.Date("2020/01/04")
  ), 2),
  report = rep(c(
    as.Date("2020/01/01"), as.Date("2020/01/02"),
    as.Date("2020/01/02"), as.Date("2020/01/04"),
    as.Date("2020/01/05")
  ), 2),
  n = rpois(10, lambda = 5),
  sex = c(rep("Male", 5), rep("Female", 5))
)
ndata <- tbl_now(ndata,
  event_date = event, report_date = report,
  verbose = FALSE, strata = sex, case_count = n, data_type = "count-incidence"
)

# Nothing happened on 2020-01-03, so the data has no row for it at all.
sort(unique(ndata$event))

## complete_zeroes() writes that absence down as an explicit zero, for every
# stratum, so a model can tell "no cases" from "not reported yet".
filled <- complete_zeroes(ndata)
sort(unique(filled$event))
nrow(ndata)
nrow(filled)

# Also works for count-cumulative
ndata |>
  to_count("count-cumulative") |>
  complete_zeroes() |>
  dplyr::arrange(event, sex, report)

COVID-19 Notifications – Colombia 2020-2023

Description

Daily case counts of COVID-19 from Colombia's national epidemiological surveillance system (INS), aggregated by notification date, diagnosis date, and sex. Each row is a unique combination of these three variables together with the number of cases n.

Usage

covid_colombia

Format

A data frame with 35,501 rows and 4 variables:

notification_date

Date. The event date – when the case was notified / symptom onset was recorded.

diagnosis_date

Date. The report date – when the laboratory diagnosis was registered in the national system.

sex

character. Biological sex of the case: "Female" or "Male".

n

integer. Number of cases with this (notification_date, diagnosis_date, sex) combination.

Details

In the nowcasting context the event date is notification_date (when the case symptom onset was recorded in the system) and the report date is diagnosis_date (when the laboratory result was entered). The delay between the two reflects the time from symptom onset to laboratory revision and data entry – the quantity the nowcasting model estimates and corrects for.

The dataset covers the full first three years of the Colombian epidemic (2020-03-02 to 2023-03-03) and was used in the NobBS comparison study to benchmark the diseasenowcasting / diseasenowcast2 engine.

Source

Instituto Nacional de Salud (INS), Colombia. Data accessed via the SIVIGILA open-data platform and pre-processed for the diseasenowcasting benchmarking study.

See Also

tbl_now() to declare the date columns; summary() and diagnose() to inspect the result; the package's other datasets – denguedat, mpoxdat, flusight, covid_colombia, covid_us and hai_bucaramanga.

Examples

data(covid_colombia)

# A stratified tbl_now: event = notification, report = diagnosis.
tn <- tbl_now(
  covid_colombia,
  event_date  = notification_date,
  report_date = diagnosis_date,
  strata      = sex,
  case_count  = n,
  data_type   = "count-incidence",
  verbose     = FALSE
)
tn

covid_us: CDC COVID-19 Case Surveillance Public Use Data (2020)

Description

A compact aggregation of the U.S. CDC's individual-level COVID-19 case surveillance database. It is the package's worked example for two different things: batch reporting, and the revision process – the optional third date a surveillance record can carry.

Usage

data(covid_us)

Format

A data frame with 192,953 rows and six variables:

onset_dt

Date. The event date – symptom onset.

pos_spec_dt

Date. The report date – collection of the first positive specimen.

cdc_report_dt

Date. The revision date – when the case was registered at CDC.

current_status

character. CDC's classification, either "Laboratory-confirmed case" or "Probable Case". Map it with revision_levels (see above).

sex

character. "Female", "Male", "Other", "Unknown" or "Missing".

n

integer. Number of cases sharing that combination.

Details

Each row is a unique (onset date, specimen date, CDC report date, status, sex) combination with the number of cases n.

The three dates

The source file carries four date columns. cdc_case_earliest_dt is derived by CDC as the earliest of the others, and equals onset_dt for 99.997% of the rows kept here, so it is dropped as redundant. The three that remain are the only chain that runs forward in time, and they map onto the three roles a tbl_now() knows about:

onset_dt

the event – symptoms begin.

pos_spec_dt

the report – the first positive specimen is collected, which is when the surveillance system first sees the case.

cdc_report_dt

the revision – the case is registered at CDC with a status.

current_status and revision_levels

current_status is kept in CDC's own words rather than recoded, because translating it is exactly what tbl_now(revision_levels = ) is for:

revision_levels = c(
  "Laboratory-confirmed case" = "confirmed",
  "Probable Case"             = "pending"
)

A probable case is one that met the clinical and epidemiological criteria without meeting the laboratory-confirmed definition. Every row here has a positive specimen, so "probable" means the specimen was collected and the case was never laboratory-settled – "pending" in this package's vocabulary. Note what is not there: CDC does not withdraw cases, so "retracted" does not occur in this dataset. It is a two-outcome revision process, and code that needs a retraction has to look elsewhere.

The relationship between the outcome and the revision delay is real rather than fabricated: probable cases are registered a median of 2 days after the specimen, laboratory-confirmed ones 4 days.

What was kept

Cases where all three dates are present, correctly ordered (⁠onset_dt <= pos_spec_dt <= cdc_report_dt⁠) and falling entirely within 2020 – a self-consistent "as of the end of 2020" snapshot. Rows out of order are data-entry errors; rows missing a date cannot be placed on the chain at all. See data-raw/covid_us.R for the exact duckdb aggregation of the 14 GB source file.

The reporting delay is enormous and heavily right-skewed: cases reached CDC not smoothly but in large backlog dumps – a textbook batch-reporting pattern that diagnose_batches() and transport_discriminant() recover.

Source

Centers for Disease Control and Prevention (CDC), COVID-19 Response. COVID-19 Case Surveillance Public Use Data (version date: June 21, 2024). https://data.cdc.gov/Case-Surveillance/COVID-19-Case-Surveillance-Public-Use-Data/vbim-akqf/about_data. COVID-19 case surveillance data are collected by jurisdictions and reported voluntarily to CDC.

See Also

tbl_now() to declare the date columns; add_revision_date() to attach the third one to an object that has none; revised_cases to count the outcomes; summary() and diagnose() to inspect the result; the package's other datasets – denguedat, mpoxdat, flusight, covid_colombia and hai_bucaramanga.

Examples

library(dplyr)

data(covid_us)

# The three dates with CDC's labels translated to this package's vocabulary.
covid_us <- covid_us |>
  filter(onset_dt <= as.Date("2020-02-01"))

tn3 <- tbl_now(
  covid_us,
  event_date       = onset_dt,
  report_date      = pos_spec_dt,
  revision_date    = cdc_report_dt,
  revision_type    = current_status,
  revision_levels = c(
    "Laboratory-confirmed case" = "confirmed",
    "Probable Case"             = "pending"
  ),
  case_count = n,
  strata     = sex,
  data_type  = "count-incidence",
  verbose    = FALSE
)

denguedat: Dengue fever individual-level reporting data from Puerto Rico

Description

Surveillance data from CDC Division of Vector-Borne Diseases. 1990-2010 case reporting data included.

Usage

data(denguedat)

Format

A data frame with 52,987 rows (one per case) and 3 variables:

onset_week

Date. The week symptoms began – the event date.

report_week

Date. The week the case reached the surveillance system – the report date. Always on or after onset_week.

gender

character. "Male" or "Female". Synthetic; see the note.

Details

Each row represents a case with the columns indicating the following:

• onset_week: the week of symptom onset.

• report_week: the week of case report.

• gender: the gender of the infected individual (randomly assigned with 0.5:0.5 probability of "Male"/"Female").

Note

Data originally from the NobBS package. While onset_week and report_week correspond to actual observed data the gender was constructed exclusively for the examples of NobBS. It is a synthetic (simulated) variable and does not correspond to any reality.

References

MCGOUGH, Sarah F., et al. Nowcasting by Bayesian Smoothing: A flexible, generalizable model for real-time epidemic tracking. PLoS computational biology, 2020, vol. 16, no 4, p. e1007735.

See Also

tbl_now() to declare the date columns; summary() and diagnose() to inspect the result; the package's other datasets – denguedat, mpoxdat, flusight, covid_colombia, covid_us and hai_bucaramanga.

Examples

data(denguedat)
head(denguedat)

# The two dates every nowcast needs. Weekly data, twenty years of it.
range(denguedat$onset_week)

# Declaring them turns the data frame into a tbl_now.
dengue <- tbl_now(denguedat,
  event_date = onset_week, report_date = report_week,
  strata = gender, verbose = FALSE
)
dengue

# Most cases arrive within a week or two of onset; a few take much longer.
summary(as.numeric(dengue$.delay))

Diagnose a tbl_now

Description

diagnose() is a structural health check. It looks for the things that make a nowcast wrong before any model is fitted – dates out of order, missing values, repeated rows, units that disagree, data after now, event dates too recent to be complete – and returns them as a tibble of findings, sorted worst first.

It is deterministic and runs no statistical test. Whether the reporting delay drifts, and whether reports arrive in batches, are questions about a distribution, not about the object's structure, so diagnose() leaves them to diagnose_drift() and diagnose_batches() rather than quietly running a test whose method, window and multiplicity correction you did not choose.

Every block is also available on its own – see nowcast_diagnose_components – and diagnose() is exactly the dplyr::bind_rows() of those pieces.

Usage

diagnose(x, ...)

## Default S3 method:
diagnose(x, ...)

## S3 method for class 'tbl_now'
diagnose(
  x,
  ...,
  checks = NULL,
  by_strata = NULL,
  strata = NULL,
  warn_non_uniqueness = TRUE
)

Arguments

Value

A tibble with the columns described above, sorted worst first.

The columns

Every function in this family returns the same schema, so results can be stacked with dplyr::bind_rows() and filtered with dplyr::filter().

check

Which block the row belongs to: "declarations", "ordering", "missing", "duplicates", "units", "negatives", "now", "truncation" or "strata".

scope

What the row is about: a column name, a time axis, a pair of axes, or "all".

stratum

Which subset of the data the row describes: "all" for the pooled rows, or the stratum label otherwise.

status

An ordered factor, worst first, so the tibble sorts itself: error > warning > note > ok > skipped. See the section below.

n_affected

How many rows (or cases, or dates) the finding is about.

n_total

How many were considered.

prop

n_affected / n_total.

message

One human sentence, already formatted.

hint

What to do about it, or NA.

rows

A list-column of offending row indices, so x[result$rows[[1]], ] goes straight to the bad rows. Empty when the finding is not about particular rows, or when it was computed on a de-accumulated view whose rows are not the object's own.

What the statuses mean

error

validate_tbl_now() aborts on this. The object is not a usable tbl_now.

warning

validate_tbl_now() warns about this.

note

A diagnose()-only observation worth your attention. It is deliberately never promoted to a warning: validate_tbl_now() runs on every dplyr verb, and a new warning there would turn a quiet construction into a noisy one for data that has always been accepted.

ok

The check ran and found nothing.

skipped

Could not be assessed – no revision process, the wrong data type, or an optional package that is not installed.

See Also

nowcast_diagnose_components for the individual blocks; summary() for the descriptive counterpart – what is in the data rather than what is wrong with it; validate_tbl_now() for the same findings raised as errors and warnings; diagnostic_plot() for the picture version. The Diagnosing a tbl_now article goes through the findings one at a time.

Examples

data(denguedat)
# The last five years. The full twenty-year series gives the same shape
# of answer, it just takes longer to compute.
recent <- denguedat[denguedat$onset_week >= as.Date("2006-01-01"), ]
ndata <- tbl_now(recent,
  event_date = "onset_week",
  report_date = "report_week",
  strata = "gender",
  verbose = FALSE
)

# Everything, worst first
diagnose(ndata)

# Only what needs acting on
diagnose(ndata) |> dplyr::filter(status <= "note")

# One block on its own
diagnose(ndata, checks = "units")

## `diagnose()` never stops your pipeline -- it hands back a table for you to
## read. Use validate_tbl_now() when you want a broken object to be an error.
nrow(diagnose(ndata))

Screen the report axis for batched reporting

Description

Detects batches: report dates at which a stalled reporting system releases a backlog. A batch moves reports along the report axis without creating them, so it shows up as a spike preceded by a deficit, while the total over a window spanning both is unchanged. diagnose_batches() is completely model-free – it needs only a tbl_now(), not a fitted model – which makes it the right tool for exploratory data analysis before any nowcasting model is chosen.

Usage

diagnose_batches(
  x,
  lookback = 7L,
  baseline_window = NULL,
  period = NULL,
  null_model = c("auto", "poisson", "robust"),
  axis = c("report", "revision"),
  alpha = 0.05,
  drop_censored = TRUE
)

Arguments

Details

The idea is that a batch moves reports along the report axis without creating them. Over a window of report dates that spans both the lull and the release, the total is therefore unchanged – every report you would have seen in the window you still see, just on a different day. A genuine surge instead adds reports and inflates the window total. So two quantities separate the two cases: the deficit (how many reports the days just before the spike were missing) picks up transport, and the window total (relative to a baseline) picks up creation. A batch has a large deficit but a conserved window total; a surge has an inflated window total but no deficit.

The baseline for each candidate window is refit from report dates lying strictly outside that window, using a robust local line (Siegel's repeated median). Smoothing through the episode instead would let the deficit drag the baseline down and the batch would mask itself as a surge.

The batch flag is the trustworthy verdict: it compares dates within one window (insensitive to the overall level) and is Benjamini-Hochberg corrected across every (report date, stratum) pair, so it controls the false discovery rate rather than firing on every point that crosses a raw threshold. A per-point creation ("surge") label is deliberately not returned: it only compares the window total against the baseline, so on a steeply curved epidemic curve it fires on ordinary growth. If you need genuine surges, fit a model. For "count-cumulative" data the increments are signed and reported can be negative (a down-revision).

A reporting system that is always closed at weekends produces every batch symptom, every week, so diagnose_batches() needs the length of any scheduled cycle. It reads that from the object's temporal effects when it can: a day-of-week effect sets period = 7, a week-of-year effect period = 52 (see add_temporal_effects()). Pass period yourself to override; if the data is daily and carries no temporal effect, the function suggests period = 7. With a period set, the baseline is corrected by per-phase medians across cycles, so an irregular batch reads as an excursion relative to the schedule.

Value

A tibble of class diagnose_batches, one row per (report date, stratum), with a print() method that summarises the flagged dates. Columns:

report_date

The report (registration) date the row describes.

stratum

The stratum label, or "all" when the data is unstratified.

reported

Reports recorded on report_date (a signed increment for "count-cumulative" data, so it can be negative).

baseline

The robust expected number of reports on report_date under "no batch", from the leave-window-out local line.

deficit

How many reports the lookback days before report_date were missing relative to baseline – the transport signal. Large and positive when a stall preceded a spike.

delta

The window total minus its baseline mean – the creation signal. Near zero for a pure batch (mass only moved), large for a surge.

p_transport

One-sided p-value that the deficit is larger than noise (the raw, per-point transport test).

p_transport_bh

p_transport after a Benjamini-Hochberg correction across all rows; the flag below thresholds this.

batch

Logical verdict: TRUE when p_transport_bh < alpha and the window is not still depleted (a hold). This is the column to trust.

See Also

diagnose_batches2() for the complementary test on which event dates a flagged report date drew from; transport_discriminant() for the two coordinates behind the test, without the hypothesis test on top; simulate_batch() to plant a known batch and check the screen finds it; plot_reporting_process() and plot_reporting_triangle() to see it; add_is_censored_report() to record a batch once you believe it. The Diagnosing a tbl_now article works through a real example.

Examples

data(denguedat)

dengue_tbl <- tbl_now(
  denguedat,
  event_date  = onset_week,
  report_date = report_week,
  data_type   = "linelist",
  verbose     = FALSE
)

# Scan every report date for an unusually large arrival.
screened <- diagnose_batches(dengue_tbl, lookback = 2)
head(screened)

# The dates it flagged, strongest evidence first. Treat these as candidates to
# look into, not as confirmed backlog releases. `reported` against `baseline`
# says how much bigger the arrival was than the surrounding days led you to
# expect.
flagged <- screened[screened$batch, ]
nrow(flagged)
flagged[order(flagged$p_transport_bh), c("report_date", "reported", "baseline")]

Test whether one report date drew from unusually old event dates

Description

A complement to diagnose_batches(), which sees only report volumes. This test asks whether the reports that arrived on a candidate date came from systematically older event dates – the signature of a released backlog – by comparing their delays with those of neighbouring report dates. It is model-free and, under the conditions below, exactly distribution-free.

Usage

diagnose_batches2(
  x,
  at,
  neighbours = 3L,
  guard = 1L,
  permute = c("items", "blocks"),
  n_permutations = 999L,
  axis = c("report", "revision"),
  drop_censored = TRUE,
  seed = NULL
)

Arguments

Details

The delays of the reports arriving on at are compared with the pooled delays of the reports arriving on nearby dates, using a one-sided rank-sum (Wilcoxon) statistic directed at longer delays on at. The p-value comes from a permutation, so no asymptotic approximation is used.

The test is model-free: as long as the epidemic curve is locally smooth, neighbouring report dates share one common delay profile, so their delay labels are exchangeable and the permutation test is (essentially) distribution-free – it needs neither the delay distribution nor the epidemic curve. With Poisson counts permute = "items" is exact; if the counts are overdispersed (neighbouring report dates share event dates, so individual items are not exchangeable) use permute = "blocks", which permutes whole report dates.

at need not be a date that carries rows. A line list cannot represent a zero, so a report date on which nothing arrived has no rows at all; that is the observation "no arrivals", not a missing one, and the test reports zero arrivals for it instead of aborting. Only a date off the object's report grid is an error.

The guard argument omits report dates immediately adjacent to at from the comparison set: if a batch is present, its own deficit dates sit right beside the spike and would contaminate the reference group. For "count-cumulative" data only positive increments carry a meaningful delay; negative increments (down-revisions) are dropped with a message.

Value

A tibble, one row per stratum, with stratum, n_at, n_reference, mean_delay_at, mean_delay_reference, statistic (standardised rank-sum) and p_value (one-sided: longer delays on at).

See Also

diagnose_batches(), which finds the report dates worth passing to at; simulate_batch() to plant a batch of known shape and check it is recovered; plot_delay_profiles() to see the delay profile this tests.

Examples

data(denguedat)

dengue_tbl <- tbl_now(
  denguedat,
  event_date  = onset_week,
  report_date = report_week,
  data_type   = "linelist",
  verbose     = FALSE
)

# Pick a report date to interrogate. A real workflow takes this from
## diagnose_batches(); here we simply name one.
diagnose_batches2(dengue_tbl, at = as.Date("1990-06-25"), n_permutations = 99)

# `n_permutations` sets the resolution of the p-value: 99 keeps the example
## fast, but use the default (999) for anything you intend to report.

Detect an abrupt change point in the reporting-delay distribution

Description

Complements diagnose_drift(). Where that tests for a gradual monotonic trend, this tests for a single abrupt shift (e.g. a reporting-system change on some date) in the per-period delay summaries, using Pettitt's nonparametric change-point test. As with diagnose_drift() it works on both a location statistic (median / mean) and a dispersion statistic (IQR / 10-90 spread), on mature data only, and — being rank-based — it is robust to the skew and serial dependence of a delay series.

Usage

diagnose_changepoint(
  x,
  ...,
  stat = c("median", "spread"),
  by_strata = FALSE,
  strata = NULL,
  mature_only = TRUE,
  level = 0.95,
  alpha = 0.05,
  axis = c("report", "revision")
)

Arguments

Value

A tibble with one row per requested stat per stratum, and the following columns:

strata

character. The stratum the row refers to. When by_strata = FALSE (the default) there is a single stratum labelled "all"; otherwise one level per observed combination of strata.

stat

character. Which delay summary was tested — one of "median", "mean", "iqr" or "spread". As in diagnose_drift(), the first two are location statistics and the last two dispersion statistics.

n

integer. Length of the tested series: the number of event dates contributing a non-missing value after the mature_only filter — periods, not cases. Series shorter than 8 periods, or with zero variance, are not tested and return NA throughout.

changepoint

Date. The event date of the last period before the estimated change; the shift is taken to occur immediately after it. NA when the series was too short to test. Note this is reported even when changepoint_detected is FALSE — Pettitt's test always returns the most extreme candidate split, so this field is only meaningful once the p-value supports it.

statistic

numeric. Pettitt's K, the maximum absolute value of the rank statistic U_t over all candidate split points. Larger means a cleaner separation between the two sides. It is not standardised, so it grows with n and is not comparable across series of different lengths.

p_value

numeric. Two-sided p-value for the null of no change point, from the standard approximation 2\exp(-6K^2 / (n^3 + n^2)), capped at 1. This approximation is known to be conservative for small n.

before, after

numeric. The mean of the statistic on each side of changepoint, in the object's delay units. These are plain means of the per-period summaries, so they describe the two regimes directly.

shift

numeric. after - before: the estimated size and direction of the jump, in delay units. Positive means delays got longer after the change point. This is the number to judge operational relevance by.

changepoint_detected

logical. The verdict: TRUE when p_value < alpha. NA p-values give FALSE.

Interpreting the result

Judge shift first and p_value second. A statistically detected change point with a shift far smaller than the day-to-day noise in the delay series is not worth acting on; a large shift is, even at a marginal p-value.

Two structural caveats matter in practice:

• Pettitt's test assumes exactly one change point. Given several, it returns the most prominent and silently ignores the rest. If you suspect more, re-run on each side of the first changepoint to search recursively.

• A slow monotonic drift will often trip this test too, with the change point landing near the middle of the series. Running diagnose_drift() alongside disambiguates: a genuine step shows up here and not necessarily there, while a gradual drift shows up in both.

A confirmed change point usually has an operational explanation — a new laboratory information system, a change in case definition, a reporting mandate, a holiday backlog being cleared. Where it lands is a strong hint about the cause, and about how far back a nowcasting model can safely be fitted: data before the change point comes from a different reporting regime.

Unlike diagnose_drift(), this test has no third-party dependency and no meaningful runtime cost, so it is cheap to run routinely.

See Also

diagnose_drift() for a gradual trend rather than a single break; plot_delay_drift() to see the series and where the break was found; diagnose_batches() for a one-day spike rather than a lasting shift.

Examples

data(denguedat)
dengue <- tbl_now(denguedat,
  event_date = "onset_week", report_date = "report_week", verbose = FALSE
)
diagnose_changepoint(dengue)

Test whether the reporting-delay distribution drifts over time

Description

Runs an autocorrelation-robust monotonic-trend test on the per-period, count-weighted delay summaries, to answer "do delays drift over time?" in a way that respects the fact that a delay series is correlated with itself.

Usage

diagnose_drift(
  x,
  ...,
  stat = c("median", "spread"),
  method = c("hamed-rao", "yue-pilon", "block-bootstrap"),
  by_strata = FALSE,
  strata = NULL,
  mature_only = TRUE,
  level = 0.95,
  alpha = 0.05,
  axis = c("report", "revision")
)

Arguments

Details

For each requested stat (and each stratum) it builds the per-event-date series of that statistic and tests it for a monotonic trend with the modifiedmk package, which corrects the Mann-Kendall variance for serial autocorrelation. A plain Mann-Kendall (or an OLS slope) would be anti-conservative here, because positive autocorrelation shrinks the effective sample size.

By default the test uses only mature event dates (those on or before the level incompleteness cutoff), because the recent, not-yet-fully-reported dates would otherwise inject a spurious downward trend.

Value

A tibble with one row per requested stat per stratum, and the following columns:

strata

character. The stratum the row refers to. When by_strata = FALSE (the default) there is a single stratum labelled "all"; otherwise one level per observed combination of strata.

stat

character. Which delay summary was tested — one of "median", "mean", "iqr" or "spread". "median"/"mean" are location statistics (are delays getting longer?); "iqr"/"spread" are dispersion statistics (are delays getting more erratic?).

n

integer. Length of the tested series, i.e. the number of event dates contributing a non-missing value after the mature_only filter. This is a count of periods, not a count of cases. Series with n < 10 (or with zero variance) are not tested and return NA for every test column.

tau

numeric in ⁠[-1, 1]⁠. Kendall's rank correlation between the statistic and time — the effect size. Positive means delays are growing, negative means they are shrinking. Roughly, ⁠|tau|⁠ below 0.1 is a negligible trend even when p_value is small.

sens_slope

numeric. Sen's slope: the median pairwise rate of change, expressed in delay units per period — so for weekly data with delays measured in weeks, "weeks of delay gained per week elapsed". Multiply by n for the total drift implied across the series. Unlike an OLS slope this is robust to outlying periods.

statistic

numeric. The autocorrelation-corrected Mann-Kendall Z score. Under the null it is standard normal, so ⁠|Z| > 1.96⁠ corresponds to p_value < 0.05.

p_value

numeric. Two-sided p-value for the null hypothesis of no monotonic trend, after the serial-correlation correction implied by method. NA when the series was too short or constant.

method

character. The method actually used, echoed back so the result is self-documenting when several runs are bound together.

drift

logical. The verdict: TRUE when p_value < alpha. NA p-values give FALSE, so a FALSE means "no drift detected" and not necessarily "no drift".

Interpreting the result

Read tau and sens_slope before p_value. On long surveillance series a tiny, operationally irrelevant trend will still be highly significant, so drift = TRUE on its own is not a reason to act. The question to ask is whether sens_slope * n — the total drift implied over the observed window — is large relative to the delays themselves.

The location and dispersion statistics answer different questions and can disagree, which is informative rather than contradictory:

• median drifting up, spread flat: reporting is uniformly slower.

• median flat, spread drifting up: the typical case is unaffected but the tail is getting worse — often a subset of reporting sites degrading.

• both drifting up: broad deterioration in reporting timeliness.

A detected drift means a nowcasting model fitted on a fixed delay distribution will be biased, because it is averaging over delay regimes that are not exchangeable. Consider a model with a time-varying delay, or fitting only to the recent, homogeneous stretch of data.

Because this is a trend test it will not find an abrupt one-off shift; a step change can even cancel out to a non-significant monotonic trend. Pair it with diagnose_changepoint(), which is built for exactly that case.

Choosing a method

All three options are Mann-Kendall tests that correct for the serial correlation of a delay series; they differ in what they assume about that correlation, and in cost.

"hamed-rao" (default)

Inflates the Mann-Kendall variance using all significant autocorrelation lags of the detrended ranks. It makes no AR(1) assumption, is deterministic, and is effectively instantaneous, so it is the sensible default for a routine diagnostic. Its variance correction is known to be unstable on short series — treat results with n below roughly 30 as indicative only.

"yue-pilon"

Trend-free pre-whitening, which effectively assumes the series is AR(1). That assumption is a poor fit for daily reporting delays, which carry strong day-of-week periodicity, and pre-whitening is known to remove part of the very trend being tested. Offered for comparability with the hydrology literature; rarely the right choice here.

"block-bootstrap"

Resamples contiguous blocks, so it accommodates arbitrary dependence within a block — including weekly periodicity, if the block length covers it. Statistically the most defensible for daily data, and the best cross-check when a hamed-rao result is borderline. Two caveats: it is stochastic, so call set.seed() first for a reproducible p-value, and it is thousands of times slower — it scales at roughly the square of the series length, so a multi-year daily series can take many minutes per statistic. Reduce nsim (passed through ...) or restrict to a shorter window before reaching for it.

When a decision matters, run the default first and confirm a borderline result with method = "block-bootstrap" on a restricted window.

See Also

diagnose_changepoint() for an abrupt shift rather than a gradual trend; plot_delay_drift() to see the series being tested; censor_reporting_delays_above() once you decide some delays are not to be believed; diagnose() for the structural checks, which this test deliberately sits outside of.

Examples

data(denguedat)
dengue <- tbl_now(denguedat,
  event_date = "onset_week", report_date = "report_week", verbose = FALSE
)
diagnose_drift(dengue)

Diagnostic plots of the reporting process

Description

Lays out a gallery of complementary views of a tbl_now's reporting process, all aimed at spotting reporting artefacts – especially batch reporting. Each view is also available on its own (see See also); diagnostic_plot() picks the ones named in panels and combines them with patchwork. Selecting a single panel returns it as a plain plot. Every view is facetted by stratum when the tbl_now declares strata.

Usage

diagnostic_plot(
  x,
  panels = "all",
  by = c("report", "event"),
  max_delay = NULL,
  ...,
  plotly = FALSE,
  axis = c("report", "revision"),
  size = 1,
  linewidth = 1,
  grid_linewidth = 0.3,
  palette = .tbl_now_palette()
)

Arguments

Value

A patchwork object, or a single plot when one panel is selected (or a plotly widget when plotly = TRUE).

See Also

Every panel is also a function of its own: plot_reporting_process() and plot_epidemic_process() (when reports arrived, versus when cases happened), plot_reporting_triangle() (the full event-by-delay grid), plot_delay_profiles() (each date's delay curve), plot_delay_drift() (whether delays are getting longer), plot_transport_discriminant().

Examples

data(denguedat)
# The two and a half years around the 1996 and 1997 backlog dumps: enough
# for the transport panel to have something to flag, quick enough to draw.
window <- denguedat[
  denguedat$onset_week >= as.Date("1995-06-01") &
    denguedat$onset_week <= as.Date("1998-01-01"),
]
dn <- tbl_now(window, onset_week, report_week, verbose = FALSE)
diagnostic_plot(dn, panels = c("triangle", "transport"))

Specify a nowcasting model and its arguments

Description

An engine is one modelling package plus every argument that package needs. It is what run_nowcast() and nowcast_backtest() take, and it is the object nowcast_fit() and nowcast_tidy() dispatch on.

engine() is the general constructor and works for any registered method, including one you wrote yourself. The ⁠engine_*()⁠ functions are its package-specific counterparts: each names the arguments of the modelling function it drives, so the ones that matter are visible in the signature and at ?engine_nobbs rather than buried in a ... that silently swallows a typo. Anything a named argument does not cover still goes through ....

Usage

engine(
  method,
  ...,
  min_date = NULL,
  quantile_levels = nowcast_quantile_levels(),
  label = NULL
)

Arguments

Value

An object of class c(method, "nowcast_engine").

See Also

run_nowcast(), nowcast_backtest(), list_nowcast_methods(), and the Adding your own nowcasting model article for writing a backend of your own.

Examples

engine("baselinenowcast", draws = 500)

# Built-in names are matched case-insensitively
engine("nobbs", max_D = 10)

# The same package twice, told apart by `label`
engine("diseasenowcasting", label = "default")

A toy engine for examples

Description

A deliberately naive nowcasting engine that needs no modelling package. It exists so that the examples in this package can actually run: every real engine depends on epinowcast, NobBS, EpiNow2 or another optional package, and an example that cannot run teaches nothing.

Do not nowcast with this. It does not model the reporting delay at all – it reports the counts that have arrived so far and puts a fixed percentage band around them. Because late reports are exactly what it ignores, it under-predicts recent dates by design, which is a useful thing to see and a terrible thing to rely on. For real work use one of the engines – engine_baselinenowcast(), engine_epinowcast(), engine_nobbs() and the rest – or write your own.

Usage

example_engine(
  ...,
  spread = 0.2,
  min_date = NULL,
  quantile_levels = nowcast_quantile_levels(),
  label = NULL
)

## S3 method for class 'example'
nowcast_fit(
  engine,
  x,
  ...,
  spread = 0.2,
  quantile_levels = nowcast_quantile_levels(),
  verbose = TRUE
)

## S3 method for class 'example'
nowcast_tidy(engine, fit, x, ..., quantile_levels)

Arguments

Details

For each event date (and stratum) it takes the cumulative count reported by now, from get_latest_reported_cases(), and reports that as the median. The other quantile levels are that median scaled linearly by spread, so the 2.5% and 97.5% levels sit at roughly ⁠1 -/+ spread⁠ times it.

No random numbers are involved, so it gives the same answer every time and does not disturb the RNG stream.

Value

A nowcast_engine object, as engine() returns, that run_nowcast() and nowcast_backtest() accept.

See Also

nowcast_engines for the engines you would actually nowcast with; engine() for the general constructor; nowcast_fit() and nowcast_tidy(), the two methods this implements – read its source for the shortest possible complete backend. The Adding your own nowcasting model article walks through writing a real one.

Examples

data(denguedat)
recent <- subset(denguedat, onset_week >= as.Date("2010-06-01"))
dengue <- tbl_now(recent,
  event_date = onset_week, report_date = report_week, verbose = FALSE
)

# It is an ordinary engine, so it goes where a real one goes.
example_engine()

nc <- run_nowcast(dengue, example_engine(), verbose = FALSE)
nc

## `spread` controls how wide the (made-up) interval is.
run_nowcast(dengue, example_engine(spread = 0.5), verbose = FALSE)

flusight: NHSN Weekly Hospital Respiratory Data from FluSight

Description

FluSight's weekly hospital admission prediction targets based on the 'total number of new hospital admissions of patients with confirmed influenza captured during the reporting week' reported through CDC's NHSN (the dataset formerly known as HHS-Protect), Weekly Hospital Respiratory Data. Data was downloaded on November 12th 2025.

Usage

data(flusight)

Format

A data frame with 452,567 rows and 4 variables:

as_of

Date. The date this row's value was published – the report date. The same week appears many times, once per publication.

target_end_date

Date. The week being reported on – the event date.

location_name

character. US state or territory.

observation

numeric. Hospital admissions reported for that week as of that publication date.

Details

Data represents how many cases were considered influenza during the week of target_end_date given the information known until week as_of. Note that as_of is always one week ahead of target_end_date.

This is count data with 452,567 rows and 4 columns:

• as_of: The report date – the date the snapshot was taken, i.e. what was known as of that week.

• target_end_date: The event date – the week the admissions occurred.

• location_name: State, district or territory (53 levels).

• observation: Case counts for those dates. NA for 1,152 rows.

Together, as_of, target_end_date and location_name form a unique key.

Duplicate rows removed

The upstream FluSight time-series.csv ships exact duplicate rows – 39,139 of them in this snapshot. They were dropped with dplyr::distinct() before the dataset was saved (issue #25), taking it from 491,706 to 452,567 rows.

The removal is lossless: every repeated (as_of, target_end_date, location_name) key carried an identical observation, with no conflicting values anywhere in the file, so no information was discarded and the key became unique. If you download the upstream file yourself you will still see the duplicates and should distinct() them before use.

References

Target data from Flusight. Online: https://github.com/cdcepi/FluSight-forecast-hub/blob/main/target-data/time-series.csv

See Also

tbl_now() to declare the date columns; summary() and diagnose() to inspect the result; the package's other datasets – denguedat, mpoxdat, flusight, covid_colombia, covid_us and hai_bucaramanga.

Examples

data(flusight)
head(flusight)

## This is count data: one row per (week, publication date, state).
nrow(flusight)
length(unique(flusight$location_name))

# One state is enough to see the reporting process.
texas <- flusight[flusight$location_name == "Texas", ]
flu <- tbl_now(texas,
  event_date = target_end_date, report_date = as_of,
  case_count = observation, verbose = FALSE
)
flu

# `as_of` is not always the same weekday, so some delays are not whole weeks.
## `align_weeks()` fixes that.
mean(flu$.delay != round(flu$.delay))

Cases at a chosen point in the reporting process

Description

The same event date has more than one count, depending on when you look. A week of dengue onsets might show 12 cases the day reporting starts, 40 a week later, and 47 once everything has arrived. These functions let you pick which of those numbers you want.

Usage

get_latest_reported_cases(x, type = "total")

get_initial_reported_cases(x, type = "total")

get_nth_reported_cases(x, delay, type = "total")

Arguments

Details

• get_initial_reported_cases() – the count as first seen: the earliest report for that event date. This is what a dashboard would have shown you at the time, and it is always an undercount.

• get_latest_reported_cases() – the count as latest seen: the most recent report. This is the current best estimate of what really happened, and it is what you score a nowcast against.

• get_nth_reported_cases() – the count accumulated within a given delay. delay = 0 gives the cases reported on the event date itself, delay = 1 adds those reported one period later, and so on. delay = Inf is the same as get_latest_reported_cases().

The gap between the first and the latest count is the reporting delay problem that nowcasting exists to solve.

Value

A count-cumulative tbl_now with one row per event date (and stratum, and grouping column), containing:

• the event-date column – when the cases happened. Its numeric version is .event_num.

• the report-date column – the report that was selected for that event date. Its numeric version is .report_num.

• n – the number of cases reported for that event date at the selected point.

• .delay – the delay of the selected report.

• any strata, covariate, censoring indicator and temporal-effect columns the object carried, plus the caller's grouping columns.

The revision columns are not carried: the count pools over many revision dates, so the result has no single one and does not pretend to. type = "by_type" is the exception – it keeps the outcome column, declared as a covariate, because that is the whole point of the call and an undeclared column is one to_count() would pool away. Use get_latest_revised_cases() when you want the third date on the result.

Grouping is respected

Unlike to_count(), these functions keep the caller's grouping and answer by it: the grouping columns join the event date and the strata as keys, and come back on the result. That is what lets you ask for the latest count by a covariate – a column that matters but is not something you nowcast by – which grouping is the only way to express.

They can do this because they select a point in the process rather than reshaping the object: one row in is still one case (or one cell) out. to_count() cannot, and warns instead.

See Also

get_latest_revised_cases() and friends for the same idea on the revision process; to_count() for the underlying data shapes; score_nowcast(), which uses the latest counts as truth.

Examples

data(denguedat)
# The last five years. The counters work the same on the full twenty-year
# series, they just have more weeks to walk.
recent <- denguedat[denguedat$onset_week >= as.Date("2006-01-01"), ]
dengue <- tbl_now(recent,
  report_date = "report_week",
  event_date = "onset_week",
  strata = "gender",
  verbose = FALSE
)

# What the surveillance system showed the very first time it reported each
# week -- an undercount, because the late reports had not arrived yet.
first <- get_initial_reported_cases(dengue)
first

# What it shows now, after all the corrections.
latest <- get_latest_reported_cases(dengue)
latest

# The difference between them is what a nowcast tries to predict.
sum(latest$n) - sum(first$n)

# Everything known within two weeks of onset.
get_nth_reported_cases(dengue, delay = 2)

# A grouping is answered by, not dropped.
dengue |>
  dplyr::group_by(gender) |>
  get_latest_reported_cases() |>
  dplyr::group_vars()

Healthcare-Associated Infections – Bucaramanga, Colombia 2020-2023

Description

A line list of healthcare-associated infections (IAAS, Infecciones Asociadas a la Atencion en Salud) notified in the municipality of Bucaramanga, Santander, Colombia, between January 2020 and January 2023 as reported by March 19th 2026. Each row is one notified infection: a specimen taken from a hospitalised patient, the laboratory result, and the isolated microorganism.

Usage

hai_bucaramanga

Format

A tibble with 1,423 rows and 13 variables:

id

integer. The source's Orden autonumber. Documented as unique but repeated for 100 records (see above), so it is not a reliable key.

specimen_date

Date. The event date – when the sample was taken from the patient. NA for 318 records.

received_date

Date. When the laboratory received the sample. NA for 1,194 records and absent after 2019-12-01.

report_date

Date. The report date – when the laboratory result was issued. NA for 583 records.

specimen

factor, 6 levels. Sample type: "Whole blood", "Urine", "Secretions", "Other sterile fluids", "Sputum", "Bronchoalveolar lavage". NA for clinically-confirmed cases.

test

factor, 7 levels. Laboratory test performed, e.g. "Blood culture", "Urine culture". NA for clinically-confirmed cases.

microorganism

character, 85 distinct values. The isolated organism, in conventional binomial form, e.g. "Klebsiella pneumoniae". NA for clinically-confirmed cases. Where the source recorded only a genus the value is "<Genus> spp.".

sex

factor. "Female" or "Male".

age_group

⁠ordered factor⁠, 11 levels from "<1" to "70+".

case_type

factor. "Laboratory-confirmed" (1,090) or "Clinically-confirmed" (333).

final_condition

factor. Patient status on discharge: "Alive" or "Dead". NA for 8 records.

icu_type

factor. Intensive-care unit where the case occurred: "Adult", "Paediatric" or "Neonatal".

institution

integer, 1-10. Anonymised code of the reporting health institution.

Details

In the nowcasting context the event date is specimen_date (when the sample was taken from the patient) and the report date is report_date (when the laboratory result was issued). The delay between the two is the specimen-to-result turnaround that a nowcasting model would estimate and correct for. A third date, received_date, records when the laboratory received the sample and splits the delay into a transport and a processing leg – but see the warnings below before relying on it.

The column names and all categorical values have been translated from the original Spanish. The dataset has been trimmed to the columns relevant to a delay or nowcasting analysis; see data-raw/hai_bucaramanga.R in the package sources for the full translation tables and the list of dropped columns.

Data quality – read this first

This is a real, unpolished open-data extract, and it is included partly because it is messy: it is a realistic exercise for the delay diagnostics in this package (diagnose_drift(), diagnose_changepoint(), plot_delay_profiles()). Nothing below has been silently repaired.

Missing dates

The source uses 1900-01-01 as an undocumented missing-date sentinel. It has been converted to NA here – left in, it produces delays of about -45,000 days. After conversion specimen_date is present for 1,105 records (77.7%), report_date for 840 (59.0%) and received_date for only 229 (16.1%).

received_date is largely unusable

Beyond being 84% missing, it stops entirely at 2019-12-01, so the three-date chain exists only for the first third of the study period. Prefer the specimen_date -> report_date pair.

Negative delays

88 records (10.7% of the otherwise-valid pairs) have a report_date before their specimen_date, by up to 331 days. These are kept exactly as recorded. Filter them out before fitting anything.

Exact duplicates

100 records are byte-identical duplicates of another record, sharing the id that the source documents as a unique autonumber. All 1,423 rows are shipped for fidelity; use dplyr::distinct() to reduce to the 1,323 unique records.

Clinically-confirmed cases

For the 333 cases confirmed on clinical grounds rather than by laboratory, the source wrote the literal string "CONFIRMADO POR CLINICA" into the muestra, ⁠nombre prueba⁠ and microorganismo fields. Those are NA here; the information is preserved in case_type.

Sparsity

Only 738 records (51.9%) support a non-negative specimen_date -> report_date delay, spread over 488 distinct event dates – roughly 1.5 cases per event date. That is thin for fitting a nowcasting model, though ample for delay diagnostics. Aggregating to weeks or months is usually necessary.

The delay distribution is strongly bimodal: the median is 3 days but the 90th percentile is 92 days, which makes this a useful test case for delay diagnostics that assume a unimodal delay.

Source

Secretaria de Salud y Ambiente de Bucaramanga, via the Colombian national open-data portal https://www.datos.gov.co/Salud-y-Protecci-n-Social/48-Infecciones-asociadas-a-la-atenci-n-en-salud-IA/w4zx-wbff/about_data. Retrieved 2026-08-17. Column names and categorical values translated from Spanish; see data-raw/hai_bucaramanga.R for the mapping.

See Also

tbl_now() to declare the date columns, and add_revision_date() for the third one this dataset has; diagnose() and diagnose_drift(), which this dataset is deliberately messy enough to exercise; the package's other datasets – denguedat, mpoxdat, flusight, covid_colombia and covid_us.

Examples

data(hai_bucaramanga)

# The source ships 100 exact duplicates and a number of unusable rows.
# Reduce to unique records with a valid, non-negative reporting delay.
iaas_clean <- dplyr::distinct(hai_bucaramanga) |>
  dplyr::filter(
    !is.na(specimen_date), !is.na(report_date),
    report_date >= specimen_date
  )
nrow(iaas_clean)

# Roughly 1.5 cases per event date, so aggregate to weeks before building a
# tbl_now for anything model-shaped.
iaas_now <- tbl_now(
  iaas_clean,
  event_date  = "specimen_date",
  report_date = "report_date",
  verbose     = FALSE
)
iaas_now

# The delay is strongly bimodal: a 3-day median with a long secondary mode.
quantile(iaas_now$.delay, c(0.5, 0.75, 0.9, 0.99), na.rm = TRUE)

Is this an engine?

Description

Tests whether an object is a nowcasting engine – the specification built by engine() that says which modelling package to use and how to configure it. A bare package name is not an engine, which is what this is usually used to check.

Usage

is_nowcast_engine(x)

Arguments

Value

A single TRUE or FALSE.

See Also

engine() to build one; nowcast_engines for the engines that ship with the package; run_nowcast(), which takes one.

Examples

## Built by engine(): yes.
is_nowcast_engine(engine("baselinenowcast"))

# The name of a package on its own: no.
is_nowcast_engine("baselinenowcast")

Is this object a tbl_nowcast?

Description

Tests whether an object is a fitted nowcast – the thing run_nowcast() returns – rather than the data a nowcast is fitted to (for which see is_tbl_now()).

Usage

is_tbl_nowcast(x)

Arguments

Value

A single TRUE when x is a tbl_nowcast, FALSE otherwise.

See Also

tbl_nowcast for the class itself; run_nowcast() which produces one; is_tbl_now() for the input side.

Examples

# A number is not a nowcast.
is_tbl_nowcast(1)

## The object run_nowcast() returns is.
predictions <- data.frame(
  onset_week = as.Date("2020-01-05"),
  .quantile_level = c(0.5, 0.9), .value = c(10, 14)
)
nc <- tbl_nowcast(
  predictions = predictions, method = "toy", event_date = "onset_week"
)
is_tbl_nowcast(nc)

Is a date a weekday or a weekend?

Description

Reporting almost always slows down at the weekend, which is one of the strongest and most predictable patterns in surveillance data. This tells you which days are which, and lets you say what "weekend" means – it is Friday and Saturday in much of the Middle East, and Sunday alone in some countries.

Usage

is_weekday(date, weekend_days = c("Sat", "Sun"))

Arguments

Value

A logical vector, TRUE where the date is a weekday and FALSE where it falls on the weekend.

See Also

temporal_effects() and add_temporal_effects(), which use this to build the day-of-week and weekend terms a model can fit; plot_day_of_week_effects() to see the effect in the data; align_weeks(), whose align_on_day uses this same ISO numbering.

Examples

is_weekday(as.Date("2020-04-22")) # TRUE (Wed)
is_weekday(as.Date("2020-04-19")) # FALSE (Sun)

## Middle East weekend (Fri - Sat)
is_weekday(as.Date("2020-04-17"), weekend_days = c("Fri", "Sat"))

# Weekend only on Friday
is_weekday(as.Date("2020-04-17"), weekend_days = "Friday")
is_weekday(as.Date("2020-04-18"), weekend_days = "Friday")

## Weekend on Sun - Mon (numeric: 7 = Sun, 1 = Mon)
is_weekday(as.Date("2020-04-20"), weekend_days = c(7, 1))

## Day names of the session's own locale are understood too
locale_weekend <- format(as.Date(c("2020-04-18", "2020-04-19")), "%a")
is_weekday(as.Date("2020-04-18"), weekend_days = locale_weekend)

List the available nowcasting methods

Description

Scans the S3 methods registered for nowcast_fit() in every loaded namespace, so any backend you (or another package) defined shows up here as soon as it is loaded.

"example" in the list is example_engine(), the toy used to keep this package's examples runnable. It is not a nowcasting method – ignore it when choosing one.

Usage

list_nowcast_methods(installed_only = TRUE)

Arguments

Value

A character vector of method names, suitable for passing to engine().

See Also

engine() to turn one of these names into a configured model; nowcast_engines for the engines that ship with the package; nowcast_fit() and nowcast_tidy(), the two functions a new method must define. The Adding your own nowcasting model article walks through writing one.

Examples

## What can this installation actually fit right now? ("example" is the toy
# engine, not a method you would nowcast with.)
list_nowcast_methods()

# Including methods whose modelling package is not installed
list_nowcast_methods(installed_only = FALSE)

mpoxdat: Mpox reporting data from the 2022 New York City outbreak

Description

Surveillance line list data provided by the New York City (NYC) Health Department at https://github.com/nychealth/mpox_nowcast_eval, to accompany a nowcasting performance evaluation (doi: 10.2196/56495). Patients with a confirmed or probable mpox diagnosis or illness onset from July 8 through September 30, 2022 were included. The original dataset was aggregated and pre-processed as described in the note below.

Usage

data(mpoxdat)

Format

A data frame with 1,417 rows and 4 variables:

dx_date

Date. Specimen collection date of the first positive result – the event date.

dx_report_date

Date. When the Health Department received that result – the report date.

race

character. Synthetic; see the note.

n

integer. Number of cases with that combination.

Details

This is count data: each row holds the number of cases sharing a diagnosis date, a report date and a race. The columns are as follows:

• dx_date: is the specimen collection date of the first positive mpox laboratory result,

• dx_report_date: is the date the report of first positive mpox laboratory result was received by the NYC Health Department,

• n: the case count of individuals within those dates.

• race: the race corresponding to those cases. Race was randomly assigned with probabilities "Non-Hispanic White" = 0.309, "Hispanic" = 0.283, "Black" = 0.202, "Asian" = 0.156, and "Other" = 0.05 which follow what has been reported for the US Census.

Note

While dx_date, dx_report_date and n correspond to actual observed data the race was constructed exclusively for the examples of this package. It is a synthetic (simulated) variable and does not correspond to any reality.

References

ROHRER, Rebecca, et al. Nowcasting to Monitor Real-Time Mpox Trends During the 2022 Outbreak in New York City: Evaluation Using Reportable Disease Data Stratified by Race or Ethnicity. Online Journal of Public Health Informatics, 2025, vol. 17, no 1, p. e56495.

See Also

tbl_now() to declare the date columns; summary() and diagnose() to inspect the result; the package's other datasets – denguedat, mpoxdat, flusight, covid_colombia, covid_us and hai_bucaramanga.

Examples

data(mpoxdat)
head(mpoxdat)

# Count data, and daily rather than weekly -- unlike denguedat.
mpox <- tbl_now(mpoxdat,
  event_date = dx_date, report_date = dx_report_date,
  case_count = n, strata = race, verbose = FALSE
)
mpox

# A short, sharp outbreak: about three months of data.
range(mpoxdat$dx_date)
sum(mpoxdat$n)

Refit several methods at past now dates and score them

Description

Walks back through time: for every date in now_dates, the tbl_now is truncated to the reports that were available then, each method is refitted on that snapshot, and the resulting nowcast is scored against the resolved truth defined by truth_axis and truth_type (reported totals by default). This is what turns a set of models into ensemble weights (see nowcast_weights() and nowcast_ensemble()).

Be aware that this refits every model once per date: with Bayesian backends and a long now_dates it is genuinely expensive.

Usage

nowcast_backtest(
  x,
  ...,
  now_dates = NULL,
  horizon = 4,
  n_dates = 4L,
  seed = NULL,
  keep_draws = FALSE,
  on_error = c("warn", "abort"),
  verbose = TRUE,
  truth_axis = c("report", "revision"),
  truth_type = "total"
)

Arguments

Value

An object of class nowcast_backtest: a list with

scores

A tibble of per-date scores with an extra .now column.

predictions

A tibble of every retrospective quantile prediction.

draws

When keep_draws = TRUE, a tibble of the retained draws; otherwise NULL.

timings

A tibble with one row per attempted engine/date fit, its elapsed time in seconds, whether it succeeded, and any error text.

truth

The observed counts used for scoring.

methods

The labels that produced at least one nowcast.

now_dates

The dates that were nowcast.

Use the result directly with scoringutils

A nowcast_backtest has methods for scoringutils::as_forecast_quantile(), scoringutils::as_forecast_point(), and scoringutils::as_forecast_sample(), so no manual reshaping is needed:

quantile_forecast <- scoringutils::as_forecast_quantile(bt)
point_forecast <- scoringutils::as_forecast_point(bt)

For sample forecasts, create the backtest with keep_draws = TRUE and use scoringutils::as_forecast_sample(bt). The returned forecast objects can be passed to any compatible scoringutils workflow. For example, relative WIS is obtained with:

relative_scores <- quantile_forecast |>
  scoringutils::score() |>
  scoringutils::add_relative_skill(metric = "wis")

model, now, the event-date column, and declared strata are retained as forecast units, allowing scores to be extended, regrouped, or summarised without returning to the internal tbl.now representation.

Every engine must report the same quantile levels

A backtest exists to compare models, and two models summarised at different levels are not comparable: the weighted interval score is an average over the levels reported, so a model asked for three of them and one asked for nine are scoring different quantities. Mismatched engines are therefore an error rather than a warning.

This matters most for the engines where the levels are a fit-time argument. NobBS computes exactly the quantiles it is handed and keeps no draws, so a level it was never asked for cannot be recovered afterwards – and an ensemble weighted from such a backtest would silently fall back to whatever levels its members happened to share.

See Also

engine() to specify each model being compared, and its min_date argument, which matters here because now moves between fits; score_nowcast() for the scores computed at each now; nowcast_weights() to turn the result into ensemble weights, and nowcast_ensemble() to use them; scoringutils::score() and scoringutils::add_relative_skill() for an extensible scoring workflow. The One call, many models article compares several packages this way.

Examples

data(denguedat)

# A short recent window keeps the example quick.
recent <- subset(denguedat, onset_week >= as.Date("2010-06-01"))
dengue <- tbl_now(recent,
  event_date = onset_week, report_date = report_week, verbose = FALSE
)

## `example_engine()` is a toy that ignores the reporting delay entirely; it
# is used here only so the example runs without a modelling package.
## Swap in a real one -- `engine_baselinenowcast()`, `engine_epinowcast()`,
## `engine_nobbs()` -- for anything you intend to act on.

# Refit at two past `now` dates and score each against what is known now.
bt <- nowcast_backtest(dengue,
  example_engine(label = "carry forward"),
  now_dates = as.Date(c("2010-10-04", "2010-11-15")),
  verbose = FALSE
)
head(bt$scores)

# Naming several engines compares them on identical data and dates.
bt$methods

# With a real model the call is the same, with a real engine.
if (requireNamespace("baselinenowcast", quietly = TRUE)) {
  nowcast_backtest(dengue,
    engine_baselinenowcast(draws = 100),
    now_dates = as.Date("2010-11-15"), verbose = FALSE
  )$scores
}

Read what a tbl_now was told about itself

Description

When you build a tbl_now() you tell it which column is the event date, which is the report date, which are strata, and so on. These functions read that information back.

They are how the rest of the package – and any modelling code you write yourself – finds the right columns without hard-coding names. Rather than assuming a column is called onset_week, write x[[get_event_date(x)]] and your code works on any tbl_now.

Usage

get_event_date(x)

get_report_date(x)

get_strata(x)

get_num_strata(x)

get_covariates(x)

get_num_covariates(x)

get_now(x)

get_report_units(x)

get_event_units(x)

get_data_type(x)

get_temporal_effects(x)

get_temporal_effect_cols(x)

get_is_censored_report(x)

get_case_count(x)

get_revision_date(x)

get_revision_type(x)

get_revision_units(x)

get_is_censored_revision(x)

get_revision_levels(x)

has_revision(x)

Arguments

Details

Most of these return a column name, not the column itself. To get the values, index with the name: x[[get_event_date(x)]].

A getter returns NULL when the object was never told about that attribute, so is.null(get_strata(x)) is the test for "unstratified". The two counting helpers, get_num_strata() and get_num_covariates(), return 0 instead, which is usually easier to work with.

Value

A column name, a count, or a metadata value, depending on the function:

get_event_date(), get_report_date()

Character. The name of the column holding the date the event happened / was reported.

get_case_count()

Character, or NULL for linelist data. The name of the column holding the number of cases.

get_strata(), get_covariates()

Character vector of column names, or NULL when there are none.

get_num_strata(), get_num_covariates()

Integer count, 0 when there are none.

get_is_censored_report()

Character, or NULL. The name of the column flagging reports whose date is only an upper bound.

get_is_censored_revision()

Character, or NULL. The same on the revision axis: the column flagging rows whose revision delay is a bound rather than a measurement.

get_now()

The Date (or number) the nowcast is anchored on.

get_event_units(), get_report_units()

One of "days", "weeks", "months", "years" or "numeric" – the grid each date lives on.

get_data_type()

One of "linelist", "count-incidence" or "count-cumulative". See to_count().

get_temporal_effects()

The temporal_effects() specification the object carries, or NULL. This is the request, not the data.

get_temporal_effect_cols()

Character vector of the temporal-effect columns actually materialised in the data by compute_temporal_effects(); character(0) when none have been.

get_revision_date(), get_revision_type()

Character, or NULL. The name of the column holding the date a case was resolved, and of the column holding how it resolved.

get_revision_units()

The grid the revision date lives on, or NULL when the object carries no revision process.

get_revision_levels()

The named dictionary translating the labels in the data into the canonical outcomes, or NULL when the column was already canonical.

has_revision()

TRUE when the object carries a revision date. Every code path must work when it is FALSE, because most objects have no third date.

The revision process, the optional third date

A tbl_now may carry a third date beyond the event and the report: the date a case was resolved, either confirmed or retracted. Think of influenza: symptom onset is the event, the medical visit is the report, and the laboratory result is the revision – which can come back negative, in which case the case is retracted rather than confirmed.

It is optional and most objects do not have one, so has_revision() gates the four getters below it: they all return NULL on an object that was never given a third date.

See Also

tbl_now_attributes() to get all of them at once; add(), change() and remove() to set them, including add_revision_date(); get_latest_revised_cases() and get_latest_revised_cases(type = "net") to count the outcomes; revision_delay for how long resolution takes; get_latest_reported_cases() and friends for reading the counts rather than the metadata.

Examples

data(denguedat)
ndata <- denguedat |>
  tbl_now(
    event_date = onset_week,
    report_date = report_week,
    strata = gender,
    t_effects = temporal_effects(month_of_year = TRUE),
    verbose = FALSE
  ) |>
  compute_temporal_effects()

# The two dates every nowcast needs.
get_event_date(ndata)
get_report_date(ndata)

# Use the name to reach the column, so the code does not depend on it.
head(ndata[[get_event_date(ndata)]])

# Strata are groups you want separate nowcasts for; covariates are not.
get_strata(ndata)
get_num_strata(ndata)

## Nothing was declared a covariate, so this is NULL (and the count is 0).
get_covariates(ndata)
get_num_covariates(ndata)

# Likewise for a censoring indicator that was never supplied.
get_is_censored_report(ndata)
get_is_censored_revision(ndata)

# The as-of moment, and the calendar grid the dates live on.
get_now(ndata)
get_event_units(ndata)
get_report_units(ndata)

# Linelist means one row per case; there is no count column yet.
get_data_type(ndata)
get_case_count(ndata)

## After to_count() there is one.
counts <- to_count(ndata, to = "count-incidence")
get_data_type(counts)
get_case_count(counts)

# The temporal-effects request, versus the columns it actually produced.
get_temporal_effects(ndata)
get_temporal_effect_cols(ndata)

# The third date is optional, so ask before you read it.
has_revision(ndata)
get_revision_date(ndata)

## Once one is attached, the same name-then-index pattern applies.
data(hai_bucaramanga)
hai <- hai_bucaramanga |>
  dplyr::filter(!is.na(specimen_date), !is.na(report_date)) |>
  tbl_now(
    event_date = specimen_date, report_date = report_date,
    data_type = "linelist", verbose = FALSE
  ) |>
  add_revision_date(received_date) |>
  suppressWarnings()

has_revision(hai)
get_revision_date(hai)
get_revision_units(hai)
head(hai[[get_revision_date(hai)]])

Individual blocks of a tbl_now diagnosis

Description

Each function returns one block of diagnose(), in the same schema, so they can be stacked with dplyr::bind_rows() or used on their own.

• diagnose_declarations() – the attributes and the columns they name: types, existence, collisions, columns the object was never told about, and temporal effects that were added but never materialised.

• diagnose_ordering() – the ⁠event <= report <= revision⁠ timeline.

• diagnose_missing() – NA values, per column and per stratum. An NA count is reported neutrally: in a reporting triangle it means not yet observed, which is correct data rather than a defect.

• diagnose_duplicates() – rows that repeat on the full key: the declared dates, the revision type, the strata, the covariates and the censoring flags. A row with an NA in that key is not compared, because unobserved cannot be shown to equal unobserved.

• diagnose_units() – the declared units against each other, against the calendar the dates actually land on, and against the delay they produce.

• diagnose_negatives() – negative counts, and the negative increments a downward revision leaves behind when cumulative data is de-accumulated.

• diagnose_now() – anything dated after now, and how stale the object is.

• diagnose_truncation() – how many recent event dates are still immature, and how much of their eventual total is probably still missing.

• diagnose_strata() – the smallest and the sparsest stratum, and the revisions still pending.

Usage

diagnose_declarations(x, by_strata = NULL, strata = NULL)

diagnose_ordering(x, by_strata = NULL, strata = NULL)

diagnose_missing(x, by_strata = NULL, strata = NULL)

diagnose_duplicates(
  x,
  by_strata = NULL,
  strata = NULL,
  warn_non_uniqueness = TRUE
)

diagnose_units(x, by_strata = NULL, strata = NULL)

diagnose_negatives(x, by_strata = NULL, strata = NULL)

diagnose_now(x, by_strata = NULL, strata = NULL)

diagnose_truncation(x, by_strata = NULL, strata = NULL)

diagnose_strata(x, by_strata = NULL, strata = NULL)

Arguments

Value

A tibble in the schema documented in diagnose().

See Also

diagnose(), which stacks all of these and sorts them worst-first; validate_tbl_now() for the same findings raised as errors and warnings; nowcast_summary_components for what is in the data rather than what is wrong with it; diagnose_drift(), diagnose_changepoint() and diagnose_batches() for the statistical tests diagnose() deliberately does not run. The Diagnosing a tbl_now article explains how to read each finding.

Examples

data(denguedat)
# The last five years. The full twenty-year series gives the same shape
# of answer, it just takes longer to compute.
recent <- denguedat[denguedat$onset_week >= as.Date("2006-01-01"), ]
ndata <- tbl_now(recent,
  event_date = "onset_week",
  report_date = "report_week",
  strata = "gender",
  verbose = FALSE
)

# Is the object described correctly, and do the dates make sense?
diagnose_declarations(ndata)
diagnose_ordering(ndata)
diagnose_units(ndata)
diagnose_now(ndata)

# Is anything missing, repeated, negative, or cut off at the recent edge?
diagnose_missing(ndata)
diagnose_duplicates(ndata)
diagnose_negatives(ndata)
diagnose_truncation(ndata)

# Are the strata usable?
diagnose_strata(ndata)

## Each returns the same schema, so they stack the way diagnose() stacks them.
dplyr::bind_rows(
  diagnose_units(ndata),
  diagnose_now(ndata)
)

Engines for the built-in nowcasting packages

Description

One constructor per supported modelling package. Each is engine() with the arguments of that package's own entry point spelled out, so the ones that matter are visible in the signature and a typo is an error rather than a silently ignored extra.

The One dataset, many nowcasts article documents each package's own API; this page is about driving it through run_nowcast().

Usage

engine_diseasenowcasting(
  ...,
  model = NULL,
  type = NULL,
  n_draws = NULL,
  min_date = NULL,
  quantile_levels = nowcast_quantile_levels(),
  label = NULL
)

engine_baselinenowcast(
  ...,
  draws = 1000,
  delays_unit = NULL,
  max_delay = NULL,
  strata_sharing = "none",
  min_date = NULL,
  quantile_levels = nowcast_quantile_levels(),
  label = NULL
)

engine_epinowcast(
  ...,
  preprocess_args = list(),
  expectation = NULL,
  reference = NULL,
  report = NULL,
  fit = NULL,
  min_date = NULL,
  quantile_levels = nowcast_quantile_levels(),
  label = NULL
)

engine_nobbs(
  ...,
  max_D = NULL,
  moving_window = NULL,
  specs = NULL,
  min_date = NULL,
  quantile_levels = nowcast_quantile_levels(),
  label = NULL
)

engine_surveillance(
  ...,
  D = NULL,
  when = NULL,
  fit_method = NULL,
  control = NULL,
  min_date = NULL,
  quantile_levels = nowcast_quantile_levels(),
  label = NULL
)

engine_epinow2(
  ...,
  generation_time = NULL,
  delays = NULL,
  truncation = NULL,
  rt = NULL,
  obs = NULL,
  stan = NULL,
  convert_args = list(),
  min_date = NULL,
  quantile_levels = nowcast_quantile_levels(),
  label = NULL
)

Arguments

Value

A nowcast_engine, as engine() returns.

See Also

engine(), run_nowcast(), nowcast_backtest()

Examples

engine_baselinenowcast(draws = 500)
engine_nobbs(max_D = 10, moving_window = 64)

# Fit epinowcast on the last 180 periods only; it scales with the number of
# reference dates, while the two engines below take the whole series.
engine_epinowcast(preprocess_args = list(max_delay = 30), min_date = 180)
engine_baselinenowcast()
engine_diseasenowcasting()

Combine several nowcasts into an ensemble

Description

Takes the nowcasts produced by different modelling packages on the same tbl_now and combines them into a single tbl_nowcast. Ensembles are routinely better calibrated than any of their members, and because run_nowcast() puts every backend on the same tidy footing, combining them needs no reshaping on your side.

Usage

nowcast_ensemble(
  ...,
  type = c("quantile", "linear_pool"),
  weights = "equal",
  backtest = NULL,
  include_now = FALSE,
  quantile_levels = NULL,
  n_draws = 4000L,
  name = "ensemble",
  verbose = TRUE
)

Arguments

Details

An ensemble assumes that all members predict the same epidemiological quantity. The target dates, strata and quantile levels are checked here, but reporting-versus-revision semantics are currently a modelling convention: combine members that target the same quantity, and score the result with the matching truth_axis and truth_type in score_nowcast().

Value

A tbl_nowcast whose fit property is the list of member nowcasts and whose metadata holds the weights and the combination type.

See Also

run_nowcast() to produce the nowcasts being combined; nowcast_backtest() and nowcast_weights() to decide how much to trust each one, instead of weighting them equally; score_nowcast() to check the ensemble beats its members. The One call, many models article builds one end to end.

Examples

toy <- function(method, shift) {
  predictions <- data.frame(
    onset_week = as.Date("2020-01-05"),
    .quantile_level = c(0.25, 0.5, 0.75),
    .value = c(8, 10, 13) + shift
  )
  tbl_nowcast(predictions = predictions, method = method, event_date = "onset_week")
}

nowcast_ensemble(toy("a", 0), toy("b", 4), verbose = FALSE)

# Unequal weights
nowcast_ensemble(toy("a", 0), toy("b", 4), weights = c(a = 0.75, b = 0.25), verbose = FALSE)

Fit a nowcast with one modelling package

Description

nowcast_fit() and nowcast_tidy() are the two extension points of the nowcasting framework. Together they teach run_nowcast() about a new modelling package: nowcast_fit() runs the model, nowcast_tidy() turns whatever it returned into the tidy quantile format every other function in tbl.now understands.

Dispatch happens on the object built by engine(), so a method for "mypackage" is a function called nowcast_fit.mypackage(). It can live in any package.

Usage

## S3 method for class 'diseasenowcasting'
nowcast_fit(
  engine,
  x,
  ...,
  quantile_levels = nowcast_quantile_levels(),
  verbose = TRUE
)

## S3 method for class 'baselinenowcast'
nowcast_fit(
  engine,
  x,
  ...,
  draws = 1000,
  delays_unit = NULL,
  max_delay = NULL,
  strata_sharing = "none",
  quantile_levels = nowcast_quantile_levels(),
  verbose = TRUE
)

## S3 method for class 'epinowcast'
nowcast_fit(
  engine,
  x,
  ...,
  preprocess_args = list(),
  quantile_levels = nowcast_quantile_levels(),
  verbose = TRUE
)

## S3 method for class 'NobBS'
nowcast_fit(
  engine,
  x,
  ...,
  specs = list(),
  quantile_levels = nowcast_quantile_levels(),
  verbose = TRUE
)

## S3 method for class 'surveillance'
nowcast_fit(
  engine,
  x,
  ...,
  when = NULL,
  D = NULL,
  fit_method = "bayes.notrunc.bnb",
  control = list(),
  quantile_levels = nowcast_quantile_levels(),
  verbose = TRUE
)

## S3 method for class 'EpiNow2'
nowcast_fit(
  engine,
  x,
  ...,
  convert_args = list(),
  quantile_levels = nowcast_quantile_levels(),
  verbose = TRUE
)

nowcast_fit(
  engine,
  x,
  ...,
  quantile_levels = nowcast_quantile_levels(),
  verbose = TRUE
)

## Default S3 method:
nowcast_fit(
  engine,
  x,
  ...,
  quantile_levels = nowcast_quantile_levels(),
  verbose = TRUE
)

Arguments

Value

nowcast_fit() returns the modelling package's own object, verbatim. It is stored in the fit property of the resulting tbl_nowcast, and it is the only thing nowcast_tidy() is given besides the tbl_now itself, so put whatever the tidying step will need into it.

See Also

nowcast_tidy(), run_nowcast(), list_nowcast_methods() and the Adding your own nowcasting model article for a worked example of a new backend.

Examples

# A minimal backend: two S3 methods and you are done.
nowcast_fit.constant <- function(engine, x, ..., quantile_levels, verbose = TRUE) {
  counts <- get_latest_reported_cases(x)
  list(dates = counts[[get_event_date(x)]], value = counts[[ncol(counts)]])
}

nowcast_tidy.constant <- function(engine, fit, x, ..., quantile_levels) {
  predictions <- tidyr::expand_grid(
    event_date = fit$dates, .quantile_level = quantile_levels
  )
  predictions$.value <- rep(fit$value, each = length(quantile_levels))
  names(predictions)[1] <- get_event_date(x)
  list(predictions = predictions, draws = NULL)
}

Default quantile levels for a nowcast

Description

The quantile levels run_nowcast() summarises a nowcast at by default: nine probabilities, symmetric about the median, spanning the 50%, 80%, 90% and 95% central intervals.

They are a subset of the 23 levels the US and European COVID-19 forecast hubs and FluSight ask for (0.01, 0.025, 0.05, then 0.10 to 0.90 in steps of 0.05, then 0.975 and 0.99). Nine cover the intervals people actually read at a fraction of the storage, and every one of them is a hub level, so the output still scores against hub submissions in scoringutils without interpolation. Pass quantile_levels explicitly when you need the full hub set:

hub_levels <- c(0.01, 0.025, seq(0.05, 0.95, by = 0.05), 0.975, 0.99)
run_nowcast(x, engine("baselinenowcast", quantile_levels = hub_levels))

The levels live on the engine(), not on run_nowcast(), because for some backends they are a fit-time model argument rather than a way of summarising afterwards.

Backends that expose draws can honour any levels you ask for. Ones that report a point estimate and a single interval ("surveillance", "EpiNow2") cannot, and say so rather than interpolating.

Usage

nowcast_quantile_levels()

Value

A numeric vector of nine probabilities in ⁠(0, 1)⁠, sorted increasingly.

See Also

engine(), whose quantile_levels argument this is the default for; run_nowcast() and nowcast_backtest(), which report at these levels; score_nowcast() and scoringutils::as_forecast_point(), which score them.

Examples

nowcast_quantile_levels()

# The 50%, 80%, 90% and 95% central intervals, as lower/upper pairs.
matrix(nowcast_quantile_levels()[-5], ncol = 2)

# Ask an engine for something else -- here the full forecast-hub set.
hub_levels <- c(0.01, 0.025, seq(0.05, 0.95, by = 0.05), 0.975, 0.99)
engine("baselinenowcast", quantile_levels = hub_levels)

Individual blocks of a tbl_now summary

Description

summary() answers a dozen questions about a tbl_now at once. When you only want one of them – for a report, a dashboard, or a check inside a script – call that block directly instead of computing the rest and filtering it away.

Every one of these returns the same schema as summary() itself, so they can be stacked with dplyr::bind_rows(), compared across datasets, or used alone.

• cases_per_date() – case counts per date on one axis.

• delay_summary() – the case-weighted delay distribution.

• zero_run_summary() – lengths of the runs of consecutive zero dates.

• prop_censored() – proportion of cases flagged censored.

• prop_revision_type() – proportion of cases per revision outcome.

• prop_strata() – proportion of cases per stratum.

• prop_covariate_levels() – proportion of cases per level of each categorical covariate.

• date_ranges() – totals, date ranges and now.

• triangle_occupancy() – how full the reporting triangle is, and how stale the object is.

• cumulative_growth() – ratio of one delay's running total to the previous one's.

Usage

cases_per_date(
  x,
  axis = c("event", "report", "revision"),
  by_strata = NULL,
  strata = NULL
)

delay_summary(
  x,
  delay = c("event_to_report", "event_to_revision", "report_to_revision"),
  by_strata = NULL,
  strata = NULL
)

zero_run_summary(
  x,
  axis = c("event", "report", "revision"),
  by_strata = NULL,
  strata = NULL
)

prop_censored(x, by_strata = NULL, strata = NULL)

prop_revision_type(x, by_strata = NULL, strata = NULL)

prop_strata(x, strata = NULL)

prop_covariate_levels(x, by_strata = NULL, strata = NULL)

date_ranges(x, by_strata = NULL, strata = NULL)

triangle_occupancy(x, by_strata = NULL, strata = NULL)

cumulative_growth(x, k = 7, by_strata = NULL, strata = NULL)

Arguments

Value

A tibble in the schema documented in tbl_now_summary: one row per quantity and stratum, with component, quantity and stratum identifying the row and the remaining columns holding whichever statistics apply.

See Also

summary(), which stacks all of these into one table and documents the schema; diagnose() for what is wrong with the data rather than what is in it; autoplot() for the same information as pictures. The Diagnosing a tbl_now article walks through them in order.

Examples

data(denguedat)
# The last five years. The full twenty-year series gives the same shape
# of answer, it just takes longer to compute.
recent <- denguedat[denguedat$onset_week >= as.Date("2006-01-01"), ]
ndata <- tbl_now(recent,
  event_date = "onset_week",
  report_date = "report_week",
  strata = "gender",
  verbose = FALSE
)

# How many cases per week of onset, and how long they took to be reported.
cases_per_date(ndata, axis = "event")
delay_summary(ndata)

# How sparse the series is.
zero_run_summary(ndata, axis = "event")

# What the data is made of, and how far it reaches.
prop_strata(ndata)
prop_censored(ndata)
date_ranges(ndata)
triangle_occupancy(ndata)

# How fast the running total is still growing. This is a distribution over
# event dates, so it fills `mean`/`q50` rather than the scalar `value`.
cumulative_growth(ndata, k = 3)

# Every block shares one schema, so they stack.
dplyr::bind_rows(
  date_ranges(ndata),
  delay_summary(ndata)
)

Standardise a fitted nowcast

Description

The second extension point of the nowcasting framework (see nowcast_fit()). It receives the object the modelling package returned and must express its predictions in the tidy format tbl.now uses everywhere else.

Usage

## S3 method for class 'diseasenowcasting'
nowcast_tidy(engine, fit, x, ..., quantile_levels)

## S3 method for class 'baselinenowcast'
nowcast_tidy(engine, fit, x, ..., quantile_levels)

## S3 method for class 'epinowcast'
nowcast_tidy(engine, fit, x, ..., quantile_levels)

## S3 method for class 'NobBS'
nowcast_tidy(engine, fit, x, ..., quantile_levels)

## S3 method for class 'surveillance'
nowcast_tidy(engine, fit, x, ..., quantile_levels)

## S3 method for class 'EpiNow2'
nowcast_tidy(engine, fit, x, ..., quantile_levels)

nowcast_tidy(engine, fit, x, ..., quantile_levels)

## Default S3 method:
nowcast_tidy(engine, fit, x, ..., quantile_levels)

Arguments

Value

A list with two elements:

predictions

A data frame with one row per (event date, stratum, quantile level) and the columns ⁠<event_date>⁠, the strata columns, .quantile_level and .value.

draws

Either NULL, or a data frame with one row per (event date, stratum, draw) and the columns ⁠<event_date>⁠, the strata columns, .draw and .value.

When draws is supplied and predictions is NULL, run_nowcast() derives the quantiles from the draws for you.

See Also

nowcast_fit(), run_nowcast()

Examples

# See `?nowcast_fit` for a complete two-method backend.
methods(nowcast_tidy)

Ensemble weights from a backtest

Description

Turns the retrospective scores of a nowcast_backtest() into a vector of weights for nowcast_ensemble().

Usage

nowcast_weights(
  backtest,
  type = c("inverse_score", "optim", "equal"),
  now = NULL,
  include_now = FALSE,
  ...
)

Arguments

Value

A named numeric vector of weights summing to 1.

See Also

nowcast_backtest(), which produces the scores these weights come from; nowcast_ensemble(), which consumes them; engine()'s label argument, which is what tells two configurations of the same package apart in the result.

Examples

data(denguedat)

# A short recent window keeps the example quick.
recent <- subset(denguedat, onset_week >= as.Date("2010-06-01"))
dengue <- tbl_now(recent,
  event_date = onset_week, report_date = report_week, verbose = FALSE
)

## `example_engine()` is a toy that ignores the reporting delay entirely; it
# is used here only so the example runs without a modelling package.
## Swap in a real one -- `engine_baselinenowcast()`, `engine_epinowcast()`,
## `engine_nobbs()` -- for anything you intend to act on.

# Two engines that differ in how wide they claim their intervals are.
bt <- nowcast_backtest(dengue,
  example_engine(spread = 0.2, label = "narrow"),
  example_engine(spread = 0.5, label = "wide"),
  now_dates = as.Date(c("2010-10-04", "2010-11-15")),
  verbose = FALSE
)

# Weights sum to one, and the better-scoring engine takes the larger share.
nowcast_weights(bt)
sum(nowcast_weights(bt))

## Hand them to nowcast_ensemble() to pool the nowcasts they came from.

Periodogram of the case counts or of the reporting delay

Description

The "seasonality" / "delay_seasonality" panels of autoplot(), drawn on their own: a periodogram whose dominant peak is marked. For type = "epidemic" (green) the peak suggests a Fourier season length to pass to temporal_effects(); for type = "report" (red) it marks a cycle in the reporting delay itself, such as a weekly reporting rhythm.

Usage

plot_cycles(x, type = c("epidemic", "report", "revision"), ...)

Arguments

Value

A ggplot2 object (or a plotly widget when plotly = TRUE).

See Also

autoplot.tbl_now(), calendar_effect_plots.

Examples

data(denguedat)
dengue_now <- tbl_now(denguedat, onset_week, report_week, verbose = FALSE)
plot_cycles(dengue_now)

Empirical distribution of the reporting or revision delay

Description

The "delay_distribution" panel of autoplot(), drawn on its own: a case-count weighted histogram of .delay. For count-cumulative data it becomes the cumulative growth by delay instead — boxplots, on a log scale, of the ratio of each event date's cumulative count at a delay to its count at the previous delay.

axis = "revision" draws the same histogram of .revision_delay, the time from a report to its resolution, in the revision process's colours. A case still "pending" has no resolution, and so no revision delay, and does not appear.

Usage

plot_delay_distribution(
  x,
  axis = c("report", "revision"),
  by_revision_type = TRUE,
  ...
)

Arguments

Value

A ggplot2 object (or a plotly widget when plotly = TRUE).

See Also

autoplot.tbl_now(), plot_delay_profiles(), plot_delay_drift(); diagnose_revision_delay() for the test behind the outcome split.

Examples

data(denguedat)
dengue_now <- tbl_now(denguedat, onset_week, report_week, verbose = FALSE)
plot_delay_distribution(dengue_now)

# On the revision axis, split by how each case resolved.
cases <- data.frame(
  onset = as.Date("2021-01-04") + rep(0:9, each = 4),
  visit = as.Date("2021-01-05") + rep(0:9, each = 4),
  result = as.Date("2021-01-05") + rep(0:9, each = 4) +
    rep(c(1, 1, 5, 6), times = 10),
  outcome = rep(c("confirmed", "confirmed", "retracted", "retracted"), times = 10)
)
flu <- tbl_now(cases,
  event_date = onset, report_date = visit,
  revision_date = result, revision_type = outcome,
  data_type = "linelist", verbose = FALSE
)
plot_delay_distribution(flu, axis = "revision")

Visualise whether the reporting-delay distribution drifts over time

Description

Draws a rolling fan chart of the count-weighted reporting-delay distribution indexed by event date: a solid line for the rolling median, a dashed line for the rolling mean, and two shaded bands (the 25-75% and 10-90% quantile ranges). Reading it left to right answers "do delays drift?" — a rising/falling centre line is location drift, widening/narrowing bands are spread drift.

Usage

plot_delay_drift(
  x,
  ...,
  window = NULL,
  step = NULL,
  min_n = 1,
  by_strata = FALSE,
  strata = NULL,
  changepoint = FALSE,
  level = 0.95,
  plotly = FALSE,
  axis = c("report", "revision"),
  linewidth = 1,
  grid_linewidth = 0.5,
  palette = .tbl_now_palette()
)

Arguments

Details

Because recent event dates have not had time to be fully reported, their delay summaries are downward-biased (only short delays are observable yet). That immature region — event dates after the level incompleteness cutoff — is shaded grey and should not be read as drift. Pair the plot with diagnose_drift() for a formal test.

Value

A ggplot2 object.

See Also

diagnose_drift() for the formal trend test behind this picture, and diagnose_changepoint() for an abrupt shift rather than a gradual one; plot_delay_distribution() for the delay pooled over the whole period; autoplot() and diagnostic_plot() for the galleries this belongs to.

Examples

data(denguedat)
dengue <- tbl_now(denguedat,
  event_date = "onset_week", report_date = "report_week", verbose = FALSE
)
plot_delay_drift(dengue)

Plot the per-date delay profiles

Description

One translucent curve per date (see by) giving that date's share of reports at each delay, coloured by its mean delay. A batch is a lone right-shifted (long-delay) curve.

Usage

plot_delay_profiles(
  x,
  by = c("report", "event"),
  max_delay = NULL,
  plotly = FALSE,
  axis = c("report", "revision"),
  linewidth = 1,
  palette = .tbl_now_palette()
)

Arguments

Value

A ggplot2 object (or a plotly widget when plotly = TRUE).

See Also

plot_delay_distribution() for the pooled delay distribution rather than one curve per date; plot_delay_drift() for whether those curves move over time; diagnose_batches2() for the test behind the eyeball; diagnostic_plot() for the whole gallery.

Examples

data(denguedat)
dn <- tbl_now(denguedat, onset_week, report_week, verbose = FALSE)
plot_delay_profiles(dn)

The epidemic process and the reporting process

Description

The same cases, counted on two different clocks. Comparing the two is the single most useful thing you can do to tell a real outbreak from a reporting artifact.

• plot_epidemic_process() counts by event date – when the cases actually happened. Epidemics grow and shrink smoothly, so this curve should be smooth.

• plot_reporting_process() counts by report date – when news of them arrived. Reporting is administrative, so this curve is spiky: weekends, holidays and backlog releases all show up here.

A lone spike in the reporting process with nothing under it in the epidemic process is a batch – a day the system cleared its inbox, not a day people got sick. A spike in both is a genuine surge.

Usage

plot_reporting_process(
  x,
  plotly = FALSE,
  axis = c("report", "revision"),
  by_revision_type = TRUE,
  palette = .tbl_now_palette()
)

plot_epidemic_process(
  x,
  plotly = FALSE,
  axis = c("report", "revision"),
  palette = .tbl_now_palette()
)

Arguments

Details

Both are facetted by stratum when the object has strata.

Value

A ggplot2 object (or a plotly widget when plotly = TRUE).

See Also

diagnostic_plot(), which draws these alongside the rest of the reporting-process gallery; plot_observed_cases() for the epidemic process with the incompleteness cutoff marked; diagnose_batches() to test a suspicious spike rather than eyeball it.

Examples

data(denguedat)
dn <- tbl_now(denguedat, onset_week, report_week, verbose = FALSE)

# When cases happened: smooth, because epidemics are.
plot_epidemic_process(dn)

# When news of them arrived: spikier, because reporting is administrative.
plot_reporting_process(dn)

Observed epidemic process with the incompleteness line

Description

The "epidemic" panel of autoplot(), drawn on its own: the latest reported counts per event_date, with a dashed vertical line marking where the data become incomplete (less than level of the delay distribution has arrived). Holidays from an attached temporal_effects() spec are marked with red dots.

plot_epidemic_process() draws the same curve without the incompleteness line, next to its reporting twin plot_reporting_process().

Usage

plot_observed_cases(x, ...)

Arguments

Value

A ggplot2 object (or a plotly widget when plotly = TRUE).

See Also

autoplot.tbl_now(), plot_epidemic_process(), plot_reporting_process().

Examples

data(denguedat)
dengue_now <- tbl_now(denguedat, onset_week, report_week, verbose = FALSE)
plot_observed_cases(dengue_now)

Plot the reporting triangle as an age-period-cohort hexamap

Description

Draws the reporting triangle as a hexagonal age-period-cohort map, using the projection of Jalal and Burke (2020). Event date, report date and reporting delay are the cohort, period and age of the map (report = event + delay), and each ⁠(event, delay)⁠ cell is one point on the hexagonal lattice, coloured by its report count. Because a batch is a single report date, it appears as a clean vertical stripe; the fast-reporting bulk sits along the short-delay bottom edge.

Usage

plot_reporting_hexamap(
  x,
  max_delay = NULL,
  complete = FALSE,
  iso = NULL,
  iso_minor = NULL,
  format = "%d/%b/%y",
  max_cells = 12000L,
  trans = "sqrt",
  axis = c("report", "revision"),
  size = 1.5,
  shape = 16,
  text_size = 2.3,
  grid_linewidth_major = 0.3,
  grid_linewidth_minor = 0.15,
  axis_linewidth = 0.4,
  legend_width = 7,
  legend_height = 0.4,
  palette = .tbl_now_palette()
)

Arguments

Details

The three axes are read off three families of iso-lines: report date (period) runs vertically, delay (age) up the right-hand spine, and event date (cohort) up the left. A major/minor triangular grid is drawn so any point can be traced back to its event date, report date and delay.

The number of points is ⁠#\{observed (event, delay) cells\}⁠, which grows with the delay range. To stay responsive the delay axis is capped so at most max_cells points are drawn (raise max_cells, or set max_delay, to change this). complete = TRUE first fills the whole observable triangle with explicit zeros (via complete_zeroes()) so the empty cells are shown too.

A point is sized in millimetres and the lattice is sized in data units, so no default size can be right for every combination of cell count and figure size – which is exactly why size exists. Raise it until the points nearly touch for the figure you are actually drawing.

Value

A ggplot2 object.

References

Jalal, H. and Burke, D. S. (2020). Hexamaps for Age-Period-Cohort Data Visualization. Epidemiology 31, e47-e49.

See Also

plot_reporting_triangle() for the same data on ordinary axes, where the third quantity has to be read off the diagonals; diagnostic_plot() for the whole gallery.

Examples

data(denguedat)
dn <- tbl_now(denguedat, onset_week, report_week, verbose = FALSE)
plot_reporting_hexamap(dn)

Plot the reporting triangle

Description

Tiles over (event date, delay), filled by the reported count. Cells that are observable but empty (a genuine reported zero) are drawn in a muted blue; cells that are not yet reportable (report date beyond now, the upper-right wedge) are left blank. A third axis for report date is drawn as evenly spaced dashed diagonals (report = event + delay) running up-right at 45 degrees, so all three quantities – event date, delay and report date – can be read off one plot. A batch is a single report date, i.e. one such diagonal.

Usage

plot_reporting_triangle(
  x,
  max_delay = NULL,
  report_ticks = 6L,
  mark_batches = 0L,
  plotly = FALSE,
  axis = c("report", "revision"),
  size = 1,
  grid_linewidth = 0.3,
  palette = .tbl_now_palette()
)

Arguments

Value

A ggplot2 object (or a plotly widget when plotly = TRUE).

See Also

plot_reporting_hexamap() for the same grid drawn so that event date, report date and delay are all read the same way; plot_delay_profiles() for one curve per date instead of a grid; complete_zeroes() to fill the cells that are genuinely zero; diagnostic_plot() for the whole gallery.

Examples

data(denguedat)
dn <- tbl_now(denguedat, onset_week, report_week, verbose = FALSE)

# Rows are event dates, columns are delays. The blank upper-right wedge is
# the future: those reports cannot have arrived yet. That wedge is what a
# nowcast fills in.
plot_reporting_triangle(dn)

How much of each day has been resolved

Description

The share of each event date's cases that are confirmed, retracted or still pending, as of the object's now.

This is the picture of the resolution front. The oldest dates are almost entirely resolved; the most recent ones are mostly pending, because the laboratory has not caught up yet. Where that front sits tells you how far back the confirmed counts can be trusted – and a day that is 80% pending is a day whose confirmed count means very little.

Usage

plot_revision_status(
  x,
  by = NULL,
  proportion = TRUE,
  palette = .tbl_now_palette()
)

Arguments

Value

A ggplot.

Reading it

The pending band widening towards the right is normal and expected – it is the same right-truncation a nowcast exists to correct, one axis over. What is not normal is a pending band that stays wide far from the now: those cases were reported and then never resolved, and they will never be. Consider censor_revision_delays_above().

A retracted share that changes over time is worth investigating: it usually means the testing criteria or the case definition changed, not that the disease did.

Colours

confirmed is drawn with the palette's epidemic role (it is a real case – the epidemic process), retracted with reporting (it was removed by the reporting process), and pending with the neutral pending role (not yet known either way). Override any of them through palette.

See Also

diagnose_revision_delay(), revised_cases.

Examples

cases <- data.frame(
  onset = as.Date("2021-01-04") + rep(0:9, each = 4),
  visit = as.Date("2021-01-05") + rep(0:9, each = 4),
  result = as.Date("2021-01-07") + rep(0:9, each = 4),
  outcome = rep(c("confirmed", "confirmed", "retracted", "pending"), times = 10)
)
cases$result[cases$outcome == "pending"] <- as.Date(NA)
flu <- tbl_now(cases,
  event_date = onset, report_date = visit,
  revision_date = result, revision_type = outcome,
  data_type = "linelist", verbose = FALSE
)

plot_revision_status(flu)

Plot the transport-discriminant plane

Description

Places each report date by its creation score (x) and transport / deficit score (y) from transport_discriminant(), shading the region that decides the batch call. Surges are not distinguished here (they fold into the quiet background) since only the batch call is of interest.

Usage

plot_transport_discriminant(
  x,
  ...,
  plotly = FALSE,
  size = 1,
  grid_linewidth = 0.3,
  palette = .tbl_now_palette()
)

Arguments

Details

Only the diagnose_batches()-confirmed batches (Benjamini-Hochberg-corrected) are coloured red; the dashed lines and shaded region are a reference for where a batch sits (deficit cleared, and significant), not the flagging rule. The most extreme-looking points (far left, far up) are holds – windows still depleted because the release has not happened yet – not batches. A genuine batch sits in the band just to the right of the vertical line, once the window total recovers.

Value

A ggplot2 object (or a plotly widget when plotly = TRUE).

See Also

transport_discriminant() for the numbers behind the plane; diagnose_batches() for the hypothesis test that flags the red points; plot_reporting_process() for the series they come from; diagnostic_plot() for the whole gallery.

Examples

data(denguedat)
# The two and a half years around the 1996 and 1997 backlog dumps, so that
# the plane has red points on it without scanning the whole series.
window <- denguedat[
  denguedat$onset_week >= as.Date("1995-06-01") &
    denguedat$onset_week <= as.Date("1998-01-01"),
]
dn <- tbl_now(window, onset_week, report_week, verbose = FALSE)
plot_transport_discriminant(dn)

Print a tbl_now diagnosis

Description

Prints the findings diagnose() returned as a report: the errors, warnings and notes in full, each with its hint, and the checks that passed and that could not be assessed as one line each.

The object is an ordinary tibble underneath, so print(tibble::as_tibble(x)) gives the table and every dplyr verb still works on it.

Usage

## S3 method for class 'tbl_now_diagnosis'
print(x, ..., all = FALSE)

Arguments

Value

x, invisibly.

See Also

diagnose(), nowcast_diagnose_components

Examples

data(denguedat)
# The last five years. The full twenty-year series gives the same shape
# of answer, it just takes longer to compute.
recent <- denguedat[denguedat$onset_week >= as.Date("2006-01-01"), ]
ndata <- tbl_now(recent,
  event_date = "onset_week", report_date = "report_week",
  strata = "gender", verbose = FALSE
)

diagnose(ndata)

# Every finding, including the ones that passed.
print(diagnose(ndata), all = TRUE)

# Still a tibble.
print(tibble::as_tibble(diagnose(ndata)))

Print a palette, one role per line

Description

Registered on base::print rather than with a plain ⁠@export⁠: the package namespace defines an S7 print generic, which shadows base::print for an attached session, so a plainly-exported ⁠print.*⁠ method never dispatches on auto-print. (print.diagnose_batches and print.transport_discriminant still have that bug.)

Usage

## S3 method for class 'tbl_now_palette'
print(x, ...)

Arguments

Value

x, invisibly.

Print a tbl_now summary

Description

Prints the table summary() returned one component at a time, dropping the columns that component does not populate. The full schema is wide because it has to hold every block's statistics at once; no single block fills more than a handful of them, and a table that is mostly NA is hard to read for a reason that has nothing to do with the data.

The object is an ordinary tibble underneath, so print(tibble::as_tibble(x)) gives the whole schema back and every dplyr verb still works on it.

Usage

## S3 method for class 'tbl_now_summary_table'
print(x, ..., n = 10)

Arguments

Value

x, invisibly.

See Also

summary(), nowcast_summary_components

Examples

data(denguedat)
# The last five years. The full twenty-year series gives the same shape
# of answer, it just takes longer to compute.
recent <- denguedat[denguedat$onset_week >= as.Date("2006-01-01"), ]
ndata <- tbl_now(recent,
  event_date = "onset_week", report_date = "report_week",
  strata = "gender", verbose = FALSE
)

summary(ndata)

# One block on its own prints the same way.
delay_summary(ndata)

# Still a tibble.
print(tibble::as_tibble(summary(ndata)))

Objects exported from other packages

Description

These objects are imported from other packages. Follow the links below to see their documentation.

ggplot2

autoplot()

tibble

as_tibble()

Cases at a chosen point in the revision process

Description

The same three questions as get_latest_reported_cases(), asked of the third date: not when the system heard about a case, but when the laboratory settled it.

• get_initial_revised_cases() – the count as of the first result to come back for that event date.

• get_latest_revised_cases() – the count as of the most recent result: everything settled so far.

• get_nth_revised_cases() – the count settled within a given delay of the event.

A case that is still "pending" has no revision date, so it has not arrived on this axis and none of these count it. That is the point: the gap between get_latest_reported_cases() and get_latest_revised_cases() is the backlog the laboratory still owes you.

Usage

get_latest_revised_cases(x, type = "total")

get_initial_revised_cases(x, type = "total")

get_nth_revised_cases(x, delay, type = "total")

Arguments

Value

A count-cumulative tbl_now with one row per event date (and stratum, grouping column, and outcome when type = "by_type"), carrying the event, report and revision dates of the selected arrival, the generated numeric columns, and the count.

Which date the count is indexed by

By the event date, as every other ⁠get_*_cases()⁠ function is. A case confirmed three weeks after onset still belongs to the week it began. If you want counts by revision date instead, group on get_revision_date(x) yourself – that is a different question (how busy was the laboratory) and this package does not silently answer it.

Which delay get_nth_revised_cases() counts

The delay from the event, so that get_nth_reported_cases(x, 7) and get_nth_revised_cases(x, 7) describe the same seven days and can be read against each other. It is deliberately not .revision_delay, which is the laboratory's turnaround measured from the report. diagnose_drift() and summary() make the same choice for the same reason.

Grouping is respected

As with the reporting-axis getters: the caller's grouping becomes a key and comes back on the result. See get_latest_reported_cases().

See Also

get_latest_reported_cases() for the same counts on the reporting process; add_revision_date() to attach a revision; revision_delay for how long resolution takes; plot_revision_status() to see confirmed, retracted and pending over time.

Examples

cases <- data.frame(
  onset = as.Date("2021-01-04") + c(0, 0, 1, 1, 2),
  visit = as.Date("2021-01-05") + c(0, 0, 1, 1, 2),
  result = as.Date("2021-01-06") + c(0, 0, 1, 1, 2),
  outcome = c("confirmed", "retracted", "confirmed", "confirmed", "retracted")
)
flu <- tbl_now(cases,
  event_date = onset, report_date = visit,
  revision_date = result, revision_type = outcome,
  data_type = "linelist", verbose = FALSE
)

# Three answers to "how many cases were there?".
get_latest_reported_cases(flu) # everything reported
get_latest_revised_cases(flu, type = "confirmed") # only the positives
get_latest_revised_cases(flu, type = "net") # positives minus withdrawals

# Every outcome side by side.
get_latest_revised_cases(flu, type = "by_type")

# And the same question asked earlier in the process: what had come back by
# the first result, and within two days of onset.
get_initial_revised_cases(flu)
get_nth_revised_cases(flu, delay = 2)

Compare revision delays between confirmed and retracted cases

Description

A negative result often comes back faster than a positive one – or slower, if positives are prioritised. Either way the delay from report to resolution is not the same distribution for the two outcomes, and a nowcast that assumes it is will be wrong about how many pending cases are still to be confirmed.

diagnose_revision_delay() compares the two delay distributions; plot_delay_distribution() with axis = "revision" shows them.

Usage

diagnose_revision_delay(x, by = NULL)

Arguments

Value

diagnose_revision_delay() returns a one-row-per-comparison tibble with stratum, n_confirmed, n_retracted, median_confirmed, median_retracted, difference, statistic and p.value.

The test

A two-sided Wilcoxon rank-sum test on the revision delays. It is used rather than a t-test because reporting delays are strongly right-skewed and frequently have a point mass at zero, so a difference in means is neither robust nor the quantity of interest – what matters is whether one outcome resolves systematically sooner.

A small p-value says the two delay distributions differ. It does not say the difference matters: with tens of thousands of records a one-hour difference is significant and irrelevant, so read difference (the gap in median days) alongside it.

Rows with a missing or negative delay are dropped, and how many is reported in the dropped attribute of the result. A negative revision delay means the record is revised before it was reported, which the timeline forbids.

See Also

add_revision_date() to attach a revision process; censor_revision_delays_above() for resolutions that never arrive; revised_cases for counting the outcomes; plot_delay_distribution() with axis = "revision" for the picture of the same comparison; diagnose_drift() for the same question about the reporting delay over time. The Diagnosing a tbl_now article puts this alongside the other checks.

Examples

cases <- data.frame(
  onset = as.Date("2021-01-04") + rep(0:9, each = 4),
  visit = as.Date("2021-01-05") + rep(0:9, each = 4),
  result = as.Date("2021-01-05") + rep(0:9, each = 4) +
    rep(c(1, 1, 5, 6), times = 10),
  outcome = rep(c("confirmed", "confirmed", "retracted", "retracted"), times = 10)
)
flu <- tbl_now(cases,
  event_date = onset, report_date = visit,
  revision_date = result, revision_type = outcome,
  data_type = "linelist", verbose = FALSE
)

# Retractions here come back about four days later than revisions, and
# the test says so.
diagnose_revision_delay(flu)

# The same comparison as a picture.
plot_delay_distribution(flu, axis = "revision")

Nowcast a tbl_now with any supported modelling package

Description

Fits a nowcasting model to a tbl_now and returns the result in a package-agnostic shape, so that models from different packages can be compared, scored and combined (nowcast_ensemble()) without any manual reshaping.

The function is named run_nowcast() rather than nowcast() because diseasenowcasting already exports a nowcast() function; keeping the names distinct means both packages can be attached at once.

Usage

run_nowcast(x, engine = engine_diseasenowcasting(), verbose = TRUE)

Arguments

Value

A tbl_nowcast object.

Engines

engine_diseasenowcasting()

Bayesian structural time series. The tbl_now is passed in directly, so strata and temporal effects are picked up automatically.

engine_baselinenowcast()

Fast, assumption-light baseline built from the reporting triangle. Stratified objects are nowcast one triangle per stratum.

engine_epinowcast()

Bayesian model with separate delay and reference modules; preprocess_args controls tbl_now_to_epinowcast().

engine_surveillance()

Höhle & an der Heiden's nowcast, fed by tbl_now_to_surveillance(). The package models one series, so a stratified object is fitted one stratum at a time.

engine_epinow2()

EpiNow2::estimate_infections(), fed by tbl_now_to_EpiNow2(); EpiNow2::regional_epinow() when the object declares strata.

engine_nobbs()

Nowcasting by Bayesian Smoothing, fed by tbl_now_to_nobbs(), which expands counts to the one row per case NobBS counts.

engine() covers any other registered method, including one you wrote yourself. Every modelling package is an optional dependency: it is only needed when you ask for its engine.

What the object contributes, and what it does not

run_nowcast() reads the tbl_now's declarations and hands each package the shape it wants. Three of those declarations behave differently enough to be worth stating plainly.

Strata. How many strata a backend can honour is a property of the package, not of this one. Where a backend cannot, it warns and pools rather than pretending:

Every backend takes any number of strata. The two that model one series at a time ("surveillance") or accept a single column ("NobBS.strat()") are given the interaction of the declared columns, which is what nowcasting each combination separately means; the label is split back into its columns on the way out.

The one thing that can go wrong is a stratum value that already contains the " | " used to join them. That is an error rather than a guess, because silently mis-assigning strata is worse than refusing.

Whatever strata columns come back are what the result reports as its strata.

Columns you did not declare are summed away by the converters (see tbl_now_to_baselinenowcast()). A tbl_now built from covid_colombia without strata = sex is nowcast as one pooled series, not silently split.

Temporal effects. add_temporal_effects() specs are lazy; the converters materialise them into ordinary columns, so they travel with the data. Whether the model then uses them is a separate question, and mostly the answer is "only if you say so":

• "diseasenowcasting" receives the tbl_now itself and reads the effects off it, so they enter the model with no further work.

• "epinowcast" carries them as covariates you name in a module formula, e.g. reference = epinowcast::enw_reference(~ 1 + day_of_week, data = ...). Without a formula referring to them they are inert.

• every other backend ignores them: the columns ride along so you can split on them, and nothing else happens.

Censored delays. A per-case censoring flag (see add_is_censored_report()) puts a censored and an uncensored row in the same ⁠(event date, report date)⁠ cell, and a reporting triangle has one slot per cell. Every backend that goes through a converter therefore collapses the flag with a warning — counts are summed over it, line lists drop the column. "diseasenowcasting" is the exception: it is handed the object untouched, so the flag reaches the package intact. To estimate a delay distribution from censored data, use tbl_now_to_epidist() instead; nowcasting and delay estimation are different jobs.

How each model is specified, and how to change it

run_nowcast() does not invent priors or model structure: it calls each package with that package's own defaults and passes the engine's arguments straight through. The defaults are not always the ones you want, and two are worth knowing before you read the output.

engine_epinowcast() runs three modules, all at their package defaults. The expectation module is enw_expectation(r = ~ 0 + (1 | day:.group)) – a random effect per day on the growth rate, which is a random walk on the log expected counts in all but name. The reference module is enw_reference(parametric = ~ 1, distribution = "lognormal") – a single lognormal reporting delay, constant over time. The report module is enw_report(non_parametric = ~ 0) – no day-of-week reporting effect. Each is a named argument of the engine, and preprocess_args carries tbl_now_to_epinowcast()'s own arguments:

run_nowcast(nowobj, engine_epinowcast(
  preprocess_args = list(max_delay = 30),
  report = epinowcast::enw_report(~ 1 + day_of_week, data = pobs),
  fit    = epinowcast::enw_fit_opts(chains = 4, iter_sampling = 1000)
))

engine_epinow2() – read this before trusting the output. EpiNow2::estimate_infections() defaults to delays = delay_opts(), which is Fixed(0): no reporting delay at all. Its generation_time = gt_opts() is Fixed(1), a one-day generation time. Those defaults describe a process with nothing to nowcast, so supply the epidemiology yourself. EpiNow2 also models R_t with a Gaussian process by default (rt_opts(rw = 0, gp_on = "R_t-1")) rather than a random walk:

run_nowcast(nowobj, engine_epinow2(
  generation_time = EpiNow2::gt_opts(EpiNow2::example_generation_time),
  delays          = EpiNow2::delay_opts(EpiNow2::example_reporting_delay),
  rt              = EpiNow2::rt_opts(rw = 7)   # weekly random walk instead
))

engine_diseasenowcasting() uses the package's own defaults, reading strata, covariates and temporal effects off the object. model, type and n_draws go to diseasenowcasting::nowcast().

engine_baselinenowcast() is not Bayesian and has no priors: the delay is estimated from the reporting triangle and applied. draws sets the number of nowcast samples, and max_delay caps the triangle's width.

engine_nobbs() has a fixed model; what you tune is max_D (maximum delay) and moving_window (how much history is fitted). moving_window counts event periods and must not exceed the history you hand it – ask for more and NobBS pads its grid backwards and returns zero for every date, with no error. specs takes its prior list.

engine_surveillance() takes fit_method, which is surveillance's own method argument renamed so it cannot collide with the engine's method; it defaults to "bayes.notrunc.bnb". D, when and control are derived from the object when you do not give them.

See Also

The Get started vignette for where a fit sits in the workflow; engine() and nowcast_engines to specify which model to fit and how; autoplot() and tidy() to look at the result; nowcast_ensemble() to combine several nowcasts; nowcast_backtest() and score_nowcast() to find out whether they are any good; nowcast_fit() and nowcast_tidy() to add a backend of your own, and example_engine() for the shortest complete one. The One dataset, many nowcasts article fits the same data with every supported package.

Examples

data(denguedat)

# A short recent window keeps the example quick.
recent <- subset(denguedat, onset_week >= as.Date("2010-06-01"))
dengue <- tbl_now(recent,
  event_date = onset_week, report_date = report_week, verbose = FALSE
)

# Every nowcast goes the same way: describe the model with an engine, then
## hand it and the data to `run_nowcast()`.
#
## `example_engine()` is a toy that ignores the reporting delay entirely; it is
# used here only so the example runs without a modelling package. Swap in a
## real one -- `engine_baselinenowcast()`, `engine_epinowcast()`,
## `engine_nobbs()` -- for anything you intend to act on.
nc <- run_nowcast(dengue, example_engine(), verbose = FALSE)
nc

# The result is a tbl_nowcast: one row per event date and quantile level.
head(tibble::as_tibble(nc))

# A real model is the same call with a real engine.
if (requireNamespace("baselinenowcast", quietly = TRUE)) {
  nc <- run_nowcast(dengue, engine_baselinenowcast(draws = 100), verbose = FALSE)
  nc
}

sari_bh: Severe Acute Respiratory Illness data from Belo Horizonte (Brazil), 2020-2022

Description

An anonymized dataset containing cases of Severe Acute Respiratory Illness (SARI) cases from the Brazilian municipality, Belo Horizonte, with symptom onset varying from 2019-12-29 to 2022-03-27.

Usage

sari_bh

Format

A data frame with 65404 rows and 5 variables:

symptom_onset_date

Date of onset symptoms, formatted in ISO8601 as YYYY-MM-DD.

record_date

Date the case was recorded in the surveillance system, formatted in ISO8601 as YYYY-MM-DD. May be NA when no record date was available.

final_classification

Final classification of the case: one of "Influenza", "Other respiratory virus", "Other ethiological agent", "Not specified", or "COVID-19".

case_evolution

Case evolution outcome: one of "Cured", "Dead", "Dead (other causes)", "Unknown", or "Not in dictionary".

age_yrs

Age of the patient in years.

Source

Redistributed from the nowcaster package (https://github.com/covid19br/nowcaster), which derives it from the Brazilian Ministry of Health's SIVEP-Gripe respiratory surveillance system, published on the OpenDataSUS portal.

References

From the nowcaster package. Bastos, S L, Economou, Theodoros, Gomes, FC M, Villela, AM D, Coelho, C F, Cruz, G O, Stoner, Oliver, Bailey, Trevor, Codeço, T C (2019). “A Modelling approach for correcting reporting delays in disease surveillance data.” Statistics in medicine, 38(22), 4363-4377. doi:10.1002/sim.8303

Score a nowcast against observed data

Description

A nowcast is a claim about numbers that are not in yet. Once the late reports arrive you can ask how good the claim was.

• score_nowcast() scores it here: the weighted interval score (WIS, lower is better), the absolute error of the median, and whether the truth fell inside the 50% and 90% intervals – one row per event date and stratum.

• scoringutils::as_forecast_point() hands the median prediction and the same truth to scoringutils, so you can use its point-score functions and plots.

• scoringutils::as_forecast_quantile() and scoringutils::as_forecast_sample() accept the same objects directly.

All three are scoringutils generics; this package only supplies the methods, so call them qualified (or after library(scoringutils)).

In each case truth is a tbl_now seen later, after the information the nowcast was predicting has arrived. The observed counts are computed from truth_axis and truth_type: by default this is get_latest_reported_cases() with type = "total", while truth_axis = "revision" uses get_latest_revised_cases(). There is no observed column to name; the count column is read from the tbl_now.

Usage

score_nowcast(
  x,
  truth = NULL,
  truth_axis = c("report", "revision"),
  truth_type = "total"
)

## S3 method for class 'nowcast_backtest'
as_forecast_quantile(
  data,
  ...,
  truth = NULL,
  truth_axis = c("report", "revision"),
  truth_type = "total"
)

## S3 method for class 'nowcast_backtest'
as_forecast_point(
  data,
  ...,
  truth = NULL,
  truth_axis = c("report", "revision"),
  truth_type = "total"
)

## S3 method for class 'nowcast_backtest'
as_forecast_sample(
  data,
  ...,
  truth = NULL,
  truth_axis = c("report", "revision"),
  truth_type = "total"
)

Arguments

Value

score_nowcast() returns a tibble with the event-date column, the strata columns, and the columns .observed, wis, ae_median, coverage_50 and coverage_90 – one row per event date and stratum.

The ⁠scoringutils::as_forecast_*()⁠ methods return the corresponding forecast_quantile, forecast_sample or forecast_point object from scoringutils. Each accepts a tbl_nowcast, an ensemble and a nowcast_backtest(); scoringutils::as_forecast_point() keeps the nowcast's median quantile as predicted and the resolved truth as observed. A backtest already carries the truth it was scored against, so its truth can normally be omitted.

scoringutils::as_forecast_sample() also accepts those objects when they carry posterior draws. Draws are retained by a linear_pool ensemble, but not by a quantile ensemble. A backtest retains them only when run with keep_draws = TRUE; every engine in the backtest must return draws.

References

Bracher, J., Ray, E. L., Gneiting, T., & Reich, N. G. (2021). Evaluating epidemic forecasts in an interval format. PLoS Computational Biology, 17(2), e1008618.

See Also

nowcast_backtest() to score many nowcasts at many now dates at once; nowcast_weights() to turn those scores into ensemble weights; get_latest_reported_cases(), which is how the truth is read off truth; nowcast_quantile_levels() for the levels being scored.

Examples

# A nowcast and the truth it should be judged against. Both are built by
# hand here so that the example needs no modelling package; in practice `nc`
## comes from run_nowcast() and `truth` is the same data seen later, once the
# late reports have arrived.
truth_df <- data.frame(
  onset  = rep(as.Date("2024-03-04") + 7 * (0:3), each = 3),
  report = rep(as.Date("2024-03-04") + 7 * (0:3), each = 3) + c(0, 7, 14),
  n      = c(5, 3, 2, 8, 4, 1, 6, 5, 3, 9, 2, 2)
)
truth <- tbl_now(truth_df,
  event_date = onset, report_date = report, case_count = n,
  data_type = "count-incidence", verbose = FALSE
)

# What eventually turned out to be true for each week.
get_latest_reported_cases(truth)

# A nowcast that predicted 8 / 10 / 13 for every week.
levels <- c(0.25, 0.5, 0.75)
preds <- tidyr::expand_grid(
  onset = unique(truth_df$onset), .quantile_level = levels
)
preds$.value <- rep(c(8, 10, 13), times = 4)
nc <- tbl_nowcast(predictions = preds, method = "toy", event_date = "onset")

# Lower `wis` is better. `coverage_50` says whether the truth fell inside
# the 50% interval, which it should about half the time.
score_nowcast(nc, truth = truth)

# The same comparison handed to scoringutils as a point forecast.
if (requireNamespace("scoringutils", quietly = TRUE)) {
  scoringutils::as_forecast_point(nc, truth = truth)
}

# With a real model, `truth` is the full object and the nowcast is fitted to
# a snapshot of it taken at an earlier `now`.
data(denguedat)

recent <- subset(denguedat, onset_week >= as.Date("2010-06-01"))
dengue <- tbl_now(recent,
  event_date = onset_week, report_date = report_week, verbose = FALSE
)
snapshot <- change_now(
  dplyr::filter(dengue, report_week <= as.Date("2010-10-04")),
  now = as.Date("2010-10-04")
)

if (requireNamespace("baselinenowcast", quietly = TRUE)) {
  nc <- run_nowcast(snapshot, engine_baselinenowcast(draws = 100), verbose = FALSE)
  # The FULL object is the truth: it still holds the reports that arrived
  # after the snapshot's `now`.
  score_nowcast(nc, truth = dengue)
}

Inject a batch into a tbl_now by withholding and then releasing reports

Description

Simulates a reporting system that is closed on a given set of report dates and releases its accumulated backlog on the next open date. Reports keep their event dates and merely move later on the report axis, so no cases are created or destroyed – the defining property of a batch. Useful for checking that diagnose_batches() and diagnose_batches2() recover a batch you planted.

Usage

simulate_batch(
  x,
  closed_dates,
  held_fraction = 1,
  drop_unreleased = TRUE,
  verbose = TRUE
)

Arguments

Details

A batch is a transport: it moves an item's report date later while leaving its event date untouched, creating and destroying nothing. Every report whose report date lies in closed_dates (or, with held_fraction < 1, a random share of them) is re-stamped with the first report date at or after it that is not closed. Consequently:

• the closed dates report nothing (the deficit);

• the release date reports its own items plus the whole backlog (the spike);

• items released late have inflated delays;

• the release date draws on an unusually large number of distinct event dates.

All four symptoms come from the one mechanism, which is why they should not be treated as four independent pieces of evidence.

Value

A new tbl_now with the same event dates, strata and data type, and modified report dates.

Reports that never come back

If a closed run extends to the end of the report axis there is no open date to release into. Those reports are then unobservable – a stall that has not yet cleared is indistinguishable from data loss, an honest identification failure. drop_unreleased = TRUE (default) discards them, reproducing exactly what a real analyst would see.

Cumulative data

For "count-cumulative" data a report announces a running total. When two reports for the same event date are pushed onto the same release date, only the later one survives: it is that report date's final word on the total.

See Also

diagnose_batches() and diagnose_batches2(), the tests this exists to validate; censor_reporting_delays_above() for recording a real batch rather than planting one. The Diagnosing a tbl_now article uses this to calibrate the screen.

Examples

data(denguedat)

dengue_tbl <- tbl_now(
  denguedat,
  event_date  = onset_week,
  report_date = report_week,
  data_type   = "linelist",
  verbose     = FALSE
)

# Pretend the reporting desk was shut for three consecutive weeks: everything
# that would have been reported then is held, and released together afterwards.
closed <- as.Date(c("1990-06-04", "1990-06-11", "1990-06-18"))
batched_tbl <- simulate_batch(dengue_tbl, closed_dates = closed, verbose = FALSE)

# No cases are lost -- they are only moved later in the reporting process.
nrow(dengue_tbl)
nrow(batched_tbl)

# Which is the point: you now have data with a batch you planted yourself, so
# you can check whether the screen finds it.
found <- suppressWarnings(diagnose_batches(batched_tbl, lookback = 2))
found$report_date[found$batch]

The date grids surveillance::nowcast() needs

Description

surveillance::nowcast() takes three dates and two date grids, and none of them have defaults you can rely on. These two helpers build the grids from the tbl_now itself, so the object stays the single source of truth for what "now" is and how wide a time step is:

• get_surveillance_when() – the dates you want estimated, passed as when. The most recent length steps up to and including get_now().

• get_surveillance_range() – the whole time axis the model is laid on, passed as control$dRange. Every step from the first event to now. Its last element is also what now itself should be: get_now() can fall mid-epoch, and surveillance::nowcast() refuses that (see below).

sur_fit <- surveillance::nowcast(
  now  = max(get_surveillance_range(x)),
  when = get_surveillance_when(x, length = 30),
  data = tbl_now_to_surveillance(x, verbose = FALSE),
  dEventCol = "dHospital", dReportCol = "dReport",
  control = list(dRange = get_surveillance_range(x))
)

Usage

get_surveillance_when(x, length = 30L, ..., to = NULL, by = NULL)

get_surveillance_range(x, ..., from = NULL, to = NULL, by = NULL)

Arguments

Value

A Date vector, in increasing order.

Why dRange has to be given explicitly

Left to itself, surveillance::nowcast() infers the time axis from the data it was handed – and a line list cannot express a zero. A day on which nothing was reported has no rows, so it is not in the line list, so it is not in the inferred axis. That is exactly the situation at the now edge, which is the part you are nowcasting: the last few days are quiet precisely because their reports have not arrived yet, and the axis silently stops short of now. Passing dRange states the grid instead of letting it be guessed, so the quiet days at the end are modelled as zeros observed so far rather than as days that do not exist.

This is also why complete_zeroes() is no help here: it can only add zero counts, and a line list has no count column to put a zero in.

Which weekday the grid lands on

surveillance::nowcast() refuses a grid that does not sit at the first day of an epoch: a Monday for "1 week", the first of the month for "1 month". Epidemiological weeks routinely start on a Sunday instead, so both grids are snapped back to the epoch start, and both may therefore begin a few days before the dates in x. run_nowcast() shifts surveillance's estimates back onto the object's own weekday when they return, so a nowcast fitted through the engine is still indexed by the event dates you gave it.

See Also

tbl_now_to_surveillance(), get_now(), get_event_units()

Examples

data(denguedat)
nowobj <- tbl_now(denguedat,
  event_date = "onset_week", report_date = "report_week", verbose = FALSE
)
get_surveillance_when(nowobj, length = 4)
range(get_surveillance_range(nowobj))

Create a tbl_now object

Description

Surveillance data arrives late. A case that happened on Monday may only reach the surveillance system on Thursday, so counts for the most recent days always look artificially low. Nowcasting corrects that artifact: it estimates how many cases have already happened but have not been reported yet.

To do that, a model needs two dates for every case – when it happened (event_date) and when it was reported (report_date) – together with the date you are standing on (now). tbl_now() takes an ordinary data.frame and records which of its columns play those roles, so you only have to say it once.

The result still behaves like a tibble: dplyr verbs, $, [ and ggplot2 keep working, and every tbl.now function knows where to find the dates without being told again.

Usage

tbl_now(
  data,
  event_date = NULL,
  report_date = NULL,
  delay = NULL,
  strata = NULL,
  covariates = NULL,
  case_count = NULL,
  is_censored_report = NULL,
  revision_date = NULL,
  revision_type = NULL,
  revision_units = units,
  revision_levels = NULL,
  is_censored_revision = NULL,
  now = NULL,
  event_units = units,
  report_units = units,
  units = "auto",
  data_type = "auto",
  t_effects = character(0),
  verbose = TRUE,
  force = FALSE,
  warn_non_uniqueness = TRUE,
  align_weeks = FALSE,
  ...
)

Arguments