| Title: | Conformal Prediction Methods for Multistep-Ahead Time Series Forecasting |
| Version: | 0.2.0 |
| Description: | Methods and tools for performing multistep-ahead time series forecasting using conformal prediction methods including classical conformal prediction, adaptive conformal prediction, conformal PID (Proportional-Integral-Derivative) control, and autocorrelated multistep-ahead conformal prediction. The methods were described by Wang and Hyndman (2024) <doi:10.48550/arXiv.2410.13115>. |
| License: | GPL-3 |
| URL: | https://github.com/xqnwang/conformalForecast, https://xqnwang.github.io/conformalForecast/ |
| BugReports: | https://github.com/xqnwang/conformalForecast/issues |
| Depends: | R (≥ 4.1.0), forecast |
| Imports: | ggdist, rlang, stats, zoo |
| Suggests: | dplyr, ggplot2, knitr, rmarkdown, testthat (≥ 3.0.0), tibble, tsibble |
| Config/testthat/edition: | 3 |
| Encoding: | UTF-8 |
| RoxygenNote: | 7.3.3 |
| VignetteBuilder: | knitr |
| NeedsCompilation: | no |
| Packaged: | 2026-10-15 05:03:21 UTC; xiaoqianwang |
| Author: | Xiaoqian Wang |
| Maintainer: | Xiaoqian Wang <Xiaoqian.Wang@amss.ac.cn> |
| Repository: | CRAN |
| Date/Publication: | 2026-10-15 06:10:40 UTC |
conformalForecast: Conformal Prediction Methods for Multistep-Ahead Time Series Forecasting
Description
Methods and tools for performing multistep-ahead time series forecasting using conformal prediction methods including classical conformal prediction, adaptive conformal prediction, conformal PID (Proportional-Integral-Derivative) control, and autocorrelated multistep-ahead conformal prediction. The methods were described by Wang and Hyndman (2024) doi:10.48550/arXiv.2410.13115.
Author(s)
Maintainer: Xiaoqian Wang Xiaoqian.Wang@amss.ac.cn (ORCID) [copyright holder]
Authors:
Rob Hyndman Rob.Hyndman@monash.edu (ORCID)
Daniele Girolimetto daniele.girolimetto@unipd.it (ORCID)
See Also
Useful links:
Report bugs at https://github.com/xqnwang/conformalForecast/issues
Point estimate accuracy measures
Description
Accuracy measures for point forecast residuals.
Usage
ME(resid, na.rm = TRUE)
MAE(resid, na.rm = TRUE, ...)
MSE(resid, na.rm = TRUE, ...)
RMSE(resid, na.rm = TRUE, ...)
MPE(resid, actual, na.rm = TRUE, ...)
MAPE(resid, actual, na.rm = TRUE, ...)
MASE(
resid,
train,
demean = FALSE,
na.rm = TRUE,
period,
d = period == 1,
D = period > 1,
...
)
RMSSE(
resid,
train,
demean = FALSE,
na.rm = TRUE,
period,
d = period == 1,
D = period > 1,
...
)
point_measures
Arguments
resid |
A numeric vector of residuals from either the validation or the test data. |
na.rm |
If |
... |
Additional arguments for each measure. |
actual |
A numeric vector of the realised values matching the forecasts. Required by the percentage measures. |
train |
A numeric vector of responses used to train the model. Required by the scaled measures. |
demean |
Should the response be demeaned (for |
period |
The seasonal period of the data. Required by the scaled measures. |
d |
Should the response model include a first difference? Defaults to
|
D |
Should the response model include a seasonal difference? Defaults
to |
Format
An object of class list of length 8.
Value
For the individual functions (ME, MAE, MSE,
RMSE, MPE, MAPE, MASE, RMSSE), a
single numeric scalar giving the requested accuracy measure.
For the exported object point_measures, a named list of functions
that can be supplied to higher-level accuracy routines.
See Also
interval_measures for the interval counterparts, and
accuracy.cvforecast, which applies these measures to a
cross-validation or a conformal prediction object.
Examples
# Toy residuals and data
set.seed(1)
y_train <- rnorm(50)
y_test <- rnorm(10)
fcast <- y_test + rnorm(10, sd = 0.2)
resid <- y_test - fcast
# Basic measures
ME(resid)
MAE(resid)
RMSE(resid)
# Percentage measures require 'actual'
MPE(resid, actual = y_test)
MAPE(resid, actual = y_test)
# Scaled measures require training data (and seasonal period if applicable)
MASE(resid, train = y_train, period = 1)
RMSSE(resid, train = y_train, period = 1)
Interval estimate accuracy measures
Description
Accuracy measures for interval forecasts.
Usage
MSIS(
lower,
upper,
actual,
train,
level = 95,
period,
d = period == 1,
D = period > 1,
na.rm = TRUE,
...
)
winkler_score(lower, upper, actual, level = 95, na.rm = TRUE, ...)
interval_measures
Arguments
lower |
A numeric vector of lower bounds of the interval forecasts. |
upper |
A numeric vector of upper bounds of the interval forecasts. |
actual |
A numeric vector of realised values. |
train |
A numeric vector of responses used to train the model. Required by the scaled scores. |
level |
The nominal level of the forecast interval (e.g., 95 or 0.95).
Defaults to |
period |
The seasonal period of the data. Required by the scaled scores. |
d |
Should the response model include a first difference? Defaults to
|
D |
Should the response model include a seasonal difference? Defaults
to |
na.rm |
If |
... |
Additional arguments for each measure. |
Format
An object of class list of length 2.
Value
For winkler_score and MSIS, a single numeric scalar
giving the average interval score (Winkler or mean scaled interval
score).
For the exported object interval_measures, a named list of
functions that can be supplied to higher-level accuracy routines.
See Also
point_measures for the point counterparts, and
accuracy.cvforecast, which applies these measures to a
cross-validation or a conformal prediction object.
Examples
set.seed(1)
actual <- rnorm(10)
lower <- actual - runif(10, 0.5, 1)
upper <- actual + runif(10, 0.5, 1)
train <- rnorm(50)
# Winkler score at 95%
winkler_score(lower, upper, actual, level = 95)
# Mean scaled interval score (needs training data and period)
MSIS(lower, upper, actual, train, level = 95, period = 1)
Accuracy measures for a cross-validation model and a conformal prediction model
Description
Return a range of summary measures of the out-of-sample forecast accuracy.
If x is given, the function also measures the test set forecast
accuracy. If x is not given, the function only produces accuracy
measures on the validation set.
Usage
## S3 method for class 'cvforecast'
accuracy(
object,
x,
CV = TRUE,
period = NULL,
measures = interval_measures,
byhorizon = FALSE,
...
)
## S3 method for class 'cpforecast'
accuracy(object, ...)
Arguments
object |
An object of class |
x |
An optional numerical vector containing the actual values of the
same length as |
CV |
If |
period |
The seasonal period of the data. Defaults to |
measures |
A list of accuracy measure functions to compute (such as
point_measures or interval_measures). Defaults to
|
byhorizon |
If |
... |
Additional arguments depending on the specific measure. |
Details
The measures calculated are:
ME: Mean Error
MAE: Mean Absolute Error
MSE: Mean Squared Error
RMSE: Root Mean Squared Error
MPE: Mean Percentage Error
MAPE: Mean Absolute Percentage Error
MASE: Mean Absolute Scaled Error
RMSSE: Root Mean Squared Scaled Error
Winkler: Winkler Score
MSIS: Mean Scaled Interval Score
Value
A matrix giving mean out-of-sample forecast accuracy measures.
See Also
point_measures and interval_measures
for the measures that can be supplied through measures.
Other evaluation functions:
coverage(),
width()
Examples
# Simulate time series from an AR(2) model
library(forecast)
set.seed(1)
series <- arima.sim(n = 200, list(ar = c(0.8, -0.5)), sd = sqrt(1))
# Cross-validation forecasting with a rolling window
far2 <- function(x, h, level) {
Arima(x, order = c(2, 0, 0)) |>
forecast(h = h, level)
}
fc <- cvforecast(series, forecastfun = far2, h = 3, level = 95, window = 50)
# Out-of-sample forecast accuracy on validation set
accuracy(fc, measures = point_measures, byhorizon = TRUE)
accuracy(fc, measures = interval_measures, level = 95, byhorizon = TRUE)
# Accuracy of conformal prediction intervals
scpfc <- conformal(fc, method = "scp", ncal = 50, rolling = TRUE)
accuracy(scpfc, measures = interval_measures, level = 95,
byhorizon = TRUE)
# Out-of-sample forecast accuracy on test set
accuracy(fc, x = c(1, 0.5, 0), measures = interval_measures,
CV = FALSE, level = 95, byhorizon = TRUE)
Autocorrelated multistep-ahead conformal prediction method
Description
Compute prediction intervals and other information by applying the Autocorrelated Multistep-ahead Conformal Prediction (AcMCP) method. The method can only deal with asymmetric nonconformity scores, i.e., forecast errors.
Usage
acmcp(
object,
alpha = 1 - 0.01 * object$level,
ncal = 10,
rolling = FALSE,
integrate = TRUE,
scorecast = TRUE,
lr = 0.1,
Tg = NROW(object$ERROR),
delta = 0.01,
Csat = NULL,
KI = max(abs(object$ERROR), na.rm = TRUE),
update = FALSE,
ma_method = c("CSS-ML", "CSS"),
...
)
Arguments
object |
An object of class |
alpha |
A numeric vector of significance levels to achieve a desired
coverage level |
ncal |
Length of the burn-in period for training the scorecaster.
If |
rolling |
If |
integrate |
If |
scorecast |
If |
lr |
A positive initial learning rate used for quantile tracking.
Defaults to |
Tg |
The time that is set to achieve the target absolute coverage
guarantee before this. It must be greater than 1 when |
delta |
A number in |
Csat |
A positive constant ensuring that by time |
KI |
A non-negative constant to place the integrator on the same scale
as the scores. Defaults to |
update |
If |
ma_method |
Estimation method for the MA |
... |
Not used. |
Details
Similar to the PID method, the AcMCP method also integrates three modules
(P, I, and D) to form the final iteration. However, instead of performing
conformal prediction for each individual forecast horizon h
separately, AcMCP employs a combination of an MA(h-1) model and a
linear regression model of e_{t+h|t} on
e_{t+h-1|t},\dots,e_{t+1|t} as the scorecaster. This allows the AcMCP
method to capture the relationship between the h-step-ahead forecast
error and the past errors.
Scorecasts are constructed recursively, so longer forecast horizons require
more history before all scorecaster inputs are available. For horizon
h, the first scorecast can be computed at cross-validation error index
ncal + h(h-1)/2. The number of successful scorecasts is
reported in scorecast_times.
Value
A list of class
c("acmcp", "cpforecast", "cvforecast", "forecast") with the
following components:
x |
The original time series. |
series |
The name of the series |
xreg |
Exogenous predictor variables used, if applicable. |
method |
A character string "acmcp". |
cp_times |
An integer vector giving the number of conformal predictions performed in cross-validation for each forecast horizon. |
scorecast_times |
An integer vector giving the number of successful
scorecasts for each forecast horizon. Returned when
|
MEAN |
Point forecasts as a multivariate time series, where the
|
ERROR |
Forecast errors given by
|
LOWER |
A list containing lower bounds for prediction intervals for
each |
UPPER |
A list containing upper bounds for prediction intervals for
each |
level |
The confidence values associated with the prediction intervals. |
call |
The matched call. |
model |
A list containing information about the conformal prediction
model: the resolved arguments in |
If mean is included in the object, the components
mean, lower, and upper will also be returned, showing
the information about the test set forecasts generated using all
available observations.
References
Wang, X., and Hyndman, R. J. (2024). "Online conformal inference for multi-step time series forecasting", arXiv preprint arXiv:2410.13115.
See Also
cvforecast to produce object, and
update.cpforecast to extend the results with new
observations.
Other conformal prediction methods:
acp(),
conformal(),
pid(),
scp()
Examples
# Simulate time series from an AR(2) model
library(forecast)
set.seed(1)
series <- arima.sim(n = 200, list(ar = c(0.8, -0.5)), sd = sqrt(1))
# Cross-validation forecasting
far2 <- function(x, h, level) {
Arima(x, order = c(2, 0, 0)) |>
forecast(h = h, level)
}
fc <- cvforecast(series, forecastfun = far2, h = 3, level = 95, window = 50)
# AcMCP setup
Tg <- 200; delta <- 0.01
Csat <- 2 / pi * (ceiling(log(Tg) * delta) - 1 / log(Tg))
KI <- 2
# AcMCP with integrator and scorecaster
acmcpfc <- acmcp(fc, ncal = 50, rolling = TRUE, KI = KI, Csat = Csat)
print(acmcpfc)
summary(acmcpfc)
acmcpfc$scorecast_times
Adaptive conformal prediction method
Description
Compute prediction intervals and other information by applying the adaptive conformal prediction (ACP) method.
Usage
acp(
object,
alpha = 1 - 0.01 * object$level,
gamma = 0.005,
symmetric = FALSE,
ncal = 10,
rolling = FALSE,
quantiletype = 1,
update = FALSE,
na.rm = TRUE,
...
)
Arguments
object |
An object of class |
alpha |
A numeric vector of significance levels to achieve a desired
coverage level |
gamma |
The step size parameter |
symmetric |
If |
ncal |
Length of the calibration set. If |
rolling |
If |
quantiletype |
An integer between 1 and 9 determining the type of
quantile estimator to be used. Types 1 to 3 are for discontinuous
quantiles, types 4 to 9 are for continuous quantiles. See the
|
update |
If |
na.rm |
If |
... |
Other arguments are passed to the
|
Details
The ACP method considers the online update:
\alpha_{t+h|t}:=\alpha_{t+h-1|t-1}+\gamma(\alpha-\mathrm{err}_{t|t-h}),
for each individual forecast horizon h, respectively, where
\mathrm{err}_{t|t-h}=1 if s_{t|t-h}>q_{t|t-h}, and
\mathrm{err}_{t|t-h}=0 if s_{t|t-h} \leq q_{t|t-h}.
Value
A list of class
c("acp", "cpforecast", "cvforecast", "forecast") with the
following components:
x |
The original time series. |
series |
The name of the series |
xreg |
Exogenous predictor variables used, if applicable. |
method |
A character string "acp". |
cp_times |
An integer vector giving the number of conformal predictions performed in cross-validation for each forecast horizon. |
MEAN |
Point forecasts as a multivariate time series, where the
|
ERROR |
Forecast errors given by
|
LOWER |
A list containing lower bounds for prediction intervals for
each |
UPPER |
A list containing upper bounds for prediction intervals for
each |
level |
The confidence values associated with the prediction intervals. |
call |
The matched call. |
model |
A list containing information about the conformal prediction
model: the resolved arguments in |
If mean is included in the object, the components
mean, lower, and upper will also be returned, showing
the information about the forecasts generated using all available
observations.
References
Gibbs, I., and Candes, E. (2021). "Adaptive conformal inference under distribution shift", Advances in Neural Information Processing Systems, 34, 1660–1672.
See Also
cvforecast to produce object, and
update.cpforecast to extend the results with new
observations.
Other conformal prediction methods:
acmcp(),
conformal(),
pid(),
scp()
Examples
# Simulate time series from an AR(2) model
library(forecast)
set.seed(1)
series <- arima.sim(n = 200, list(ar = c(0.8, -0.5)), sd = sqrt(1))
# Cross-validation forecasting
far2 <- function(x, h, level) {
Arima(x, order = c(2, 0, 0)) |>
forecast(h = h, level)
}
fc <- cvforecast(series, forecastfun = far2, h = 3, level = 95, window = 50)
# ACP with asymmetric nonconformity scores and rolling calibration sets
acpfc <- acp(fc, ncal = 50, rolling = TRUE)
print(acpfc)
summary(acpfc)
Conformal prediction
Description
This function allows you to specify the method used to perform conformal prediction.
Usage
conformal(object, ...)
## S3 method for class 'cvforecast'
conformal(object, method = c("scp", "acp", "pid", "acmcp"), ...)
Arguments
object |
An object of class |
... |
Additional arguments to be passed to the selected conformal method. |
method |
A character string specifying the conformal method to be
applied. Possible options include |
Value
An object whose class depends on the method invoked.
See Also
cvforecast to produce object, and
update.cpforecast to extend the results with new
observations.
Other conformal prediction methods:
acmcp(),
acp(),
pid(),
scp()
Examples
# Simulate time series from an AR(2) model
library(forecast)
set.seed(1)
series <- arima.sim(n = 200, list(ar = c(0.8, -0.5)), sd = sqrt(1))
# Cross-validation forecasting
far2 <- function(x, h, level) {
Arima(x, order = c(2, 0, 0)) |>
forecast(h = h, level)
}
fc <- cvforecast(series, forecastfun = far2, h = 3, level = 95, window = 50)
# Classical conformal prediction with equal weights
scpfc <- conformal(fc, method = "scp", ncal = 50, rolling = TRUE)
summary(scpfc)
# ACP with asymmetric nonconformity scores and rolling calibration sets
acpfc <- conformal(fc, method = "acp", gamma = 0.005,
ncal = 50, rolling = TRUE)
summary(acpfc)
Calculate interval forecast coverage
Description
Calculate the mean coverage and the ifinn matrix for the prediction
intervals on the validation set. If window is not NULL, a
matrix of the rolling means of the interval forecast coverage is also
returned.
Usage
coverage(object, ..., level = 95, window = NULL, na.rm = FALSE)
Arguments
object |
An object of class |
... |
Arguments |
level |
Target confidence level for the prediction intervals. Only one
level can be specified. Defaults to |
window |
If not |
na.rm |
A logical indicating whether |
Value
A list of class "coverage" with the following components:
mean |
Mean coverage across the validation set. |
ifinn |
An indicator matrix as a multivariate time series, where the
|
rollmean |
If |
See Also
cvforecast and the conformal methods, which produce
object.
Other evaluation functions:
accuracy.cvforecast(),
width()
Examples
# Simulate time series from an AR(2) model
library(forecast)
set.seed(1)
series <- arima.sim(n = 200, list(ar = c(0.8, -0.5)), sd = sqrt(1))
# Cross-validation forecasting with a rolling window
far2 <- function(x, h, level) {
Arima(x, order = c(2, 0, 0)) |>
forecast(h = h, level)
}
fc <- cvforecast(series, forecastfun = far2, h = 3, level = 95, window = 50)
# Mean and rolling mean coverage for interval forecasts on validation set
cov_fc <- coverage(fc, level = 95, window = 50)
str(cov_fc)
# Coverage calculated directly from interval components
coverage(x = fc$x, LOWER = fc$LOWER, UPPER = fc$UPPER, level = 95)
Time series cross-validation forecasting
Description
Compute forecasts and other information by applying forecastfun to
subsets of the time series y using a rolling forecast origin.
Usage
cvforecast(
y,
forecastfun,
h = 1,
level = c(80, 95),
forward = TRUE,
xreg = NULL,
initial = 1,
window = NULL,
...
)
Arguments
y |
Univariate time series. |
forecastfun |
Function to return an object of class |
h |
Forecast horizon. Defaults to |
level |
Confidence level for prediction intervals, as a numeric vector
of percentages. It must not be |
forward |
If |
xreg |
Exogenous predictor variables passed to |
initial |
Initial period of the time series where no cross-validation
forecasting is performed. Defaults to |
window |
Length of the rolling window. Defaults to |
... |
Other arguments are passed to |
Details
Let y denote the time series y_1,\dots,y_T and let
t_0 denote the initial period.
Suppose forward = TRUE. If window is NULL,
forecastfun is applied successively to the subset time series
y_{1},\dots,y_t, for t=t_0,\dots,T, generating forecasts
\hat{y}_{t+1|t},\dots,\hat{y}_{t+h|t}. If window is
not NULL and has a length of t_w, then forecastfun is
applied successively to the subset time series
y_{t-t_w+1},\dots,y_{t}, for
t=\max(t_0, t_w),\dots,T.
If forward is FALSE, the last observation used for training
will be y_{T-1}.
Value
A list of class c("cvforecast", "forecast") with components:
x |
The original time series. |
series |
The name of the series |
xreg |
Exogenous predictor variables used in the model, if applicable. |
method |
A character string "cvforecast". |
fit_times |
The number of times the model is fitted in cross-validation. |
MEAN |
Point forecasts as a multivariate time series, where the
|
ERROR |
Forecast errors given by
|
LOWER |
A list containing lower bounds for prediction intervals for
each |
UPPER |
A list containing upper bounds for prediction intervals for
each |
level |
The confidence values associated with the prediction intervals. |
call |
The matched call. |
forward |
Whether |
If forward is TRUE, the components mean,
lower, upper, and model will also be returned,
showing the information about the final fitted model and the forecasts
using all available observations, see e.g.
forecast.ets for more details.
See Also
conformal and the conformal methods
scp, acp, pid and
acmcp, which all take the result of cvforecast() as
their input; update.cpforecast to extend the results with
new observations.
Examples
# Simulate time series from an AR(2) model
library(forecast)
set.seed(1)
series <- arima.sim(n = 200, list(ar = c(0.8, -0.5)), sd = sqrt(1))
# Example with a rolling window
far2 <- function(x, h, level) {
Arima(x, order = c(2, 0, 0)) |>
forecast(h = h, level)
}
fc <- cvforecast(series, forecastfun = far2, h = 3, level = 95, window = 50)
print(fc)
summary(fc)
# Example with exogenous predictors
far2_xreg <- function(x, h, level, xreg, newxreg) {
Arima(x, order = c(2, 0, 0), xreg = xreg) |>
forecast(h = h, level = level, xreg = newxreg)
}
fc_xreg <- cvforecast(series, forecastfun = far2_xreg, h = 3, level = 95,
xreg = matrix(rnorm(406), ncol = 2, nrow = 203),
window = 50)
Create lags or leads of a matrix
Description
Find a shifted version of a matrix, adjusting the time base backward (lagged) or forward (leading) by a specified number of observations for each column.
Usage
lagmatrix(x, lag)
Arguments
x |
A matrix or a multivariate time series. |
lag |
A vector of finite integer lags (positive values) or leads
(negative values) with a length equal to the number of columns of
|
Value
A matrix with the same class and size as x.
See Also
cvforecast, which uses this function to align the
forecasts and the forecast errors by horizon.
Examples
set.seed(1)
x <- matrix(rnorm(20), nrow = 5, ncol = 4)
# Create lags of a matrix
lagmatrix(x, c(0, 1, 2, 3))
# Create leads of a matrix
lagmatrix(x, c(0, -1, -2, -3))
Conformal PID control method
Description
Compute prediction intervals and other information by applying the conformal PID (Proportional-Integral-Derivative) control method.
Usage
pid(
object,
alpha = 1 - 0.01 * object$level,
symmetric = FALSE,
ncal = 10,
rolling = FALSE,
integrate = TRUE,
scorecast = !symmetric,
scorecastfun = NULL,
lr = 0.1,
Tg = NROW(object$ERROR),
delta = 0.01,
Csat = NULL,
KI = max(abs(object$ERROR), na.rm = TRUE),
update = FALSE,
...
)
Arguments
object |
An object of class |
alpha |
A numeric vector of significance levels to achieve a desired
coverage level |
symmetric |
If |
ncal |
Length of the burn-in period for training the scorecaster.
If |
rolling |
If |
integrate |
If |
scorecast |
If |
scorecastfun |
A scorecaster function to return an object of class
|
lr |
A positive initial learning rate used for quantile tracking.
Defaults to |
Tg |
The time that is set to achieve the target absolute coverage
guarantee before this. It must be greater than 1 when |
delta |
A number in |
Csat |
A positive constant ensuring that by time |
KI |
A non-negative constant to place the integrator on the same scale
as the scores. Defaults to |
update |
If |
... |
Other arguments are passed to the |
Details
The PID method combines three modules to make the final iteration:
q_{t+h|t}=\underbrace{q_{t+h-1|t-1} + \eta(\mathrm{err}_{t|t-h}-\alpha)}_{\mathrm{P}}+\underbrace{r_t\left(\sum_{i=1}^t\left(\mathrm{err}_{i|i-h}-\alpha\right)\right)}_{\mathrm{I}}+\underbrace{\hat{s}_{t+h|t}}_{\mathrm{D}}
for each individual forecast horizon h, respectively, where
Quantile tracking part (P) is
q_{t+h-1|t-1} + \eta(\mathrm{err}_{t|t-h}-\alpha), whereq_{1+h|1}is set to 0 without a loss of generality,\mathrm{err}_{t|t-h}=1ifs_{t|t-h}>q_{t|t-h}, and\mathrm{err}_{t|t-h}=0ifs_{t|t-h} \leq q_{t|t-h}.Error integration part (I) is
r_t\left(\sum_{i=1}^t\left(\mathrm{err}_{i|i-h}-\alpha\right)\right). Here we use a nonlinear saturation functionr_t(x)=K_{\mathrm{I}} \tan \left(x \log (t) /\left(t C_{\text {sat }}\right)\right), where we set\tan (x)=\operatorname{sign}(x) \cdot \inftyforx \notin[-\pi / 2, \pi / 2], andC_{\text {sat }}, K_{\mathrm{I}}>0are constants that we choose heuristically.Scorecasting part (D) is
\hat{s}_{t+h|t}, the forecast generated by training a scorecaster based on the nonconformity scores available at timet.
Value
A list of class
c("pid", "cpforecast", "cvforecast", "forecast") with the
following components:
x |
The original time series. |
series |
The name of the series |
xreg |
Exogenous predictor variables used, if applicable. |
method |
A character string "pid". |
cp_times |
An integer vector giving the number of conformal predictions performed in cross-validation for each forecast horizon. |
MEAN |
Point forecasts as a multivariate time series, where the
|
ERROR |
Forecast errors given by
|
LOWER |
A list containing lower bounds for prediction intervals for
each |
UPPER |
A list containing upper bounds for prediction intervals for
each |
level |
The confidence values associated with the prediction intervals. |
call |
The matched call. |
model |
A list containing information about the conformal prediction
model: the resolved arguments in |
If mean is included in the object, the components
mean, lower, and upper will also be returned, showing
the information about the forecasts generated using all available
observations.
References
Angelopoulos, A., Candes, E., and Tibshirani, R. J. (2023). "Conformal PID control for time series prediction", Advances in Neural Information Processing Systems, 36, 23047–23074.
See Also
cvforecast to produce object, and
update.cpforecast to extend the results with new
observations.
Other conformal prediction methods:
acmcp(),
acp(),
conformal(),
scp()
Examples
# Simulate time series from an AR(2) model
library(forecast)
set.seed(1)
series <- arima.sim(n = 200, list(ar = c(0.8, -0.5)), sd = sqrt(1))
# Cross-validation forecasting
far2 <- function(x, h, level) {
Arima(x, order = c(2, 0, 0)) |>
forecast(h = h, level)
}
fc <- cvforecast(series, forecastfun = far2, h = 3, level = 95, window = 50)
# PID setup
Tg <- 200; delta <- 0.01
Csat <- 2 / pi * (ceiling(log(Tg) * delta) - 1 / log(Tg))
KI <- 2
# PID without scorecaster
pidfc_nsf <- pid(fc, ncal = 50, rolling = TRUE, scorecast = FALSE,
KI = KI, Csat = Csat)
print(pidfc_nsf)
summary(pidfc_nsf)
# PID with a Naive model for the scorecaster
naivefun <- function(x, h) {
naive(x) |> forecast(h = h)
}
pidfc <- pid(fc, ncal = 50, rolling = TRUE, scorecastfun = naivefun,
KI = KI, Csat = Csat)
Classical split conformal prediction method
Description
Compute prediction intervals and other information by applying the classical split conformal prediction (SCP) method.
Usage
scp(
object,
alpha = 1 - 0.01 * object$level,
symmetric = FALSE,
ncal = 10,
rolling = FALSE,
quantiletype = 1,
weightfun = NULL,
kess = FALSE,
update = FALSE,
na.rm = TRUE,
...
)
Arguments
object |
An object of class |
alpha |
A numeric vector of significance levels to achieve a desired
coverage level |
symmetric |
If |
ncal |
Length of the calibration set. If |
rolling |
If |
quantiletype |
An integer between 1 and 9 determining the type of
quantile estimator to be used. Types 1 to 3 are for discontinuous
quantiles, types 4 to 9 are for continuous quantiles. See the
|
weightfun |
Function to return a vector of weights used for the sample
quantile computation. Its first argument must be an integer indicating
the number of observations for which weights are generated. Defaults to
|
kess |
If |
update |
If |
na.rm |
If |
... |
Other arguments are passed to |
Details
Consider a vector s_{t+h|t} that contains the nonconformity scores for
the h-step-ahead forecasts.
If symmetric is TRUE, s_{t+h|t}=|e_{t+h|t}|. When
rolling is FALSE, the (1-\alpha)-quantiles
\hat{q}_{t+h|t} are computed successively on expanding calibration
sets s_{1+h|1},\dots,s_{t|t-h}, for t=\mathrm{ncal}+h,\dots,T.
Then the prediction intervals will be
[\hat{y}_{t+h|t}-\hat{q}_{t+h|t}, \hat{y}_{t+h|t}+\hat{q}_{t+h|t}].
When rolling is TRUE, the calibration sets will be of the same
length ncal.
If symmetric is FALSE, s_{t+h|t}^{u}=e_{t+h|t} for the
upper interval bounds and s_{t+h|t}^{l} = -e_{t+h|t} for the lower
bounds. Instead of computing the (1-\alpha)-quantile, the
(1-\alpha/2)-quantiles for the lower bound \hat{q}_{t+h|t}^{l}
and the upper bound \hat{q}_{t+h|t}^{u} are calculated based on their
nonconformity scores, respectively. Then the prediction intervals will be
[\hat{y}_{t+h|t}-\hat{q}_{t+h|t}^{l}, \hat{y}_{t+h|t}+\hat{q}_{t+h|t}^{u}].
Value
A list of class
c("scp", "cpforecast", "cvforecast", "forecast") with the
following components:
x |
The original time series. |
series |
The name of the series |
xreg |
Exogenous predictor variables used, if applicable. |
method |
A character string "scp". |
cp_times |
An integer vector giving the number of conformal predictions performed in cross-validation for each forecast horizon. |
MEAN |
Point forecasts as a multivariate time series, where the
|
ERROR |
Forecast errors given by
|
LOWER |
A list containing lower bounds for prediction intervals for
each |
UPPER |
A list containing upper bounds for prediction intervals for
each |
level |
The confidence values associated with the prediction intervals. |
call |
The matched call. |
model |
A list containing information about the conformal prediction
model: the resolved arguments in |
If mean is included in the object, the components
mean, lower, and upper will also be returned, showing
the information about the test set forecasts generated using all
available observations.
See Also
cvforecast to produce object,
update.cpforecast to extend the results with new
observations, and weighted_quantile for the
weighted quantile estimator used here.
Other conformal prediction methods:
acmcp(),
acp(),
conformal(),
pid()
Examples
# Simulate time series from an AR(2) model
library(forecast)
set.seed(1)
series <- arima.sim(n = 200, list(ar = c(0.8, -0.5)), sd = sqrt(1))
# Cross-validation forecasting
far2 <- function(x, h, level) {
Arima(x, order = c(2, 0, 0)) |>
forecast(h = h, level)
}
fc <- cvforecast(series, forecastfun = far2, h = 3, level = 95, window = 50)
# Classical conformal prediction with equal weights
scpfc <- scp(fc, ncal = 50, rolling = TRUE)
print(scpfc)
summary(scpfc)
# Classical conformal prediction with exponential weights
expweight <- function(n) {
0.99^{n+1-(1:n)}
}
scpfc_exp <- scp(fc, ncal = 50, rolling = TRUE,
weightfun = expweight, kess = TRUE, quantiletype = 7)
Update and reperform cross-validation forecasting and conformal prediction
Description
Update conformal prediction intervals and other information by applying the
cvforecast and conformal functions. Only the newly added time
steps are computed: the prediction intervals produced earlier are carried
over unchanged.
Usage
## S3 method for class 'cpforecast'
update(object, new_data, forecastfun, new_xreg = NULL, ...)
Arguments
object |
An object of class |
new_data |
A non-empty numeric vector of newly available data. |
forecastfun |
Function to return an object of class |
new_xreg |
Newly available exogenous predictor variables passed to
|
... |
Other arguments are passed to |
Value
A refreshed object of class "cpforecast" with updated fields
(e.g., x, MEAN, ERROR, LOWER, UPPER,
and any method-specific components), reflecting the newly appended data
and the re-computed cross-validation forecasts and conformal prediction
intervals.
See Also
cvforecast and conformal, which
produce the object to be updated.
Examples
# Simulate time series from an AR(2) model
library(forecast)
set.seed(1)
series <- arima.sim(n = 200, list(ar = c(0.8, -0.5)), sd = sqrt(1))
# Cross-validation forecasting
far2 <- function(x, h, level) {
Arima(x, order = c(2, 0, 0)) |>
forecast(h = h, level)
}
fc <- cvforecast(series, forecastfun = far2, h = 3, level = 95, window = 50)
# Classical conformal prediction with equal weights
scpfc <- conformal(fc, method = "scp", ncal = 50, rolling = TRUE)
# Update conformal prediction using newly available data
scpfc_update <- update(scpfc, forecastfun = far2,
new_data = c(1.5, 0.8, 2.3))
print(scpfc_update)
summary(scpfc_update)
Calculate interval forecast width
Description
Calculate the mean width of the prediction intervals on the validation set.
If window is not NULL, a matrix of the rolling means of the
interval width is also returned. If includemedian is TRUE,
the information of the median interval width will be returned.
Usage
width(
object,
...,
level = 95,
includemedian = FALSE,
window = NULL,
na.rm = FALSE
)
Arguments
object |
An object of class |
... |
Time-series matrices |
level |
Target confidence level for the prediction intervals. Only one
level can be specified. Defaults to |
includemedian |
If |
window |
If not |
na.rm |
A logical indicating whether |
Value
A list of class "width" with the following components:
width |
Forecast interval width as a multivariate time series, where
the |
mean |
Mean interval width across the validation set. |
rollmean |
If |
median |
Median interval width across the validation set. |
rollmedian |
If |
See Also
cvforecast and the conformal methods, which produce
object.
Other evaluation functions:
accuracy.cvforecast(),
coverage()
Examples
# Simulate time series from an AR(2) model
library(forecast)
set.seed(1)
series <- arima.sim(n = 200, list(ar = c(0.8, -0.5)), sd = sqrt(1))
# Cross-validation forecasting with a rolling window
far2 <- function(x, h, level) {
Arima(x, order = c(2, 0, 0)) |>
forecast(h = h, level)
}
fc <- cvforecast(series, forecastfun = far2, h = 3, level = 95, window = 50)
# Mean and rolling mean width for interval forecasts on validation set
wid_fc <- width(fc, level = 95, window = 50)
str(wid_fc)
# Width calculated directly from interval components
width(LOWER = fc$LOWER, UPPER = fc$UPPER, level = 95)