---
title: "Post-Processing Multi-Output and Multistep Models with tailor and probably"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Post-Processing Multi-Output and Multistep Models with tailor and probably}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---



## Why single-output models "just work" and multi-output/multistep models don't

`tailor` and `probably` are the tidymodels tools for post-processing model predictions: calibrating probabilities, adjusting classification thresholds, and building conformal prediction intervals. Both packages are built around a single assumption: **one outcome column, one estimate column**. `tailor::fit()` selects them with `[[`, which requires exactly one column each.

For a kerasnip model with a single outcome, this is exactly what `predict()` already produces (`.pred` for regression; `.pred_class`/`.pred_<level>` for classification), so `tailor` and `probably` work unmodified — see the [Prediction Intervals with Conformal Inference](conformal_intervals.html) vignette for `probably`, and the examples below for `tailor`.

kerasnip also supports two shapes that go beyond a single outcome, and neither fits `tailor`/`probably`'s assumption:

* **Multi-output models** (a recipe like `output_1 + output_2 ~ .`, each outcome its own Keras head) produce `.pred_output_1`, `.pred_output_2`, ... — this is the standard `parsnip::maybe_multivariate()` shape for multivariate regression, not something kerasnip invented, but downstream post-processing tooling hasn't caught up to it for *any* engine yet.
* **Multistep forecasting models** (see `vignette("multistep_forecasting")`) produce a nested `.pred` list-column: one inner tibble per row, holding `.step` and the forecasted value at each step. `tailor::check_variable_type()` requires `is.numeric()` on the outcome/estimate columns, which a list-column fails outright.

kerasnip cannot make `workflows::add_tailor()` work directly on either shape — `tailor::fit()`'s single-column selection is baked into its own code, not something kerasnip's prediction format can work around. Instead, kerasnip provides:

* `kerasnip_output_view()` / `kerasnip_step_view()`: present one output (or one forecast step) as an ordinary single-output fit, so you can use `tailor`/`probably` exactly as documented, one output/step at a time.
* `kerasnip_add_tailor()`: a `workflows::add_tailor()`-alike built on top of the views, for when you want the tailor trained and applied automatically as part of `fit()`/`predict()`.

## Setup


``` r
library(kerasnip)
library(tidymodels)
#> ── Attaching packages ───────────────── tidymodels 1.5.0 ──
#> ✔ broom        1.0.13     ✔ recipes      1.3.3 
#> ✔ dials        1.4.4      ✔ rsample      1.3.2 
#> ✔ dplyr        1.2.1      ✔ tailor       0.1.0 
#> ✔ ggplot2      4.0.3      ✔ tidyr        1.3.2 
#> ✔ infer        1.1.0      ✔ tune         2.1.0 
#> ✔ modeldata    1.5.1      ✔ workflows    1.3.0 
#> ✔ parsnip      1.6.0      ✔ workflowsets 1.1.1 
#> ✔ purrr        1.2.2      ✔ yardstick    1.4.0
#> ── Conflicts ──────────────────── tidymodels_conflicts() ──
#> ✖ purrr::discard()         masks scales::discard()
#> ✖ dplyr::filter()          masks stats::filter()
#> ✖ parsnip::get_model_env() masks kerasnip::get_model_env()
#> ✖ dplyr::lag()             masks stats::lag()
#> ✖ recipes::step()          masks stats::step()
library(tailor)
library(probably)
#> 
#> Attaching package: 'probably'
#> The following objects are masked from 'package:base':
#> 
#>     as.factor, as.ordered
```

---

## Multi-output models

### Defining a multi-output regression workflow


``` r
input_block <- function(input_shape) keras3::layer_input(shape = input_shape)
dense_block <- function(tensor, units = 16) {
  tensor |> keras3::layer_dense(units = units, activation = "relu")
}
output_1_block <- function(tensor) keras3::layer_dense(tensor, units = 1, name = "temperature")
output_2_block <- function(tensor) keras3::layer_dense(tensor, units = 1, name = "humidity")

create_keras_functional_spec(
  model_name = "climate_mlp",
  layer_blocks = list(
    main_input = input_block,
    dense      = inp_spec(dense_block, "main_input"),
    temperature = inp_spec(output_1_block, "dense"),
    humidity    = inp_spec(output_2_block, "dense")
  ),
  mode = "regression"
)

spec <- climate_mlp(dense_units = 16, fit_epochs = 30) |>
  set_engine("keras")

set.seed(1)
n <- 300
climate_data <- tibble(
  pressure    = rnorm(n),
  wind_speed  = rnorm(n),
  temperature = pressure + rnorm(n, sd = 0.2),
  humidity    = -0.5 * wind_speed + rnorm(n, sd = 0.2)
)

rec <- recipe(temperature + humidity ~ pressure + wind_speed, data = climate_data)

split <- initial_split(climate_data, prop = 0.7)
train_dat <- training(split)
cal_dat   <- testing(split)

wflow <- workflow(rec, spec)
fit_obj <- fit(wflow, data = train_dat)
#> 7/7 - 0s - 19ms/step
#> 7/7 - 0s - 17ms/step

predict(fit_obj, new_data = cal_dat[1:5, ])
#> 1/1 - 0s - 97ms/step
#> # A tibble: 5 × 2
#>   .pred_temperature .pred_humidity
#>               <dbl>          <dbl>
#> 1           -0.172          -0.247
#> 2           -0.528           0.814
#> 3            0.234          -0.541
#> 4            0.0706          0.245
#> 5           -0.337           0.232
```

Both outcomes come back from a single `predict()` call, `.pred_temperature`/`.pred_humidity` — exactly parsnip's own convention for multivariate regression, and exactly what breaks `tailor`/`probably`, which need one estimate column to work with.

### `kerasnip_output_view()`: one output as a standard single-output fit


``` r
temp_view <- kerasnip_output_view(fit_obj, "temperature")
predict(temp_view, new_data = cal_dat[1:5, ])
#> 1/1 - 0s - 48ms/step
#> # A tibble: 5 × 1
#>     .pred
#>     <dbl>
#> 1 -0.172 
#> 2 -0.528 
#> 3  0.234 
#> 4  0.0706
#> 5 -0.337
```

`temp_view` behaves like an ordinary single-output fit to anything that calls `predict()` on it. That is enough for manual `tailor` usage:


``` r
cal_preds <- predict(temp_view, new_data = cal_dat)
#> 3/3 - 0s - 41ms/step
cal_data_for_tailor <- bind_cols(temperature = cal_dat$temperature, cal_preds)

tlr <- tailor() |> adjust_numeric_calibration(method = "linear")
tlr_fit <- fit(tlr, cal_data_for_tailor, outcome = temperature, estimate = .pred)
#> Registered S3 method overwritten by 'butcher':
#>   method                 from    
#>   as.character.dev_topic generics

new_preds <- predict(temp_view, new_data = cal_dat[1:5, ])
#> 1/1 - 0s - 81ms/step
predict(tlr_fit, new_preds)
#> # A tibble: 5 × 1
#>     .pred
#>     <dbl>
#> 1 -0.239 
#> 2 -0.481 
#> 3  0.178 
#> 4 -0.0144
#> 5 -0.353
```

It is also enough for `probably`'s conformal methods, because `kerasnip_output_view()` implements `hardhat::extract_mold()` and `generics::augment()` for the view, which is all `probably::int_conformal_split()` needs:


``` r
conformal <- int_conformal_split(temp_view, cal_data = cal_dat)
#> 3/3 - 0s - 42ms/step
predict(conformal, new_data = cal_dat[1:5, ], level = 0.90)
#> 1/1 - 0s - 59ms/step
#> # A tibble: 5 × 3
#>     .pred .pred_lower .pred_upper
#>     <dbl>       <dbl>       <dbl>
#> 1 -0.172       -0.555      0.211 
#> 2 -0.528       -0.911     -0.145 
#> 3  0.234       -0.149      0.617 
#> 4  0.0706      -0.312      0.454 
#> 5 -0.337       -0.720      0.0465
```

### `probably::int_conformal_full()`: supported, with one documented assumption

`int_conformal_full()` refits the model once per candidate value of every new observation. For a multi-output model, refitting means retraining the *whole* multi-head network — so what should the *other* output(s) be during that refit, for a row where they were never observed in the first place? `kerasnip_output_view()`'s `int_conformal_full()` support substitutes the model's own current point-prediction for the other output(s) as a placeholder. Because the placeholder equals what that head already predicts, its loss contribution for that one synthetic row is ~zero, so it should not be measurably disturbed while the target head still responds to the candidate value under test. This is a reasonable choice, not a proven one — treat the resulting intervals accordingly.


``` r
# Small subset to keep runtime reasonable in this vignette.
small_train <- train_dat[1:40, ]
small_new   <- cal_dat[1:3, ]

fit_small <- fit(wflow, data = small_train)
#> 2/2 - 0s - 119ms/step
#> 2/2 - 0s - 108ms/step
temp_view_small <- kerasnip_output_view(fit_small, "temperature")

conformal_full <- int_conformal_full(
  temp_view_small,
  train_data = small_train,
  control = control_conformal_full(method = "grid", trial_points = 15)
)
#> 2/2 - 0s - 121ms/step
predict(conformal_full, new_data = small_new, level = 0.90)
#> 1/1 - 0s - 71ms/step
#> 1/1 - 0s - 82ms/step
#> 2/2 - 0s - 102ms/step
#> 2/2 - 0s - 78ms/step
#> 2/2 - 0s - 80ms/step
#> 2/2 - 0s - 64ms/step
#> 2/2 - 0s - 83ms/step
#> 2/2 - 0s - 70ms/step
#> 2/2 - 0s - 55ms/step
#> 2/2 - 0s - 55ms/step
#> 2/2 - 0s - 63ms/step
#> 2/2 - 0s - 65ms/step
#> 2/2 - 0s - 56ms/step
#> 2/2 - 0s - 62ms/step
#> 2/2 - 0s - 58ms/step
#> 2/2 - 0s - 56ms/step
#> 2/2 - 0s - 61ms/step
#> 2/2 - 0s - 70ms/step
#> 2/2 - 0s - 58ms/step
#> 2/2 - 0s - 66ms/step
#> 2/2 - 0s - 56ms/step
#> 2/2 - 0s - 59ms/step
#> 2/2 - 0s - 66ms/step
#> 2/2 - 0s - 72ms/step
#> 2/2 - 0s - 58ms/step
#> 2/2 - 0s - 76ms/step
#> 2/2 - 0s - 65ms/step
#> 2/2 - 0s - 65ms/step
#> 2/2 - 0s - 60ms/step
#> 2/2 - 0s - 53ms/step
#> 2/2 - 0s - 100ms/step
#> 2/2 - 0s - 69ms/step
#> 2/2 - 0s - 59ms/step
#> 2/2 - 0s - 53ms/step
#> 2/2 - 0s - 69ms/step
#> 2/2 - 0s - 68ms/step
#> 2/2 - 0s - 73ms/step
#> 2/2 - 0s - 62ms/step
#> 2/2 - 0s - 57ms/step
#> 2/2 - 0s - 56ms/step
#> 2/2 - 0s - 63ms/step
#> 2/2 - 0s - 54ms/step
#> 2/2 - 0s - 49ms/step
#> 2/2 - 0s - 66ms/step
#> 2/2 - 0s - 64ms/step
#> 2/2 - 0s - 45ms/step
#> 2/2 - 0s - 61ms/step
#> 1/1 - 0s - 49ms/step
#> 2/2 - 0s - 52ms/step
#> 2/2 - 0s - 53ms/step
#> 2/2 - 0s - 63ms/step
#> 2/2 - 0s - 53ms/step
#> 2/2 - 0s - 55ms/step
#> 2/2 - 0s - 64ms/step
#> 2/2 - 0s - 63ms/step
#> 2/2 - 0s - 51ms/step
#> 2/2 - 0s - 63ms/step
#> 2/2 - 0s - 56ms/step
#> 2/2 - 0s - 57ms/step
#> 2/2 - 0s - 69ms/step
#> 2/2 - 0s - 59ms/step
#> 2/2 - 0s - 51ms/step
#> 2/2 - 0s - 74ms/step
#> 2/2 - 0s - 55ms/step
#> 2/2 - 0s - 58ms/step
#> 2/2 - 0s - 64ms/step
#> 2/2 - 0s - 51ms/step
#> 2/2 - 0s - 67ms/step
#> 2/2 - 0s - 85ms/step
#> 2/2 - 0s - 52ms/step
#> 2/2 - 0s - 65ms/step
#> 2/2 - 0s - 59ms/step
#> 2/2 - 0s - 57ms/step
#> 2/2 - 0s - 47ms/step
#> 2/2 - 0s - 51ms/step
#> 2/2 - 0s - 53ms/step
#> 2/2 - 0s - 52ms/step
#> 2/2 - 0s - 65ms/step
#> 2/2 - 0s - 55ms/step
#> 2/2 - 0s - 58ms/step
#> 2/2 - 0s - 67ms/step
#> 2/2 - 0s - 53ms/step
#> 2/2 - 0s - 53ms/step
#> 2/2 - 1s - 312ms/step
#> 2/2 - 0s - 56ms/step
#> 2/2 - 0s - 53ms/step
#> 2/2 - 0s - 68ms/step
#> 2/2 - 0s - 55ms/step
#> 2/2 - 0s - 55ms/step
#> 2/2 - 0s - 71ms/step
#> 2/2 - 0s - 66ms/step
#> 2/2 - 0s - 53ms/step
#> 2/2 - 0s - 52ms/step
#> 1/1 - 0s - 44ms/step
#> 2/2 - 0s - 43ms/step
#> 2/2 - 0s - 57ms/step
#> 2/2 - 0s - 67ms/step
#> 2/2 - 0s - 53ms/step
#> 2/2 - 0s - 48ms/step
#> 2/2 - 0s - 60ms/step
#> 2/2 - 0s - 61ms/step
#> 2/2 - 0s - 49ms/step
#> 2/2 - 0s - 51ms/step
#> 2/2 - 0s - 51ms/step
#> 2/2 - 0s - 50ms/step
#> 2/2 - 0s - 61ms/step
#> 2/2 - 0s - 56ms/step
#> 2/2 - 0s - 54ms/step
#> 2/2 - 0s - 59ms/step
#> 2/2 - 0s - 60ms/step
#> 2/2 - 0s - 61ms/step
#> 2/2 - 0s - 61ms/step
#> 2/2 - 0s - 53ms/step
#> 2/2 - 0s - 52ms/step
#> 2/2 - 0s - 46ms/step
#> 2/2 - 0s - 48ms/step
#> 2/2 - 0s - 53ms/step
#> 2/2 - 0s - 50ms/step
#> 2/2 - 0s - 51ms/step
#> 2/2 - 0s - 51ms/step
#> 2/2 - 0s - 64ms/step
#> 2/2 - 0s - 52ms/step
#> 2/2 - 0s - 64ms/step
#> 2/2 - 0s - 64ms/step
#> 2/2 - 0s - 53ms/step
#> 2/2 - 0s - 61ms/step
#> 2/2 - 0s - 67ms/step
#> 2/2 - 0s - 53ms/step
#> 2/2 - 0s - 52ms/step
#> 2/2 - 0s - 62ms/step
#> 2/2 - 0s - 48ms/step
#> 2/2 - 0s - 52ms/step
#> 2/2 - 0s - 62ms/step
#> 2/2 - 0s - 54ms/step
#> 2/2 - 0s - 53ms/step
#> 2/2 - 0s - 63ms/step
#> 2/2 - 0s - 51ms/step
#> 2/2 - 0s - 51ms/step
#> 2/2 - 0s - 58ms/step
#> # A tibble: 3 × 2
#>   .pred_lower .pred_upper
#>         <dbl>       <dbl>
#> 1      -0.610      0.241 
#> 2      -1.11       0.0851
#> 3      -0.375      0.665
```

Only `method = "grid"` is supported; `"iterative"` relies on `probably`'s private root-finding internals and is out of scope.

### `kerasnip_add_tailor()`: attach and forget

For routine use, `kerasnip_add_tailor()` wraps the view + fit + splice-back steps into the same `add_tailor()`-style workflow you would use for a single-output model — except it targets one named output, and every other output's columns pass through untouched:


``` r
tlr2 <- tailor() |> adjust_numeric_calibration(method = "linear")
tailored_wf <- kerasnip_add_tailor(wflow, tlr2, output = "temperature")

tailored_fit <- fit(tailored_wf, data = train_dat, data_calibration = cal_dat)
#> 7/7 - 0s - 15ms/step
#> 7/7 - 0s - 15ms/step
#> 3/3 - 0s - 41ms/step
predict(tailored_fit, new_data = cal_dat[1:5, ])
#> 1/1 - 0s - 47ms/step
#> 1/1 - 0s - 63ms/step
#> # A tibble: 5 × 2
#>   .pred_temperature .pred_humidity
#>               <dbl>          <dbl>
#> 1           -0.261          -0.165
#> 2           -0.451           0.805
#> 3            0.201          -0.515
#> 4            0.0167          0.160
#> 5           -0.308           0.191
```

`.pred_temperature` is calibrated; `.pred_humidity` is exactly what a plain, un-tailored `predict()` would have returned.

---

## Multistep forecasting models

A multistep model (see `vignette("multistep_forecasting")`) has a *single* outcome conceptually — but `predict()` returns it as a nested `.pred` list-column (one row per sample, one inner tibble per row holding `.step` and the forecasted value), which `tailor`'s `is.numeric()` check rejects just as firmly as a genuine multi-output shape, for an unrelated reason.


``` r
set.seed(42)
n_steps <- 200
timesteps <- 12
horizon <- 4
series <- tibble(value = sin(seq_len(n_steps) / 10) + rnorm(n_steps, sd = 0.05))

rec_step <- recipe(series) |>
  step_lead(value, lead = seq_len(horizon), prefix = "lead_") |>
  step_naomit(starts_with("lead_")) |>
  step_sequence(value, timesteps = timesteps, new_col = "window")

window_input <- function(input_shape) keras3::layer_input(shape = input_shape, name = "window_input")
lstm_block   <- function(tensor, units = 16) tensor |> keras3::layer_lstm(units = units)
step_output  <- function(tensor, units = 1) tensor |> keras3::layer_dense(units = units)

create_keras_functional_spec(
  model_name = "forecast_lstm",
  layer_blocks = list(
    window = window_input,
    lstm   = inp_spec(lstm_block, "window"),
    output = inp_spec(step_output, "lstm")
  ),
  mode = "regression"
)

step_spec <- forecast_lstm(lstm_units = 16, output_units = horizon, fit_epochs = 30) |>
  set_engine("keras")

split_step <- initial_time_split(series, prop = 0.8)
train_series <- training(split_step)
test_series  <- testing(split_step)

step_wflow <- workflow(rec_step, step_spec)
step_fit <- fit(step_wflow, data = train_series)
#> 5/5 - 0s - 85ms/step

# step_sequence() needs `timesteps` rows of leading history to produce a
# single prediction, so a preview slice must include at least that much
# context; this gives 6 rows with a full window.
preview_data <- test_series[seq_len(timesteps + 5), , drop = FALSE]

predict(step_fit, new_data = preview_data)
#> 1/1 - 0s - 205ms/step
#> # A tibble: 6 × 1
#>   .pred           
#>   <list>          
#> 1 <tibble [4 × 2]>
#> 2 <tibble [4 × 2]>
#> 3 <tibble [4 × 2]>
#> 4 <tibble [4 × 2]>
#> 5 <tibble [4 × 2]>
#> 6 <tibble [4 × 2]>
```

### `kerasnip_step_view()`: one forecast step as a standard single-output fit


``` r
step_2_view <- kerasnip_step_view(step_fit, step = 2)
predict(step_2_view, new_data = preview_data)
#> 1/1 - 0s - 45ms/step
#> # A tibble: 6 × 1
#>    .pred
#>    <dbl>
#> 1 -0.997
#> 2 -0.973
#> 3 -0.935
#> 4 -0.884
#> 5 -0.830
#> 6 -0.767
```

Unlike a multi-output model, a multistep model's per-step outcome columns (`lead_2_value`, ...) are recipe-*engineered* from a single raw column by `step_lead()` — they do not exist in raw data the way `output_1`/`output_2` do for a genuine multi-output model. `kerasnip_step_truth()` recovers the true future value at a given step by re-baking the fitted recipe:


``` r
truth <- kerasnip_step_truth(step_2_view, test_series)
head(truth)
#> [1] -0.9795517 -0.9710722 -0.9111726 -0.8897428 -0.8943562 -0.8710572
```

That is enough to calibrate manually, same as the multi-output case:


``` r
preds_step <- predict(step_2_view, new_data = train_series)
#> 5/5 - 0s - 49ms/step
truth_step <- kerasnip_step_truth(step_2_view, train_series)

cal_tbl <- tibble(truth = truth_step, .pred = preds_step$.pred) |>
  filter(!is.na(truth))

tlr_step <- tailor() |> adjust_numeric_calibration(method = "linear")
tlr_step_fit <- fit(tlr_step, cal_tbl, outcome = truth, estimate = .pred)

new_preds_step <- predict(step_2_view, new_data = preview_data)
#> 1/1 - 0s - 53ms/step
predict(tlr_step_fit, new_preds_step)
#> # A tibble: 6 × 1
#>    .pred
#>    <dbl>
#> 1 -1.01 
#> 2 -0.986
#> 3 -0.946
#> 4 -0.893
#> 5 -0.837
#> 6 -0.772
```

...and enough for `probably::int_conformal_split()`, exactly as with a multi-output view:


``` r
conformal_step <- int_conformal_split(step_2_view, cal_data = train_series)
#> 5/5 - 0s - 15ms/step
predict(conformal_step, new_data = preview_data, level = 0.90)
#> 1/1 - 0s - 46ms/step
#> # A tibble: 6 × 3
#>    .pred .pred_lower .pred_upper
#>    <dbl>       <dbl>       <dbl>
#> 1 -0.997      -1.09       -0.908
#> 2 -0.973      -1.06       -0.884
#> 3 -0.935      -1.02       -0.846
#> 4 -0.884      -0.974      -0.795
#> 5 -0.830      -0.919      -0.740
#> 6 -0.767      -0.856      -0.677
```

`probably::int_conformal_full()` is also supported for step views, with a different design from the multi-output case: a multistep model's step targets are not independent raw columns — every `lead_k_value` column is derived from the *same* single raw column by `step_lead()`. Testing a candidate value for one step means writing that candidate into the raw column at the appropriate future offset, which also (partially) supplies the targets for the *other* steps forecast from the same origin; those other steps' placeholders are the current model's own forecast, the same idea as the multi-output case's "other output(s)" placeholder. This is only supported when `step_lead()` and `step_sequence()` share a single source column, true of the model built above (and every multistep example in this package).


``` r
# Small subset to keep runtime reasonable in this vignette, but wide enough
# for the residual-variance model to see a representative range of
# predictions — too narrow a range makes it extrapolate wildly for new
# observations outside it. small_new needs at least `timesteps` rows of
# leading context, same as preview_data above.
small_train <- train_series[1:80, , drop = FALSE]
small_new <- test_series[seq_len(timesteps + 2), , drop = FALSE]

fit_small <- fit(step_wflow, data = small_train)
#> 3/3 - 0s - 140ms/step
step_2_view_small <- kerasnip_step_view(fit_small, step = 2)

conformal_step_full <- int_conformal_full(
  step_2_view_small,
  train_data = small_train,
  control = control_conformal_full(method = "grid", trial_points = 10)
)
#> 3/3 - 0s - 147ms/step
predict(conformal_step_full, new_data = small_new, level = 0.90)
#> 1/1 - 0s - 67ms/step
#> 1/1 - 0s - 44ms/step
#> 3/3 - 0s - 144ms/step
#> 3/3 - 0s - 119ms/step
#> 3/3 - 0s - 144ms/step
#> 3/3 - 0s - 141ms/step
#> 3/3 - 0s - 129ms/step
#> 3/3 - 0s - 129ms/step
#> 3/3 - 0s - 123ms/step
#> 3/3 - 0s - 137ms/step
#> 3/3 - 0s - 136ms/step
#> 3/3 - 0s - 157ms/step
#> 3/3 - 0s - 130ms/step
#> 3/3 - 0s - 129ms/step
#> 3/3 - 1s - 171ms/step
#> 3/3 - 0s - 135ms/step
#> 3/3 - 0s - 143ms/step
#> 3/3 - 0s - 140ms/step
#> 3/3 - 0s - 136ms/step
#> 3/3 - 0s - 117ms/step
#> 3/3 - 0s - 132ms/step
#> 3/3 - 1s - 404ms/step
#> 3/3 - 0s - 136ms/step
#> 3/3 - 0s - 130ms/step
#> 3/3 - 0s - 124ms/step
#> 3/3 - 0s - 124ms/step
#> 3/3 - 0s - 120ms/step
#> 3/3 - 0s - 116ms/step
#> 3/3 - 0s - 125ms/step
#> 3/3 - 0s - 124ms/step
#> 3/3 - 0s - 123ms/step
#> 3/3 - 0s - 107ms/step
#> 3/3 - 0s - 106ms/step
#> 3/3 - 0s - 110ms/step
#> 3/3 - 0s - 134ms/step
#> 3/3 - 0s - 112ms/step
#> 3/3 - 0s - 100ms/step
#> 3/3 - 0s - 116ms/step
#> 3/3 - 0s - 103ms/step
#> 3/3 - 0s - 121ms/step
#> 3/3 - 0s - 127ms/step
#> 3/3 - 0s - 110ms/step
#> 3/3 - 0s - 124ms/step
#> 3/3 - 0s - 104ms/step
#> 3/3 - 0s - 122ms/step
#> 3/3 - 0s - 118ms/step
#> 3/3 - 0s - 109ms/step
#> 3/3 - 0s - 91ms/step
#> 3/3 - 0s - 101ms/step
#> 3/3 - 0s - 105ms/step
#> 3/3 - 0s - 113ms/step
#> 3/3 - 0s - 119ms/step
#> 3/3 - 0s - 98ms/step
#> 3/3 - 0s - 101ms/step
#> 3/3 - 0s - 96ms/step
#> 3/3 - 0s - 100ms/step
#> 3/3 - 0s - 110ms/step
#> 3/3 - 0s - 111ms/step
#> 3/3 - 0s - 95ms/step
#> 3/3 - 0s - 104ms/step
#> 3/3 - 0s - 94ms/step
#> 3/3 - 0s - 86ms/step
#> # A tibble: 3 × 2
#>   .pred_lower .pred_upper
#>         <dbl>       <dbl>
#> 1       -1.49      -0.610
#> 2       -1.45      -0.607
#> 3       -1.40      -0.603
```

As with the multi-output case, only `method = "grid"` is supported.

### `kerasnip_add_tailor()` for one forecast step


``` r
tlr_step2 <- tailor() |> adjust_numeric_calibration(method = "linear")
tailored_step_wf <- kerasnip_add_tailor(step_wflow, tlr_step2, step = 2)

tailored_step_fit <- fit(tailored_step_wf, data = train_series)
#> 5/5 - 0s - 67ms/step
#> 5/5 - 0s - 71ms/step
predict(tailored_step_fit, new_data = preview_data)
#> 1/1 - 0s - 30ms/step
#> 1/1 - 0s - 41ms/step
#> # A tibble: 6 × 1
#>   .pred           
#>   <list>          
#> 1 <tibble [4 × 2]>
#> 2 <tibble [4 × 2]>
#> 3 <tibble [4 × 2]>
#> 4 <tibble [4 × 2]>
#> 5 <tibble [4 × 2]>
#> 6 <tibble [4 × 2]>
```

Step 2's forecasted value is calibrated in every row's nested tibble; every other step is left exactly as a plain `predict()` would have returned it.

---

## Cleanup


``` r
remove_keras_spec("climate_mlp")
#> Removed from parsnip registry objects: climate_mlp, climate_mlp_args, climate_mlp_encoding, climate_mlp_fit, climate_mlp_modes, climate_mlp_pkgs, climate_mlp_predict
#> Removed 'climate_mlp' from parsnip:::get_model_env()$models
remove_keras_spec("forecast_lstm")
#> Removed from parsnip registry objects: forecast_lstm, forecast_lstm_args, forecast_lstm_encoding, forecast_lstm_fit, forecast_lstm_modes, forecast_lstm_pkgs, forecast_lstm_predict
#> Removed 'forecast_lstm' from parsnip:::get_model_env()$models
```