Package {pmsims}

Title: Simulation-Based Sample Size Tools for Prediction Models
Version: 1.0.0
Description: Provides a flexible, simulation-based toolkit for exploring how much data are needed to develop reliable prediction models. It works by repeatedly generating data, fitting models, and evaluating performance to show how sample size affects predictive accuracy, calibration, and overfitting. The package supports continuous, binary, and time-to-event outcomes and can be used with both regression-based modelling approaches and machine-learning methods. It is designed to help researchers plan studies, assess feasibility, and build more robust and generalisable models. The methods are described in Olaniran et al. (2026) <doi:10.1186/s12874-026-02935-9> and Shamsutdinova et al. (2026) <doi:10.48550/arXiv.2602.23507>.
License: GPL (≥ 3)
URL: https://pmsims-package.github.io/pmsims/, https://github.com/pmsims-package/pmsims
BugReports: https://github.com/pmsims-package/pmsims/issues
Depends: R (≥ 4.1.0)
Imports: cli, ggplot2, lifecycle, mlpwr, pROC, stats, survival, timeROC, utils
Suggests: covr, DescTools, doParallel, foreach, glmnet, knitr, mlbench, mlr, randomForestSRC, ranger, rmarkdown, synthpop, testthat (≥ 3.0.0), tuneRanger, xgboost
Encoding: UTF-8
LazyData: true
LazyDataCompression: xz
VignetteBuilder: knitr
Config/testthat/edition: 3
Config/roxygen2/version: 8.0.0
NeedsCompilation: no
Packaged: 2026-09-08 19:30:59 UTC; k1644956
Author: Ewan Carr ORCID iD [aut, cre], Gordon Forbes ORCID iD [aut], Ridwan Olaniran ORCID iD [aut], Diana Shamsutdinova ORCID iD [aut], Daniel Stahl ORCID iD [aut], Sarah Markham ORCID iD [aut], Felix Zimmer ORCID iD [aut]
Maintainer: Ewan Carr <ewan.carr@kcl.ac.uk>
Repository: CRAN
Date/Publication: 2026-09-17 09:30:02 UTC

pmsims: Simulation-Based Sample Size Tools for Prediction Models

Description

logo

Provides a flexible, simulation-based toolkit for exploring how much data are needed to develop reliable prediction models. It works by repeatedly generating data, fitting models, and evaluating performance to show how sample size affects predictive accuracy, calibration, and overfitting. The package supports continuous, binary, and time-to-event outcomes and can be used with both regression-based modelling approaches and machine-learning methods. It is designed to help researchers plan studies, assess feasibility, and build more robust and generalisable models. The methods are described in Olaniran et al. (2026) doi:10.1186/s12874-026-02935-9 and Shamsutdinova et al. (2026) doi:10.48550/arXiv.2602.23507.

Author(s)

Maintainer: Ewan Carr ewan.carr@kcl.ac.uk (ORCID)

Authors:

See Also

Useful links:

default_model_generators Generate appropriate model based on input arguments

Description

default_model_generators Generate appropriate model based on input arguments

Usage

.xgb_cv_nrounds(
  dtrain,
  params,
  nrounds_max = 500L,
  nfold = 5L,
  early_stopping_rounds = 20L
)

Format

A named list of default model generator functions grouped by outcome type.

Value

default_models is a list containing built-in model generators for binary, continuous, and survival outcomes.

adaptive_startvalues Derive adaptive sample sizes

Description

adaptive_startvalues Derive adaptive sample sizes

Usage

adaptive_startvalues(
  output,
  aggregate_fun,
  var_bootstrap,
  target,
  ci_q = 0.975
)

Arguments

output

List-like object containing track_bisection, produced by calculate_bisection().

aggregate_fun

Function used to summarise replicate performance values (for example, mean or a quantile function).

var_bootstrap

Function returning the bootstrap variance of the aggregated performance.

target

Numeric target performance threshold.

ci_q

Numeric quantile for confidence-interval construction (default 0.975 gives a two-sided 95% interval).

Tuning function for a binary outcome model

Description

Finds the beta_signal and log-odds intercept (mu_lp) that jointly produce the target AUC and prevalence under the exact data-generating settings (complexity, nonlinear_strength, correlation, distribution, predictor_type).

Usage

binary_tuning(
  target_prevalence,
  target_performance,
  candidate_features,
  proportion_noise_features,
  complexity = 1,
  nonlinear_strength = NULL,
  correlation = 0.3,
  distribution = "normal",
  predictor_type = "continuous",
  binary_prevalence = 0,
  n_sim = 3e+05,
  n_validate = NULL,
  beta_interval = c(1e-04, 20),
  beta_tol = 1e-04,
  mu_tol = 1e-06,
  tolerance = 0.02
)

Arguments

target_prevalence

Target outcome prevalence. Must be in (0, 1).

target_performance

Target AUC (c-statistic). Must be in (0.5, 1).

candidate_features

Total number of predictors (signal + noise).

proportion_noise_features

Proportion of candidate_features that are noise predictors. Must be in [0, 1).

complexity

Integer 1-4. Default = 1.

nonlinear_strength

Fraction of signal variance carried by the nonlinear component (C2/C3 only), in [0, 1). When NULL (default), the complexity-level default is used: C1 = 0, C2 = 0.2, C3 = 0.3, C4 = 0.

correlation

Common pairwise predictor correlation. Defaults to the SAME value as generate_binary_data() so the two never silently disagree. Whatever you pass here must match the generator call.

distribution

Global continuous predictor distribution family. Defaults to match generate_binary_data().

predictor_type

"continuous" (default) or "binary". Defaults to match generate_binary_data().

binary_prevalence

Bernoulli probability for binary predictors. Defaults to match generate_binary_data().

n_sim

Sample size for the internal LP simulation. Larger values give more stable AUC estimates within the bisection. Default = 300 000.

n_validate

Sample size for the independent end-to-end validation. Defaults to n_sim.

beta_interval

Search interval c(lo, hi) for beta_signal. Default = c(1e-4, 20). The upper bound is doubled automatically if the AUC at hi is still below target_performance.

beta_tol

Absolute convergence tolerance for the beta bisection. Default = 1e-4.

mu_tol

Absolute convergence tolerance for the mu_lp root-finding. Default = 1e-6.

tolerance

Acceptable absolute deviation of the validated prevalence and AUC from their targets before a warning is raised. Default = 0.02.

Value

A named numeric vector:

mu_lp

Tuned log-odds intercept.

beta_signal

Tuned effect size.

prevalence_achieved

Prevalence from an independent generate_binary_data() validation draw.

auc_achieved

AUC of the oracle LP from the same draw.

var_lp_unit

Estimated Var(LP) at beta_signal = 1.

The Bisection Engine

Description

Runs a bisection search over sample size using repeated simulations and summaries of the chosen performance metric.

Usage

calculate_bisection(
  data_function = data_function,
  model_function = model_function,
  metric_function = metric_function,
  value_on_error = value_on_error,
  min_sample_size = min_sample_size,
  max_sample_size = max_sample_size,
  test_n = test_n,
  n_reps_total = n_reps_total,
  n_reps_per = n_reps_per,
  target_performance = target_performance,
  c_statistic,
  mean_or_assurance = mean_or_assurance,
  tol = 0.001,
  parallel = FALSE,
  cores = 20,
  verbose = FALSE,
  budget = TRUE
)

Arguments

data_function

Function taking a single argument, n, giving the training sample size, and returning a dataset that can be passed to model_function.

model_function

Function that fits a model to the dataset returned by data_function. It must take the generated dataset as its only argument and return a fitted model object.

metric_function

Function that evaluates predictive performance on test data. It must take three positional arguments in the order ⁠(test_data, fitted_model, model_name)⁠ and return a single numeric value. Optionally, users may set attr(metric_function, "value_on_error") to a single numeric fallback value to be returned if model fitting or metric evaluation fails during a simulation run.

value_on_error

Numeric fallback returned when a simulation run fails.

min_sample_size

Optional integer lower bound for the sample-size search. If supplied, max_sample_size must also be supplied.

max_sample_size

Optional integer upper bound for the sample-size search. If supplied, min_sample_size must also be supplied. Supplying both bounds defines the search space directly, so the adaptive starting-value search is skipped. Because the runtime estimate is extrapolated from that stage, no long-run warning is issued either.

test_n

Integer size of the test dataset used to evaluate model performance. This should usually be large enough that test-set variability is negligible relative to the training-sample search.

n_reps_total

Integer total number of simulation replications allocated to the search. The search evaluates approximately n_reps_total / n_reps_per candidate sample sizes.

n_reps_per

Integer number of simulation replications performed at each candidate sample size.

target_performance

Numeric target value for the chosen performance metric. The search aims to find the smallest sample size n for which the selected criterion is met relative to this threshold.

c_statistic

Optional numeric value used only by the internal start-value heuristics for some outcome and metric combinations. In most custom workflows this should be left as NULL.

mean_or_assurance

Character string specifying the criterion used to define the minimum sample size. Must be either "mean" or "assurance".

tol

Numeric tolerance controlling when the bisection loop stops.

parallel

Logical; if TRUE the per-sample-size simulations run in parallel via foreach.

cores

Integer number of cores to use when parallel = TRUE.

verbose

Logical flag passed to mlpwr; when TRUE verbose output is printed.

budget

Logical; if TRUE the algorithm halts once the evaluation budget is exhausted instead of using tol.

Value

A list containing the simulation results, performance summaries, optional tracking history, and the track_bisection records.

Calculate performance metrics Calculates the performance metrics for a model given a sample size n.

Description

Calculate performance metrics Calculates the performance metrics for a model given a sample size n.

Usage

calculate_metrics_perf(
  n,
  data_function,
  model_function,
  metric_function,
  value_on_error
)

Arguments

n

Integer sample size.

value_on_error

Numeric fallback returned if the metric cannot be computed.

Value

The calculated performance metric

mlpwr engine

Description

mlpwr engine

Usage

calculate_mlpwr(
  test_n,
  n_reps_total,
  n_reps_per,
  se_final,
  min_sample_size,
  max_sample_size,
  target_performance,
  c_statistic,
  mean_or_assurance,
  n_init,
  progress = TRUE,
  verbose,
  data_function,
  model_function,
  metric_function,
  value_on_error,
  ...
)

Arguments

test_n

Integer size of the test dataset used to evaluate model performance. This should usually be large enough that test-set variability is negligible relative to the training-sample search.

n_reps_total

Integer total number of simulation replications allocated to the search. The search evaluates approximately n_reps_total / n_reps_per candidate sample sizes.

n_reps_per

Integer number of simulation replications performed at each candidate sample size.

min_sample_size

Optional integer lower bound for the sample-size search. If supplied, max_sample_size must also be supplied.

max_sample_size

Optional integer upper bound for the sample-size search. If supplied, min_sample_size must also be supplied. Supplying both bounds defines the search space directly, so the adaptive starting-value search is skipped. Because the runtime estimate is extrapolated from that stage, no long-run warning is issued either.

target_performance

Numeric target value for the chosen performance metric. The search aims to find the smallest sample size n for which the selected criterion is met relative to this threshold.

c_statistic

Optional numeric value used only by the internal start-value heuristics for some outcome and metric combinations. In most custom workflows this should be left as NULL.

mean_or_assurance

Character string specifying the criterion used to define the minimum sample size. Must be either "mean" or "assurance".

n_init

Integer number of initial sample sizes simulated before the Gaussian process search begins.

progress

Logical flag controlling whether the mlpwr progress bar is shown.

verbose

Logical flag passed to mlpwr; when TRUE verbose output is printed.

data_function

Function taking a single argument, n, giving the training sample size, and returning a dataset that can be passed to model_function.

model_function

Function that fits a model to the dataset returned by data_function. It must take the generated dataset as its only argument and return a fitted model object.

metric_function

Function that evaluates predictive performance on test data. It must take three positional arguments in the order ⁠(test_data, fitted_model, model_name)⁠ and return a single numeric value. Optionally, users may set attr(metric_function, "value_on_error") to a single numeric fallback value to be returned if model fitting or metric evaluation fails during a simulation run.

value_on_error

Numeric fallback value used if model fitting or metric calculation fails.

...

Additional options passed to mlpwr::find.design().

mlpwr-bs Hybrid engine using bisection to determine initial range and mlpwr for search

Description

mlpwr-bs Hybrid engine using bisection to determine initial range and mlpwr for search

Usage

calculate_mlpwr_bs(
  test_n,
  n_reps_total,
  n_reps_per,
  se_final,
  min_sample_size,
  max_sample_size,
  target_performance,
  c_statistic,
  mean_or_assurance,
  progress = TRUE,
  verbose,
  data_function,
  model_function,
  metric_function,
  value_on_error,
  ...
)

Arguments

test_n

Integer size of the test dataset used to evaluate model performance. This should usually be large enough that test-set variability is negligible relative to the training-sample search.

n_reps_total

Integer total number of simulation replications allocated to the search. The search evaluates approximately n_reps_total / n_reps_per candidate sample sizes.

n_reps_per

Integer number of simulation replications performed at each candidate sample size.

min_sample_size

Optional integer lower bound for the sample-size search. If supplied, max_sample_size must also be supplied.

max_sample_size

Optional integer upper bound for the sample-size search. If supplied, min_sample_size must also be supplied. Supplying both bounds defines the search space directly, so the adaptive starting-value search is skipped. Because the runtime estimate is extrapolated from that stage, no long-run warning is issued either.

target_performance

Numeric target value for the chosen performance metric. The search aims to find the smallest sample size n for which the selected criterion is met relative to this threshold.

c_statistic

Optional numeric value used only by the internal start-value heuristics for some outcome and metric combinations. In most custom workflows this should be left as NULL.

mean_or_assurance

Character string specifying the criterion used to define the minimum sample size. Must be either "mean" or "assurance".

progress

Logical flag controlling whether the mlpwr progress bar is shown.

verbose

Logical flag passed to mlpwr; when TRUE verbose output is printed.

data_function

Function taking a single argument, n, giving the training sample size, and returning a dataset that can be passed to model_function.

model_function

Function that fits a model to the dataset returned by data_function. It must take the generated dataset as its only argument and return a fitted model object.

metric_function

Function that evaluates predictive performance on test data. It must take three positional arguments in the order ⁠(test_data, fitted_model, model_name)⁠ and return a single numeric value. Optionally, users may set attr(metric_function, "value_on_error") to a single numeric fallback value to be returned if model fitting or metric evaluation fails during a simulation run.

value_on_error

Numeric fallback value used if model fitting or metric calculation fails.

...

Additional options passed to mlpwr::find.design().

Value

List containing the combined bisection and mlpwr results (results, summaries, min_n, perf_n, and mlpwr_ds).

Tuning function for a continuous outcome model

Description

Finds the beta_signal value that produces a target large-sample R^2 under the exact data-generating settings (complexity, nonlinear_strength, correlation, distribution, predictor_type).

Usage

continuous_tuning(
  r2,
  candidate_features,
  proportion_noise_features,
  complexity = 1,
  nonlinear_strength = NULL,
  correlation = 0,
  distribution = "normal",
  predictor_type = "continuous",
  binary_prevalence = 0,
  n_sim = 1e+05
)

Arguments

r2

Target large-sample R^2 (proportion of variance explained). Must be in (0, 1).

candidate_features

Total number of predictors (signal + noise).

proportion_noise_features

Proportion of candidate_features that are noise predictors (zero coefficient). Must be in [0, 1).

complexity

Integer 1-4 controlling the functional form of the linear predictor passed to the data generator. Default = 1.

nonlinear_strength

Fraction of signal variance carried by the nonlinear component (C2/C3 only), in [0, 1). When NULL (default), the complexity-level default is used: C1 = 0, C2 = 0.2, C3 = 0.3, C4 = 0.

correlation

Common pairwise predictor correlation. Default = 0.

distribution

Global continuous predictor distribution family passed to the data generator. Default = "normal".

predictor_type

"continuous" (default) or "binary".

binary_prevalence

Bernoulli probability for binary predictors. Required (and used) only when predictor_type = "binary".

n_sim

Sample size used to estimate Var(LP). Larger values give more stable estimates. Default = 100 000.

Value

A named numeric vector:

beta_signal

Tuned effect size.

r2_achieved

Empirical R^2 verified in a large simulation.

var_lp_unit

Estimated Var(LP) at beta_signal = 1 (the scaling constant).

Create default data generating functions

Description

Create default data generating functions

Usage

default_data_generators(opts)

Arguments

opts

A list with two elements:

type

Outcome type: "continuous", "binary", or "survival".

args

Named list of arguments to pre-set on the corresponding generator function.

Value

A partially-applied generator function whose formals have been set to the values in opts$args.

Simulate binary outcome data

Description

Simulate binary outcome data

Usage

generate_binary_data(
  n,
  n_signal_parameters,
  noise_parameters,
  beta_signal,
  complexity = 1,
  nonlinear_strength = NULL,
  predictor_type = "continuous",
  binary_prevalence = 0,
  correlation = 0.3,
  distribution = "normal",
  mu_lp = 0,
  baseline_prob = 0.5
)

Arguments

n

Sample size.

n_signal_parameters

Number of signal predictors. These occupy the first n_signal_parameters columns (x1 ... x_S).

noise_parameters

Number of noise predictors (zero coefficient).

beta_signal

Base effect size / overall scale of the signal.

complexity

Integer 1-4 specifying the functional form of the linear predictor:

  1. Linear — lp = \alpha + \beta\sum_j x_j.

  2. Quadratic — linear + a quadratic nonlinear component.

  3. Quadratic + Interaction — linear + quadratic and pairwise interaction nonlinear component.

  4. Friedman — canonical Friedman (1991) benchmark.

nonlinear_strength

Fraction of signal variance carried by the nonlinear component (C2/C3 only), in [0, 1). When NULL (default), the complexity-level default is used: C1 = 0, C2 = 0.2, C3 = 0.3, C4 = 0. Ignored for C1 (pure linear) and C4 (canonical Friedman).

predictor_type

Type of predictors: "continuous" (default) or "binary". When "binary", all predictors are drawn as Bernoulli(binary_prevalence); binary_prevalence must be in (0, 1] and distribution is ignored.

binary_prevalence

Scalar in (0, 1]. Bernoulli probability applied to all predictors when predictor_type = "binary". Default = 0.

correlation

Scalar in [-1, 1]. Common pairwise correlation applied via a Gaussian copula (equicorrelation, rank-based Cholesky). Default = 0.3. Set to 0 for independence.

distribution

Distribution family for all continuous predictors. Default = "normal". For complexity 4, if left at "normal" the framework uses "uniform" (Friedman canonical).

mu_lp

Intercept on the log-odds scale. Default = 0.

baseline_prob

Nominal baseline event probability (documentation only; the realised probability is determined by mu_lp).

Value

A data frame with columns y (0/1), x1, x2, ...

Simulate continuous outcome data

Description

Simulate continuous outcome data

Usage

generate_continuous_data(
  n,
  n_signal_parameters,
  noise_parameters,
  beta_signal,
  complexity = 1,
  nonlinear_strength = NULL,
  predictor_type = "continuous",
  binary_prevalence = 0,
  correlation = 0.3,
  distribution = "normal",
  intercept = 0
)

Arguments

n

Sample size.

n_signal_parameters

Number of signal predictors. These occupy the first n_signal_parameters columns (x1 ... x_S).

noise_parameters

Number of noise predictors (zero coefficient).

beta_signal

Base effect size / overall scale of the signal.

complexity

Integer 1-4 specifying the functional form of the linear predictor:

  1. Linear — lp = \alpha + \beta\sum_j x_j.

  2. Quadratic — linear + a quadratic nonlinear component.

  3. Quadratic + Interaction — linear + quadratic and pairwise interaction nonlinear component.

  4. Friedman — canonical Friedman (1991) benchmark.

nonlinear_strength

Fraction of signal variance carried by the nonlinear component (C2/C3 only), in [0, 1). When NULL (default), the complexity-level default is used: C1 = 0, C2 = 0.2, C3 = 0.3, C4 = 0. Ignored for C1 (pure linear) and C4 (canonical Friedman).

predictor_type

Type of predictors: "continuous" (default) or "binary". When "binary", all predictors are drawn as Bernoulli(binary_prevalence); binary_prevalence must be in (0, 1] and distribution is ignored.

binary_prevalence

Scalar in (0, 1]. Bernoulli probability applied to all predictors when predictor_type = "binary". Default = 0.

correlation

Scalar in [-1, 1]. Common pairwise correlation applied via a Gaussian copula (equicorrelation, rank-based Cholesky). Default = 0.3. Set to 0 for independence.

distribution

Distribution family for all continuous predictors. Default = "normal". For complexity 4, if left at "normal" the framework uses "uniform" (Friedman canonical).

intercept

Scalar intercept added to the linear predictor. Default = 0.

Value

A data frame with columns y, x1, x2, ...

References

Friedman, J. H. (1991). Multivariate adaptive regression splines. The Annals of Statistics, 19(1), 1-67. doi:10.1214/aos/1176347963

Construct the linear predictor

Description

Construct the linear predictor

Usage

generate_linear_predictor(
  X,
  n_signal_parameters,
  noise_parameters,
  intercept,
  beta_signal,
  complexity,
  nonlinear_strength = NULL
)

Arguments

X

n x p predictor matrix (colnames: x1, x2, ...).

n_signal_parameters

Signal predictor count.

noise_parameters

Noise predictor count.

intercept

Scalar intercept (log-odds intercept for binary, log-hazard intercept for survival, mean intercept for continuous).

beta_signal

Base effect size (overall scale). The tuning routines scale this to hit the target oracle metric.

complexity

Integer 1-4.

nonlinear_strength

Fraction of signal variance carried by the nonlinear component (C2/C3 only), in [0, 1). NULL uses the complexity-level default in COMPLEXITY_NONLINEAR_STRENGTH_DEFAULTS.

Details

Why variance fraction rather than an R^2 split. For a continuous outcome whose target is R^2, splitting the signal by latent-scale variance is exact: R^2 is a variance ratio, so a correctly specified linear model recovers exactly the linear share. For binary (AUC) and survival (Harrell's C) outcomes that logic is only approximate, because AUC and C are rank/link-mediated. What carries over is the key property: the nonlinear component is built to be inaccessible to any linear-in-X model, so the discrimination it carries genuinely requires a nonlinear learner and more sample size.

Construction. The linear predictor is

lp = \alpha + \beta\,\Bigl(\,\underbrace{\textstyle\sum_j x_j}_{L} \; + \; \kappa\,\mathrm{sd}(L)\, \underbrace{\tilde N}_{N\,\mathrm{std}} \Bigr)

where L = \sum_j x_j is the linear score (identical to complexity 1) and \tilde N is the nonlinear aggregate (\sum_j x_j^2 for C2; plus \sum_{j<k} x_j x_k for C3) residualised against the full design [1, x_1, \ldots, x_S] and standardised to unit SD. Because L and \tilde N are orthogonal, the nonlinear variance fraction is

f = \frac{\kappa^2}{1 + \kappa^2},\qquad\text{equivalently}\qquad \kappa = \sqrt{\frac{f}{1 - f}}.

The user supplies nonlinear_strength = f (e.g. 0.2 = 20% of the signal variance is nonlinear) and \kappa is computed internally so the realised fraction equals f exactly.

Because the bracket is fixed given X, lp is linear in beta_signal, so the tuning routines (which scale beta_signal to hit the target oracle AUC / C-index) work unchanged.

Calibrating difficulty. f sets the difficulty order but not, by itself, a precise AUC/C-index gap: the metric reduction a given f produces also depends on the tuned scale, the prevalence/censoring, and the predictor family. To target a specific linear-accessible metric, calibrate f per complexity by simulation (raise it until a fitted GLM/Cox on the linear terms reaches the desired value while beta_signal keeps the oracle metric on target).

Complexity 1 — Linear: lp = \alpha + \beta \sum_j x_j.

Complexity 4 — Friedman (1991) (canonical, nonlinear_strength ignored): lp = \alpha + \beta[10\sin(\pi x_1 x_2) + 20(x_3-0.5)^2 + 10x_4 + 5x_5]. Friedman contains linear-accessible terms (10x_4 + 5x_5); to place C4 on the same controlled ladder as C2/C3, residualise the Friedman vector against Xs and apply the same kappa split.

Value

Numeric vector of length n.

References

Friedman, J. H. (1991). Multivariate adaptive regression splines. The Annals of Statistics, 19(1), 1-67.

Generate the n x p predictor matrix

Description

Draws all predictors from a single global distribution family, then optionally applies an equicorrelation structure via a Gaussian copula.

Usage

generate_predictors(
  n,
  n_signal_parameters,
  noise_parameters,
  complexity = 1,
  predictor_type = "continuous",
  binary_prevalence = 0,
  correlation = 0.3,
  distribution = "normal"
)

Arguments

n

Sample size.

n_signal_parameters

Number of signal predictors.

noise_parameters

Number of noise predictors.

complexity

Integer 1-4 (used to resolve the C4 distribution default).

predictor_type

"continuous" (default) or "binary".

binary_prevalence

Bernoulli probability; used when predictor_type = "binary".

correlation

Scalar common pairwise correlation; 0 = independent.

distribution

Global continuous distribution family; used when predictor_type = "continuous".

Value

Named n x p numeric matrix (column names: x1, x2, ...).

Simulate survival outcome data

Description

Simulate survival outcome data

Usage

generate_survival_data(
  n,
  n_signal_parameters,
  noise_parameters,
  beta_signal,
  baseline_hazard,
  censoring_rate,
  complexity = 1,
  nonlinear_strength = NULL,
  predictor_type = "continuous",
  binary_prevalence = 0,
  correlation = 0.3,
  distribution = "normal",
  intercept = 0
)

Arguments

n

Sample size.

n_signal_parameters

Number of signal predictors. These occupy the first n_signal_parameters columns (x1 ... x_S).

noise_parameters

Number of noise predictors (zero coefficient).

beta_signal

Base effect size / overall scale of the signal.

baseline_hazard

Baseline hazard rate (exponential survival model).

censoring_rate

Administrative censoring proportion in (0, 1).

complexity

Integer 1-4 specifying the functional form of the linear predictor:

  1. Linear — lp = \alpha + \beta\sum_j x_j.

  2. Quadratic — linear + a quadratic nonlinear component.

  3. Quadratic + Interaction — linear + quadratic and pairwise interaction nonlinear component.

  4. Friedman — canonical Friedman (1991) benchmark.

nonlinear_strength

Fraction of signal variance carried by the nonlinear component (C2/C3 only), in [0, 1). When NULL (default), the complexity-level default is used: C1 = 0, C2 = 0.2, C3 = 0.3, C4 = 0. Ignored for C1 (pure linear) and C4 (canonical Friedman).

predictor_type

Type of predictors: "continuous" (default) or "binary". When "binary", all predictors are drawn as Bernoulli(binary_prevalence); binary_prevalence must be in (0, 1] and distribution is ignored.

binary_prevalence

Scalar in (0, 1]. Bernoulli probability applied to all predictors when predictor_type = "binary". Default = 0.

correlation

Scalar in [-1, 1]. Common pairwise correlation applied via a Gaussian copula (equicorrelation, rank-based Cholesky). Default = 0.3. Set to 0 for independence.

distribution

Distribution family for all continuous predictors. Default = "normal". For complexity 4, if left at "normal" the framework uses "uniform" (Friedman canonical).

intercept

Scalar intercept added to the linear predictor. Default = 0.

Value

A data frame with columns time, event (0 = censored, 1 = event), x1, x2, ...

get_min_sample_size: Heuristic starting-n for binary/continuous/survival prediction

Description

get_min_sample_size: Heuristic starting-n for binary/continuous/survival prediction

Usage

get_min_sample_size(
  npar,
  prevalence = NULL,
  c_stat = NULL,
  calibration_slope = NULL,
  epv_value = NULL,
  outcome_type = c("binary", "survival", "continuous")
)

Arguments

npar

Integer; number of predictors in the model.

prevalence

Numeric in ⁠[0, 1]⁠; optional event rate or case fraction used for EPV calculations.

c_stat

Numeric in (0.5, 1]; anticipated discrimination (C-statistic). Lower values inflate the heuristic.

calibration_slope

Numeric; anticipated calibration slope. Values below 1 trigger a modest inflation.

epv_value

Numeric; target events-per-variable (EPV) value applied when prevalence is supplied.

outcome_type

Character string; must be one of "binary", "survival", or "continuous".

Value

Integer recommended starting value from which to calculate the minimum sample size.

Objective function for pmsims

Description

Computes the objective value

\left| M(n) - M^* - \lambda\,C(n) \right|,

where M(n) is performance at sample size n, M^* is the target performance, C(n) is a cost term increasing with n, and \lambda (set via penalty_weight) controls the penalty on large samples.

Usage

objective_function(
  n,
  penalty_weight,
  target_performance,
  min_sample_size,
  max_sample_size,
  value_on_error,
  data_function = NULL,
  model_function = NULL,
  metric_function = NULL
)

Arguments

n

Integer sample size.

penalty_weight

Penalty weight \lambda on the cost term C(n); 0 minimises only the absolute performance gap.

target_performance

Target performance M^* (numeric).

min_sample_size

Minimum sample size considered (lower bound of search).

max_sample_size

Maximum sample size considered (upper bound of search).

value_on_error

Value to return if the objective cannot be evaluated.

data_function

Data generator passed to calculate_metrics_perf().

model_function

Model-fitting function passed to calculate_metrics_perf().

metric_function

Metric function passed to calculate_metrics_perf().

Value

A single numeric: the objective value at n.

Examples

# Example usage (assuming helper functions exist):
# objective_function(
#   n = 500,
#   penalty_weight = 0.1,
#   target_performance = 0.75,
#   min_sample_size = 100,
#   max_sample_size = 5000,
#   value_on_error = Inf
# )

Parse and validate input specifications

Description

This function validates the provided data, model, and metric specifications, and returns corresponding generator functions for each. It ensures that all required inputs are provided and correctly configured.

Usage

parse_inputs(data_spec, metric, model)

Arguments

data_spec

A list containing two elements:

type

A character string indicating the outcome type.

args

A list of arguments to be passed to the data-generating function.

metric

A character vector specifying one or more metrics to be used. Currently, only the first element is used.

model

A character string specifying the model to be used.

Details

This function calls default_data_generators(), default_model_generators(), and default_metric_generator() to construct the appropriate functions based on the supplied inputs.

Value

A list containing three elements:

data_function

The data-generating function.

model_function

The model-generating function.

metric_function

The metric function corresponding to the chosen metric.

Plot sample-size learning curves for pmsims outputs

Description

Produces a ggplot showing the simulated points and the fitted Gaussian-process learning curve stored inside a pmsims object. Optionally returns the underlying data instead of drawing the plot.

Usage

## S3 method for class 'pmsims'
plot(x, metric_label = NULL, plot = TRUE, ...)

Arguments

x

A pmsims object returned by simulate_binary(), simulate_continuous(), simulate_survival(), or simulate_custom().

metric_label

Optional string used for the y-axis label when the object does not already record the metric name.

plot

Logical; if TRUE (default) the function prints the plot. If FALSE, the data used to build the plot are returned instead of drawing anything.

...

Currently unused.

Value

Invisibly returns the ggplot object when plot = TRUE. When plot = FALSE, returns a list with two data frames: observed_data (simulated points) and predicted_data (Gaussian-process predictions).

Precomputed sample-size simulations

Description

Results from four completed sample-size searches, one for each of the package's entry points. A search repeatedly simulates datasets, fits a prediction model to each and evaluates its performance across a range of candidate sample sizes, so a realistic run takes minutes to hours. These objects were computed once, with the calls shown under Source, so that the examples and the vignette can demonstrate the output without recomputing it.

Usage

binary_example

continuous_example

survival_example

custom_example

Format

A list of class "pmsims". The most useful components are:

min_n

The estimated minimum sample size.

perf_n

Expected performance at min_n.

target_performance

The performance target the search aimed for.

metric

The performance metric used.

model

The model fitted to each simulated dataset.

mlpwr_ds

The sampled designs and simulated performance values behind the fitted curve, used by plot().

summaries

Aggregated performance summaries across replications.

simulation_time

Elapsed time of the original run, in seconds.

Objects produced by the wrapper functions additionally record the data-generating configuration (for example outcome_prevalence, correlation and complexity); see simulate_binary() for the full set.

An object of class pmsims of length 36.

An object of class pmsims of length 35.

An object of class pmsims of length 37.

An object of class pmsims of length 24.

Details

Each object is a "pmsims" object, as returned by simulate_binary(), simulate_continuous(), simulate_survival() and simulate_custom(), and can be used with print() and plot() in the usual way.

Source

Generated by data-raw/precomputed-examples.R.

binary_example: 

set.seed(123)
simulate_binary(
  signal_parameters = 20,
  noise_parameters = 0,
  complexity = 1,
  data_control = list(correlation = 0.3),
  outcome_prevalence = 0.30,
  maximum_achievable_cstatistic = 0.80,
  model = "glm",
  metric = "calibration_slope",
  target_performance = 0.85,
  n_reps_total = 1000,
  mean_or_assurance = "assurance"
)

continuous_example: 

set.seed(123)
simulate_continuous(
  signal_parameters = 15,
  noise_parameters = 0,
  complexity = 1,
  data_control = list(correlation = 0.3),
  maximum_achievable_rsquared = 0.50,
  model = "lm",
  metric = "calibration_slope",
  target_performance = 0.95,
  n_reps_total = 1000,
  mean_or_assurance = "assurance"
)

survival_example: 

set.seed(123)
simulate_survival(
  signal_parameters = 15,
  noise_parameters = 0,
  complexity = 1,
  data_control = list(correlation = 0.3),
  maximum_achievable_cindex = 0.70,
  baseline_hazard = 0.01,
  censoring_rate = 0.30,
  model = "coxph",
  metric = "calibration_slope",
  target_performance = 0.90,
  n_reps_total = 1000,
  mean_or_assurance = "assurance"
)

custom_example: a linear model with three independent predictors and a population R^2 of 0.5, targeting a calibration slope of 0.9 with 80% assurance. The search uses bounds of 25 to 1,000 participants, 1,000 replications and 30,000 test observations. See simulate_custom() for the generating functions and executable call (seed 123).

See Also

simulate_binary(), simulate_continuous(), simulate_survival(), simulate_custom()

Examples

binary_example
binary_example$min_n

Print a pmsims result

Description

Print a pmsims result

Usage

## S3 method for class 'pmsims'
print(x, ..., max_width = 80, verbose = FALSE)

Arguments

x

A pmsims object.

...

Currently unused.

max_width

Maximum console width used for the summary.

verbose

Logical. If TRUE, add implementation-level detail: internal metric identifiers, the engine settings used for the search, and any quantities recorded on an internal search scale. summary.pmsims() is the same display with verbose = TRUE by default.

Value

x, invisibly.

Minimum sample size for binary-outcome prediction models

Description

Compute the minimum sample size required to develop a prediction model with a binary outcome. The function wraps a simulation-based engine that combines a bisection search with Gaussian-process curve fitting. From user inputs (outcome prevalence, maximum achievable performance, target performance, etc.) it constructs a data-generating function, a model-fitting function, and a metric function, then searches for the smallest n that meets the chosen performance criterion.

Usage

simulate_binary(
  signal_parameters,
  noise_parameters = 0,
  complexity = 1,
  data_control = NULL,
  outcome_prevalence,
  maximum_achievable_cstatistic,
  model = c("glm", "lasso", "ridge", "rf", "xgboost"),
  metric = "calibration_slope",
  target_performance,
  n_reps_total = 1000,
  mean_or_assurance = "assurance",
  ...
)

Arguments

signal_parameters

Integer. Number of candidate predictors associated with the outcome (i.e., true signal features).

noise_parameters

Integer. Number of candidate predictors not associated with the outcome (noise features). Default is 0.

complexity

Integer in 1:4 selecting the data-generating signal structure (see Data control). Default 1.

data_control

Optional named list controlling the predictors (see Data control). Default NULL (generator defaults).

outcome_prevalence

Numeric in (0, 1). Target prevalence of the binary outcome in the intended modelling context.

maximum_achievable_cstatistic

Numeric in (0, 1). Maximum achievable C-statistic with effectively unlimited data. This is used to calibrate the data-generating mechanism and is not the minimum acceptable threshold.

model

Character string specifying the modelling algorithm. One of "glm" (logistic regression), "lasso", "ridge", "rf" (random forest), or "xgboost" (gradient-boosted trees).

metric

Character string naming the performance metric used to assess the sample size; defaults to "calibration_slope". Metric identifiers use one canonical form throughout the package, such as "calibration_slope", "calibration_in_the_large", "auc", "r2", "cindex", and "csse".

"calibration_slope" is the slope from regressing the observed outcome on the model's linear predictor in held-out data; 1 indicates perfect calibration, and values below 1 indicate overfitting. Note that for the machine-learning models ("lasso", "ridge", "rf", "xgboost") this is converted internally to the calibration slope squared error for optimisation and translated back before results are returned; you don't need to do anything, and target_performance is still given on the calibration slope scale. Results derived this way carry a footnote marker in the printed output.

"csse" is the calibration slope squared error, -(1 - s)^2 for a calibration slope s, so that larger is better and 0 is perfect calibration. It can be requested directly, which is mainly useful for advanced use and for comparison against the internal conversion described above. When requesting it directly you are responsible for supplying target_performance on the CSSE scale: a calibration slope target of 0.9 corresponds to a CSSE target of -0.01. No adjustment is applied on your behalf, and results are reported on the CSSE scale.

target_performance

Numeric. Minimum acceptable value of the selected performance metric M^*; the algorithm searches for the smallest n meeting the chosen criterion with respect to this threshold.

n_reps_total

Integer. Total number of simulation replications used by the engine across the search.

mean_or_assurance

Character string, either "mean" or "assurance". Controls whether the minimum n is defined by the mean-based criterion or the assurance-based criterion (with the assurance level \delta controlled by the engine's defaults or additional arguments in ...).

...

Additional options passed to simulate_custom() (e.g., assurance level \delta, per-iteration settings).

Value

An object of class "pmsims" containing the estimated minimum sample size and simulation diagnostics (inputs, fitted GP curve, intermediate evaluations, and summary metrics).

Criteria

Two formulations are supported.

Here, M is the chosen performance metric and the probability/expectation is over repeated samples of training data of size n. The assurance criterion explicitly accounts for variability across training sets; models with higher variance typically require larger n to satisfy it.

Data control

complexity selects the signal structure of the data-generating mechanism: 1 purely linear, 2 linear + quadratic, 3 linear + quadratic + interaction, 4 the Friedman function. data_control is an optional list fine-tuning the predictors:

nonlinear_strength

Numeric in ⁠[0, 1)⁠. Fraction of the signal variance carried by the nonlinear, linearly-inaccessible component. Applies to complexity 2 and 3 only; ignored (with a warning) for 1 and 4. If omitted, the generator's per-complexity default is used.

correlation

Numeric in [-1, 1]. Pairwise correlation among the candidate predictors. Default 0.3.

predictor_distribution

One of "normal", "uniform", "binary", "exponential", "lognormal", "t", "laplace". "binary" selects 0/1 predictors and requires binary_predictor_prevalence; any other value selects continuous predictors from that family. Default "normal".

binary_predictor_prevalence

Numeric in ⁠(0, 1)⁠. Prevalence of the binary predictors; required when predictor_distribution = "binary", ignored (with a warning) otherwise. Note: binary predictors are incompatible with complexity 2/3 because squaring a 0/1 variable returns itself.

See Also

simulate_continuous(), simulate_survival(), simulate_custom()

Examples


set.seed(123)
est <- simulate_binary(
  signal_parameters = 3,
  noise_parameters = 0,
  complexity = 1,
  data_control = list(correlation = 0),
  outcome_prevalence = 0.50,
  maximum_achievable_cstatistic = 0.80,
  model = "glm",
  metric = "calibration_slope",
  target_performance = 0.9,
  mean_or_assurance = "assurance",
  min_sample_size = 50,
  max_sample_size = 1000,
  n_reps_total = 1000,
  test_n = 30000,
  progress = FALSE
)
est
est$min_n
plot(est)

Minimum sample size for continuous‐outcome prediction models

Description

Compute the minimum sample size required to develop a prediction model with a continuous outcome. This wraps the same simulation engine as simulate_binary(), combining bisection search with Gaussian-process learning-curve modelling. From user inputs (maximum achievable performance, target performance, etc.) it constructs a data-generating function, model-fitting function, and metric function, then searches for the smallest n meeting the chosen criterion.

Usage

simulate_continuous(
  signal_parameters,
  noise_parameters = 0,
  complexity = 1,
  data_control = NULL,
  maximum_achievable_rsquared,
  model = c("lm", "lasso", "ridge", "rf", "xgboost"),
  metric = "calibration_slope",
  target_performance,
  n_reps_total = 1000,
  mean_or_assurance = "assurance",
  ...
)

Arguments

signal_parameters

Integer. Number of candidate predictors associated with the outcome (i.e., true signal features).

noise_parameters

Integer. Number of candidate predictors not associated with the outcome (noise features). Default is 0.

complexity

Integer in 1:4 selecting the data-generating signal structure (see Data control). Default 1.

data_control

Optional named list controlling the predictors (see Data control). Default NULL (generator defaults).

maximum_achievable_rsquared

Numeric in (0, 1). Maximum achievable R^2 with effectively unlimited data. This is used to calibrate the data-generating mechanism and is not the minimum acceptable threshold.

model

Character string specifying the modelling algorithm. One of "lm" (linear regression), "lasso", "ridge", "rf" (random forest), or "xgboost" (gradient-boosted trees).

metric

Character string naming the performance metric used to assess the sample size; defaults to "calibration_slope". Metric identifiers use one canonical form throughout the package, such as "calibration_slope", "calibration_in_the_large", "auc", "r2", "cindex", and "csse".

"calibration_slope" is the slope from regressing the observed outcome on the model's linear predictor in held-out data; 1 indicates perfect calibration, and values below 1 indicate overfitting. Note that for the machine-learning models ("lasso", "ridge", "rf", "xgboost") this is converted internally to the calibration slope squared error for optimisation and translated back before results are returned; you don't need to do anything, and target_performance is still given on the calibration slope scale. Results derived this way carry a footnote marker in the printed output.

"csse" is the calibration slope squared error, -(1 - s)^2 for a calibration slope s, so that larger is better and 0 is perfect calibration. It can be requested directly, which is mainly useful for advanced use and for comparison against the internal conversion described above. When requesting it directly you are responsible for supplying target_performance on the CSSE scale: a calibration slope target of 0.9 corresponds to a CSSE target of -0.01. No adjustment is applied on your behalf, and results are reported on the CSSE scale.

target_performance

Numeric. Minimum acceptable value of the selected performance metric M^*; the algorithm searches for the smallest n meeting the chosen criterion with respect to this threshold.

n_reps_total

Integer. Total number of simulation replications used by the engine across the search.

mean_or_assurance

Character string, either "mean" or "assurance". Controls whether the minimum n is defined by the mean-based criterion or the assurance-based criterion (with the assurance level \delta controlled by the engine's defaults or additional arguments in ...).

...

Additional options passed to simulate_custom() (e.g., assurance level \delta, per-iteration settings).

Value

An object of class "pmsims" containing the estimated minimum sample size and simulation diagnostics (inputs, fitted GP curve, intermediate evaluations, and summary metrics).

Criteria

Two formulations are supported.

Here, M is the chosen performance metric and the probability/expectation is over repeated samples of training data of size n. The assurance criterion explicitly accounts for variability across training sets; models with higher variance typically require larger n to satisfy it.

Data control

complexity selects the signal structure of the data-generating mechanism: 1 purely linear, 2 linear + quadratic, 3 linear + quadratic + interaction, 4 the Friedman function. data_control is an optional list fine-tuning the predictors:

nonlinear_strength

Numeric in ⁠[0, 1)⁠. Fraction of the signal variance carried by the nonlinear, linearly-inaccessible component. Applies to complexity 2 and 3 only; ignored (with a warning) for 1 and 4. If omitted, the generator's per-complexity default is used.

correlation

Numeric in [-1, 1]. Pairwise correlation among the candidate predictors. Default 0.3.

predictor_distribution

One of "normal", "uniform", "binary", "exponential", "lognormal", "t", "laplace". "binary" selects 0/1 predictors and requires binary_predictor_prevalence; any other value selects continuous predictors from that family. Default "normal".

binary_predictor_prevalence

Numeric in ⁠(0, 1)⁠. Prevalence of the binary predictors; required when predictor_distribution = "binary", ignored (with a warning) otherwise. Note: binary predictors are incompatible with complexity 2/3 because squaring a 0/1 variable returns itself.

See Also

simulate_binary(), simulate_survival(), simulate_custom()

Examples


set.seed(123)
est <- simulate_continuous(
  signal_parameters = 3,
  noise_parameters = 0,
  complexity = 1,
  data_control = list(correlation = 0),
  maximum_achievable_rsquared = 0.50,
  model = "lm",
  metric = "calibration_slope",
  target_performance = 0.9,
  mean_or_assurance = "assurance",
  min_sample_size = 50,
  max_sample_size = 1000,
  n_reps_total = 1000,
  test_n = 30000,
  progress = FALSE
)
est
est$min_n
plot(est)

Minimum sample size for custom simulation workflows

Description

Compute the minimum sample size required to achieve a target level of predictive performance using user-defined simulation components. simulate_custom() is the low-level interface in pmsims: users supply a data-generating function, a model-fitting function, and a metric function, and the chosen search engine estimates the smallest n meeting the selected performance criterion.

Usage

simulate_custom(
  data_function,
  model_function,
  metric_function,
  target_performance,
  c_statistic = NULL,
  mean_or_assurance = "assurance",
  test_n = 30000,
  min_sample_size = NULL,
  max_sample_size = NULL,
  n_reps_total = 1000,
  n_reps_per = 20,
  method = "mlpwr",
  progress = TRUE,
  verbose = FALSE,
  ...
)

Arguments

data_function

Function taking a single argument, n, giving the training sample size, and returning a dataset that can be passed to model_function.

model_function

Function that fits a model to the dataset returned by data_function. It must take the generated dataset as its only argument and return a fitted model object.

metric_function

Function that evaluates predictive performance on test data. It must take three positional arguments in the order ⁠(test_data, fitted_model, model_name)⁠ and return a single numeric value. Optionally, users may set attr(metric_function, "value_on_error") to a single numeric fallback value to be returned if model fitting or metric evaluation fails during a simulation run.

target_performance

Numeric target value for the chosen performance metric. The search aims to find the smallest sample size n for which the selected criterion is met relative to this threshold.

c_statistic

Optional numeric value used only by the internal start-value heuristics for some outcome and metric combinations. In most custom workflows this should be left as NULL.

mean_or_assurance

Character string specifying the criterion used to define the minimum sample size. Must be either "mean" or "assurance".

test_n

Integer size of the test dataset used to evaluate model performance. This should usually be large enough that test-set variability is negligible relative to the training-sample search.

min_sample_size

Optional integer lower bound for the sample-size search. If supplied, max_sample_size must also be supplied.

max_sample_size

Optional integer upper bound for the sample-size search. If supplied, min_sample_size must also be supplied. Supplying both bounds defines the search space directly, so the adaptive starting-value search is skipped. Because the runtime estimate is extrapolated from that stage, no long-run warning is issued either.

n_reps_total

Integer total number of simulation replications allocated to the search. The search evaluates approximately n_reps_total / n_reps_per candidate sample sizes.

n_reps_per

Integer number of simulation replications performed at each candidate sample size.

method

Character string specifying the search engine. Defaults to "mlpwr".

progress

Logical flag controlling whether the mlpwr progress bar is shown for mlpwr-based methods.

verbose

Logical flag controlling engine-specific diagnostic output when supported. For the bisection engine, setting verbose = TRUE stores the iteration history on the returned object.

...

Additional arguments passed to the selected search engine.

Value

An object of class "pmsims" containing the estimated minimum sample size.

See Also

simulate_binary(), simulate_continuous(), simulate_survival()

Examples

# Three independent predictors with a population R-squared of 0.5.
data_fun <- function(n) {
  x1 <- rnorm(n)
  x2 <- rnorm(n)
  x3 <- rnorm(n)
  y <- (x1 + x2 + x3) / sqrt(3) + rnorm(n)
  data.frame(y = y, x1 = x1, x2 = x2, x3 = x3)
}

model_fun <- function(dat) {
  stats::lm(y ~ ., data = dat)
}

# Calibration slope evaluated on independent test data.
metric_fun <- function(test_data, fit, model) {
  preds <- stats::predict(fit, newdata = test_data)
  unname(stats::coef(stats::lm(test_data$y ~ preds))[2])
}
attr(metric_fun, "metric") <- "calibration_slope"


set.seed(123)
est <- simulate_custom(
  data_function = data_fun,
  model_function = model_fun,
  metric_function = metric_fun,
  target_performance = 0.9,
  mean_or_assurance = "assurance",
  min_sample_size = 25,
  max_sample_size = 1000,
  n_reps_total = 1000,
  test_n = 30000,
  progress = FALSE
)
est
est$min_n
plot(est)

Minimum sample size for survival‐outcome prediction models

Description

Compute the minimum sample size required to develop a prediction model with a time-to-event (survival) outcome. As with the other wrappers, this uses a simulation-based learning-curve approach with Gaussian-process surrogate modelling to locate the smallest n meeting the chosen performance criterion.

Usage

simulate_survival(
  signal_parameters,
  noise_parameters = 0,
  complexity = 1,
  data_control = NULL,
  maximum_achievable_cindex,
  baseline_hazard = 1,
  censoring_rate,
  model = c("coxph", "lasso", "ridge", "rf", "xgboost"),
  metric = "calibration_slope",
  target_performance,
  n_reps_total = 1000,
  mean_or_assurance = "assurance",
  ...
)

Arguments

signal_parameters

Integer. Number of candidate predictors associated with the outcome (i.e., true signal features).

noise_parameters

Integer. Number of candidate predictors not associated with the outcome (noise features). Default is 0.

complexity

Integer in 1:4 selecting the data-generating signal structure (see Data control). Default 1.

data_control

Optional named list controlling the predictors (see Data control). Default NULL (generator defaults).

maximum_achievable_cindex

Numeric in (0, 1). Maximum achievable C-index with effectively unlimited data. This is used to calibrate the data-generating mechanism and is not the minimum acceptable threshold.

baseline_hazard

Numeric greater than 0. Baseline hazard level used by the data-generating mechanism (e.g., the constant hazard in an exponential baseline). Larger values imply shorter event times, all else equal.

censoring_rate

Numeric in [0, 1). Proportion of individuals expected to be censored in the simulated datasets (administrative or random censoring). Higher values imply fewer observed events for a fixed n.

model

Character string specifying the modelling algorithm. One of "coxph" (Cox proportional hazards), "lasso", "ridge", "rf" (random survival forest), or "xgboost" (gradient boosting with a Cox objective).

metric

Character string naming the performance metric used to assess the sample size; defaults to "calibration_slope". Metric identifiers use one canonical form throughout the package, such as "calibration_slope", "calibration_in_the_large", "auc", "r2", "cindex", and "csse".

"calibration_slope" is the slope from regressing the observed outcome on the model's linear predictor in held-out data; 1 indicates perfect calibration, and values below 1 indicate overfitting. Note that for the machine-learning models ("lasso", "ridge", "rf", "xgboost") this is converted internally to the calibration slope squared error for optimisation and translated back before results are returned; you don't need to do anything, and target_performance is still given on the calibration slope scale. Results derived this way carry a footnote marker in the printed output.

"csse" is the calibration slope squared error, -(1 - s)^2 for a calibration slope s, so that larger is better and 0 is perfect calibration. It can be requested directly, which is mainly useful for advanced use and for comparison against the internal conversion described above. When requesting it directly you are responsible for supplying target_performance on the CSSE scale: a calibration slope target of 0.9 corresponds to a CSSE target of -0.01. No adjustment is applied on your behalf, and results are reported on the CSSE scale.

target_performance

Numeric. Minimum acceptable value of the selected performance metric M^*; the algorithm searches for the smallest n meeting the chosen criterion with respect to this threshold.

n_reps_total

Integer. Total number of simulation replications used by the engine across the search.

mean_or_assurance

Character string, either "mean" or "assurance". Controls whether the minimum n is defined by the mean-based criterion or the assurance-based criterion (with the assurance level \delta controlled by the engine's defaults or additional arguments in ...).

...

Additional options passed to simulate_custom() (e.g., assurance level \delta, per-iteration settings).

Value

An object of class "pmsims" containing the estimated minimum sample size and simulation diagnostics (inputs, fitted GP curve, intermediate evaluations, and summary metrics).

Criteria

Two formulations are supported.

Here, M is the chosen performance metric and the probability/expectation is over repeated samples of training data of size n. The assurance criterion explicitly accounts for variability across training sets; models with higher variance typically require larger n to satisfy it.

Data control

complexity selects the signal structure of the data-generating mechanism: 1 purely linear, 2 linear + quadratic, 3 linear + quadratic + interaction, 4 the Friedman function. data_control is an optional list fine-tuning the predictors:

nonlinear_strength

Numeric in ⁠[0, 1)⁠. Fraction of the signal variance carried by the nonlinear, linearly-inaccessible component. Applies to complexity 2 and 3 only; ignored (with a warning) for 1 and 4. If omitted, the generator's per-complexity default is used.

correlation

Numeric in [-1, 1]. Pairwise correlation among the candidate predictors. Default 0.3.

predictor_distribution

One of "normal", "uniform", "binary", "exponential", "lognormal", "t", "laplace". "binary" selects 0/1 predictors and requires binary_predictor_prevalence; any other value selects continuous predictors from that family. Default "normal".

binary_predictor_prevalence

Numeric in ⁠(0, 1)⁠. Prevalence of the binary predictors; required when predictor_distribution = "binary", ignored (with a warning) otherwise. Note: binary predictors are incompatible with complexity 2/3 because squaring a 0/1 variable returns itself.

See Also

simulate_binary(), simulate_continuous(), simulate_custom()

Examples


set.seed(123)
est <- simulate_survival(
  signal_parameters = 1,
  noise_parameters = 0,
  complexity = 1,
  data_control = list(correlation = 0),
  maximum_achievable_cindex = 0.70,
  baseline_hazard = 0.01,
  censoring_rate = 0.30,
  model = "coxph",
  metric = "calibration_slope",
  target_performance = 0.9,
  mean_or_assurance = "assurance",
  min_sample_size = 25,
  max_sample_size = 500,
  n_reps_total = 1000,
  test_n = 30000,
  progress = FALSE
)
est
est$min_n
plot(est)

Detailed summary of a pmsims result

Description

The same display as print.pmsims(), with the implementation-level detail that the default print method hides: internal metric identifiers, the engine settings used for the search, and any quantities recorded on an internal search scale.

Usage

## S3 method for class 'pmsims'
summary(object, ..., max_width = 80, verbose = TRUE)

Arguments

object

A pmsims object.

...

Currently unused.

max_width

Maximum console width used for the summary.

verbose

Logical. Set to FALSE for the default print display.

Value

object, invisibly.

Cox calibration slope at a landmark horizon

Description

Predicted survival at eval_time is obtained exactly as in the reference script – via survfit() for every proportional-hazards model, and read off the ensemble matrix for the two forests – then mapped to the complementary log-log scale and used as the sole covariate in a Cox model fitted to the validation outcome. The coefficient is the calibration slope.

Usage

survival_calib_slope_cox(
  data,
  fit,
  model,
  eval_time = NULL,
  train_data = NULL,
  eps = 1e-06
)

Arguments

data

Validation data with columns time, event and the predictors.

fit

Fitted model object.

model

Model string, as passed by the engines. Used only for messages and for the glmnet lambda; dispatch is on class.

eval_time

Landmark horizon. Defaults to median(data$time), as in the script.

train_data

Training data, same columns as data. REQUIRED for ridge, lasso and xgboost, which cannot reconstruct their baseline hazard from the fit object alone. Ignored for coxph, ranger and rfsrc.

eps

Clamp applied to predicted survival before the cloglog transform.

Details

Baselines come from the TRAINING data (the model's own survfit baseline for coxph; x/y at lambda.min for glmnet; a training-fitted coxph on the xgboost linear predictor).

Value

A single numeric calibration slope (1 = calibrated), or NaN.

Tuning function for a survival outcome model

Description

Finds the beta_signal that produces the target C-index under the exact data-generating settings (complexity, nonlinear_strength, correlation, distribution, predictor_type). The event rate is set exactly by quantile censoring, and the baseline hazard only fixes the time scale.

Usage

survival_tuning(
  target_prevalence,
  target_performance,
  candidate_features,
  proportion_noise_features,
  complexity = 1,
  nonlinear_strength = NULL,
  correlation = 0.3,
  distribution = "normal",
  predictor_type = "continuous",
  binary_prevalence = 0,
  n_sim = 50000,
  n_validate = 1e+05,
  beta_interval = c(1e-04, 20),
  beta_tol = 1e-04,
  tolerance = 0.02
)

Arguments

target_prevalence

Target event rate (proportion of events) in (0, 1). Equivalently 1 - censoring_rate.

target_performance

Target C-index in (0.5, 1).

candidate_features

Total number of predictors (signal + noise).

proportion_noise_features

Proportion of candidate_features that are noise predictors. Must be in [0, 1).

complexity

Integer 1-4. Default = 1.

nonlinear_strength

Fraction of signal variance carried by the nonlinear component (C2/C3 only), in [0, 1). When NULL (default), the complexity-level default is used: C1 = 0, C2 = 0.2, C3 = 0.3, C4 = 0.

correlation

Common pairwise predictor correlation. Default = 0.3 (to match generate_survival_data()).

distribution

Global continuous predictor distribution family. Default = "normal".

predictor_type

"continuous" (default) or "binary".

binary_prevalence

Bernoulli probability for binary predictors.

n_sim

Sample size for LP simulation and the bisection. Default 50000.

n_validate

Sample size for the independent validation. Default 100000.

beta_interval

Search interval c(lo, hi) for beta_signal. The upper bound is doubled automatically if the C-index at hi is below target.

beta_tol

Absolute convergence tolerance for the beta bisection.

tolerance

Acceptable absolute deviation of the validated event rate and C-index from their targets before a warning is raised. Default 0.02.

Value

A named numeric vector: lambda_opt (baseline hazard / time scale), beta_signal, event_rate, cindex, var_lp_unit.