Tutorial: Eliciting Bayesian Priors with ElicitationWizard

Setup

Launch the app:

library(ElicitationWizard)
run_elicitation()

If you have not set BASE_URL and API_KEY as environment variables, expand the API Connection panel and enter them now.

Step 1 — Define the Clinical Context

Expand the Clinical Context panel. The default text describes a metastatic hormone-resistant prostate cancer (HRPC) multi-center RCT. Edit this to match your own study if needed.

Disease: Metastatic hormone-resistant prostate cancer (HRPC)
Treatment: Control arm (placebo/standard care)
Population: Adult oncology patients
Study: Multi-center RCT

The clinical context is injected verbatim into every expert’s elicitation prompt. Be as specific as possible — endpoint definition, patient population, and comparator all influence the prior a domain expert would choose.

Step 2 — Specify the Model and Parameters

Expand the Result Expectations panel. This tells the experts exactly which parameters to provide priors for, and in what format.

The default specifies the Poisson-Gamma hierarchical model:

Model:
Each patient i in site j has AE count:
y_ij ~ Poisson(lambda_j)
Site-specific rates: lambda_j ~ Gamma(alpha, beta)
REQUIRED: alpha ~ Exponential(rate_alpha), beta ~ Exponential(rate_beta)

The REQUIRED: line instructs the LLM to provide priors specifically for rate_alpha and rate_beta, the hyperprior rates of the Exponential distributions on alpha and beta.

Step 3 — Configure Experts

In the Expert Configuration section, choose how many LLM experts to query (1–4). Each expert is assigned an independent model and temperature:

• Model — select from the available elicitation models
• Temperature — controls response diversity; higher values (e.g. 1.0) produce more varied priors; lower values (e.g. 0.1) produce more conservative, deterministic responses

Tip: When using the same model for multiple experts, set different temperatures to encourage diverse prior proposals rather than identical responses.

Step 4 — Elicit Priors

Click Elicit Priors. All experts are queried in parallel — the button is disabled and each expert’s card shows a loading spinner until its response arrives.

When all responses have arrived, the button re-enables and the status clears.

Step 5 — Review Expert Responses

Navigate to the Expert Responses tab. Each expert’s full response is rendered as a card with markdown formatting and LaTeX equations (MathJax).

At the top of the tab, an Extracted Prior Distributions box shows the distributions parsed from all responses — useful for a quick sanity check.

Step 6 — Inspect the Prior Summary

Navigate to the Prior Summary tab. This shows:

• A table of all parsed distributions with a Valid column (✓ valid / ⚠ invalid)
• A density plot with one subplot per distribution family, overlaying all experts’ curves

Invalid distributions (e.g. out-of-range parameters) are flagged in the table and excluded from the density plot.

Step 7 — Pool Expert Opinions

Navigate to the Opinion Pooling tab. This applies linear opinion pooling:

\[f(\theta) = \sum_{i} w_i f_i(\theta)\]

with equal weights across all valid experts. Distributions are grouped by family and parameter index, so e.g. Exponential(rate_alpha) and Exponential(rate_beta) are pooled separately.

The table shows a Valid column for each expert row and a summary row for the pooled distribution. If any experts are excluded (invalid distributions), the pooled row shows “K/N valid”.

Click Generate Stan Code to produce a Stan model incorporating the pooled hyperpriors.

Step 8 (Optional) — Delphi Iterative Elicitation

The Delphi Method tab implements a structured consensus process. Instead of a single round, experts iteratively revise their priors after seeing each other’s responses.

Exporting Results

Troubleshooting