---
title: "Tutorial: Eliciting Bayesian Priors with ElicitationWizard"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Tutorial: Eliciting Bayesian Priors with ElicitationWizard}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment  = "#>",
  eval     = FALSE
)
```

This tutorial walks through a complete prior elicitation session using the
built-in default example: a hierarchical Poisson-Gamma model for adverse event
counts in a multi-center clinical trial.

---

## Setup

Launch the app:

```{r launch}
library(ElicitationWizard)
run_elicitation()
```

If you have not set `BASE_URL` and `API_KEY` as environment variables, expand
the **API Connection** panel and enter them now.

---

## Step 1 — Define the Clinical Context

Expand the **Clinical Context** panel. The default text describes a metastatic
hormone-resistant prostate cancer (HRPC) multi-center RCT. Edit this to match
your own study if needed.

```
Disease: Metastatic hormone-resistant prostate cancer (HRPC)
Treatment: Control arm (placebo/standard care)
Population: Adult oncology patients
Study: Multi-center RCT
```

The clinical context is injected verbatim into every expert's elicitation
prompt. Be as specific as possible — endpoint definition, patient population,
and comparator all influence the prior a domain expert would choose.

---

## Step 2 — Specify the Model and Parameters

Expand the **Result Expectations** panel. This tells the experts exactly which
parameters to provide priors for, and in what format.

The default specifies the Poisson-Gamma hierarchical model:

```
Model:
Each patient i in site j has AE count:
y_ij ~ Poisson(lambda_j)
Site-specific rates: lambda_j ~ Gamma(alpha, beta)
REQUIRED: alpha ~ Exponential(rate_alpha), beta ~ Exponential(rate_beta)
```

The `REQUIRED:` line instructs the LLM to provide priors specifically for
`rate_alpha` and `rate_beta`, the hyperprior rates of the Exponential
distributions on `alpha` and `beta`.

---

## Step 3 — Configure Experts

In the **Expert Configuration** section, choose how many LLM experts to query
(1–4). Each expert is assigned an independent model and temperature:

- **Model** — select from the available elicitation models
- **Temperature** — controls response diversity; higher values (e.g. `1.0`)
  produce more varied priors; lower values (e.g. `0.1`) produce more
  conservative, deterministic responses

> **Tip:** When using the same model for multiple experts, set different
> temperatures to encourage diverse prior proposals rather than identical
> responses.

---

## Step 4 — Elicit Priors

Click **Elicit Priors**. All experts are queried in parallel — the button is
disabled and each expert's card shows a loading spinner until its response
arrives.

When all responses have arrived, the button re-enables and the status clears.

---

## Step 5 — Review Expert Responses

Navigate to the **Expert Responses** tab. Each expert's full response is
rendered as a card with markdown formatting and LaTeX equations (MathJax).

At the top of the tab, an **Extracted Prior Distributions** box shows the
distributions parsed from all responses — useful for a quick sanity check.

### Generating Stan Code from an Expert Response

Click the **`</>`** (code) icon in any expert card header to send that expert's
full response plus the model specification to the Stan code generation model.
The generated Stan model appears in a dark card below the panels.

---

## Step 6 — Inspect the Prior Summary

Navigate to the **Prior Summary** tab. This shows:

- A **table** of all parsed distributions with a **Valid** column (✓ valid /
  ⚠ invalid)
- A **density plot** with one subplot per distribution family, overlaying all
  experts' curves

Invalid distributions (e.g. out-of-range parameters) are flagged in the table
and excluded from the density plot.

---

## Step 7 — Pool Expert Opinions

Navigate to the **Opinion Pooling** tab. This applies **linear opinion
pooling**:

$$f(\theta) = \sum_{i} w_i f_i(\theta)$$

with equal weights across all valid experts. Distributions are grouped by
family and parameter index, so e.g. `Exponential(rate_alpha)` and
`Exponential(rate_beta)` are pooled separately.

The table shows a **Valid** column for each expert row and a summary row for
the pooled distribution. If any experts are excluded (invalid distributions),
the pooled row shows "K/N valid".

Click **Generate Stan Code** to produce a Stan model incorporating the pooled
hyperpriors.

---

## Step 8 (Optional) — Delphi Iterative Elicitation

The **Delphi Method** tab implements a structured consensus process. Instead of
a single round, experts iteratively revise their priors after seeing each
other's responses.

### How it works

| Round   | Expert sees                                                                 |
|---------|-----------------------------------------------------------------------------|
| Round 1 | Clinical context + model spec only                                          |
| Round 2 | Clinical context + model spec + all other experts' Round 1 parsed distributions |
| Round 3 | Clinical context + model spec + all other experts' Round 2 parsed distributions |

Up to **3 rounds** are supported.

### Running a Delphi session

1. Click **Run Delphi Round** to start Round 1. Expert configuration is locked
   from this point.

2. Click **Run Delphi Round** again to run Round 2. Each expert now receives
   the other experts' Round 1 distributions as peer feedback.

3. After any round completes, the **Converged Distributions** section appears
   at the bottom, showing the current consensus: a table, density plot, and
   **Generate Stan Code** button.

4. Click **Reset** to clear all rounds and start a new Delphi session. Expert
   configuration is unlocked.

> **Note:** The Delphi tab is independent from the main **Elicit Priors**
> button. Expert model and count selections apply to both, but Delphi rounds
> must be triggered separately.

---

## Exporting Results

### Stan Model

Any of the three **Generate Stan Code** buttons (Expert Responses, Opinion
Pooling, Delphi) will produce a complete Stan model. Copy the code from the
dark card — it includes the model structure from **Result Expectations** and
incorporates the elicited hyperpriors.

### Downstream Validation

The generated hyperpriors can be passed directly to Stan as `rate_alpha` and
`rate_beta` data to run cross-validation and sample efficiency experiments
against real data.

---

## Troubleshooting

| Symptom | Likely cause | Fix |
|---------|-------------|-----|
| "Please provide a Base URL and API Key" notification | Missing credentials | Set `BASE_URL` / `API_KEY` env vars or fill in the API Connection panel |
| Expert card shows an error (red) | API call failed | Check Base URL format (must end in `/v1`), API key validity, and model availability |
| All distributions marked ⚠ Invalid | LLM did not follow the required format | Try a lower temperature, or use a model with stronger instruction-following (e.g. MedGemma, Llama-3.3) |
| Delphi round 2 shows "partial peer feedback" warning | One or more Round 1 experts returned no valid distributions | Review the invalid expert's response in the round accordion; proceed or Reset and re-run |
| No distributions appear in Prior Summary | Parsing found nothing | Check that expert responses contain distribution strings in the form `Family(param1, param2)` |