Getting Started with ElicitationWizard

Overview

ElicitationWizard is an R package that provides an interactive Shiny application for eliciting Bayesian prior distributions using large language models (LLMs). It queries multiple LLM “experts” in parallel, parses their responses into named probability distributions, and aggregates them via opinion pooling or the Delphi method.

Installation

Install the package from CRAN:

install.packages("ElicitationWizard")

Or install the development version from GitHub:

# install.packages("pak")
pak::pak("JefferyAyiti/Automated_Elicitation")

Dependencies

The following R packages are required and will be installed automatically:

Package	Purpose
`shiny`	Web application framework
`bslib`	Bootstrap theming (flatly theme)
`shinyjs`	Button enable/disable during async calls
`ellmer`	LLM API client (`chat_openai_compatible()`)
`future`	Parallel async execution
`promises`	Non-blocking promise handling
`commonmark`	Markdown rendering

API Requirements

ElicitationWizard connects to any OpenAI-compatible API endpoint. You need:

A Base URL — the root of the API (e.g. https://your-endpoint/v1)
An API key — for authentication

We recommend SAIA (Scalable AI Accelerator), which provides OpenAI-compatible access to a range of open models including those used by this package. Other compatible providers include:

Groq — https://api.groq.com/openai/v1
Together AI — https://api.together.ai/v1
OpenRouter — https://openrouter.ai/api/v1

These can be provided in two ways:

Option 1 — Environment variables (recommended):

Sys.setenv(BASE_URL = "https://your-endpoint/v1")
Sys.setenv(API_KEY  = "sk-...")

Or add them to your .Renviron file for persistence across sessions:

BASE_URL=https://your-endpoint/v1
API_KEY=sk-...

Option 2 — Enter directly in the app via the “API Connection” panel in the sidebar.

Launching the App

library(ElicitationWizard)
run_elicitation()

Interface Overview

The app has a sidebar for configuration and a main panel with four tabs.

Sidebar

Panel	Description
API Connection	Base URL and API key; collapsible; pre-filled from environment variables
Clinical Context	Free-text description of the disease, population, and trial setting
Result Expectations	Model structure and the parameter(s) to elicit priors for
Expert Configuration	Number of LLM experts (1–4) and model assignment per expert
Elicit Priors	Button to run elicitation; disabled while requests are in flight

Main Panel Tabs

Tab	Description
Expert Responses	Full markdown-rendered responses from each expert, with LaTeX support
Prior Summary	Parsed distributions in a table + density overlay plots
Opinion Pooling	Linear pooled distribution across all valid experts
Delphi Method	Multi-round iterative elicitation with peer feedback

Available Models

Elicitation Models

These models act as the LLM “experts”:

Label	Model ID
MedGemma-27b-it	`medgemma-27b-it`
Gemma-3-27b-it	`gemma-3-27b-it`
Deepseek-R1-L-70b	`deepseek-r1-distill-llama-70b`
Llama-3.3-70b	`llama-3.3-70b-instruct`
Qwen-3-I-30b	`qwen3-30b-a3b-instruct-2507`
Mistral-3-675B	`mistral-large-3-675b-instruct-2512`

Stan Code Generation Models

Used by the “Generate Stan Code” buttons:

Label	Model ID
Qwen-Coder (default)	`qwen3-coder-30b-a3b-instruct`

Supported Distribution Families

The parser recognises the following families in expert responses:

Normal, Log-Normal, Beta, Gamma, Exponential, Half-Normal, Cauchy, Half-Cauchy, Uniform, Weibull, Poisson, Student-t, Inverse-Gamma, Log-Logistic, Pareto

Note: Dirichlet and Binomial are recognised but not supported by the density pipeline and will be marked as invalid.

Next Steps

See vignette("tutorial", package = "ElicitationWizard") for a step-by-step walkthrough of a complete prior elicitation session using the default Poisson-Gamma clinical trial example.