In the SimTOST R package, which is specifically designed
for sample size estimation for bioequivalence studies, hypothesis
testing is based on the Two One-Sided Tests (TOST) procedure. (Sozu et al.
2015) In TOST, the equivalence test is framed as a comparison
between the the null hypothesis of ‘new product is worse by a clinically
relevant quantity’ and the alternative hypothesis of ‘difference between
products is too small to be clinically relevant’.
The null and alternative hypotheses for the equivalence test are presented below for two different approaches:
Individual equivalence between a test treatment (\(T\)) and a reference treatment (\(R\)) is assessed by comparing the difference between their means, \(\mu_T - \mu_R\), with lower and upper equivalence limits \(L\) and \(U\). The TOST procedure evaluates two one-sided hypotheses:
\[H_{01}: \mu_T - \mu_R \leq L \qquad\text{versus}\qquad H_{11}: \mu_T - \mu_R > L,\]
and
\[H_{02}: \mu_T - \mu_R \geq U \qquad\text{versus}\qquad H_{12}: \mu_T - \mu_R < U.\] The TOST formulation follows the standard equivalence-testing framework (Sozu et al. 2015; Schuirmann 1987).
Equivalence is established if and only if both \(H_{01}\) and \(H_{02}\) are rejected at the chosen significance level \(\alpha\). Equivalently, the alternative equivalence hypothesis is
\[H_1: L < \mu_T - \mu_R < U,\]
against the composite null hypothesis
\[H_0: \mu_T - \mu_R \leq L \qquad\text{or}\qquad \mu_T - \mu_R \geq U.\]
For multiple endpoints, these hypotheses are evaluated for each endpoint, and the trial-level decision requires the specified number of endpoint hypotheses to be rejected.
The DOM test can be implemented in sampleSize() by setting
ctype = "DOM" and distribution = "norm".
For pharmacokinetic (PK) outcomes, such as the area under the curve
(AUC) and maximum concentration (Cmax), log-transformation is commonly
applied to achieve normality. To perform this transformation, the
logarithm of the geometric mean should be provided to
mu_list, while the logarithmic variance can be derived from
the coefficient of variation (CV) using the formula:
\[ \text{Logarithmic Variance} = \log\left(1 + {\text{CV}^2}\right) \]
The use of the logarithmic scale for bioequivalence analysis is described in standard sample-size and regulatory guidance (Sozu et al. 2015; Committee for Medicinal Products for Human Use (CHMP) 2010). The equation itself follows from the definition of a log-normal distribution.
Equivalence limits must also be specified on the log scale to align with the transformed data.
The same two one-sided structure can be expressed for the ratio of means, \(\mu_T / \mu_R\), using lower and upper ratio limits \(L\) and \(U\):
\[H_{01}: \frac{\mu_T}{\mu_R} \leq L \qquad\text{versus}\qquad H_{11}: \frac{\mu_T}{\mu_R} > L,\]
and
\[H_{02}: \frac{\mu_T}{\mu_R} \geq U \qquad\text{versus}\qquad H_{12}: \frac{\mu_T}{\mu_R} < U.\] The ratio formulation is the corresponding TOST on the ratio-of-means scale (Sozu et al. 2015; Schuirmann 1987).
Thus, equivalence requires rejection of both \(H_{01}\) and \(H_{02}\), or equivalently:
\[H_1: L < \frac{\mu_T}{\mu_R} < U,\]
against
\[H_0: \frac{\mu_T}{\mu_R} \leq L \qquad\text{or}\qquad \frac{\mu_T}{\mu_R} \geq U.\]
Here, \(\mu_T\) and \(\mu_R\) represent the arithmetic mean endpoints for the test and reference products, respectively.
The ROM test can be implemented in sampleSize() by setting
ctype = "ROM" and distribution = "lnorm". Note
that the mu_list argument should contain the arithmetic
means of the endpoints, while sigma_list should contain
their corresponding variances.
The ROM test is converted to a Difference of Means (DOM) tests by log-transforming the data and equivalence limits. The variance on the log scale is calculated using the normalized variance formula:
\[ \text{Logarithmic Variance} = \log\left(1 + \frac{\text{Arithmetic Variance}}{\text{Arithmetic Mean}^2}\right) \]
The logarithmic mean is then calculated as:
\[\text{Logarithmic Mean} = \log(\text{Arithmetic Mean}) - \frac{1}{2}(\text{Logarithmic Variance})\] These transformations use the standard log-normal parameterization.
For count outcomes, such as the number of events observed during a specified exposure period, SimTOST uses the event-rate ratio rather than the continuous outcome ratio of means. The event-rate ratio is
\[RR = \frac{\lambda_T}{\lambda_R},\] The event-rate ratio is the estimand used for count-outcome equivalence comparisons (Chang et al. 2017; Zhu 2017), where \(\lambda_T\) and \(\lambda_R\) are the event rates in the test and reference groups. For lower and upper rate-ratio limits \(L\) and \(U\), the hypotheses are:
\[H_{01}: \frac{\lambda_T}{\lambda_R} \leq L \qquad\text{versus}\qquad H_{11}: \frac{\lambda_T}{\lambda_R} > L,\]
and
\[H_{02}: \frac{\lambda_T}{\lambda_R} \geq U \qquad\text{versus}\qquad H_{12}: \frac{\lambda_T}{\lambda_R} < U.\]
Equivalence is concluded only when both one-sided null hypotheses are rejected. Equivalently:
\[H_1: L < \frac{\lambda_T}{\lambda_R} < U,\]
against the composite null
\[H_0: \frac{\lambda_T}{\lambda_R} \leq L \qquad\text{or}\qquad \frac{\lambda_T}{\lambda_R} \geq U.\]
The count test is implemented on the log-rate-ratio scale, using
\(\log(L)\) and \(\log(U)\) as the boundaries. It is distinct
from the continuous ROM estimand. In sampleSize() and simPower(), use
distribution = "pois" or
distribution = "nbinom" and provide a named
rate_list. Count outcomes also require
exposure; negative-binomial outcomes additionally require a
positive dispersion value. The ctype argument
is not used for count outcomes because the estimand is automatically the
event-rate ratio.
For multiple endpoints, list_comparator identifies the
treatment pairs and list_lequi.tol and
list_uequi.tol provide the rate-ratio limits for each
selected endpoint. A simulated trial is successful when the required
number of endpoints passes for every required comparator. Endpoint
dependence can be specified with cor_mat. For count
outcomes, cor_mat specifies the dependence between latent
normal variables used to generate correlated counts. The resulting
Pearson correlation between the observed discrete counts is generally
different and depends on the marginal count distributions. For normally
distributed continuous outcomes, the latent correlation and Pearson
correlation coincide because the transformation is linear. This follows
directly from the definition of Pearson correlation: a common linear
transformation preserves the correlation, apart from a possible change
in sign if the transformation reverses orientation.
When evaluating bioequivalence, certain statistical and methodological requirements must be adhered to, as outlined in the European Medicines Agency’s bioequivalence guidelines (Committee for Medicinal Products for Human Use (CHMP) 2010). These requirements ensure that the test and reference products meet predefined criteria for equivalence in terms of PK parameters. The key considerations are summarized below:
When conducting a DOM test, the FDA recommends that the equivalence acceptance criterion (EAC) be defined as \(\delta = EAC = 1.5 \sigma_R\), where \(\sigma_R\) represents the variability of the log-transformed endpoint for the reference product.
Assessment of equivalence is often required for more than one primary variable. (Sozu et al. 2015) For example, the EMA recommends showing equivalence both for AUC and Cmax
A decision must be made as to whether it is desirable to
When a trial defines multiple co-primary endpoints, equivalence must be demonstrated for all of them to claim overall treatment equivalence. In this setting, each endpoint is tested separately at the usual significance level (\(\alpha\)), and equivalence is established only if all individual tests are statistically significant. Because conclusions require rejecting all null hypotheses, the decision is an intersection–union test and a formal endpoint-wise multiplicity adjustment is not needed to control the Type I error rate (Berger and Hsu 1996; Hamasaki et al. 2018). However, as the number of co-primary endpoints (\(K\)) increases, the likelihood of failing to meet equivalence on at least one endpoint also rises, resulting in a higher Type II error rate (i.e., a greater risk of incorrectly concluding non-equivalence) (Mielke et al. 2018)
This has several important implications:
When a trial aims to establish equivalence for at least \(k\) of \(m\) primary endpoints, the decision is a partial-conjunction test. When \(k < m\), an endpoint-wise adjustment is generally needed to control the Type I error across the possible successful endpoint subsets (Benjamini and Heller 2008; Mielke et al. 2018). The appropriate adjustment depends on the prespecified \(k\)-out-of-\(m\) rule.
For example, if a study includes \(m = 3\) independent primary endpoints and uses a significance level of \(\alpha = 5%\) for each test, the overall probability of falsely concluding equivalence for at least one endpoint is:
\[ 1 – (1-\alpha)^m = 1 - (1-0.05)^3 = 0.1426. \] This is the probability of at least one false rejection under independent tests (Dunn 1961; Sidak 1967). This means that the overall probability of making any false positive error, also known as the family-wise error rate (FWER), increases to approximately 14%.
To address this issue, adjustments to the significance level are necessary for multiple endpoint comparisons for which various methods have been proposed. In SimTOST, the following approaches are included:
The most common and easiest procedure for multiplicity adjustment to control the FWER is the Bonferroni method (Bonferroni 1936). Each hypothesis is tested at level
\[\alpha_{bon}= \alpha/m\]
where \(m\) is the total number of tests. Although simple, this method is highly conservative, particularly when tests are correlated, as it assumes all tests are independent. This conservativeness remains pronounced even for \(k=1\), where only one of the \(m\) hypotheses needs to be rejected. (Mielke et al. 2018)
In the sampleSize()
function, the Bonferroni correction can be applied by setting
adjust = "bon".
The Sidak correction (Sidak 1967) is an alternative method for controlling the FWER. Like the Bonferroni correction, it assumes that tests are independent. However, the Sidak correction accounts for the joint probability of all tests being non-significant, making it mathematically less conservative than the Bonferroni method. The adjusted significance level is calculated as:
\[\alpha_{sid}= 1-(1-\alpha)^ {1/m}\]
The Sidak correction can be implemented by specifying
adjust = "sid" in the sampleSize() function.
The K adjustment (Mielke et al. 2018) explicitly accounts for the scenario where equivalence is required for only \(k\) out of \(m\) endpoints. Unlike the Bonferroni and Sidak corrections, which assume that all \(m\) tests contribute equally to the overall Type I error rate, the k-adjustment directly incorporates the number of endpoints (\(k\)) required for equivalence into the adjustment. The adjusted significance level is calculated as:
\[\alpha_k= \frac{k*\alpha}{m}\]
where \(k\) is the number of endpoints required for equivalence, and \(m\) is the total number of endpoints evaluated.
The t adjustment (Mielke et al. 2018) described
below is the strong k-out-of-m procedure for partial-conjunction
decisions and is distinct from the weak k adjustment.
When the decision rule requires equivalence for at least
k of m endpoints, the null hypothesis is a
partial-conjunction hypothesis. In this setting,
adjust = "t" applies Mielke et al.’s strong k-out-of-m
calibration. The significance level for each relevant boundary endpoint
is:
\[\alpha_{t}= \frac{\alpha}{m-k+1}\]
where m is the number of selected endpoints and
k is the minimum number that must demonstrate equivalence.
This adjustment controls the Type I error rate for the
partial-conjunction decision, including cases in which only some
endpoints satisfy the null hypothesis (Mielke et al. 2018).
Hierarchical testing is an approach to multiple endpoint testing where endpoints are tested in a predefined order, typically based on their clinical or regulatory importance. A fallback testing strategy is applied, allowing sequential hypothesis testing. If a hypothesis earlier in the sequence fails to be rejected, testing stops, and subsequent hypotheses are not evaluated. (Chowdhry et al. 2024)
To implement hierarchical testing in simTOST, the user
specifies adjust = "seq" in the sampleSize() function and
defines primary and secondary endpoints using the type_y
vector argument. The significance level (\(\alpha\)) is adjusted separately for each
group of endpoints, ensuring strong control of the Family-Wise Error
Rate (FWER) while maintaining interpretability.
k
secondary endpoints.k
secondary endpoints demonstrate equivalence.An example of hierarchical testing can be found in this vignette.
In certain cases, it may be necessary to compare multiple treatments
simultaneously. This can be achieved by specifying multiple comparators
in the mu_list and sigma_list parameters. The
sampleSize() function can
accommodate multiple treatments, allowing for the evaluation of
equivalence across different products or formulations.
Although trials with multiple arms are common, there is no clear consensus in the literature as to whether statistical corrections should be applied for testing multiple primary hypotheses in such analyses. In SimTOST, no adjustments are made for trials involving more than two treatment arms.