---
title: "Comparing groups"
author: "Written by [Mohammed Saqr](https://saqr.me) and [Sonsoles López-Pernas](https://sonsoles.me)"
output:
  rmarkdown::html_vignette:
    toc: true
    toc_depth: 2
vignette: >
  %\VignetteIndexEntry{Comparing groups}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r options, include = FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>", fig.width = 7,
                      fig.height = 4.5, fig.align = "center", dpi = 96)
old_options <- options(digits = 3, width = 90)
```

Many questions about sequential behavior are comparative. Do high and low
achievers regulate their collaboration in the same order? Do students who
persist move between engagement states differently from those who drop
out? Two groups can use the same states at similar rates and still differ
in what follows a given history, and they can also differ in how often
they reach that history at all. These are distinct questions, and a single
distance between two transition matrices conflates them.

A variable-order transition tree fitted to each group gives both pieces of
information for every context: the next-state distribution
$P(x_t \mid c)$ and the number of times the context occurs. The
`transitiontrees` package compares groups on these two axes. The
**behavioral** axis asks whether, given the same context, the groups
predict a different next state; it is measured per context by the
Jensen-Shannon divergence between the groups' next-state distributions,

$$
\mathrm{JSD}(c) = H\Big(\sum_g w_g P_g(\cdot \mid c)\Big) -
\sum_g w_g H\big(P_g(\cdot \mid c)\big), \qquad
w_g = \frac{N_g(c)}{\sum_h N_h(c)},
$$

where $H$ is the Shannon entropy in bits and $N_g(c)$ the count of the
context in group $g$. The **usage** axis asks whether the groups reach the
context at different rates; it is measured by a $G^2$ homogeneity
statistic on the context's share of each group's positions at the same
depth. The reference distribution of every statistic is built by permuting
the group labels of whole sequences, and per-context p-values are adjusted
for multiplicity with the Benjamini-Hochberg procedure. The model itself is
introduced in the
[variable-order transition trees](https://pak.dynasite.org/transitiontrees/articles/transitiontrees.html)
vignette.

## Data

`group_regulation_long` is an event log of collaborative regulation: one
row per action, 27,533
rows from 2,000 students.

- **Actor:** `Actor` identifies the student whose actions form a sequence.
- **Time:** `Time` orders the actions; a gap of more than 15 minutes
  starts a new sequence.
- **State:** `Action`, one of nine regulation moves (`adapt`, `cohesion`,
  `consensus`, `coregulate`, `discuss`, `emotion`, `monitor`, `plan`,
  `synthesis`).
- **Group:** `Achiever`, the student's achievement level (`High` or
  `Low`), constant within a student.
- **Stratum:** `Course`, the course the student took (`A`, `B` or `C`).

```{r setup}
library(transitiontrees)
```

Achievement and course are related. The cross-tabulation of actions by
course and achievement level shows that course `A` contains only high
achievers, course `C` only low achievers, and course `B` both.

```{r course-by-achievement}
xtabs(~ Course + Achiever, data = group_regulation_long)
```

## Fitting one tree per group

Naming the group column in `context_tree()` fits one tree per group over a
shared alphabet and returns a `transitiontrees_group`. The trees below
hold contexts of one action, which keeps the permutation tests fast; the
[collaborative regulation case study](https://pak.dynasite.org/transitiontrees/articles/case-regulation-achievement.html)
compares deeper trees.

```{r fit}
grp <- context_tree(group_regulation_long,
                    actor = "Actor", time = "Time", action = "Action",
                    group = "Achiever", max_depth = 1, min_count = 10)
grp
```

`summary()` returns one row per group with the size and fit of its tree.

```{r summary-group}
summary(grp)
```

Each group contributes 1,000 sequences and about 13,700 actions. The two
trees fit their own data about equally well: a perplexity of 5.35 for
high achievers and 5.36 for low achievers, against nine possible
actions.

`state_frequencies()` lists the shares of the actions in each group.

```{r frequencies}
state_frequencies(grp)
```

High achievers use `consensus` most (26.6% of their actions) and low
achievers `plan` (25.5%). Low achievers use `adapt`, `monitor` and
`synthesis` more often; high achievers use `emotion` and `cohesion` more
often.

## Overall difference between two trees

`compare_trees()` tests whether two trees predict differently overall. Its
statistic is the count-weighted symmetric Kullback-Leibler divergence
between the trees' next-state distributions, in nats; the null
distribution comes from refitting both trees after reassigning the
sequences at random to the two groups.

```{r compare-trees}
cmp <- compare_trees(grp, iter = 99)
summary(cmp)
```

The observed distance of 0.0471 is many times the permutation mean of
0.0029 and exceeds every one of the 99 permuted distances, so the p-value
is at its floor of $1 / (99 + 1) = 0.01$. The per-context breakdown shows
which histories contribute most.

```{r compare-trees-pathways}
as.data.frame(cmp, what = "pathways")
```

What follows `synthesis` differs most between the groups (a symmetric
divergence of 0.305), then what follows `discuss` (0.191) and `cohesion`
(0.117). What follows `monitor` hardly differs (0.005).

## Behavioral and usage differences

`compare_groups()` separates the two axes, tests every context, and works
for any number of groups. `summary()` reduces the result to one row with
a verdict.

```{r compare-groups}
cg <- compare_groups(grp, iter = 99)
summary(cg)
```

Both omnibus tests reach the permutation floor of 0.01: given the same
preceding action the groups choose different next actions, and they also
reach the preceding actions at different rates. Eight of the ten contexts,
the root among them, show a behavioral difference after false discovery
rate adjustment, and eight of the nine contexts with a usage test (the
root has none) show a usage difference.

The omnibus table and the pairwise distance are available as data frames.

```{r omnibus}
as.data.frame(cg, what = "omnibus")
as.data.frame(cg, what = "distance")
```

The behavioral statistic is the count-weighted sum of the per-context
divergences, 874 bits; the usage statistic is the sum of the per-context
$G^2$ values, 283. The pairwise distance is the `compare_trees()`
statistic, 0.0471.

The per-context table gives the counts, the modal next action in each
group, whether the modal actions differ (`flips`), and both tests with
raw and adjusted p-values.

```{r pathways}
as.data.frame(cg, what = "pathways")
```

After `discuss`, high achievers most often move to `consensus` and low
achievers continue to `discuss`. After `coregulate` the modal action is
`plan` for high achievers and `discuss` for low achievers, and after
`emotion` it is `consensus` against `cohesion`. `synthesis` has the
largest behavioral divergence (0.102 bits) although both groups most
often follow it with `consensus`; the difference lies in the rest of the
distribution. `monitor` (adjusted p = 0.88) and `adapt` (0.089) show no
behavioral difference, and `discuss` is the one context with no usage
difference (adjusted p = 0.21). `adapt` differs in usage ($G^2$ = 105)
more than any other action but not in what follows it.

## Where the groups differ

`plot_difference()` draws one row per context and one column per next
action. Each cell is the Pearson residual of the first group against the
no-difference null, so red cells are transitions high achievers make more
often than expected and blue cells transitions they make less often. With
`comparison = cg`, contexts whose behavioral difference survives the
adjustment are starred.

```{r plot-difference, fig.width = 8, fig.height = 5}
plot_difference(grp, comparison = cg)
```

The largest residuals sit in the `discuss` row: after discussing, high
achievers move to `consensus` more often than expected (+8.2) and to
`adapt` less often (-7.9). High achievers also move from `consensus` to
`discuss` (+6.0) and from `plan` to `emotion` (+5.0) more often, and from
`synthesis` to `plan` (+4.2). The `adapt` and `monitor` rows are pale and
unstarred: what follows these actions does not differ between the
groups.

## Stratified permutation

The permutation tests above reassign achievement labels freely across all
students. Because course `A` contains only high achievers and course `C`
only low achievers, part of the observed difference may be a difference
between courses. Naming `Course` as `block` restricts the permutation to
reassign labels only within each course, so the null distribution keeps
the course composition of each group fixed. The block is stored on the
fitted object, and `compare_groups()` uses it without further arguments.
The same argument handles repeated measures: with several sequences per
person, `block` set to the person column keeps each person's sequences
together in the null.

```{r fit-blocked}
grp_course <- context_tree(group_regulation_long,
                           actor = "Actor", time = "Time", action = "Action",
                           group = "Achiever", block = "Course",
                           max_depth = 1, min_count = 10)
cg_course <- compare_groups(grp_course, iter = 99)
summary(cg_course)
```

The observed statistics are unchanged, and both omnibus tests remain at
the permutation floor. The per-context results are more conservative:
six contexts keep a behavioral difference and four a usage difference.

```{r pathways-blocked}
as.data.frame(cg_course, what = "pathways")
```

`coregulate` and `emotion`, two of the contexts whose modal next action
differs between the groups, no longer pass the behavioral adjustment
(adjusted p = 0.057 and 0.062), and the usage differences after
`cohesion`, `coregulate`, `plan` and `monitor` are no longer significant.
The behavioral differences after `synthesis`, `discuss`, `cohesion`,
`consensus` and `plan` remain under a null that holds the course
composition of each group fixed.

## Assumptions

- Sequences are exchangeable under the null hypothesis, within blocks
  when `block` is given.
- With `block`, only blocks that contain more than one group contribute
  to the permutation; here only course `B` does.
- The tested contexts are those in the fitted trees; a context rarer than
  `min_count` in every group is not tested.
- Weighted trees cannot be compared, because the permutation refits from
  the raw sequences.

```{r restore-options, include = FALSE}
options(old_options)
```
