Package {transforEmotion}

Title:	Transformer Emotion Analysis for Text, Image, Video, and Speech
Version:	0.1.9
Date:	2026-10-26
Author:	Alexander Christensen [aut], Hudson Golino [aut], Aleksandar Tomašević [aut, cre]
Maintainer:	Aleksandar Tomašević <atomashevic@gmail.com>
Description:	Zero-shot emotion classification for text, images, video and speech, and tools to find faces and code them by hand. Text is scored with 'Hugging Face' transformer pipelines (default 'cross-encoder/nli-distilroberta-base'). Images and video are scored with 'OpenAI CLIP'; faces can be limited to those found by 'OpenCV YuNet'. Speech segments are scored with 'LAION CLAP'. Frame scores can be calibrated and combined into posteriors, and detected faces or other items can be turned into a local coding task served from R. Any other zero-shot model can be selected by its name on https://huggingface.co/models. For details see Tomašević, Golino, and Christensen (2026) <doi:10.5117/CCR2026.2.2.TOMA>.
Depends:	R (≥ 3.5.0)
License:	GPL (≥ 3.0)
Copyright:	See file inst/COPYRIGHTS
Encoding:	UTF-8
Imports:	dplyr, googledrive, Matrix, methods, pbapply, reticulate (≥ 1.45.0), textdata, jsonlite, ggplot2, reshape2, httr, httpuv (≥ 1.6.0)
Suggests:	IRdisplay, knitr, LSAfun, markdown, remotes, rmarkdown, rstudioapi, testthat (≥ 3.0.0), withr
VignetteBuilder:	knitr
Config/testthat/edition:	3
Config/roxygen2/version:	8.1.0
NeedsCompilation:	no
Packaged:	2026-10-26 11:26:54 UTC; aleksandar
Repository:	CRAN
Date/Publication:	2026-10-26 15:20:02 UTC

transforEmotion–package

Description

Zero-shot emotion classification for text, images, video and speech, and tools to find faces and code them by hand. Text is scored with Hugging Face transformer pipelines (default cross-encoder/nli-distilroberta-base). Images and video are scored with OpenAI CLIP; faces can be limited to those found by OpenCV YuNet. Speech segments are scored with LAION CLAP. Frame scores can be calibrated and combined into posteriors, and detected faces or other items can be turned into a local coding task served from R. Any other zero-shot model can be selected by its name on Hugging Face.

Author(s)

Alexander P. Christensen <alexpaulchristensen@gmail.com>, Hudson Golino <hfg9s@virginia.edu> and Aleksandar Tomasevic <atomashevic@gmail.com>

References

Tomašević, A., Golino, H., & Christensen, A. P. (2026). transforEmotion: An open-source R package for emotion analysis using transformer-based generative AI models. Computational Communication Research, 8(2). doi:10.5117/CCR2026.2.2.TOMA

Yin, W., Hay, J., & Roth, D. (2019). Benchmarking zero-shot text classification: Datasets, evaluation and entailment approach. arXiv preprint arXiv:1909.00161.

See Also

citation("transforEmotion")

Initialize Built-in Vision Models

Description

Register the default/built-in vision models that come with transforEmotion. This function is automatically called when the package is loaded.

Usage

.init_builtin_models()

Value

Invisibly returns TRUE

Vision Model Registry for transforEmotion Package

Description

Central registry system for managing vision models in transforEmotion. Provides extensible architecture allowing users to register custom vision models beyond the default CLIP-based models.

Usage

.vision_model_registry

Details

The registry maintains a list of available vision models with their configurations. Each model entry includes the HuggingFace model ID, architecture type, and metadata needed for proper initialization and processing.

Author(s)

Aleksandar Tomasevic <atomashevic@gmail.com>

Multivariate Normal (Gaussian) Distribution

Description

This function generates a random sample from the multivariate normal distribution with mean mu and covariance matrix Sigma.

Usage

MASS_mvrnorm(n = 1, mu, Sigma, tol = 1e-06, empirical = FALSE, EISPACK = FALSE)

Arguments

Value

A (n X p) matrix of random observations from the multivariate normal distribution. Updated: 26.10.2023.

Keep the attributes of a detect_faces() table when it is subset

Description

Keep the attributes of a detect_faces() table when it is subset

Usage

## S3 method for class 'te_faces'
x[...]

Arguments

Value

The subset, with the "media" and "settings" attributes face_scores needs to find video frames again.

User-Friendly Vision Model Management Functions

Description

High-level functions for managing vision models in transforEmotion, providing an easy interface for extending the package with custom models.

User-friendly wrapper for registering custom vision models with automatic validation and helpful error messages.

Usage

add_vision_model(
  name,
  model_id,
  description = NULL,
  architecture = "clip",
  test_labels = NULL,
  force = FALSE
)

Arguments

Value

Invisibly returns TRUE if successful

Author(s)

Aleksandar Tomasevic <atomashevic@gmail.com> Add a Custom Vision Model

Examples

## Not run: 
# Add a fine-tuned CLIP model for emotion recognition
add_vision_model(
  name = "emotion-clip",
  model_id = "openai/clip-vit-large-patch14",
  description = "Large CLIP model for better emotion recognition",
  test_labels = c("happy", "sad", "angry"),
  force = TRUE
)

# Add a local model
add_vision_model(
  name = "my-local-model",
  model_id = "/path/to/my/model",
  description = "My custom fine-tuned model"
)

# Add experimental BLIP model
add_vision_model(
  name = "blip-base",
  model_id = "Salesforce/blip-image-captioning-base",
  architecture = "blip",
  description = "BLIP model for image captioning"
)

# Now use any of them in analysis
result <- image_scores("photo.jpg", c("happy", "sad"), model = "emotion-clip")
batch_results <- image_scores_dir("photos/", c("positive", "negative"), 
                                 model = "my-local-model")

## End(Not run)

Code faces in the browser

Description

Starts the coding page of an annotation_task from R and opens it in the web browser. Each answer is saved in the task's results folder as soon as the coder gives it, so closing the browser or R loses nothing, and a coder who comes back continues where they stopped.

Usage

annotate_faces(
  task,
  coder = NULL,
  open = interactive(),
  port = NULL,
  inline = .te_in_notebook(),
  max_side = 640
)

Arguments

Details

The page is served by R (with httpuv) on your own computer only (127.0.0.1), at an address with a random part, and keeps running while you use R. Coders press the number keys to answer, Enter for the next item, and Backspace to undo.

Coders on other computers can open page/index.html of the task from a copy of the folder instead. Their page keeps answers in that browser and ends with Export annotations; put the exported files in the task's results folder.

In a Jupyter notebook, including Google Colab, the coding page is shown below the cell instead (inline = TRUE, the default there), because a page served from R cannot be reached from a notebook running on another computer. Its Copy R code button gives a line such as answers <- read_annotations(task, "te-annotations:...") to run in a new cell. The line holds every answer given so far, so copy it again after more coding; keep the last one, as the answers are not saved in the task's results folder.

Value

Invisibly, the page: a list with its url and stop(), which stops the server. With inline = TRUE, invisibly, the path of the page as one self-contained file.

See Also

read_annotations, annotation_progress

Examples

## Not run: 
coding <- annotate_faces("annotation-round-1", coder = "C01")
annotation_progress("annotation-round-1")
coding$stop()

## End(Not run)

Agreement between coders

Description

Krippendorff's alpha, Fleiss' kappa and percent agreement for each question of a face annotation task.

Usage

annotation_agreement(annotations, questions = NULL, level = NULL)

Arguments

Details

Only items with answers from at least two coders count; a question that was not asked (after "Not a face", say) is missing, not an answer.

alpha

Krippendorff's alpha, from the coincidence matrix. It allows for missing answers and for items coded by different numbers of coders.

fleiss_kappa

Fleiss' kappa, when every counted item has answers from the same number of coders; NA otherwise. Treats answers as categories.

percent_agreement

The share of pairs of coders who gave the same answer to an item, averaged over items.

Value

A data frame with one row per question: question, level, n_items, n_coders, n_answers, percent_agreement, alpha and fleiss_kappa.

References

Krippendorff, K. (2011). Computing Krippendorff's alpha-reliability. https://repository.upenn.edu/items/034a6030-c584-4d14-9d3d-7b7e8d16df20

Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5), 378-382.

See Also

consensus_labels

Progress of a face annotation task

Description

Progress of a face annotation task

Usage

annotation_progress(task)

Arguments

Value

A data frame with one row per coder: items assigned, answered, skipped and left, and the median seconds per answered item.

See Also

annotate_faces

Questions for a face annotation task

Description

Ready-made questions for annotation_task. Each question is a list, so you can change them or add your own (see Details).

Usage

annotation_questions(which = c("face_check", "emotion", "intensity"))

Arguments

Details

A question is a list with

id

A name in lower case: letters, digits and underscores.

type

"single" (one of a few options) or "likert" (a whole number from min to max).

prompt

The question as coders read it.

options

For "single": the answers, or options_from = "classes" for the task's classes. Optional labels: a named list of the text shown for each answer.

min, max

For "likert", with optional anchors = list(min = "...", max = "...").

show_if

Optional: ask only after another question got one of these answers, as in list(face_check = "ok").

reveal

Optional, for items with a reference label and a model prediction: TRUE asks the question only once they are shown, which with blind = TRUE is after the other questions are answered.

required

FALSE lets coders leave it unanswered.

Value

A list of questions.

See Also

annotation_task

Examples

annotation_questions(c("face_check", "emotion"))

# Add a question of your own
gaze <- list(id = "gaze", type = "single", prompt = "Where does the person look?",
             options = c("camera", "away", "closed"), show_if = list(face_check = "ok"))
questions <- c(annotation_questions(), list(gaze))

Set up a face annotation task

Description

Turns detected faces into a coding task: each face is an item that coders see outlined in its picture, with a close-up, and answer questions about, such as whether it is a face at all and which emotion it shows. Start the coding page with annotate_faces, then read the answers with read_annotations.

Usage

annotation_task(
  faces = NULL,
  classes = NULL,
  questions = NULL,
  coders = "C01",
  overlap = 1,
  dir,
  seed = 1,
  title = NULL,
  instructions = NULL,
  max_side = 1280,
  task_id = NULL,
  items = NULL,
  frames = 4,
  play = TRUE,
  blind = TRUE
)

Arguments

Details

With items instead of faces, each item is a video clip, an image or a text, for example the clips a model got wrong. Coders see sampled frames of the clip with its face boxes (the largest face, the one video_scores scores by default, drawn solid), the clip itself with its sound, and the item's text. When the items have a reference label and a model prediction, the page shows them with the model's scores once the coder has answered (blind = TRUE), then asks what explains the result (annotation_questions("why")). Unlike a face task, such a page holds the texts, labels and scores.

The task is a folder:

page/

The coding page: small copies of the images or video frames, under names that say nothing about the files, and the questions. It holds no file names, scores, or other coders' answers.

items.csv

Which face each item is: its file, frame, face number, track and person. For you, not for coders.

task.json

The questions, items and each coder's list.

results/

The answers, one file per coder.

Value

The task, invisibly: a list with its folder (dir), task_id, questions, coders and items (the contents of items.csv).

See Also

annotate_faces, read_annotations, annotation_agreement, consensus_labels

Examples

## Not run: 
faces <- detect_faces(system.file("extdata", "two-people.mp4", package = "transforEmotion"),
                      nframes = 20)
task <- annotation_task(faces, classes = c("happy", "sad", "angry", "neutral"),
                        coders = c("C01", "C02"), dir = "annotation-round-1")
annotate_faces(task)

# A model's errors on video clips, with their descriptions
emotions <- c("angry", "fearful", "happy", "sad")
items <- data.frame(id = errors$clip, media = errors$path, text = errors$description,
                    label = errors$label, prediction = errors$predicted, errors[, emotions])
review <- annotation_task(items = items, classes = emotions, dir = "error-review")
annotate_faces(review)
answers <- read_annotations(review)
table(answers$value[answers$question == "why"])

## End(Not run)

Convert RAG JSON to a table

Description

Produces a long-form data.frame with columns: 'label', 'confidence', 'intensity', 'doc_id', 'span', 'score'.

Usage

as_rag_table(x, validate = TRUE)

Arguments

Value

A data.frame suitable for statistical analysis.

Examples

j <- '{"labels":["joy","surprise"],"confidences":[0.8,0.5],
  "intensity":0.7,"evidence_chunks":[]}'
as_rag_table(j)

Calculate the moving average for a time series

Description

This function calculates the moving average for a time series.

Usage

calculate_moving_average(data, window_size)

Arguments

Value

Matrix or Data frame containing the moving average values.

Calibrate frame scores for posterior_scores()

Description

Learns how far to trust a model's frame scores, so that posterior_scores can treat them as noisy evidence instead of taking them at face value. Without labels it learns the model's average score for each class, its lean towards some labels before it has seen a face. With the true emotion of some videos it also learns the model's confusions: how often it gives each top label when the true emotion is a given one.

Usage

calibrate_scores(scores, labels = NULL, classes = NULL, alpha = NULL)

Arguments

Details

A model's scores are not probabilities to be read as they are. CLIP's scores have a lean of their own: averaged over the frames of CREMA-D clips with six bare labels, standard CLIP gives "neutral" 37% of its probability, whichever emotion the actor shows. They make the same mistakes again and again, such as reading disgust as sadness. And frames a fraction of a second apart are nearly copies of each other, so 80 frames are not 80 independent looks at a face.

The calibration turns the score vector p_t of frame t into evidence L_t(c) for each emotion c:

• without labels, L_t(c) = (p_t(c) / \pi(c))^\alpha, where \pi(c) is the model's mean score for c (prior);

• with labels, L_t(c) = (\sum_k p_t(k) M_{ck})^\alpha, where M_{ck} is the share of frames with true emotion c whose top label is k (confusion).

\alpha discounts frames that repeat each other: with \alpha = 0.1, ten frames count as one independent look. Without labels it is (1 - \rho) / (1 + \rho), where \rho is the correlation of the evidence between neighbouring frames, the share of independent observations in a series with that correlation. Videos whose frames all have the same scores are left out of the correlation; if every video is like that, \alpha is 0.01.

Scores are renormalised over classes in every row, which is the same as scoring with only those labels.

Value

An object of class te_calibration, a list with

classes

The score columns, in order.

prior

The model's mean score of each class over all frames.

confusion

With labels: a matrix whose row c gives the share of frames of videos with true emotion c that have each top label (add-one smoothed). NULL without labels.

likelihood

"prior" or "confusion": how posterior_scores turns scores into evidence.

alpha

The weight of one frame's evidence.

n_videos, n_frames, n_labelled

The videos, frames with scores and labelled videos it was fitted on.

See Also

posterior_scores

Examples

# Frame scores of nine short clips (in practice from video_scores())
set.seed(1)
clip <- function(n, emotion) {
  s <- matrix(runif(n * 3), n, 3,
              dimnames = list(NULL, c("neutral", "happy", "sad")))
  s[, "neutral"] <- s[, "neutral"] + 0.6  # the model leans to neutral
  s[, emotion] <- s[, emotion] + 0.5
  as.data.frame(s / rowSums(s))
}
emotions <- rep(c("neutral", "happy", "sad"), each = 3)
scores <- lapply(emotions, function(e) clip(40, e))

# Without labels: correct for the model's lean
calibrate_scores(scores)

# With labels: also learn its confusions
calibration <- calibrate_scores(scores, labels = emotions)
calibration

Check FindingEmo Dataset Quality

Description

Checks the quality and completeness of a downloaded FindingEmo dataset, reporting on file availability, image accessibility, and potential issues.

Usage

check_findingemo_quality(data_dir, check_images = FALSE, sample_size = 10)

Arguments

Value

A list containing:

• structure: Dataset structure type ("standard" or "flat")

• files_found: List of available files

• annotations_count: Number of annotations

• urls_count: Number of image URLs

• images_count: Number of downloaded images

• completeness: Percentage of images successfully downloaded

• image_check: Results of image accessibility check (if performed)

Examples

## Not run: 
# Check dataset quality
quality_report <- check_findingemo_quality("./findingemo_data")
print(quality_report)

# Check with image verification
quality_report <- check_findingemo_quality(
  data_dir = "./findingemo_data",
  check_images = TRUE,
  sample_size = 5
)

## End(Not run)

One label per item from several coders

Description

Combines the coders' answers to one question into one value per item: the most common answer, or the mean or median of Likert answers. Use it as the truth for evaluate_emotions.

Usage

consensus_labels(
  annotations,
  question = "emotion",
  method = c("majority", "mean", "median"),
  min_coders = 1
)

Arguments

Value

A data frame with one row per item: item_id, label (the combined answer, NA for a tie or too few answers), n_coders, agreement (the share of coders who gave the majority answer; NA for the mean or median), tie, and the item's face (source, frame, frame_index, time, face, track_id, person_id), or, for a task made from items, its id, reference, prediction and scores: compare the coders with the reference by evaluate_emotions(consensus, truth_col = "reference", pred_col = "label").

See Also

annotation_agreement

Delete a Transformer Model

Description

Large language models can be quite large and, when stored locally, can take up a lot of space on your computer. The direct paths to where the models are on your computer is not necessarily intuitive.

This function quickly identifies the models on your computer and informs you which ones can be deleted from it to open up storage space

Usage

delete_transformer(model_name, delete = FALSE)

Arguments

Value

Returns list of models or confirmed deletion

Author(s)

Alexander P. Christensen <alexpaulchristensen@gmail.com>

Examples

if(interactive()){
  delete_transformer()
}

Detect faces in images or a video

Description

Finds every face in a set of images or in frames sampled from a video, so you can see which faces the package found, pick the ones to analyse (with select_faces or ordinary subsetting) and score only those with face_scores.

Usage

detect_faces(
  x,
  face_detector = c("yunet", "haar"),
  min_confidence = 0.6,
  identify = TRUE,
  identity_threshold = 0.363,
  nframes = 100,
  start = 0,
  end = -1,
  uniform = FALSE,
  ffreq = 15,
  save_frames = FALSE,
  save_video = FALSE,
  save_dir = tempdir(),
  video_name = "temp",
  pattern = "\\.(jpg|jpeg|png|bmp)$",
  recursive = FALSE
)

Arguments

Details

Faces are numbered from left to right within each image or frame. In a video, a face is followed from frame to frame and keeps its track_id. With identify = TRUE, faces of the same person get the same person_id: across all images, or across all tracks of the video, so a person who leaves the picture and comes back, or who appears in several images, is one person.

Face recognition compares faces through SFace embeddings (128 numbers per face). Faces are grouped by average-linkage clustering on cosine similarity; a video track counts as one face, the average of its frames. Two faces in the same image can be the same person (mirrors, screens, posters). The embeddings stay in memory and are never returned or written to disk, and person_ids only label faces within one call: they do not name anyone and do not match ids from other calls.

Grouping faces by person is processing of biometric data under the EU General Data Protection Regulation (Article 9); make sure your ethics approval covers it, or use identify = FALSE.

The YuNet model ships with the package. The SFace model (face_recognition_sface_2021dec_int8.onnx, from OpenCV's model zoo, Apache 2.0 licence) is downloaded into the package cache on first use, or by setup_modules. To use a copy elsewhere, for example on a computer without internet access, set the environment variable TRANSFOREMOTION_SFACE_MODEL to its path. Without the model, person_id is NA.

Value

A data frame with one row per face and the columns:

source

The image path or URL, or the video.

frame, frame_index, time

For a video: the sample number (as in the frame file name), the frame's index in the video and its time in seconds. NA for images.

face

Face number within the image or frame, from left to right.

track_id

For a video: the face's track. NA for images.

person_id

"p1", "p2", ... numbered by first appearance; NA without recognition.

x, y, width, height

The face box in pixels, from the top-left corner of the image.

confidence

YuNet's score; NA for Haar.

detector

"yunet" or "haar".

image_width, image_height

Size of the image or frame in pixels.

right_eye_x, ..., mouth_left_y

The five YuNet landmarks (NA for Haar).

The attribute "media" holds one row per image or frame, with the number of faces found (n_faces), so images without faces are not lost. The attribute "settings" records the detector and, for a video, how frames were sampled, which face_scores uses to find the same frames again.

Data Privacy

All processing is done locally; images and frames are never sent to a remote server.

See Also

select_faces, face_scores, draw_faces

Examples

## Not run: 
image <- system.file("extdata", "two-people.jpg", package = "transforEmotion")
faces <- detect_faces(image)
faces[, c("face", "person_id", "x", "y", "width", "height", "confidence")]
draw_faces(faces)

# Every face of one person in a video, scored once per sampled frame
video <- system.file("extdata", "two-people.mp4", package = "transforEmotion")
faces <- detect_faces(video, nframes = 20)
table(faces$track_id, faces$person_id)
scores <- face_scores(select_faces(faces, persons = "p1"),
                      classes = c("happy", "sad", "angry", "neutral"))

## End(Not run)

Dynamics function of the DLO model

Description

This function calculates the dynamics of a system using the DLO (Damped Linear Oscillator) model based on Equation 1 (Ollero et al., 2023). The DLO model is a second-order differential equation that describes the behavior of a damped harmonic oscillator. The function takes in the current state of the system, the derivative of the state, the damping coefficient, the time step, and the values of the eta and zeta parameters. It returns the updated derivative of the state.

Usage

dlo_dynamics(x, dxdt, q, dt, eta, zeta)

Arguments

Value

A numeric vector containing the updated derivative of the state.

References

Ollero, M. J. F., Estrada, E., Hunter, M. D., & Cancer, P. F. (2023). Characterizing affect dynamics with a damped linear oscillator model: Theoretical considerations and recommendations for individual-level applications. Psychological Methods. doi:10.1037/met0000615

Download FindingEmo-Light Dataset

Description

Downloads the FindingEmo-Light dataset using the official PyPI package. This dataset contains 25k images with emotion annotations including valence, arousal, and discrete emotion labels, focusing on complex naturalistic scenes with multiple people in social settings.

Usage

download_findingemo_data(
  target_dir,
  max_images = NULL,
  randomize = FALSE,
  skip_existing = TRUE,
  force = FALSE
)

Arguments

Details

This function uses the findingemo-light Python package, which is installed automatically on first use (or ahead of time with setup_modules(extras = "findingemo")).

The FindingEmo dataset is described in: Mertens, L. et al. (2024). "FindingEmo: An Image Dataset for Emotion Recognition in the Wild". NeurIPS 2024 Datasets and Benchmarks Track.

The dataset uses a flat directory structure with all images stored directly in the images/ subdirectory, annotations.csv and urls.json at the root level.

**Note**: For copyright reasons, the dataset provides URLs and annotations only. Images are downloaded on-demand from their original sources.

Value

A list containing:

• success: Logical indicating if download was successful

• message: Character string with status message

• target_dir: Path to downloaded data

• annotation_file: Path to annotation file (if successful)

• urls_file: Path to URLs file (if successful)

• image_count: Number of images downloaded (if any)

• annotations: Full annotations data.frame (raw)

• evaluation_data: Data.frame filtered to downloaded images with columns suitable for evaluation workflows (id, truth, image_file, image_path, valence, arousal, emo8_label, emotion)

• evaluation_csv: Path to saved CSV of evaluation_data

• matched_count: Number of annotations matched to downloaded images

See Also

load_findingemo_annotations, setup_modules

Examples

## Not run: 
# First install required modules
setup_modules(extras = "findingemo")

# Download dataset to local directory
result <- download_findingemo_data("./findingemo_data")

if (result$success) {
  cat("Dataset downloaded to:", result$target_dir)
  cat("Images downloaded:", result$image_count)
}

# Download random subset for testing/benchmarking
result <- download_findingemo_data(
  target_dir = "./findingemo_test",
  max_images = 100,
  randomize = TRUE
)

# Download subset with flat directory structure (always used)
result <- download_findingemo_data(
  target_dir = "./findingemo_subset",
  max_images = 50
)

# Force re-download
result <- download_findingemo_data(
  target_dir = "./findingemo_data",
  force = TRUE
)

## End(Not run)

Draw detected faces

Description

Draws the box and a label for each face on its image or video frame, so you can check what detect_faces found: which faces were detected, how they are numbered, and which faces were recognised as the same person. Faces of the same person share a colour. plot() on a detect_faces result does the same.

Usage

draw_faces(faces, labels = NULL, max_images = 20, dir = tempdir(), plot = TRUE)

## S3 method for class 'te_faces'
plot(x, ...)

Arguments

Details

Only images and frames with a face in faces are drawn. The "media" attribute of a detect_faces result lists every image or frame with its number of faces, including those without any.

For a video, the frames are extracted again (as in face_scores), unless detect_faces kept them with save_frames = TRUE.

Value

The paths of the drawn images, invisibly.

See Also

detect_faces, select_faces

Examples

## Not run: 
image <- system.file("extdata", "two-people.jpg", package = "transforEmotion")
faces <- detect_faces(image)
draw_faces(faces)

# A video: a few frames, with each face's track and person
video <- system.file("extdata", "two-people.mp4", package = "transforEmotion")
plot(detect_faces(video, nframes = 20), max_images = 4)

## End(Not run)

Emotions Data

Description

A matrix containing words (n = 175,592) and the emotion category most frequently associated with each word. This dataset is a modified version of the 'DepecheMood++' lexicon developed by Araque, Gatti, Staiano, and Guerini (2018). For proper scoring, text should not be stemmed prior to using this lexicon. This version of the lexicon does not rely on part of speech tagging.

Usage

data(emotions)

Format

A data frame with 175,592 rows and 9 columns.

word

An entry in the lexicon, in English

AFRAID, AMUSED, ANGRY, ANNOYED, DONT_CARE, HAPPY, INSPIRED, SAD

The emotional category. All emotions contain either a 0 or 1. If the category is most likely to be associated with the word, it recieves a 1, otherwise, 0. Words are only associated with one category.

References

Araque, O., Gatti, L., Staiano, J., and Guerini, M. (2018). DepecheMood++: A bilingual emotion lexicon built through simple yet powerful techniques. ArXiv

Examples

data("emotions")

Emoxicon Scores

Description

A bag-of-words approach for computing emotions in text data using the lexicon compiled by Araque, Gatti, Staiano, and Guerini (2018).

Usage

emoxicon_scores(text, lexicon, exclude)

Arguments

Author(s)

Tara Valladares <tls8vx at virginia.edu> and Hudson F. Golino <hfg9s at virginia.edu>

References

Araque, O., Gatti, L., Staiano, J., and Guerini, M. (2018). DepecheMood++: A bilingual emotion lexicon built through simple yet powerful techniques. ArXiv

See Also

emotions, where we describe how we modified the original DepecheMood++ lexicon.

Examples

# Obtain "emotions" data
data("emotions")

# Obtain "tinytrolls" data
data("tinytrolls")

## Not run: 
# Obtain emoxicon scores for first 10 tweets
emotions_tinytrolls <- emoxicon_scores(text = tinytrolls$content, lexicon = emotions)

## End(Not run)

Generate and emphasize sudden jumps in emotion scores

Description

This function generates and emphasizes the effect of strong emotions expressions during the period where the derivative of the latent variable is high. The observable value of the strongest emotion from the positive or negative group will spike in the next k time steps. The probability of this happening is p at each time step in which the derivative of the latent variable is greater than 0.2. The jump is proportionate to the derivative of the latent variable and the sum of the observable values of the other emotions.

Usage

emphasize(data, num_observables, num_steps, k = 10, p = 0.5)

Arguments

Value

A data frame containing the updated observable variables.

Evaluate Emotion Classification Performance

Description

Comprehensive evaluation function for discrete emotion classification tasks. Computes standard classification metrics including accuracy, F1-scores, AUROC, calibration metrics, and inter-rater reliability measures.

Usage

evaluate_emotions(
  data,
  id_col = "id",
  truth_col = "truth",
  pred_col = "pred",
  probs_cols = NULL,
  classes = NULL,
  metrics = c("accuracy", "precision", "recall", "f1_macro", "f1_micro", "auroc", "ece",
    "krippendorff", "confusion_matrix"),
  return_plot = FALSE,
  na_rm = TRUE
)

Arguments

Details

This function implements a comprehensive evaluation pipeline for discrete emotion classification following best practices from the literature.

**Metrics computed:**

• **Accuracy**: Overall classification accuracy

• **Precision/Recall/F1**: Per-class and macro/micro averages

• **AUROC**: Area under ROC curve (requires probability scores)

• **ECE**: Expected Calibration Error for probability calibration

• **Krippendorff's alpha**: Inter-rater reliability between human and model

**Input format:** The input data should contain at minimum:

• ID column: Unique identifier for each instance

• Truth column: Ground truth emotion labels

• Prediction column: Model predicted emotion labels

• Probability columns (optional): Class probabilities for each emotion

Value

A list containing:

• metrics: Data frame with computed evaluation metrics

• confusion_matrix: Confusion matrix (if requested)

• per_class: Per-class metrics breakdown

• summary: Overall performance summary

• plot_data: Data prepared for plotting (if return_plot = TRUE)

References

Grandini, M., Bagli, E., & Visani, G. (2020). Metrics for multi-class classification: an overview. arXiv preprint arXiv:2008.05756.

Krippendorff, K. (2011). Computing Krippendorff's alpha-reliability. Scholarly commons, 25.

Naeini, M. P., Cooper, G., & Hauskrecht, M. (2015). Obtaining well calibrated probabilities using bayesian binning. In AAAI (pp. 2901-2907).

See Also

transformer_scores, nlp_scores, emoxicon_scores for emotion prediction functions.

Examples

## Not run: 
# Basic evaluation with predicted labels only
results <- evaluate_emotions(
  data = evaluation_data,
  truth_col = "human_label",
  pred_col = "model_prediction"
)

# Full evaluation with probabilities
results <- evaluate_emotions(
  data = evaluation_data,
  truth_col = "ground_truth",
  pred_col = "predicted_class",
  probs_cols = c("prob_anger", "prob_joy", "prob_sadness"),
  return_plot = TRUE
)

# Custom metrics selection
results <- evaluate_emotions(
  data = evaluation_data,
  metrics = c("accuracy", "f1_macro", "confusion_matrix")
)

## End(Not run)

Score selected faces

Description

Scores each face of a detect_faces result with a CLIP model, so you can analyse exactly the faces you picked, for example one person across a whole video.

Usage

face_scores(
  faces,
  classes,
  model = "oai-base",
  local_model_path = NULL,
  padding = 0.25
)

Arguments

Details

For a video, the frames are extracted again with the sampling settings detect_faces used (and the video downloaded again from YouTube), unless the frames were kept with save_frames = TRUE.

Value

faces without the landmark columns, plus one column of scores per class.

Data Privacy

All processing is done locally with the downloaded model.

See Also

detect_faces, select_faces

Examples

## Not run: 
image <- system.file("extdata", "boris-1.png", package = "transforEmotion")
faces <- detect_faces(image)
face_scores(faces, classes = c("happy", "sad", "angry", "neutral"))

## End(Not run)

Generate observable emotion scores data from latent variables

Description

Function to generate observable data from 2 latent variables (negative and positive affect). The function takes in the latent variable scores, the number of time steps, the number of observable variables per latent factor, and the measurement error variance. It returns a matrix of observable data. The factor loadings are not the same for all observable variables. They have uniform random noise added to them (between -0.15 and 0.15). The loadings are scaled so that the sum of the loadings for each latent factor is 2, to introduce a ceiling effect and to differentiate the dynamics of specific emotions. This is further empahsized by adding small noise to the measurement error variance for each observed variable (between -0.01 and 0.01).

Usage

generate_observables(X, num_steps, num_obs, error, loadings = 0.8)

Arguments

Value

A (num_steps X num_obs) Matrix or Data frame containing the observable variables.

Generate a matrix of Dynamic Error values for the DLO simulation

Description

This function generates a matrix of Dynamic Error values (q) for the DLO simulation.

Usage

generate_q(num_steps, sigma_q)

Arguments

Value

A (num_steps X 3) matrix of Dynamic Error values for neutral, negative and positive emotion latent score.

Get Vision Model Configuration

Description

Retrieve the configuration for a specific vision model from the registry.

Usage

get_vision_model_config(name)

Arguments

Value

A list with model configuration, or NULL if model not found

Calculate image scores using a Hugging Face CLIP model

Description

This function takes an image file and a vector of classes as input and calculates the scores for each class using a specified Hugging Face CLIP model. Primary use of the function is to calculate FER scores - Facial Expression Detection of emotions based on detected facial expression in images. In case there are more than one face in the image, the function will return the scores of the face selected using the face_selection parameter. If there is no face in the image, the function will return NA for all classes. Function uses reticulate to call the Python functions in the image.py file. If you run this package/function for the first time it will take some time for the package to setup a functioning Python virtual environment in the background. This includes installing Python libraries for facial recognition and emotion detection in text, images and video. Please be patient.

Usage

image_scores(
  image,
  classes,
  face_selection = "largest",
  model = "oai-base",
  local_model_path = NULL,
  face_detector = c("yunet", "haar")
)

Arguments

Details

Data Privacy: All processing is done locally with the downloaded model, and your images are never sent to any remote server or third-party.

Value

A data frame containing the scores for each class.

Author(s)

Aleksandar Tomasevic <atomashevic@gmail.com>

Calculate image scores for all images in a directory (fast batch)

Description

This function scans a directory for image files and computes scores for each image using a Hugging Face CLIP model. It loads the model once and reuses text embeddings for speed, returning one row per image with the filename as image_id and probability columns for each class.

Usage

image_scores_dir(
  dir,
  classes,
  face_selection = "largest",
  pattern = "\\.(jpg|jpeg|png|bmp)$",
  recursive = FALSE,
  model = "oai-base",
  local_model_path = NULL,
  face_detector = c("yunet", "haar")
)

Arguments

Value

A data.frame with columns: image_id and one column per class. With "all" or a data frame of faces, one row per face, with the columns of face_scores after image_id.

Check if Vision Model is Registered

Description

Check if a vision model is available in the registry.

Usage

is_vision_model_registered(name)

Arguments

Value

Logical indicating if the model is registered

List Available Vision Models

Description

List all vision models currently available in the transforEmotion registry.

Usage

list_vision_models(
  include_builtin = TRUE,
  architecture_filter = NULL,
  verbose = FALSE
)

Arguments

Value

A data.frame with model information, or detailed list if verbose=TRUE

Examples

# List all available models
list_vision_models()

# List only CLIP models
list_vision_models(architecture_filter = "clip")

# Get detailed information
list_vision_models(verbose = TRUE)

# See what models are available for image analysis
models <- list_vision_models()
print(paste("Available models:", paste(models$name, collapse = ", ")))

Load FindingEmo-Light Annotations

Description

Loads and preprocesses annotations from a downloaded FindingEmo-Light dataset. Returns a clean R data.frame with emotion annotations, valence/arousal scores, and associated metadata.

Usage

load_findingemo_annotations(
  data_dir,
  output_format = c("dataframe", "list"),
  python_path = NULL
)

Arguments

Details

This function loads the CSV annotation file and JSON URLs file from a downloaded FindingEmo dataset, performs basic validation and preprocessing, and returns the data in a format suitable for emotion analysis.

The function handles missing values, validates valence/arousal ranges, and provides summary statistics for the loaded data.

Value

If output_format = "dataframe": A data.frame with columns:

• image_id: Unique image identifier

• valence: Valence score (emotion positivity)

• arousal: Arousal score (emotion intensity)

• Additional columns as present in the dataset

If output_format = "list": A list containing:

• annotations: Data.frame with annotation data

• urls: List with image URL information

• metadata: List with dataset metadata

See Also

download_findingemo_data, prepare_findingemo_evaluation

Examples

## Not run: 
# Download dataset first
download_result <- download_findingemo_data("./findingemo_data")

if (download_result$success) {
  # Load annotations as data.frame
  annotations <- load_findingemo_annotations("./findingemo_data")
  
  # Examine the data
  head(annotations)
  summary(annotations)
  
  # Get full processed data including metadata
  full_data <- load_findingemo_annotations(
    data_dir = "./findingemo_data",
    output_format = "list"
  )
  
  print(full_data$metadata)
}

## End(Not run)

Map Discrete Emotions to VAD (Valence-Arousal-Dominance) Framework

Description

Maps discrete emotion classifications from image_scores(), transformer_scores(), or video_scores() functions to the Valence-Arousal-Dominance (VAD) framework using published lexicons. Automatically downloads the NRC VAD lexicon via textdata package on first use.

Usage

map_discrete_to_vad(
  results,
  mapping = "nrc_vad",
  weighted = TRUE,
  cache_lexicon = TRUE,
  vad_lexicon = NULL
)

Arguments

Details

This function bridges discrete emotion classification outputs with the continuous VAD emotion framework. The VAD model represents emotions in a three-dimensional space where:

• **Valence**: Pleasantness (positive/negative)

• **Arousal**: Activation level (excited/calm)

• **Dominance**: Control (dominant/submissive)

**Input Type Detection:** The function automatically detects the input type:

• **data.frame**: Assumes output from image_scores() or video_scores()

• **list**: Assumes output from transformer_scores()

**Weighting Methods:**

• **weighted = TRUE**: Computes weighted average VAD scores based on classification confidence scores

• **weighted = FALSE**: Uses VAD values for the highest-scoring emotion only

**VAD Mappings:** Currently supports the NRC VAD lexicon which provides VAD ratings for emotion words based on crowdsourced annotations. The lexicon must be downloaded first using 'textdata::lexicon_nrc_vad()' in an interactive session.

**Setup Required:** Before using this function, download the NRC VAD lexicon by running: 'textdata::lexicon_nrc_vad()' in an interactive R session and accepting the license.

Value

A data.frame with columns:

• valence: Valence score (positive vs negative emotion)

• arousal: Arousal score (excitement vs calmness)

• dominance: Dominance score (control vs submissiveness)

For transformer_scores() input, includes additional identifier columns. For image/video_scores() input, returns one row per input row.

Data Privacy

VAD lexicon is downloaded once and cached locally. No data is sent to external servers.

Author(s)

Aleksandar Tomasevic <atomashevic@gmail.com>

References

Mohammad, S. M. (2018). Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 English words. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 174-184.

Examples

## Not run: 
# Method 1: Auto-load from textdata (requires prior download)
textdata::lexicon_nrc_vad()  # Run once to download

# With image scores
image_path <- system.file("extdata", "boris-1.png", package = "transforEmotion")
emotions <- c("joy", "sadness", "anger", "fear", "surprise", "disgust")
img_results <- image_scores(image_path, emotions)
vad_results <- map_discrete_to_vad(img_results)

# Method 2: Download once and pass as argument (recommended)
nrc_vad <- textdata::lexicon_nrc_vad()  # Download once

# Use with different emotion results
vad_results1 <- map_discrete_to_vad(img_results, vad_lexicon = nrc_vad)

text <- "I am so happy today!"
trans_results <- transformer_scores(text, emotions)
vad_results2 <- map_discrete_to_vad(trans_results, vad_lexicon = nrc_vad)

# Simple lookup (no weighting)
vad_simple <- map_discrete_to_vad(img_results, weighted = FALSE, vad_lexicon = nrc_vad)

## End(Not run)

Map FindingEmo Emotions to Emo8 Labels

Description

Maps FindingEmo dataset emotion labels to the standard 8 basic emotions (Emo8) from Plutchik's emotion wheel. This function converts complex emotion labels to the 8 fundamental emotions using intensity-based mappings from the circumplex model.

Usage

map_to_emo8(findingemo_emotions)

Arguments

Details

The mapping is based on Plutchik's circumplex model of emotions, where complex emotions are mapped to their corresponding basic emotions:

**Basic Emotions (direct mapping):** - Joy, Trust, Fear, Surprise, Sadness, Disgust, Anger, Anticipation

**Intensity Variations:** - High intensity: Ecstasy→Joy, Admiration→Trust, Terror→Fear, etc. - Low intensity: Serenity→Joy, Acceptance→Trust, Apprehension→Fear, etc.

The 8 basic emotions (Emo8) are: joy, trust, fear, surprise, sadness, disgust, anger, anticipation.

Value

Character vector of mapped Emo8 emotion labels. Unmapped emotions return NA.

Examples

## Not run: 
# Map single emotions
map_to_emo8("Joy")           # "joy"
map_to_emo8("Ecstasy")       # "joy" 
map_to_emo8("Serenity")      # "joy"

# Map multiple emotions
findingemo_labels <- c("Joy", "Rage", "Terror", "Interest")
emo8_labels <- map_to_emo8(findingemo_labels)
# Returns: c("joy", "anger", "fear", "anticipation")

# Use in evaluation pipeline
annotations <- load_findingemo_annotations("./data")
annotations$emo8_label <- map_to_emo8(annotations$emotion)

## End(Not run)

NEO-PI-R IPIP Extraversion Item Descriptions

Description

A list (length = 6) of the NEO-PI-R IPIP item descriptions (https://ipip.ori.org/newNEOFacetsKey.htm). Each vector within the 6 list elements contains the item descriptions for the respective Extraversion facets – friendliness, gregariousness, assertiveness, activity_level, excitement_seeking, and cheerfulness

Usage

data(neo_ipip_extraversion)

Format

A list (length = 6)

Examples

data("neo_ipip_extraversion")

Natural Language Processing Scores

Description

Natural Language Processing using word embeddings to compute semantic similarities (cosine; see costring) of text and specified classes. The suggested package LSAfun is loaded on use. It is not attached with transforEmotion, because loading it also loads rgl, which needs the system OpenGL and GLU libraries.

Usage

nlp_scores(
  text,
  classes,
  semantic_space = c("baroni", "cbow", "cbow_ukwac", "en100", "glove", "tasa"),
  preprocess = TRUE,
  remove_stop = TRUE,
  keep_in_env = TRUE,
  envir = 1
)

Arguments

Value

Returns semantic distances for the text classes

Author(s)

Alexander P. Christensen <alexpaulchristensen@gmail.com>

References

Baroni, M., Dinu, G., & Kruszewski, G. (2014). Don't count, predict! a systematic comparison of context-counting vs. context-predicting semantic vectors. In Proceedings of the 52nd annual meting of the association for computational linguistics (pp. 238-247).

Landauer, T.K., & Dumais, S.T. (1997). A solution to Plato's problem: The Latent Semantic Analysis theory of acquisition, induction and representation of knowledge. Psychological Review, 104, 211-240.

Pennington, J., Socher, R., & Manning, C. D. (2014). GloVe: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (pp. 1532-1543).

Examples

# Load data
data(neo_ipip_extraversion)

# Example text 
text <- neo_ipip_extraversion$friendliness[1:5]

## Not run: 
# GloVe
nlp_scores(
 text = text,
 classes = c(
   "friendly", "gregarious", "assertive",
   "active", "excitement", "cheerful"
 )
)

# Baroni
nlp_scores(
 text = text,
 classes = c(
   "friendly", "gregarious", "assertive",
   "active", "excitement", "cheerful"
 ),
 semantic_space = "baroni"
)
 
# CBOW
nlp_scores(
 text = text,
 classes = c(
   "friendly", "gregarious", "assertive",
   "active", "excitement", "cheerful"
 ),
 semantic_space = "cbow"
)

# CBOW + ukWaC
nlp_scores(
 text = text,
 classes = c(
   "friendly", "gregarious", "assertive",
   "active", "excitement", "cheerful"
 ),
 semantic_space = "cbow_ukwac"
)

# en100
nlp_scores(
 text = text,
 classes = c(
   "friendly", "gregarious", "assertive",
   "active", "excitement", "cheerful"
 ),
 semantic_space = "en100"
)

# tasa
nlp_scores(
 text = text,
 classes = c(
   "friendly", "gregarious", "assertive",
   "active", "excitement", "cheerful"
 ),
 semantic_space = "tasa"
)

## End(Not run)

Parse RAG JSON

Description

Parses a JSON string (or list) matching the enforced RAG schema and returns a normalized list: 'list(labels=chr, confidences=num, intensity=num, evidence=data.frame(doc_id, span, score))'.

Usage

parse_rag_json(x, validate = TRUE)

Arguments

Value

A normalized list with atomic vectors and an 'evidence' data.frame.

Examples

j <- '{"labels":["joy"],"confidences":[0.9],
  "intensity":0.8,"evidence_chunks":[]}'
parse_rag_json(j)

Plot Evaluation Results

Description

Creates visualizations for emotion evaluation results including confusion matrix heatmaps and per-class metrics bar plots.

Usage

## S3 method for class 'emotion_evaluation'
plot(x, type = "both", ...)

Arguments

Value

A ggplot object or list of ggplot objects

Plot the latent or the observable emotion scores.

Description

Function to plot the latent or the observable emotion scores.

Usage

plot_sim_emotions(df, mode = "latent", title = " ")

Arguments

Value

A plot of the latent or the observable emotion scores.

Posterior emotions from frame scores

Description

Combines each frame's scores with what the earlier (and, with method = "smooth", the later) frames showed, and returns the probability of each emotion in every frame and for the whole video. The scores are treated as noisy evidence, as learned by calibrate_scores, and the emotion is assumed to stay the same from one frame to the next with probability stay.

Usage

posterior_scores(
  scores,
  calibration,
  stay = 0.99,
  method = c("filter", "smooth")
)

Arguments

Details

The posterior of emotion c in frame t is

P_t(c) \propto L_t(c) \sum_{c'} T(c' \to c) P_{t-1}(c')

where L_t is the frame's evidence (see calibrate_scores) and T(c \to c) = stay. The first frame starts from equal probabilities. This is a hidden Markov model over the emotions, with the model's scores as noisy observations.

Value

A list with

frames

scores (a data frame, or a list of them) in time order, with the class columns holding the posterior probabilities and a label column with the most probable emotion (replacing any label column already there). No class may be called label, video or n_frames.

clips

A data frame with one row per video: video (the name in the list, or its number for a video without a name; frames is named the same way), n_frames, the posterior of each class and label. The clip posterior is the posterior at the last frame, which has seen every frame; it is the same for both methods. With stay near 1 it is the posterior of one emotion for the whole clip. For long videos in which the emotion changes, summarise frames instead, for example with colMeans().

See Also

calibrate_scores, video_scores

Examples

set.seed(1)
clip <- function(n, emotion) {
  s <- matrix(runif(n * 3), n, 3,
              dimnames = list(NULL, c("neutral", "happy", "sad")))
  s[, "neutral"] <- s[, "neutral"] + 0.6
  s[, emotion] <- s[, emotion] + 0.5
  as.data.frame(s / rowSums(s))
}
scores <- list(a = clip(40, "happy"), b = clip(40, "sad"),
               c = clip(40, "neutral"), d = clip(40, "sad"))
calibration <- calibrate_scores(scores)

result <- posterior_scores(scores, calibration)
result$clips
head(result$frames$a)

# Each frame using the whole clip
smoothed <- posterior_scores(scores$a, calibration, method = "smooth")
table(smoothed$frames$label)

Prepare FindingEmo Data for Evaluation

Description

Prepares FindingEmo dataset annotations for use with evaluate_emotions(). Converts the dataset format to match the expected input structure for evaluation functions.

Usage

prepare_findingemo_evaluation(
  annotations,
  predictions,
  id_col = "image_id",
  truth_col = "emotion_label",
  pred_col = "predicted_emotion",
  include_va = TRUE
)

Arguments

Details

This function merges FindingEmo annotations with model predictions and formats the result for evaluation. It handles missing values, validates data consistency, and ensures the output matches the expected format for evaluate_emotions().

Value

A data.frame formatted for use with evaluate_emotions(), containing:

• id: Image identifiers

• truth: Ground truth emotion labels

• pred: Predicted emotion labels

• valence: Valence scores (if available and include_va = TRUE)

• arousal: Arousal scores (if available and include_va = TRUE)

• Additional probability columns if present in predictions

See Also

load_findingemo_annotations, evaluate_emotions

Examples

## Not run: 
# Load annotations
annotations <- load_findingemo_annotations("./findingemo_data")

# Create mock predictions (replace with actual model predictions)
predictions <- data.frame(
  image_id = annotations$image_id[1:100],
  predicted_emotion = sample(c("happy", "sad", "angry"), 100, replace = TRUE),
  prob_happy = runif(100),
  prob_sad = runif(100),
  prob_angry = runif(100)
)

# Prepare for evaluation
eval_data <- prepare_findingemo_evaluation(
  annotations = annotations,
  predictions = predictions
)

# Evaluate model performance
results <- evaluate_emotions(
  data = eval_data,
  probs_cols = c("prob_happy", "prob_sad", "prob_angry")
)

print(results)

## End(Not run)

Print method for emotion evaluation results

Description

Print method for emotion evaluation results

Usage

## S3 method for class 'emotion_evaluation'
print(x, ...)

Arguments

Punctuation Removal for Text

Description

Keeps the punctuations you want and removes the punctuations you don't

Usage

punctuate(
  text,
  allowPunctuations = c("-", "?", "'", "\"", ";", ",", ".", "!")
)

Arguments

Details

Coarsely removes punctuations from text. Keeps general punctuations that are used in most English language text. Apostrophes are much trickier. For example, not allowing "'" will remove apostrophes from contractions like "can't" becoming "cant"

Value

Returns text with only the allowed punctuations

Author(s)

Alexander P. Christensen <alexpaulchristensen@gmail.com>

Examples

# Load data
data(neo_ipip_extraversion)

# Example text 
text <- neo_ipip_extraversion$friendliness

# Keep only periods
punctuate(text, allowPunctuations = c("."))

Python Requirements for transforEmotion

Description

Returns the Python requirements that transforEmotion installs, in requirements.txt format. Use it to build a fixed Python environment, for example in a container image, and point transforEmotion at it with the TRANSFOREMOTION_PYTHON environment variable (see Details).

Usage

python_requirements(
  extras = character(),
  gpu = FALSE,
  sysname = Sys.info()[["sysname"]],
  machine = Sys.info()[["machine"]]
)

Arguments

Details

The PyTorch requirements are direct wheel URLs built for Python 3.12, so the environment must use Python 3.12.

When TRANSFOREMOTION_PYTHON is set to a Python executable, transforEmotion uses that interpreter and does not install anything. This is the route for read-only environments such as Apptainer images, where uv cannot write to its cache.

Value

A character vector of requirement specifiers.

Examples

# Requirements for a CPU environment with the rag() packages
python_requirements(extras = "rag")

## Not run: 
# Write a requirements file and build the environment with uv (shell):
#   uv venv --python 3.12 /opt/te-venv
#   uv pip install --python /opt/te-venv/bin/python -r requirements.txt
writeLines(python_requirements(extras = "rag"), "requirements.txt")

## End(Not run)

Retrieval-augmented Generation (RAG)

Description

Performs retrieval-augmented generation {llama-index}

Supports multiple local LLM backends via HuggingFace and llama-index.

Usage

rag(
  text = NULL,
  path = NULL,
  transformer = c("TinyLLAMA", "Gemma3-1B", "Gemma3-4B", "Qwen2.5-1.5B", "Qwen2.5-3B",
    "Qwen3-1.7B", "Ministral-3B"),
  prompt = "You are an expert at extracting themes across many texts",
  query,
  response_mode = c("accumulate", "compact", "no_text", "refine", "simple_summarize",
    "tree_summarize"),
  similarity_top_k = 5,
  retriever = c("vector", "bm25"),
  retriever_params = list(),
  output = c("text", "json", "table", "csv"),
  task = c("general", "emotion", "sentiment"),
  labels_set = NULL,
  max_labels = 5,
  global_analysis = FALSE,
  device = c("auto", "cpu", "cuda"),
  temperature = NULL,
  do_sample = NULL,
  max_new_tokens = NULL,
  top_p = NULL,
  keep_in_env = TRUE,
  envir = 1,
  progress = TRUE
)

Arguments

Value

For output = "text", returns an object of class "rag" with fields: $response (character), $content (data.frame), and $document_embeddings (matrix). For output = "json", returns a JSON character(1) string matching the enforced schema. For output = "table", returns a data.frame suitable for statistical analysis.

Data Privacy

All processing is done locally with the downloaded model, and your text is never sent to any remote server or third-party.

Author(s)

Alexander P. Christensen <alexpaulchristensen@gmail.com>

Examples

# Load data
data(neo_ipip_extraversion)

# Example text
text <- neo_ipip_extraversion$friendliness[1:5]

## Not run: 
rag(
 text = text,
 query = "What themes are prevalent across the text?",
 response_mode = "tree_summarize",
 similarity_top_k = 5
)

# Structured outputs (Gemma 3 or Qwen 2.5)
rag(text = text, query = "Extract emotions", output = "json",
    transformer = "Qwen2.5-3B")
rag(text = text, query = "Extract emotions", output = "table",
    transformer = "Qwen2.5-3B")

## End(Not run)

RAG JSON utilities

Description

Helpers for validating, parsing, and flattening structured output from 'rag(output = "json"|"table")'.

RAG retriever registry and helpers

Description

Provides a simple plugin system to add custom retrievers and built-in handlers for vector and BM25 retrieval. Used internally by 'rag()'.

Structured Emotion/Sentiment via RAG (Small LLMs)

Description

Convenience wrapper around rag() that keeps vector retrieval but simplifies getting structured outputs for emotion or sentiment analysis using small local LLMs (1–4B) with sensible defaults.

Usage

rag_sentemo(
  text = NULL,
  path = NULL,
  task = c("emotion", "sentiment"),
  labels_set = NULL,
  max_labels = 5,
  transformer = c("TinyLLAMA", "Gemma3-1B", "Gemma3-4B", "Qwen2.5-1.5B", "Qwen2.5-3B",
    "Qwen3-1.7B", "Ministral-3B"),
  similarity_top_k = 5,
  response_mode = c("compact", "refine", "simple_summarize"),
  output = c("table", "json", "csv"),
  global_analysis = FALSE,
  ...
)

Arguments

Value

For Gemma 3 and Qwen 2.5 and output = "table"/"csv", a data.frame with columns
doc_id, text, label, confidence.
For Gemma 3 and Qwen 2.5 and output = "json", a JSON array of per-doc objects with those fields.
For other models, structured outputs are not supported; the function falls back to output = "text" and returns a free-text "rag" object.

Examples

## Not run: 
texts <- c(
  "I feel so happy and grateful today!",
  "This is frustrating and makes me angry."
)
rag_sentemo(texts, task = "emotion", output = "table")
rag_sentemo(texts, task = "sentiment", output = "json")
rag_sentemo(texts, task = "emotion", output = "table", transformer = "Qwen2.5-3B")

## End(Not run)

Read the answers of a face annotation task

Description

Reads what coders answered in annotate_faces: the files in the task's results folder, and any exported files you add.

Usage

read_annotations(task, files = NULL, revisions = c("last", "all"))

Arguments

Value

A data frame with one row per item, coder and question: item_id, coder_id, question, value (the answer as text, NA when the question was not asked or the item was skipped), value_num (the answer as a number, for Likert questions), status ("answered" or "skipped"), revision, rt_ms (time on the item), shown_at, submitted_at and session_id, followed by the item's face: source, frame, frame_index, time, face, track_id and person_id. For a task made from items: the item's id, kind, media, reference (its label), prediction and score columns.

See Also

annotation_agreement, consensus_labels

Read faces chosen in the browser

Description

Reads the file that Export selection in view_faces(select = TRUE) downloads, or the code in the line that Copy R code gives in a notebook, and returns the chosen faces.

Usage

read_face_selection(file, faces)

Arguments

Value

The chosen rows of faces, which face_scores scores and face_selection in image_scores, image_scores_dir and video_scores accepts.

See Also

view_faces

Examples

## Not run: 
faces <- detect_faces(system.file("extdata", "two-people.jpg", package = "transforEmotion"))
view_faces(faces, select = TRUE)
chosen <- read_face_selection("~/Downloads/selection-faces-20261011-101500.json", faces)

## End(Not run)

Register a custom retriever

Description

Registers a retriever under a name. The handler should construct and return a query engine compatible with llama-index or a fallback with a 'query_fn'.

Usage

register_retriever(name, handler)

Arguments

Register a Vision Model

Description

Register a new vision model in the transforEmotion registry, making it available for use with image_scores(), video_scores(), and related functions.

Usage

register_vision_model(
  name,
  model_id,
  architecture = "clip",
  description = NULL,
  preprocessing_config = NULL,
  requires_special_handling = FALSE
)

Arguments

Value

Invisibly returns TRUE if registration successful

Examples

## Not run: 
# Register a custom CLIP model
register_vision_model(
  name = "my-emotion-clip",
  model_id = "j-hartmann/emotion-english-distilroberta-base",
  architecture = "clip",
  description = "Custom CLIP fine-tuned on emotion datasets"
)

# Register a local model
register_vision_model(
  name = "local-clip",
  model_id = "/path/to/local/model",
  architecture = "clip",
  description = "Locally stored fine-tuned model"
)

# Register experimental BLIP model
register_vision_model(
  name = "blip-caption",
  model_id = "Salesforce/blip-image-captioning-base",
  architecture = "blip",
  description = "BLIP model for image captioning"
)

## End(Not run)

Remove a Vision Model

Description

Remove a custom vision model from the registry. Built-in models cannot be removed.

Remove a vision model from the transforEmotion registry.

Usage

remove_vision_model(name, confirm = TRUE)

remove_vision_model(name, confirm = TRUE)

Arguments

Value

Invisibly returns TRUE if successful

Invisibly returns TRUE if removal successful

Examples

## Not run: 
# Remove a custom model
remove_vision_model("my-custom-model")

# Remove without confirmation prompt
remove_vision_model("my-custom-model", confirm = FALSE)

## End(Not run)

Select faces to analyse

Description

Keeps the faces of a detect_faces result that a rule, face numbers, tracks or people pick. The result can be passed to face_scores, or as face_selection to image_scores or video_scores. Ordinary subsetting (for example faces[faces$person_id == "p1", ]) works as well.

Usage

select_faces(
  faces,
  rule = c("all", "largest", "left", "right"),
  ids = NULL,
  tracks = NULL,
  persons = NULL
)

Arguments

Value

The selected rows of faces, with its attributes.

Examples

faces <- data.frame(
  source = "group.jpg", frame = NA, face = 1:3, person_id = c("p1", "p2", "p3"),
  x = c(10, 200, 400), y = 50, width = c(80, 120, 60), height = c(80, 120, 60)
)
select_faces(faces, rule = "largest")
select_faces(faces, persons = c("p1", "p3"))

Sentiment Analysis Scores

Description

Uses sentiment analysis pipelines from huggingface to compute probabilities that the text corresponds to the specified classes

Usage

sentence_similarity(
  text,
  comparison_text,
  transformer = c("all_minilm_l6"),
  device = c("auto", "cpu", "cuda"),
  preprocess = FALSE,
  keep_in_env = TRUE,
  envir = 1
)

Arguments

Value

Returns a n x m similarity matrix where n is length of text and m is the length of comparison_text

Author(s)

Alexander P. Christensen <alexpaulchristensen@gmail.com>

Examples

# Load data
data(neo_ipip_extraversion)

# Example text
text <- neo_ipip_extraversion$friendliness[1:5]

## Not run: 
# Example with defaults
sentence_similarity(
 text = text, comparison_text = text
)

# Example with model from 'sentence-transformers'
sentence_similarity(
 text = text, comparison_text = text,
 transformer = "sentence-transformers/all-mpnet-base-v2"
)


## End(Not run)

Use a Shared or Offline Cache

Description

Stores the Python environment, Python itself and the Hugging Face models in one folder, and optionally runs without network access. This is the route for HPC clusters: prepare the folder on a login node with internet access, then use it offline on compute nodes.

Usage

setup_cache(path, offline = FALSE, gpu = NULL)

Arguments

Details

Call setup_cache() in a new R session, after library(transforEmotion) and before any analysis. It sets these environment variables for the session:

UV_CACHE_DIR

uv package cache and Python environments

UV_PYTHON_INSTALL_DIR, UV_PYTHON_PREFERENCE

a Python interpreter kept in the cache, so compute nodes do not need one

HF_HOME

Hugging Face models

R_USER_CACHE_DIR

reticulate's own copy of uv and the NLTK data used by rag(). This also moves the cache of any other R package that uses tools::R_user_dir() in this session.

Offline sessions can use any combination of the feature sets prepared with setup_modules(), and any model it downloaded.

The cache must stay writable, because uv locks it even when reusing an environment. For read-only environments such as Apptainer images, build a fixed Python environment instead (see python_requirements).

Value

Invisibly, a named character vector of the environment variables set.

Examples

## Not run: 
# On a login node, with internet access
library(transforEmotion)
setup_cache("/project/mylab/transforEmotion-cache", gpu = TRUE)
setup_modules(extras = "rag", models = "facebook/bart-large-mnli")

# In a job on a compute node, without internet access
library(transforEmotion)
setup_cache("/project/mylab/transforEmotion-cache", offline = TRUE, gpu = TRUE)
transformer_scores(text, classes, transformer = "facebook-bart")

## End(Not run)

Install GPU Python Modules

Description

Adds bitsandbytes, used for 4-bit quantization of the EVA-CLIP-8B vision model, to the Python environment. Equivalent to setup_modules(extras = "gpu", download_models = FALSE).

Usage

setup_gpu_modules()

Details

The CUDA build of PyTorch does not need this function: it is selected automatically when an NVIDIA GPU is detected (see setup_modules). image_scores() and video_scores() add bitsandbytes themselves when EVA-CLIP-8B is used on a GPU.

Note

This function requires NVIDIA GPU and drivers to be properly installed.

Author(s)

Alexander P. Christensen <alexpaulchristensen@gmail.com>

Deprecated: Miniconda setup (use uv instead)

Description

setup_miniconda() is deprecated. The transforEmotion package now uses reticulate's uv-based ephemeral environments managed via 'py_require()'. No conda or Miniconda installation is required.

Usage

setup_miniconda()

Setup Required Python Modules

Description

Builds the Python environment for transforEmotion ahead of first use and downloads the default models, so the first analysis call does not wait on either. Calling it is optional: every function sets up what it needs on first use.

Usage

setup_modules(
  extras = character(),
  download_models = TRUE,
  models = character()
)

Arguments

Details

Python requirements are declared with py_require. reticulate resolves them with uv into a cached environment, so after the first run later sessions start in under a second. No conda installation or manual uv installation is needed; reticulate downloads uv when it is not on the PATH.

PyTorch is installed as a CPU-only build unless an NVIDIA GPU is detected. Set TE_FORCE_CPU=1 to always use the CPU build, or TRANSFOREMOTION_USE_GPU=1 to always use the CUDA build. The choice is made when the package is loaded, so set either variable in .Renviron or before library(transforEmotion), in a new R session.

Value

Invisibly returns NULL.

Author(s)

Alexander P. Christensen <alexpaulchristensen@gmail.com>

Examples

## Not run: 
# Core packages and default models
setup_modules()

# Also install the packages rag() needs
setup_modules(extras = "rag")

# Prepare an additional model for offline use
setup_modules(models = "facebook/bart-large-mnli")

## End(Not run)

Quick Setup for Popular Models

Description

Convenience function to quickly add popular vision models with pre-configured settings.

Usage

setup_popular_models(models)

Arguments

Value

Invisibly returns TRUE if all models added successfully

Examples

## Not run: 
# Add BLIP models for image captioning
setup_popular_models("blip-base")

# Add multiple experimental models at once
setup_popular_models(c("blip-base", "blip-large", "align-base"))

# Then use them in your analysis
list_vision_models()  # See all available models
result <- image_scores("image.jpg", c("happy", "sad"), model = "blip-base")

## End(Not run)

Show Available Vision Models

Description

Display all available vision models in a user-friendly format with additional details and usage hints.

Usage

show_vision_models(show_details = FALSE, filter_by = NULL)

Arguments

Value

Invisibly returns the models data.frame

Examples

# Show all models
show_vision_models()

# Show only CLIP models
show_vision_models(filter_by = "clip")

# Show detailed information
show_vision_models(show_details = TRUE)

Simulate latent and observed emotion scores for a single "video"

Description

This function simulates emotions in a video using the DLO model implemented as continuous time state space model. The function takes in several parameters, including the time step, number of steps, number of observables, and various model parameters. It returns a data frame containing the simulated emotions and their derivatives, as well as smoothed versions of the observables. The initial state of the video is always the same. Neutral score is 0.5 and both positive and negative emotion score is 0.25. To simulate more realistic time series, there is an option of including a sudden jump in the emotion scores. This is done by emphasizing the effect of the dominant emotion during the period where the derivative of the latent variable is high. The observable value of the strongest emotion from the positive or negative group will spike in the next k time step (emph.dur). The probability of this happening is p at each time step in which the derivative of the latent variable is greater than 0.2. The jump is proportionate to the derivative of the latent variable and the sum of the observable values of the other emotions.

Usage

simulate_video(
  dt,
  num_steps,
  num_observables,
  eta_n,
  zeta_n,
  eta,
  zeta,
  sigma_q,
  sd_observable,
  loadings,
  window_size,
  emph = FALSE,
  emph.dur = 10,
  emph.prob = 0.5
)

Arguments

Value

A data frame (num_steps X (6 + num_observables)) containing the latent scores for neutral score, positive emotions, negative emotions and their derivatives, as well as smoothed versions of the observables.

Examples

simulate_video(dt = 0.01, num_steps = 50, num_observables = 4, 
               eta_n = 0.5, zeta_n = 0.5,
               eta = 0.5, zeta = 0.5,
               sigma_q = 0.1, sd_observable = 0.1,
               loadings = 0.8, window_size = 10)

Speech Emotion Scores for Segments of an Audio or Video File

Description

Splits the audio track of an audio file, a video file or a YouTube video into segments and scores each segment against user-defined classes with a CLAP audio-text model (zero-shot). Each segment's scores are the softmax over the classes of the similarity between the segment's audio embedding and each class's text embedding, as image_scores does with CLIP.

Usage

speech_scores(
  audio,
  classes,
  nsegments = 100,
  segment_length = 5,
  start = 0,
  end = -1,
  uniform = FALSE,
  model = "laion-clap",
  local_model_path = NULL
)

Arguments

Details

The audio is converted to mono at the model's sampling rate (48 kHz). With uniform = FALSE, a last segment shorter than one second is not scored, unless it is the only segment. A range no longer than segment_length is scored as one segment. Segments shorter than 10 seconds are repeated to fill CLAP's 10-second input.

General-purpose CLAP models separate speech emotions much less well than CLIP separates facial expressions. On RAVDESS (1440 acted English clips, eight emotions), "laion-clap" had an unweighted average recall of 0.13 with one-word classes such as "angry" and 0.17 with classes such as "a person speaking angrily", where chance is 0.125. Check the scores against labelled examples from your own data before you use them.

Value

A data frame with one row per segment: segment (1, 2, ...), start and end (the segment's times in the file, in seconds) and one column of scores per class. The scores of a row sum to 1.

Data Privacy

All processing is done locally with the downloaded model, and your audio is never sent to any remote server or third-party.

Author(s)

Aleksandar Tomasevic <atomashevic@gmail.com>

See Also

video_scores for the frames of the same video.

Examples

## Not run: 
speech_scores("interview.wav",
              classes = c("a person speaking angrily", "a person speaking happily",
                          "a person speaking sadly", "a person speaking calmly"),
              nsegments = 20, segment_length = 5)

## End(Not run)

Stop Words from the tm Package

Description

174 English stop words in the tm package

Usage

data(stop_words)

Format

A vector (length = 174)

Examples

data("stop_words")

Summary method for emotion evaluation results

Description

Summary method for emotion evaluation results

Usage

## S3 method for class 'emotion_evaluation'
summary(object, ...)

Arguments

Remove reticulate's default virtualenv (r-reticulate)

Description

Removes the default 'reticulate' virtual environment at '~/.virtualenvs/r-reticulate' to avoid conflicts with uv-managed environments used by transforEmotion. This is optional and only needed if you want to ensure uv's environment is preferred.

Usage

te_cleanup_default_venv(confirm = TRUE)

Arguments

Value

Invisibly returns TRUE on success, FALSE otherwise.

Examples

## Not run: 
te_cleanup_default_venv()
te_cleanup_default_venv(confirm = FALSE)

## End(Not run)

Russian Trolls Data - Small Version

Description

A matrix containing a smaller subset of tweets from the trolls dataset, useful for test purposes. There are approximately 20,000 tweets from 50 authors. This dataset includes only authored tweets by each account; retweets, reposts, and repeated tweets have been removed. The original data was provided by FiveThirtyEight and Clemson University researchers Darren Linvill and Patrick Warren. For more information, visit https://github.com/fivethirtyeight/russian-troll-tweets

Usage

data(tinytrolls)

Format

A data frame with 22,143 rows and 6 columns.

content

A tweet.

author

The name of the handle that authored the tweet.

publish_date

The date the tweet was published on.

followers

How many followers the handle had at the time of posting.

updates

How many interactions (including likes, tweets, retweets) the post garnered.

account_type

Left or Right

Examples

data(tinytrolls)

Sentiment Analysis Scores

Description

Uses sentiment analysis pipelines from huggingface to compute probabilities that the text corresponds to the specified classes

Usage

transformer_scores(
  text,
  classes,
  multiple_classes = FALSE,
  transformer = c("cross-encoder-roberta", "cross-encoder-distilroberta",
    "facebook-bart"),
  device = c("auto", "cpu", "cuda"),
  preprocess = FALSE,
  keep_in_env = TRUE,
  envir = 1,
  local_model_path = NULL
)

Arguments

Value

Returns probabilities for the text classes

Data Privacy

All processing is done locally with the downloaded model, and your text is never sent to any remote server or third-party.

Author(s)

Alexander P. Christensen <alexpaulchristensen@gmail.com>

References

# BART
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., ... & Zettlemoyer, L. (2019). Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461.

# RoBERTa
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., ... & Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.

# Zero-shot classification
Yin, W., Hay, J., & Roth, D. (2019). Benchmarking zero-shot text classification: Datasets, evaluation and entailment approach. arXiv preprint arXiv:1909.00161.

# MultiNLI dataset
Williams, A., Nangia, N., & Bowman, S. R. (2017). A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426.

Examples

# Load data
data(neo_ipip_extraversion)

# Example text
text <- neo_ipip_extraversion$friendliness[1:5]

## Not run: 
# Cross-Encoder DistilRoBERTa
transformer_scores(
 text = text,
 classes = c(
   "friendly", "gregarious", "assertive",
   "active", "excitement", "cheerful"
 )
)

# Facebook BART Large
transformer_scores(
 text = text,
 classes = c(
   "friendly", "gregarious", "assertive",
   "active", "excitement", "cheerful"
 ),
 transformer = "facebook-bart"
)

# Directly from huggingface: typeform/distilbert-base-uncased-mnli
transformer_scores(
 text = text,
 classes = c(
   "friendly", "gregarious", "assertive",
   "active", "excitement", "cheerful"
 ),
 transformer = "typeform/distilbert-base-uncased-mnli"
)

## End(Not run)

Direct VAD (Valence-Arousal-Dominance) Prediction

Description

Directly predicts VAD dimensions using classification with definitional labels, bypassing the intermediate step of discrete emotion classification. This approach uses rich, educational descriptions of each VAD pole to help transformer models understand the psychological concepts and make more accurate predictions.

Usage

vad_scores(
  input,
  input_type = "auto",
  dimensions = c("valence", "arousal", "dominance"),
  label_type = "definitional",
  custom_labels = NULL,
  model = "auto",
  ...
)

Arguments

Details

This function implements direct VAD prediction using the approach: Input → VAD Classification → VAD Scores

Instead of mapping from discrete emotions, each VAD dimension is treated as a separate binary classification task using definitional labels that explain the psychological concepts.

**Definitional Labels (default):** The function uses rich descriptions that educate the model about each dimension:

• **Valence**: "Positive valence, which refers to pleasant, enjoyable..."

• **Arousal**: "High arousal, which refers to intense, energetic..."

• **Dominance**: "High dominance, which refers to feeling in control..."

**Input Type Detection:** When input_type = "auto", the function detects input type based on:

• URLs starting with "http": Video

• File paths with image extensions: Image

• Everything else: Text

**Score Interpretation:** Scores represent the probability that the input exhibits the "high" pole:

• **Valence**: 1.0 = very positive, 0.0 = very negative

• **Arousal**: 1.0 = high energy, 0.0 = very calm

• **Dominance**: 1.0 = very controlling, 0.0 = very powerless

Value

A data.frame with columns:

• input_id: Identifier for each input (text content, filename, or index)

• valence: Valence score (0-1, where 1 = positive)

• arousal: Arousal score (0-1, where 1 = high arousal)

• dominance: Dominance score (0-1, where 1 = high dominance)

Only requested dimensions are included in output.

Data Privacy

All processing is done locally with downloaded models. Data is never sent to external servers.

Author(s)

Aleksandar Tomasevic <atomashevic@gmail.com>

References

Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161-1178.

Bradley, M. M., & Lang, P. J. (1994). Measuring emotion: the self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 25(1), 49-59.

Examples

## Not run: 
# Text VAD analysis
texts <- c("I'm absolutely thrilled!", "I feel so helpless and sad", "This is boring")
text_vad <- vad_scores(texts, input_type = "text")
print(text_vad)

# Image VAD analysis  
image_path <- system.file("extdata", "boris-1.png", package = "transforEmotion")
image_vad <- vad_scores(image_path, input_type = "image")
print(image_vad)

# Single dimension prediction
valence_only <- vad_scores(texts, dimensions = "valence")

# Using simple labels for speed
simple_vad <- vad_scores(texts, label_type = "simple")

# Custom labels for domain-specific applications
custom_labels <- list(
  valence = list(
    positive = "Customer satisfaction and positive brand sentiment",
    negative = "Customer complaints and negative brand sentiment"
  )
)
brand_vad <- vad_scores(texts, dimensions = "valence", 
                        label_type = "custom", custom_labels = custom_labels)

## End(Not run)

Validate a RAG JSON structure

Description

Ensures the object has the expected fields and types: 'labels', 'confidences', 'intensity', and 'evidence_chunks'.

Usage

validate_rag_json(x, error = TRUE)

Arguments

Value

Invisibly returns TRUE when valid; otherwise FALSE or error.

Validate RAG Emotion/Sentiment Predictions

Description

Evaluates emotion/sentiment predictions from rag() or rag_sentemo() against ground truth labels using the same metrics pipeline as evaluate_emotions(). Supports table or JSON structured outputs.

Usage

validate_rag_predictions(
  rag_output,
  ground_truth,
  id_col = NULL,
  task = c("emotion", "sentiment"),
  labels_set = NULL,
  metrics = c("accuracy", "f1_macro", "confusion_matrix"),
  return_plot = FALSE
)

Arguments

Value

A list of evaluation results in the same format as evaluate_emotions(), augmented with '$rag_metadata' summarizing RAG-specific context (documents, transformer, task).

Examples

## Not run: 
texts <- c(
  "I feel so happy and grateful today!",
  "This is frustrating and makes me angry.",
  "I'm not sure how I feel about this."
)

# Get predictions (structured per-document output)
rag_results <- rag_sentemo(
  texts,
  task = "emotion",
  output = "table",
  transformer = "Gemma3-1B"
)

# Ground truth labels
ground_truth <- c("joy", "anger", "neutral")

# Validate predictions
validation_results <- validate_rag_predictions(
  rag_output = rag_results,
  ground_truth = ground_truth,
  task = "emotion",
  metrics = c("accuracy", "f1_macro", "confusion_matrix"),
  return_plot = TRUE
)

## End(Not run)

Run FER on a YouTube video using a Hugging Face CLIP model

Description

This function retrieves facial expression recognition (FER) scores from a specific number of frames extracted from a YouTube video using a specified Hugging Face CLIP model. It utilizes Python libraries for facial recognition and emotion detection in text, images, and video.

Usage

video_scores(
  video,
  classes,
  nframes = 100,
  face_selection = "largest",
  start = 0,
  end = -1,
  uniform = FALSE,
  ffreq = 15,
  save_video = FALSE,
  save_frames = FALSE,
  save_dir = "temp/",
  video_name = "temp",
  model = "oai-base",
  local_model_path = NULL,
  face_detector = c("yunet", "haar")
)

Arguments

Value

A data frame of scores, one row per frame that could be read (or one row per face; see face_selection).

Data Privacy

All processing is done locally with the downloaded model, and your video frames are never sent to any remote server or third-party.

Author(s)

Aleksandar Tomasevic <atomashevic@gmail.com>

Look at detected faces in the browser, and choose faces to analyse

Description

Starts a small web page from R that shows every image, or every sampled video frame, with its face boxes, and opens it in your web browser. Faces of the same person share a colour, and a People panel lists each person with the images, frames and tracks they appear in. With select = TRUE, you click faces, video tracks or people, then Send to R, and view_faces() returns the chosen faces.

Usage

view_faces(
  faces,
  select = FALSE,
  selected = NULL,
  live = TRUE,
  open = interactive(),
  dir = NULL,
  title = NULL,
  max_side = NULL,
  port = NULL,
  inline = .te_in_notebook()
)

Arguments

Details

The page is served by R itself (with httpuv) on your own computer only (127.0.0.1), at an address with a random part that only you get. It makes no outside requests. While R waits for a selection, the console is busy; press Esc (Ctrl+C in a terminal) to stop waiting.

With live = FALSE, no server is started: the page is written as a folder that also works when opened from disk, for example after copying it from a remote computer to yours. Its Export selection button then downloads a file that read_face_selection() reads.

In a Jupyter notebook, including Google Colab, the page is shown inside the notebook, below the cell (inline = TRUE, the default there). A page served from R cannot be reached from a notebook running on another computer, so in select mode the page gives a line of R code instead: click Copy R code, paste the line into a new cell and run it, and read_face_selection() returns the chosen faces.

Images without a face are shown too when faces is a complete detect_faces result; for a subset, only images with a face in it are shown. For a video, every sampled frame is shown, and the frames are extracted again (as in face_scores), unless detect_faces kept them with save_frames = TRUE.

In select mode, clicking a face in a still image selects that face; clicking a face in a video selects its whole track; clicking a person selects every face of that person.

When R runs on a remote computer over SSH, forward the port first (the message printed by view_faces() gives the command), or use live = FALSE and copy the folder.

The page lists file names and shows the images, so share it only with people who may see them. It contains no face embeddings and no scores.

Value

With select = TRUE: the chosen rows of faces, which face_scores scores and face_selection in image_scores, image_scores_dir and video_scores accepts, or NULL if you click Cancel. Otherwise, invisibly, the page: a list with its url, its folder dir, and stop(), which stops the server. With live = FALSE, the path of the page's index.html, invisibly. With inline = TRUE, invisibly, the path of the page as one self-contained file, faces.html in dir; in select mode the chosen faces come from the copied line of code.

See Also

draw_faces for the same boxes as plots in R, select_faces to choose faces by rule, number or person.

Examples

## Not run: 
images <- system.file("extdata", c("two-people.jpg", "boris-2.png", "trump2.jpg"),
                      package = "transforEmotion")
faces <- detect_faces(images)
view_faces(faces)

# Click the faces to keep, then Send to R
chosen <- view_faces(faces, select = TRUE)
face_scores(chosen, classes = c("happy", "sad", "angry", "neutral"))

# Without a server: a folder to copy, and a file to read back
view_faces(faces, select = TRUE, live = FALSE, dir = "faces-page")
chosen <- read_face_selection("~/Downloads/selection-faces-20261011-101500.json", faces)

# In a notebook (Colab): the page appears below the cell. Click the faces,
# then Copy R code, and run the copied line in a new cell, for example
view_faces(faces, select = TRUE)
chosen <- read_face_selection("te-selection:eyJzY2hlbWEiOi...", faces)

## End(Not run)