BIDistances: Distance computation, theory-guided selection, and scalable weighted Minkowski distances in R

Overview

BIDistances provides a common interface for calculating, comparing, and examining distances, dissimilarities, divergences, and selected similarities. The package connects four tasks that are usually treated separately:

1. Access a broad collection of distance methods through DistanceMatrix().
2. Check whether a method is a metric, pseudometric, non-metric dissimilarity, or divergence with DistanceProperties().
3. Select a candidate for distance-based cluster analysis by examining distance distributions with DistanceDistributions().
4. Scale weighted Minkowski calculations from multicore CPUs to optional OpenCL devices while retaining an optimized weighted Euclidean route for p = 2.

Additional domain-specific functionality includes a Gene Ontology-derived TF-IDF distance, time-series distances, nearest-neighbour distances, toroidal Euclidean distances, and mixed-data constructions.

Main advantages

• A unified DistanceMatrix() interface for native BIDistances methods and all predefined methods supplied by parallelDist, with optional access to philentropy and manydist.
• 60 canonical named numerical-data routes in the core installation and 94 when philentropy is installed. This count excludes aliases, user-defined C++ functions, direct-only functions, and the configurable mixed-data constructions in manydist.
• Weighted Minkowski distance calculation through parallelDist, an internal RcppParallel multicore backend, or optional OpenCL kernels.
• A dedicated weighted Euclidean GPU implementation retained as the optimized p = 2 OpenCL route.
• Memory-aware full-matrix, output-batched, and input-blocked OpenCL execution.
• DistanceDistributions() for comparing candidate distance distributions and identifying multimodality under the goal of finding distance-based cluster structures. Distance vectors above 72,000 values use the integrated batch dip-test backend, while smaller vectors retain diptest::dip.test().
• Tfidf_Distance() for deriving distances between genes from a gene-by-Gene-Ontology-term feature matrix.
• DistanceProperties() for documenting the mathematical properties and conditions of native methods.

Method coverage in release 1.0.0

manydist adds preset-based and custom mixed-type distance constructions. These are not included in the total because their specifications are configurable rather than a fixed list of independent methods. Direct-only functions such as TWED_Distance() and ToroidalEuclidean_Distance() are also outside the dispatcher count.

Installation

From CRAN:

install.packages("BIDistances")

From GitHub:

install.packages("remotes")
remotes::install_github("Mthrun/BIDistances")

System requirements are R >= 3.5.0, GNU make, and a suitable C++ toolchain. An OpenCL library and the R package OpenCL are optional and are required only for GPU acceleration. Pandoc is required when rebuilding the vignette.

Quick start

Example 1: Several distances through one interface

library(BIDistances)

data("Hepta")
Data = Hepta$Data

D_euclidean = DistanceMatrix(
  Data,
  method = "euclidean"
)

D_minkowski = DistanceMatrix(
  Data,
  method = "minkowski",
  p = 3
)

D_cosine = DistanceMatrix(
  Data,
  method = "Cosine_Distance"
)

D_euclidean[1:5, 1:5]

DistanceMatrix() first checks for a native BIDistances route. Other method names are evaluated by parallelDist; methods unavailable there can be obtained from philentropy when it is installed. Mixed numerical and categorical data can be handled through manydist.

Example 2: Weighted Minkowski distances on three backends

For finite p >= 1, Minkowski_Distance() computes

\[ d_{p,w}(x,y) = \left(\sum_k w_k |x_k-y_k|^p\right)^{1/p}. \]

Data = as.matrix(iris[1:30, 1:4])
Weights = c(1, 2, 0.5, 1)

D_parallel = Minkowski_Distance(
  Data = Data,
  p = 3,
  Weights = Weights,
  backend = "parallelDist",
  threads = 2
)

D_multicore = Minkowski_Distance(
  Data = Data,
  p = 3,
  Weights = Weights,
  backend = "multicore",
  threads = 2
)

max(abs(D_parallel - D_multicore))

The optional OpenCL route uses separate kernels for general p and delegates p = 2 to the established weighted Euclidean GPU implementation:

# Requires an installed OpenCL implementation and a usable device.
D_opencl = Minkowski_Distance(
  Data = Data,
  p = 3,
  Weights = Weights,
  backend = "opencl",
  Mem = 2
)

Use backend = "auto" to try OpenCL and fall back to the internal multicore backend with a warning. For p = Inf, unit weights are required and the maximum or Chebyshev distance is returned. Values 0 < p < 1 are available through Fractional_Distance() but are generally not metrics.

The same weighted route is available through DistanceMatrix():

D_dispatch = DistanceMatrix(
  Data,
  method = "Minkowski_Distance",
  p = 3,
  Weights = Weights,
  backend = "multicore",
  threads = 2
)

The memory planner can be inspected before a large OpenCL calculation:

MemoryPlan = calculateMemoryDemandGPU(
  n = 70000,
  d = 784,
  mem = 6
)
MemoryPlan

The Euclidean-related entry points have deliberately different roles:

Example 3: Theory-guided distance selection for clustering

library(BIDistances)

data("Hepta")
set.seed(42)

Selection = DistanceDistributions(
  Data = Hepta$Data,
  DistanceMethods = c(
    "euclidean",
    "manhattan",
    "minkowski",
    "chord"
  ),
  CosineNonParallel = TRUE,
  CorrelationDist = TRUE,
  PlotIt = FALSE
)

Selection$DistanceChoice
Selection$SelectionStatistics[, c(
  "Distance", "DipStatistic", "DipPValue", "BimodalityAmplitude", "Status"
)]
if (!is.null(Selection$DistanceMatrix)) {
  Selection$DistanceMatrix[1:5, 1:5]
}
Selection$ggobject

The workflow ranks candidate distributions using multimodality-related information under the explicit goal of finding distance-based cluster structures. SelectionStatistics reports both the Hartigan dip statistic and its p-value. For at most 72,000 pairwise values the established diptest::dip.test() path is retained; larger candidate columns are processed together by the integrated C++ batch with explicit asymptotic calibration. The result is exploratory: multimodality supports this structural goal, but it is not a universal proof of cluster validity or biological relevance.

Example 4: Gene Ontology-derived TF-IDF distances

library(BIDistances)

data("Hearingloss_N109")

Result = Tfidf_Distance(
  Hearingloss_N109$FeatureMatrix_Gene2Term,
  tf_fun = mean
)

Distance = Result$Distance
TfidfWeights = Result$TfidfWeights

Distance[1:5, 1:5]
TfidfWeights[1:5]

Rows represent genes and columns represent GO terms. The function returns pairwise absolute differences between scalar TF-IDF weights together with those weights. Because different annotation rows can receive the same scalar weight, the construction is a pseudometric on the original rows.

Example 5: Column-wise time-series distances

library(BIDistances)

TimeSeries = cbind(
  SeriesA = c(0, 1, 2, 1, 0),
  SeriesB = c(0, 1, 1.5, 1, 0),
  SeriesC = c(2, 1, 0, 1, 2)
)

D_msmd = DistanceMatrix(
  TimeSeries,
  method = "MSMD_Distance",
  ParameterC = 1
)

D_msmd

For MSMD_Distance and DTW_Distance, every column is one time series and row order is the temporal order. No separate time variable should be supplied. TWED_Distance() requires explicit timestamps and is therefore called directly rather than through DistanceMatrix().

Concepts

Data and Distance conventions

• Data: [1:n,1:d] numerical matrix with n cases, d variables
• Distance: [1:d,1:d] numerical matrix of distances

Most methods compare the rows of Data. The documented exceptions are:

• DTW_Distance and MSMD_Distance, which compare time series stored in columns.
• EndresSchindelin_Distance, which compares columns as samples of variables.
• TWED_Distance, which compares two time series directly and requires explicit time vectors.

Mathematical terminology

The word distance is used as a broad software convention. Not every returned quantity is a mathematical metric. The complete machine-readable classification is available through:

Properties = DistanceProperties()
Properties[, c(
  "Function",
  "Classification",
  "Conditions"
)]

Examples include:

• Ordinary Euclidean distance is a metric.
• Weighted Minkowski distance is a metric for p >= 1 with strictly positive weights and can become a pseudometric when zero weights remove distinguishing coordinates.
• Fractional_Distance() with 0 < p < 1 generally violates the triangle inequality.
• Mahalanobis_Distance() returns squared Mahalanobis dissimilarities; applying sqrt() yields the metric form.
• Cosine_Distance() returns 1 - cosine similarity and is not a metric in general.
• Tfidf_Distance() is a pseudometric on the original gene-annotation rows.
• Kullback_Leibler_div() returns a divergence, including when symmetrized.

See ?distance-conventions and ?DistanceProperties for details. The registry documents mathematical conditions; it is not a runtime proof for arbitrary user-supplied data.

Native DistanceMatrix() routes

Canonical package methods available through DistanceMatrix() are:

• Cosine_Distance
• EndresSchindelin_Distance
• Fractional_Distance
• Gini_Distance
• Jaccard_Distance
• Mahalanobis_Distance
• Minkowski_Distance
• SharedNeighbor_Distance
• Tfidf_Distance
• Wasserstein_Distance
• DTW_Distance
• MSMD_Distance

TWED_Distance() is intentionally excluded from the dispatcher because its explicit time vectors cannot be inferred from the standard matrix interface.

Optional functionality

Some functions require suggested packages only when they are used:

• OpenCL for the GPU backend.
• philentropy for additional probability-distribution methods.
• manydist for mixed numerical and categorical data.
• dtw and proxy for DTW_Distance().
• ineq for Gini_Distance().
• transport for Wasserstein_Distance().
• ScatterDensity for sample-based Endres-Schindelin calculations.

Manual and citation

The full manual is available at CRAN.

The package vignette can be opened from R with:

vignette("BIDistances")

Citation information is available with:

citation("BIDistances")

References

[Thrun, 2021] Thrun, M. C.: The Exploitation of Distance Distributions for Clustering, International Journal of Computational Intelligence and Applications, Vol. 20(3), 2150016, doi:10.1142/S1469026821500164, 2021.

[Stier & Thrun, 2023] Stier, Q., & Thrun, M. C.: Deriving Homogeneous Subsets from Gene Sets by Exploiting the Gene Ontology, Informatica, Vol. 34(2), pp. 357-386, doi:10.15388/23-INFOR517, 2023.