Media metadata as tibbles

library(tidymedia)

tidymedia reads media metadata into tibbles. So the metadata of a whole folder becomes a data frame that you can filter, join and summarize.

Two programs read the metadata:

You need the program installed to use its functions. The README shows how to install them. The examples use the sample clip that comes with the package:

video <- system.file("extdata", "sample.mp4", package = "tidymedia")

Which reader?

The readers differ in the program they use and in what they return. Choose by what you need back:

Functions Program Returns Use it when
probe_all(), probe_container(), probe_streams(), probe_video(), probe_audio() FFprobe tibbles, with rows for the file and for each stream you want the file and stream facts as a data frame
mediainfo_query(), mediainfo_template() MediaInfo a tibble with one row per file you want MediaInfo’s larger set of fields as a data frame
mediainfo_parameter() MediaInfo one value per file you want one MediaInfo field for several files
get_duration(), get_frame_rate(), get_width(), get_height(), get_sample_rate() MediaInfo one number per file you want one common field without naming a MediaInfo section

Some facts, such as the width of the picture, come from both probe_video() and get_width(). Then choose by the shape you want back and the program you have.

Probing with FFprobe

probe_all() returns a list of two tibbles. container has one row for each file, and streams has one row for each stream. Both start with a file column, so the results for several files stack into one table.

info <- probe_all(video)
info$container
#> # A tibble: 1 × 16
#>   file              filename nb_streams nb_programs nb_stream_groups format_name
#>   <chr>             <chr>         <int>       <int>            <int> <chr>      
#> 1 /private/var/fol… /privat…          2           0                0 mov,mp4,m4…
#> # ℹ 10 more variables: format_long_name <chr>, start_time <dbl>,
#> #   duration <dbl>, size <int>, bit_rate <int>, probe_score <int>,
#> #   `TAG:major_brand` <chr>, `TAG:minor_version` <int>,
#> #   `TAG:compatible_brands` <chr>, `TAG:encoder` <chr>
info$streams
#> # A tibble: 2 × 69
#>   file      index codec_name codec_long_name profile codec_type codec_tag_string
#>   <chr>     <int> <chr>      <chr>           <chr>   <chr>      <chr>           
#> 1 /private…     0 h264       H.264 / AVC / … High    video      avc1            
#> 2 /private…     1 aac        AAC (Advanced … LC      audio      mp4a            
#> # ℹ 62 more variables: codec_tag <chr>, mime_codec_string <chr>, width <int>,
#> #   height <int>, coded_width <int>, coded_height <int>, has_b_frames <int>,
#> #   sample_aspect_ratio <chr>, display_aspect_ratio <chr>, pix_fmt <chr>,
#> #   level <int>, color_range <chr>, color_space <chr>, color_transfer <chr>,
#> #   color_primaries <chr>, chroma_location <chr>, field_order <chr>,
#> #   is_avc <chr>, nal_length_size <int>, id <chr>, r_frame_rate <chr>,
#> #   avg_frame_rate <chr>, time_base <chr>, start_pts <int>, start_time <dbl>, …

The other probe_*() functions return one part of that result. You can give them the result of probe_all(), so FFprobe does not read the file again. Or you can give them a file with infile:

# Use the probe result, so the file is not read again
probe_video(info)
#> # A tibble: 1 × 69
#>   file      index codec_name codec_long_name profile codec_type codec_tag_string
#>   <chr>     <int> <chr>      <chr>           <chr>   <chr>      <chr>           
#> 1 /private…     0 h264       H.264 / AVC / … High    video      avc1            
#> # ℹ 62 more variables: codec_tag <chr>, mime_codec_string <chr>, width <int>,
#> #   height <int>, coded_width <int>, coded_height <int>, has_b_frames <int>,
#> #   sample_aspect_ratio <chr>, display_aspect_ratio <chr>, pix_fmt <chr>,
#> #   level <int>, color_range <chr>, color_space <chr>, color_transfer <chr>,
#> #   color_primaries <chr>, chroma_location <chr>, field_order <chr>,
#> #   is_avc <chr>, nal_length_size <int>, id <chr>, r_frame_rate <chr>,
#> #   avg_frame_rate <chr>, time_base <chr>, start_pts <int>, start_time <dbl>, …

By default, typed = TRUE gives number columns a number type. With typed = FALSE, every column is a string. FFprobe reports a frame rate as a fraction such as "30000/1001". The fraction stays a string, even with typed = TRUE.

Querying with MediaInfo

MediaInfo groups its fields in sections, such as General, Video and Audio. mediainfo_query() reads several fields from one section into a tibble:

mediainfo_query(
  video,
  section = "Video",
  parameters = c("Width", "Height", "FrameRate")
)
#> # A tibble: 1 × 4
#>   file                                                    Width Height FrameRate
#>   <chr>                                                   <int>  <int>     <dbl>
#> 1 /private/var/folders/px/frfvbz4n0sx90__c62fwwzz40000gn…   320    240        15

mediainfo_template() reads a whole set of fields at once. The package has two templates, "brief" and "extended":

mediainfo_template(video, template = "brief")
#> # A tibble: 1 × 12
#>   file           complete_name format file_size duration width height frame_rate
#>   <chr>          <chr>         <chr>      <int>    <int> <int>  <int>      <dbl>
#> 1 /private/var/… /private/var… MPEG-4     17725     1000   320    240         15
#> # ℹ 4 more variables: video_bit_rate <int>, channels <int>,
#> #   sampling_rate <int>, audio_bit_rate <int>

For one value, use the get_*() functions:

get_duration(video, unit = "sec")
#> [1] 1
get_width(video)
#> [1] 320
get_height(video)
#> [1] 240

Batching over many files

Each reader takes a vector of files, so you do not need a loop to read a whole folder. The probe_*(), mediainfo_query() and mediainfo_template() functions mark each row with its file. The get_*() functions return one value per file, in the order given.

ffm_jobs() lists the video files in a folder, in all the formats it knows. To list only one format, add extension = "mp4". If the folder has no such files, ffm_jobs() stops with an error:

files <- ffm_jobs("my/videos", type = "video")$input
probe_all(files)$container

A file that cannot be read gives a row of NA values and a warning. The other files are still read.

For a large folder, add parallel = TRUE. The files are then read in parallel with furrr. Each probe_*() function takes this argument. On the functions other than probe_all(), it has an effect only when you pass infile.

probe_all(files, parallel = TRUE)$container

The files are read in parallel only if you set a future plan. With no plan, they are read one at a time, and R gives a warning that says so. vignette("batch") shows how to set a plan.

Where to next