A figure that carries its own provenance

Backend
plot_manifest() fingerprints a plot’s data, spec, and resolved fonts, plot_svg(manifest = TRUE) embeds that in the file, and plot_verify() checks a figure still matches its data while telling a font difference apart from a data change.

Six months after a report ships, somebody asks whether figure 3 was made from the data currently in the repository. Usually there is no way to answer that except to re-run the script and compare images by eye, and that fails for reasons which have nothing to do with the data, because the machine has a different font stack now.

Since a vellumplot plot is a spec rather than a picture, it can be fingerprinted before it is drawn.

p <- vplot(mtcars, width = 6.5, height = 4) |>
  mark_point(x = wt, y = mpg, color = hp, size = 2.2) |>
  mark_smooth(x = wt, y = mpg, method = "lm", se = TRUE) |>
  scale_color_continuous() |>
  labs(title = "Weight against mileage", x = "weight (1000 lbs)", y = "mpg",
       color = "hp")

plot_manifest() returns that fingerprint: a hash of the input data (sensitive to row order and columns), the data’s shape, a structural hash of the spec, how many elements the plot emits, and the font faces it actually resolved to.

m <- plot_manifest(p)
m$data$hash
#> [1] "33f1aa2876d0ad2a1232bcd25a9a350e"
m$spec_hash
#> [1] "f830124ff2425cd2f2e7b97f00ec96ea"
m$n_elements
#> [1] 24

The fonts are the part that is easy to overlook and the part that makes the check usable:

str(m$fonts, max.level = 2, list.len = 2)
#> 'data.frame':    1 obs. of  3 variables:
#>  $ path  : chr "/System/Library/Fonts/Helvetica.ttc"
#>  $ index : int 0
#>   [list output truncated]

Embedding it in the file

plot_svg(manifest = TRUE) writes the SVG with the manifest inside it, so the figure travels with its own provenance. plot_verify() reads it back and recomputes the data hash:

svg <- plot_svg(p, manifest = TRUE)
check <- plot_verify(svg, mtcars)
check$ok
#> [1] TRUE

Hand it different data and it says so, and says which thing changed:

tampered <- mtcars
tampered$mpg[1] <- tampered$mpg[1] + 1

bad <- plot_verify(svg, tampered)
c(ok = bad$ok, data_ok = bad$data_ok, fonts_ok = bad$fonts_ok)
#>       ok  data_ok fonts_ok 
#>    FALSE    FALSE     TRUE

That split is the point. data_ok is whether the data still hashes the same; fonts_ok is whether every font the figure was drawn with is still on this machine. A pixel difference with data_ok = TRUE and fonts_ok = FALSE is the font stack rather than your analysis: a different cause with a different fix (install the font) from a changed dataset. Reporting both together as “the figure does not match” is what makes reproducibility checks so easy to ignore.

bad$expected
#> [1] "33f1aa2876d0ad2a1232bcd25a9a350e"
bad$actual
#> [1] "00a30c3af8e14eab951e12fffb03eab4"

The scene is a value too

One layer down, the same idea applies to the scene rather than the plot. vellum::scene_hash() and scene_diff() compare two scenes structurally, reporting what changed rather than which pixels. That makes a better regression test than an image diff, because it does not care what the font stack looks like on the machine running the test.

library(vellum)
a <- vl_scene(4, 2, bg = "white") |>
  draw(circle_grob(x = 0.4, y = 0.5, r = 0.2, gp = vl_gpar(fill = "#c9a874"),
                   name = "blob"))
b <- vl_scene(4, 2, bg = "white") |>
  draw(circle_grob(x = 0.4, y = 0.5, r = 0.2, gp = vl_gpar(fill = "#c1121f"),
                   name = "blob"))

scene_hash(a) == scene_hash(b)
#> [1] FALSE
scene_diff(a, b)
#> 1 difference:
#> • ~ root$children[1]$gp$fill: #c9a874 -> #c1121f

Both of these are checks you can put in a test suite and have them mean something in CI, where there is no display, no fonts you chose, and no human to look at the picture. See also One scene, three formats for why the same solved scene is what every backend walks.

Back to top