Using checkpoint for reproducible research

Andrie de Vries

1 The Reproducible R Toolkit (RRT)

The Reproducible R Toolkit provides tools to ensure the results of R code are repeatable over time, by anyone. Most R scripts rely on packages, but new versions of packages are released daily. To ensure that results from R are reproducible, it’s important to run R scripts using exactly the same package version in use when the script was written.

The Reproducible R Toolkit provides an R function checkpoint, which ensures that all of the necessary R packages are installed with the correct version. This makes it easy to reproduce your results at a later date or on another system, and makes it easier to share your code with the confidence that others will get the same results you did.

The Reproducible R Toolkit also works in conjunction with the “checkpoint-server”, which makes a daily copy of all CRAN packages, to guarantee that every package version is available to all R developers thereby ensuring reproducibility.

1.1 Components of RRT

RRT is a collection of R packages and the checkpoint server that together make your work with R packages more reproducible over time by anyone.

1.1.1 The checkpoint server

To achieve reproducibility, daily snapshots of CRAN are stored on our checkpoint server. At midnight UTC each day, we refresh our mirror of CRAN is refreshed. When the rsync process is complete, the checkpoint server takes and stores a snapshot of the CRAN mirror as it was at that very moment. These daily snapshots can then be accessed on the MRAN website or using the checkpoint package, which installs and consistently use these packages just as they existed at midnight UTC on a specified snapshot date. Daily snapshots are available as far back as 2014-09-17. For more information, visit the checkpoint server GitHub site.

checkpoint server

1.1.2 The checkpoint package

The goal of the checkpoint package is to solve the problem of package reproducibility in R. Since packages get updated on CRAN all the time, it can be difficult to recreate an environment where all your packages are consistent with some earlier state. To solve this issue, checkpoint allows you to install packages locally as they existed on a specific date from the corresponding snapshot (stored on the checkpoint server) and it configures your R session to use only these packages. Together, the checkpoint package and the checkpoint server act as a CRAN time machine so that anyone using checkpoint() can ensure the reproducibility of their scripts or projects at any time.

checkpoint package

2 Using checkpoint

Using checkpoint is simple:

2.1 Using the checkpoint function

When you create a checkpoint, the checkpoint() function:

This means the remainder of your script will run with the packages from a specific date.

2.2 Sharing your scripts for reproducibility

Sharing your script to be reproducible is as easy as:

Then send this script to your collaborators. When they run this script on their machine, checkpoint will perform the same steps of installing the necessary packages, creating the checkpoint snapshot folder and producing the same results.

2.3 Resetting the checkpoint

To reset the checkpoint, simply restart your R session.

2.4 Worked example

To create a checkpoint project, you do:

  1. Create a new folder and change your working directory to this folder. If you use an IDE like RStudio, this is identical to creating a new RStudio project.

  2. Add your R script files to this folder.

  3. Add a checkpoint to the top of the script:

    library(checkpoint)
checkpoint("<checkpoint date>")

  4. Run the script.

2.4.1 Create a checkpoint project

For example, your script may look like this:

library(checkpoint)
checkpoint("2017-04-01")

# Example from ?darts
library(darts)
x = c(12,16,19,3,17,1,25,19,17,50,18,1,3,17,2,2,13,18,16,2,25,5,5,
      1,5,4,17,25,25,50,3,7,17,17,3,3,3,7,11,10,25,1,19,15,4,1,5,12,17,16,
      50,20,20,20,25,50,2,17,3,20,20,20,5,1,18,15,2,3,25,12,9,3,3,19,16,20,
      5,5,1,4,15,16,5,20,16,2,25,6,12,25,11,25,7,2,5,19,17,17,2,12)
mod = simpleEM(x, niter=100)
e = simpleExpScores(mod$s.final)
oldpar <- par(mfrow=c(1, 2))
drawHeatmap(e)
drawBoard(new=TRUE)
drawAimSpot(e, cex = 5)
par(oldpar)

2.4.2 Run the checkpoint code

Next you want to run the script. Here is what checkpoint does:

## Create a folder to contain the checkpoint
## This is optional - the default is to use ~/.checkpoint

dir.create(file.path(tempdir(), ".checkpoint"), recursive = TRUE, showWarnings = FALSE)

## Create a checkpoint by specifying a snapshot date

library(checkpoint)
checkpoint("2017-04-01", project = example_project, 
           checkpointLocation = tempdir())
## Scanning for packages used in this project
## - Discovered 2 packages
## Installing packages used in this project
##  - Installing 'darts'
## darts
## Package which is only available in source form, and may need
##   compilation of C/C++/Fortran: 'darts'
## installing the source package 'darts'
## checkpoint process complete
## ---

2.4.3 Inspecting the results

Now inspect the results. First, check that your CRAN mirror is set to MRAN snapshot:

getOption("repos")
## [1] "https://mran.microsoft.com/snapshot/2017-04-01"

Next, check that the library path is set to ~/.checkpoint:

normalizePath(.libPaths(), winslash = "/")
## [1] "C:/Users/hongo/AppData/Local/Temp/RtmpEPckx0/.checkpoint/2017-04-01/lib/x86_64-w64-mingw32/3.6.1"
## [2] "C:/Users/hongo/AppData/Local/Temp/RtmpEPckx0/.checkpoint/R-3.6.1"                                
## [3] "C:/Program Files/R/R-3.6.1/library"

Finally, check which packages are installed in checkpoint library:

installed.packages(.libPaths()[1])[, "Package"]
## [1] "darts"