Run an R file containing tests; gather results

Usage

run_test_file(
  file,
  at_home = TRUE,
  verbose = getOption("tt.verbose", 2),
  color = getOption("tt.pr.color", TRUE),
  remove_side_effects = TRUE,
  side_effects = FALSE,
  set_env = list(),
  encoding = "unknown",
  ...
)

Arguments

file: [character] File location of a .R file.
at_home: [logical] toggle local tests.
verbose: [integer] verbosity level. 0: be quiet, 1: print status per file, 2: print status and increase counter after each test expression.
color: [logical] toggle colorize counts in verbose mode (see Note)
remove_side_effects: [logical] toggle remove user-defined side effects? See section on side effects.
side_effects: [logical|list] Either a logical, or a list of arguments to pass to report_side_effects.
set_env: [named list]. Key=value pairs of environment variables that will be set before the test file is run and reset afterwards. These are not counted as side effects of the code under scrutiny.
encoding: [character] Define encoding argument passed to parse when parsing file.
...: Currently unused

Value

A list of class tinytests, which is a list of tinytest objects.

Details

In tinytest, a test file is just an R script where some or all of the statements express an expectation. run_test_file runs the file while gathering results of the expectations in a tinytests object.

The graphics device is set to pdf(file=tempfile()) for the run of the test file.

Note

Not all terminals support ansi escape characters, so colorized output can be switched off. This can also be done globally by setting options(tt.pr.color=FALSE). Some terminals that do support ansi escape characters may contain bugs. An example is the RStudio terminal (RStudio 1.1) running on Ubuntu 16.04 (and possibly other OSs).

Side-effects caused by test code

All calls to Sys.setenv and options defined in a test file are captured and undone once the test file has run, if remove_side_effects is set to TRUE.

Tracking side effects

Certain side effects can be tracked, even when they are not explicitly evoked in the test file. See report_side_effects for side effects tracked by tinytest. Calls to report_side_effects within the test file overrule settings provided with this function.

Examples

# create a test file, in temp directory
tests <- "
addOne <- function(x) x + 2

Sys.setenv(lolz=2)

expect_true(addOne(0) > 0)
expect_equal(2, addOne(1))

Sys.unsetenv('lolz')
"
testfile <- tempfile(pattern="test_", fileext=".R")
write(tests, testfile)

# run test file
out <- run_test_file(testfile,color=FALSE)
#> 
test_35396019b4776d.R.........    0 tests    
test_35396019b4776d.R.........    0 tests    
test_35396019b4776d.R.........    1 tests OK 
test_35396019b4776d.R.........    2 tests 1 fails 
test_35396019b4776d.R.........    2 tests 1 fails 1ms
out
#> ----- FAILED[data]: test_35396019b4776d.R<7--7>
#>  call| expect_equal(2, addOne(1))
#>  diff| Expected '3', got '2'
#>  
#> Showing 1 out of 2 results: 1 fails, 1 passes (1ms)
# print everything in short format, include passes in print.
print(out, nlong=0, passes=TRUE)
#> PASSED      : test_35396019b4776d.R<6--6> expect_true(addOne(0) > 0)
#>  FAILED[data]: test_35396019b4776d.R<7--7> expect_equal(2, addOne(1))
#>  

# run test file, track supported side-effects
run_test_file(testfile, side_effects=TRUE)
#> 
test_35396019b4776d.R.........    0 tests    
test_35396019b4776d.R.........    0 tests    1 side-effects 
test_35396019b4776d.R.........    1 tests OK 1 side-effects 
test_35396019b4776d.R.........    2 tests 1 fails 1 side-effects 
test_35396019b4776d.R.........    2 tests 1 fails 2 side-effects 11ms
#> ----- SIDEFX[envv]: test_35396019b4776d.R<4--4>
#>  call| Sys.setenv(lolz = 2)
#>  diff| Changed envvar 'lolz' from '' to '2'
#>  ----- FAILED[data]: test_35396019b4776d.R<7--7>
#>  call| expect_equal(2, addOne(1))
#>  diff| Expected '3', got '2'
#>  ----- SIDEFX[envv]: test_35396019b4776d.R<9--9>
#>  call| Sys.unsetenv("lolz")
#>  diff| Removed envvar 'lolz' with value '2'
#>  
#> Showing 3 out of 4 results: 1 fails, 1 passes, 2 side effects (11ms)

# run test file, track only changes in working directory 
run_test_file(testfile, side_effects=list(pwd=TRUE, envvar=FALSE))
#> 
test_35396019b4776d.R.........    0 tests    
test_35396019b4776d.R.........    0 tests    
test_35396019b4776d.R.........    1 tests OK 
test_35396019b4776d.R.........    2 tests 1 fails 
test_35396019b4776d.R.........    2 tests 1 fails 5ms
#> ----- FAILED[data]: test_35396019b4776d.R<7--7>
#>  call| expect_equal(2, addOne(1))
#>  diff| Expected '3', got '2'
#>  
#> Showing 1 out of 2 results: 1 fails, 1 passes (5ms)