Generate Rust tests from data files

Sometimes you just have a bunch of example data laying around and you want to make sure your code works with all of them. Some of them are probably short and sweet and could live happily as doctests, which are amazing btw. But some of them are more awkward to present in such form, because, for example, of their size or number. Typically when you have an example of how the program should behave you write an example-based unit test. Ideally, each of them would represent an isolated example and they should fail independently. But, converting your source data files into a unit test one by one, manually, can be a bit tedious.

Rust build scripts to the rescue !

What if you could could just iterate over the data files you have already and then produce unit tests accordingly ? What follows is an example of such, where we iterate over directories and generate one unit test per each, assuming all of them contain files named according to our convention.

I chose to generate integration tests here, but you can generate pretty much any code using this technique.

tests/test_loader.rs

// include tests generated by `build.rs`, one test per directory in tests/data
include!(concat!(env!("OUT_DIR"), "/tests.rs"));

build.rs

use std::env;
use std::fs::read_dir;
use std::fs::DirEntry;
use std::fs::File;
use std::io::Write;
use std::path::Path;

// build script's entry point
fn main() {
    let out_dir = env::var("OUT_DIR").unwrap();
    let destination = Path::new(&out_dir).join("tests.rs");
    let mut test_file = File::create(&destination).unwrap();

    // write test file header, put `use`, `const` etc there
    write_header(&mut test_file);

    let test_data_directories = read_dir("./tests/data/").unwrap();

    for directory in test_data_directories {
        write_test(&mut test_file, &directory.unwrap());
    }
}

fn write_test(test_file: &mut File, directory: &DirEntry) {
    let directory = directory.path().canonicalize().unwrap();
    let path = directory.display();
    let test_name = format!(
            "prefix_if_needed_{}",
            directory.file_name().unwrap().to_string_lossy()
        );

    write!(
        test_file,
        include_str!("./tests/test_template"),
        name = test_name,
        path = path
    )
    .unwrap();
}

fn write_header(test_file: &mut File) {
    write!(
        test_file,
        r#"
use crate_under_test::functionality_under_test;
"#
    )
    .unwrap();
}

tests/test-template

#[test]
fn {name}() {{
    let input = include_str!("{path}/input-data");
    let expected_output = include_str!("{path}/output-data");

    let actual_output = functionality_under_test(input);

    assert_eq!(expected_output, actual_output);
}}

So to recap - first the build.rs script creates $OUT_DIR/tests.rs file containing all the generated tests code. The compiler does not know there are tests to launch using normal integration tests procedure there though, so then we use tests/test_loader.rs to tell it so, basically including the generated Rust code into that file. After the compilation proceeds normally, giving us one unit test per directory, giving us ability to pinpoint test cases that are problematic more precisely.

You can then further improve on that, e.g. add more directory structure, split tests into modules etc - you can generate any Rust code this way.

Happy hacking !

p.s. there are more Rust testing tricks and let me know if you’d like to pair program with me on anything !