silk.testing.selection.

How the built-in test runner skips a passing test whose code and inputs did not change.

When to use#

silk.testing.runner uses this module. You do not call it from normal code.

Details#

A test runner that silk test builds for an ELF or Mach-O target holds a table of every function in it: for each one a flag, which the function sets when it starts, and the digest of everything the compiler read to make it. Table reads that table. Before a test runs, the runner clears the flags; after it, the set flags are the functions the test executed.

The runner also watches the test (silk.testing.observation): the files, directory listings and environment variables it read, and whether it used the clock, randomness, the network or a child process.

A record file keeps, for each test that passed, the digests of the functions it executed and a digest of each input it read, taken after it ran. A later run skips the test when every one of those functions is in the new runner with the same digest and every input reads the same. A test that executes the same functions on the same inputs does the same thing, so it passes again.

A test is not recorded, so it runs every time, when it failed, when it used the clock, randomness, the network or a child process, listed the environment, read more than the watch holds, or executed a function the compiler gave no digest, as in a build without a cache.

The record starts with the magic SLKTREC1 and a version, then the runner's table: its count and each function's 16-byte digest. Then the working directory, and for each recorded test its identity, the table positions of its functions as increasing LEB128 differences, and its inputs: kind, root, path and a 32-byte SHA-256. Integers are little-endian u64; text is a u64 length and its bytes.

Gotchas#

  • Work a process does once, such as filling a cache on first use, is recorded only for the test that did it. A later test that relies on it can be skipped after an edit to that work.

  • A test that reads the host other than through the native providers reads unseen.

Import as TestSelection with import silk.testing.selection as TestSelection.

Public declarations: 8.

Table#

silk
pub struct Table

The table of the functions in this runner: one record per function, each a flag and a digest.

Implementation Copy for Table#

silk
impl Copy for Table

Input#

silk
pub struct Input

One input a test read, with the digest of what it read.

Field kind#

silk
pub kind: u8

The note kind, such as Observation.FILE.

Field root#

silk
pub root: Bytes

The root of the provider it was read through; empty for an environment variable.

Field path#

silk
pub path: Bytes

The path it read, or the variable's name.

Field digest#

silk
pub digest: [u8; 32]

The SHA-256 of what the input reads now, by digestInput.

Entry#

silk
pub struct Entry

What one passing test executed and read.

Field identity#

silk
pub identity: Bytes

The test's identity.

Field functions#

silk
pub functions: List<u32>

The table positions of the functions it executed, increasing.

Field inputs#

silk
pub inputs: List<Input>

The inputs it read, each once.

Previous#

silk
pub struct Previous

The entries a record file holds, with each function of the runner that wrote it mapped to the position of the same function in this runner's table.

Implementation Table#

silk
impl Table

Associated function between#

silk
pub unsafe fn between(start: ?*const u8, stop: ?*const u8) -> Table

The table that lies between start and stop, the bounds of the section that holds the records of a test runner's functions. It is empty when either is null or stop is not after start.

Details#

A test runner that silk test builds for an ELF or Mach-O target holds the records in the section silk_seen (__DATA,__silk_seen on Mach-O), whose bounds the linker names __start_silk_seen and __stop_silk_seen. Each record is 17 bytes: a flag the function sets when it starts, then its 16-byte content digest.

Safety#

The bytes between the bounds must be such records, writable, and stay valid while the table is used.

Associated function empty#

silk
pub fn empty() -> Table

The table of no functions: every test runs and none is recorded.

Method length#

silk
pub fn length(self: &Self) -> usize

The number of functions in the table.

Method clear#

silk
pub fn clear(self: &Self) -> ()

Clears every function's flag, so the flags set next are those of the code that runs next.

Method collect#

silk
pub effect fn collect(self: &Self, into: &mut List<u32>) -> bool ! OutOfMemoryError ? &mut Allocator

Appends to into the position of each function whose flag is set, in order, and clears the flags. Returns false when one of them has no digest.

Implementation Previous#

silk
impl Previous

Associated function none#

silk
pub fn none() -> Previous

No entries: every test runs.

Associated function read#

silk
pub effect fn read(bytes: &[u8], table: &Table, working: &[u8]) -> Previous ! OutOfMemoryError ? &mut Allocator

Reads the record bytes for this runner's table, run in the working directory working. A record that is not valid, or that was written in another working directory, holds no entries.

Method find#

silk
pub effect fn find(self: &mut Self, identity: &[u8]) -> Option<Entry> ! OutOfMemoryError ? &mut Allocator

The entry of the test identity with its functions moved to this runner's table, when every function it executed is still in the table with the same digest. Its inputs are not checked; unchanged checks them.

unchanged#

silk
pub effect fn unchanged(entry: &Entry) -> bool ! OutOfMemoryError ? &mut Allocator | &mut HostInput

Whether every input of entry reads now as it did when it was recorded.

digestInput#

silk
pub effect fn digestInput(kind: u8, root: &[u8], path: &[u8]) -> [u8; 32] ! OutOfMemoryError ? &mut Allocator | &mut HostInput

The SHA-256 of what the input of kind at path, through a provider rooted at root, reads now: a file's contents, an entry's kind and size, a directory's names, or a variable's value, each after a word for its kind, or a word for a failure.

inputs#

silk
pub effect fn inputs(notes: &[u8]) -> Option<List<Input>> ! OutOfMemoryError ? &mut Allocator | &mut HostInput

The inputs the notes of a watch buffer notes name, each once, with their digests now, or None when the test cannot be recorded.

Details#

Notes under a directory the test created with a unique name (Observation.TEMPORARY) are the test's own and name no input. A change to any other path (Observation.WRITE) means a later run would start from what the test left, so the result is None. So is a buffer whose notes do not parse.

record#

silk
pub effect fn record(table: &Table, working: &[u8], entries: &[Entry]) -> Bytes ! OutOfMemoryError ? &mut Allocator

The record file of entries for this runner's table, run in the working directory working.

Category: Testing.

Profiles: aarch64-apple-darwin, aarch64-unknown-linux-gnu, aarch64-unknown-linux-gnu-no-libc, x86_64-unknown-linux-gnu, x86_64-unknown-linux-gnu-no-libc, wasm32-unknown-unknown, wasm32-wasip1, wasm32-wasip2, x86_64-pc-windows-msvc, wasm64-unknown-unknown.

View the source of this page