silk.testing.selection.
How the built-in test runner skips a passing test whose code and inputs did not change.
When to use#
silk.testing.runner uses this module. You do not call it from normal code.
Details#
A test runner that silk test builds for an ELF or Mach-O target holds a table of every
function in it: for each one a flag, which the function sets when it starts, and the digest of
everything the compiler read to make it. Table reads that table. Before a test runs, the
runner clears the flags; after it, the set flags are the functions the test executed.
The runner also watches the test (silk.testing.observation): the files, directory listings and
environment variables it read, and whether it used the clock, randomness, the network or a child
process.
A record file keeps, for each test that passed, the digests of the functions it executed and a digest of each input it read, taken after it ran. A later run skips the test when every one of those functions is in the new runner with the same digest and every input reads the same. A test that executes the same functions on the same inputs does the same thing, so it passes again.
A test is not recorded, so it runs every time, when it failed, when it used the clock, randomness, the network or a child process, listed the environment, read more than the watch holds, or executed a function the compiler gave no digest, as in a build without a cache.
The record starts with the magic SLKTREC1 and a version, then the runner's table: its count
and each function's 16-byte digest. Then the working directory, and for each recorded test its
identity, the table positions of its functions as increasing LEB128 differences, and its inputs:
kind, root, path and a 32-byte SHA-256. Integers are little-endian u64; text is a u64 length
and its bytes.
Gotchas#
Work a process does once, such as filling a cache on first use, is recorded only for the test that did it. A later test that relies on it can be skipped after an edit to that work.
A test that reads the host other than through the native providers reads unseen.
Import as TestSelection with import silk.testing.selection as TestSelection.
Public declarations: 8.
Table#
pub struct TableThe table of the functions in this runner: one record per function, each a flag and a digest.
Implementation Copy for Table#
impl Copy for TableInput#
pub struct InputOne input a test read, with the digest of what it read.
Field kind#
pub kind: u8The note kind, such as Observation.FILE.
Field root#
pub root: BytesThe root of the provider it was read through; empty for an environment variable.
Field path#
pub path: BytesThe path it read, or the variable's name.
Field digest#
pub digest: [u8; 32]The SHA-256 of what the input reads now, by digestInput.
Entry#
pub struct EntryWhat one passing test executed and read.
Field identity#
pub identity: BytesThe test's identity.
Field functions#
pub functions: List<u32>The table positions of the functions it executed, increasing.
Field inputs#
pub inputs: List<Input>The inputs it read, each once.
Previous#
pub struct PreviousThe entries a record file holds, with each function of the runner that wrote it mapped to the position of the same function in this runner's table.
Implementation Table#
impl TableAssociated function between#
pub unsafe fn between(start: ?*const u8, stop: ?*const u8) -> TableThe table that lies between start and stop, the bounds of the section that holds the
records of a test runner's functions. It is empty when either is null or stop is not
after start.
Details#
A test runner that silk test builds for an ELF or Mach-O target holds the records in the
section silk_seen (__DATA,__silk_seen on Mach-O), whose bounds the linker names
__start_silk_seen and __stop_silk_seen. Each record is 17 bytes: a flag the function sets
when it starts, then its 16-byte content digest.
Safety#
The bytes between the bounds must be such records, writable, and stay valid while the table is used.
Associated function empty#
pub fn empty() -> TableThe table of no functions: every test runs and none is recorded.
Method length#
pub fn length(self: &Self) -> usizeThe number of functions in the table.
Method clear#
pub fn clear(self: &Self) -> ()Clears every function's flag, so the flags set next are those of the code that runs next.
Method collect#
pub effect fn collect(self: &Self, into: &mut List<u32>) -> bool ! OutOfMemoryError ? &mut AllocatorAppends to into the position of each function whose flag is set, in order, and clears the
flags. Returns false when one of them has no digest.
Implementation Previous#
impl PreviousAssociated function none#
pub fn none() -> PreviousNo entries: every test runs.
Associated function read#
pub effect fn read(bytes: &[u8], table: &Table, working: &[u8]) -> Previous ! OutOfMemoryError ? &mut AllocatorReads the record bytes for this runner's table, run in the working directory working.
A record that is not valid, or that was written in another working directory, holds no
entries.
Method find#
pub effect fn find(self: &mut Self, identity: &[u8]) -> Option<Entry> ! OutOfMemoryError ? &mut AllocatorThe entry of the test identity with its functions moved to this runner's table, when every
function it executed is still in the table with the same digest. Its inputs are not checked;
unchanged checks them.
unchanged#
pub effect fn unchanged(entry: &Entry) -> bool ! OutOfMemoryError ? &mut Allocator | &mut HostInputWhether every input of entry reads now as it did when it was recorded.
digestInput#
pub effect fn digestInput(kind: u8, root: &[u8], path: &[u8]) -> [u8; 32] ! OutOfMemoryError ? &mut Allocator | &mut HostInputThe SHA-256 of what the input of kind at path, through a provider rooted at root, reads
now: a file's contents, an entry's kind and size, a directory's names, or a variable's value,
each after a word for its kind, or a word for a failure.
inputs#
pub effect fn inputs(notes: &[u8]) -> Option<List<Input>> ! OutOfMemoryError ? &mut Allocator | &mut HostInputThe inputs the notes of a watch buffer notes name, each once, with their digests now, or
None when the test cannot be recorded.
Details#
Notes under a directory the test created with a unique name (Observation.TEMPORARY) are
the test's own and name no input. A change to any other path (Observation.WRITE) means a
later run would start from what the test left, so the result is None. So is a buffer whose
notes do not parse.
record#
pub effect fn record(table: &Table, working: &[u8], entries: &[Entry]) -> Bytes ! OutOfMemoryError ? &mut AllocatorThe record file of entries for this runner's table, run in the working directory working.
Category: Testing.
Profiles: aarch64-apple-darwin, aarch64-unknown-linux-gnu, aarch64-unknown-linux-gnu-no-libc, x86_64-unknown-linux-gnu, x86_64-unknown-linux-gnu-no-libc, wasm32-unknown-unknown, wasm32-wasip1, wasm32-wasip2, x86_64-pc-windows-msvc, wasm64-unknown-unknown.