silk.bench.bencher.
Benchmarks declared with bench fn, the Bencher they loop on, and the program that
runs them.
When to use#
Write a bench fn beside the code it measures when the work is a Silk function: a parser,
a hash, one step of a computation. Compare whole programs, or programs in other languages,
with a suite of silk.bench.suite instead.
Details#
A benchmark is a top-level function marked bench that takes one Bencher. It does its
setup, then loops on Bencher.next, and each turn of the loop is one iteration:
bench fn sumToOneThousand(b: Bencher) {
let mut total: u64 = 0
while b.next() {
total = 0
let mut at: u64 = 0
while at < 1000 {
total = total + at
at = at + 1
}
b.keep<u64>(&total)
}
b.check(total)
}The time that counts runs from the first call of next to the call that returns false,
so what comes before the loop and after it is free. Bencher.stop and Bencher.start
leave a stretch inside the loop out, and Bencher.reset starts the time again.
The benchmarks of a program are the bench functions of the modules its entry file
imports, directly or through other modules of the same package. Benchmarks.main runs
them. A program whose main calls it is a harness as silk.bench.harness describes: run
by hand, it finds an iteration count for each benchmark, takes ten samples and prints
result lines; declared in a suite with silk.bench.suite.Implementation.driven, it takes
its iteration counts from the benchmark task, which compares it with other programs and
with earlier runs.
A benchmark is named after its function. Two benchmarks with one name in different modules
are both named <module>.<name>.
A bench effect fn may fail. A failure ends the run of that benchmark, which is reported as
failed.
Gotchas#
The compiler removes work whose result nothing uses. Give each iteration's result to
Bencher.keep.The compiler also computes ahead of time what it can: work on a constant is done when the program is built. Give
Bencher.keepthe input too, before the work reads it.benchmarks only a top-level function with one parameter, no type parameters and no requirement row. A benchmark that needs a service provides it inside.A benchmark that never calls
nextis timed as a whole, as if its one call were every iteration.A build that does not call
Benchmarks.mainkeeps no benchmark.
Examples#
The entry file of a benchmark program:
import silk.bench.bencher { Benchmarks }
import silk.system.host_input { HostInput }
// The modules that hold the `bench` functions. A name that starts with `_` is not
// reported as unused.
import parser as _parser
import encoder as _encoder
pub effect fn main() -> i32
? &mut HostInput {
return run Benchmarks.main()
}Import as Bencher with import silk.bench.bencher { Bencher }.
Public declarations: 2.
Bencher#
pub struct BencherThe run of a benchmark that is in progress: what a bench fn loops on.
Details#
A Bencher is a handle to a run the benchmark program holds, so a copy of it is the same
run.
Gotchas#
A Bencher is good only inside the call of the benchmark it was given to.
Implementation Copy for Bencher#
impl Copy for BencherImplementation Bencher#
impl BencherMethod next#
pub fn next(self: Self) -> boolReturns whether there is another iteration to do. The first call starts the time, and the
call that returns false stops it.
Method iterations#
pub fn iterations(self: Self) -> u64Returns how many iterations the run has in all.
Method stop#
pub fn stop(self: Self) -> ()Stops the time, until Bencher.start. Stopping twice changes nothing.
Method start#
pub fn start(self: Self) -> ()Starts the time again after Bencher.stop. Starting twice changes nothing.
Method reset#
pub fn reset(self: Self) -> ()Starts the time again from zero, after work inside the loop that should not count.
Method keep#
pub fn keep<T>(self: Self, value: &T) -> ()Makes the compiler treat value as used, so it cannot remove the work that computed it.
Gotchas#
On a target without inline assembly, which is every target but Linux on x86_64 and aarch64 and macOS on aarch64, the value is only made to live in memory, which stops less.
Method check#
pub fn check(self: Self, number: u64) -> ()Reports number as what the run computed, such as a checksum. A benchmark task does not
compare implementations whose checks differ.
Benchmarks#
pub struct BenchmarksThe benchmarks of a program.
Implementation Benchmarks#
impl BenchmarksAssociated function names#
pub effect fn names() -> List<String>
! OutOfMemoryError
? &mut AllocatorReturns the names of the benchmarks of the program, in the order they run: by module, then by name.
Errors#
OutOfMemoryError if the Allocator cannot supply the names.
Associated function perform#
pub effect fn perform() -> ()
! TransportError | OutOfMemoryError
? &mut Transport | &mut Allocator | &mut MonotonicClock | &mut HostInputRuns the benchmarks as a harness: driven by a benchmark task when one started the program, and otherwise alone, printing result lines.
Errors#
TransportErrorif a line cannot be carried.OutOfMemoryErrorif theAllocatorcannot supply a line.
Associated function serve#
pub effect fn serve(driven: bool) -> ()
! TransportError | OutOfMemoryError
? &mut Transport | &mut Allocator | &mut MonotonicClockRuns the benchmarks as a harness that follows the commands arriving over the Transport
if driven, and that otherwise runs every benchmark itself and sends the result lines.
Errors#
TransportErrorif a line cannot be carried.OutOfMemoryErrorif theAllocatorcannot supply a line.
Associated function main#
pub effect fn main() -> i32
? &mut HostInputRuns the benchmarks with the standard input and output of the process, and returns the exit status: 0, or 1 if a line could not be carried.
Category: Benchmarks.
Profiles: aarch64-apple-darwin, aarch64-unknown-linux-gnu, aarch64-unknown-linux-gnu-no-libc, x86_64-unknown-linux-gnu, x86_64-unknown-linux-gnu-no-libc, wasm32-unknown-unknown, wasm32-wasip1, wasm32-wasip2, x86_64-pc-windows-msvc, wasm64-unknown-unknown.