silk.bench.bencher.

Benchmarks declared with bench fn, the Bencher they loop on, and the program that runs them.

When to use#

Write a bench fn beside the code it measures when the work is a Silk function: a parser, a hash, one step of a computation. Compare whole programs, or programs in other languages, with a suite of silk.bench.suite instead.

Details#

A benchmark is a top-level function marked bench that takes one Bencher. It does its setup, then loops on Bencher.next, and each turn of the loop is one iteration:

silk
bench fn sumToOneThousand(b: Bencher) {
  let mut total: u64 = 0
  while b.next() {
    total = 0
    let mut at: u64 = 0
    while at < 1000 {
      total = total + at
      at = at + 1
    }
    b.keep<u64>(&total)
  }
  b.check(total)
}

The time that counts runs from the first call of next to the call that returns false, so what comes before the loop and after it is free. Bencher.stop and Bencher.start leave a stretch inside the loop out, and Bencher.reset starts the time again.

The benchmarks of a program are the bench functions of the modules its entry file imports, directly or through other modules of the same package. Benchmarks.main runs them. A program whose main calls it is a harness as silk.bench.harness describes: run by hand, it finds an iteration count for each benchmark, takes ten samples and prints result lines; declared in a suite with silk.bench.suite.Implementation.driven, it takes its iteration counts from the benchmark task, which compares it with other programs and with earlier runs.

A benchmark is named after its function. Two benchmarks with one name in different modules are both named <module>.<name>.

A bench effect fn may fail. A failure ends the run of that benchmark, which is reported as failed.

Gotchas#

  • The compiler removes work whose result nothing uses. Give each iteration's result to Bencher.keep.

  • The compiler also computes ahead of time what it can: work on a constant is done when the program is built. Give Bencher.keep the input too, before the work reads it.

  • bench marks only a top-level function with one parameter, no type parameters and no requirement row. A benchmark that needs a service provides it inside.

  • A benchmark that never calls next is timed as a whole, as if its one call were every iteration.

  • A build that does not call Benchmarks.main keeps no benchmark.

Examples#

The entry file of a benchmark program:

silk
import silk.bench.bencher { Benchmarks }
import silk.system.host_input { HostInput }

// The modules that hold the `bench` functions. A name that starts with `_` is not
// reported as unused.
import parser as _parser
import encoder as _encoder

pub effect fn main() -> i32
  ? &mut HostInput {
  return run Benchmarks.main()
}

Import as Bencher with import silk.bench.bencher { Bencher }.

Public declarations: 2.

Bencher#

silk
pub struct Bencher

The run of a benchmark that is in progress: what a bench fn loops on.

Details#

A Bencher is a handle to a run the benchmark program holds, so a copy of it is the same run.

Gotchas#

A Bencher is good only inside the call of the benchmark it was given to.

Implementation Copy for Bencher#

silk
impl Copy for Bencher

Implementation Bencher#

silk
impl Bencher

Method next#

silk
pub fn next(self: Self) -> bool

Returns whether there is another iteration to do. The first call starts the time, and the call that returns false stops it.

Method iterations#

silk
pub fn iterations(self: Self) -> u64

Returns how many iterations the run has in all.

Method stop#

silk
pub fn stop(self: Self) -> ()

Stops the time, until Bencher.start. Stopping twice changes nothing.

Method start#

silk
pub fn start(self: Self) -> ()

Starts the time again after Bencher.stop. Starting twice changes nothing.

Method reset#

silk
pub fn reset(self: Self) -> ()

Starts the time again from zero, after work inside the loop that should not count.

Method keep#

silk
pub fn keep<T>(self: Self, value: &T) -> ()

Makes the compiler treat value as used, so it cannot remove the work that computed it.

Gotchas#

On a target without inline assembly, which is every target but Linux on x86_64 and aarch64 and macOS on aarch64, the value is only made to live in memory, which stops less.

Method check#

silk
pub fn check(self: Self, number: u64) -> ()

Reports number as what the run computed, such as a checksum. A benchmark task does not compare implementations whose checks differ.

Benchmarks#

silk
pub struct Benchmarks

The benchmarks of a program.

Implementation Benchmarks#

silk
impl Benchmarks

Associated function names#

silk
pub effect fn names() -> List<String>
    ! OutOfMemoryError
    ? &mut Allocator

Returns the names of the benchmarks of the program, in the order they run: by module, then by name.

Errors#

OutOfMemoryError if the Allocator cannot supply the names.

Associated function perform#

silk
pub effect fn perform() -> ()
    ! TransportError | OutOfMemoryError
    ? &mut Transport | &mut Allocator | &mut MonotonicClock | &mut HostInput

Runs the benchmarks as a harness: driven by a benchmark task when one started the program, and otherwise alone, printing result lines.

Errors#

  • TransportError if a line cannot be carried.

  • OutOfMemoryError if the Allocator cannot supply a line.

Associated function serve#

silk
pub effect fn serve(driven: bool) -> ()
    ! TransportError | OutOfMemoryError
    ? &mut Transport | &mut Allocator | &mut MonotonicClock

Runs the benchmarks as a harness that follows the commands arriving over the Transport if driven, and that otherwise runs every benchmark itself and sends the result lines.

Errors#

  • TransportError if a line cannot be carried.

  • OutOfMemoryError if the Allocator cannot supply a line.

Associated function main#

silk
pub effect fn main() -> i32
    ? &mut HostInput

Runs the benchmarks with the standard input and output of the process, and returns the exit status: 0, or 1 if a line could not be carried.

Category: Benchmarks.

Profiles: aarch64-apple-darwin, aarch64-unknown-linux-gnu, aarch64-unknown-linux-gnu-no-libc, x86_64-unknown-linux-gnu, x86_64-unknown-linux-gnu-no-libc, wasm32-unknown-unknown, wasm32-wasip1, wasm32-wasip2, x86_64-pc-windows-msvc, wasm64-unknown-unknown.

View the source of this page