silk.bench.lines.
Benchmark result lines: the text format of Go's testing package, read and written.
When to use#
Use Lines.parse to read the samples a benchmark program printed, in any language, and
Lines.fromResult to hand a run to a tool that reads the format, such as benchstat. A
program that prints these lines needs no library: one printf per sample is enough.
Details#
A result line is one sample: a name, the number of iterations the sample covers, then one or more pairs of a value and a unit. Values are for one iteration.
impl: simdjson
BenchmarkParse/file=twitter 2000 165412 ns/op 1024 B/op
BenchmarkParse/file=twitter 2000 164980 ns/op 1024 B/opA name starts with Benchmark, and the character after it is not a lowercase letter. A
configuration line is key: value, where the key starts with a lowercase letter and has no
space and no uppercase letter. It applies to every result line after it, until a line with
the same key replaces it. Every other line is ignored, so a program may print anything else.
Letters are judged as ASCII: a byte outside ASCII counts as lowercase.
A sample is never summarized by the program that prints it. It prints the line again for
the next sample, and silk.bench.statistics computes every statistic.
Examples#
import silk.bench.lines { Lines }
import silk.core.effect { Effect }
import silk.core.option { Option }
import silk.mem.allocator { Allocator, OutOfMemoryError }
effect fn check() -> i32 ! OutOfMemoryError ? &mut Allocator {
let parsed = run Lines.parse("impl: silk\nBenchmarkParse 2000 165412 ns/op\nok 0.4s\n")
let lines = parsed.lines.asSlice()
if lines.length != 1 || lines[0].iterations != 2000 {
return 1
}
return match Lines.value(&lines[0], "ns/op") {
Option<f64>.Some { value } => if value == 165412.0 { 0 } else { 2 }
Option<f64>.None => 3
}
}
effect fn failed(error: OutOfMemoryError) -> i32 {
drop error
return 4
}
pub fn main() -> i32 {
let mut allocator = Allocator.systemAllocatorProvider()
return run Effect.catchAll(check(), failed) |> Effect.provideMut<Allocator>(&mut allocator)
}Import as Lines with import silk.bench.lines { Lines }.
Public declarations: 5.
Value#
pub struct ValueOne measured value of a sample with its unit.
Field value#
pub value: f64The value, for one iteration.
Field unit#
pub unit: StringThe unit as the program wrote it, such as ns/op, B/op or MB/s.
Setting#
pub struct SettingOne configuration line.
Field name#
pub name: StringThe key, without the colon.
Field value#
pub value: StringThe value, without the space around it.
Field from#
pub from: usizeThe index in Parsed.lines of the first result line after this one. The setting applies
from that line on.
Line#
pub struct LineOne result line: one sample.
Field name#
pub name: StringThe whole name, starting with Benchmark.
Field iterations#
pub iterations: u64The number of iterations the sample covers.
Field values#
pub values: List<Value>The values, in the order they were written. There is at least one.
Parsed#
pub struct ParsedWhat a text held: its result lines and its configuration lines.
Field settings#
pub settings: List<Setting>The configuration lines, in the order they were written.
Field lines#
pub lines: List<Line>The result lines, in the order they were written.
Lines#
pub struct LinesThe owner type of the result line functions. It has no fields and is never constructed.
Implementation Lines#
impl LinesAssociated function parse#
pub effect fn parse(text: string) -> Parsed
! OutOfMemoryError
? &mut AllocatorReturns the result lines and configuration lines of text. A line that is neither is
ignored, and so is a line that starts like a result but has a part that does not parse,
such as a value that is not a number or a value with no unit.
Errors#
OutOfMemoryError if the Allocator cannot supply the lines.
Associated function setting#
pub fn setting<'parsed>(parsed: &'parsed Parsed, at: usize, name: string) -> Option<
string<'parsed>
>Returns the value of the configuration key name that applies to the result line at index
at: the last one written before that line. It is None when no line before it sets the
key.
Associated function value#
pub fn value(line: &Line, unit: string) -> Option<f64>Returns the value of line with the unit unit, or None when the line has none. When a
line repeats a unit, the first one is returned.
Associated function named#
pub effect fn named(name: string) -> String
! OutOfMemoryError
? &mut AllocatorReturns the name a benchmark of a suite has as a result line: Benchmark, then name with
its first letter in uppercase and every space replaced by _. parse/file=twitter becomes
BenchmarkParse/file=twitter.
Errors#
OutOfMemoryError if the Allocator cannot supply the text.
Associated function fromResult#
pub effect fn fromResult(result: &RunResult) -> String
! OutOfMemoryError
? &mut AllocatorReturns the samples of result as result lines, for a tool that reads the format.
Each implementation is a block that starts with the configuration line impl: <name> and,
when it has a revision, revision:, after the lines suite:, date: and, when the run
has one, label:. Each measured run
that succeeded is one line with 1 iteration and the units ns/op (time from start to
exit), user-ns/op, sys-ns/op and peak-B. Warm-up runs and failed runs are left out.
Where the programs printed result lines themselves, those lines are written in place of the process times, as they were read, with the parameters of their combination added to the name.
Errors#
OutOfMemoryError if the Allocator cannot supply the text.
Associated function fromGoogleBenchmark#
pub effect fn fromGoogleBenchmark(text: string) -> String
! OutOfMemoryError
? &mut AllocatorReturns what Google Benchmark printed in text, its console table, as result lines: one
line for each row, with the row's iterations, its Time column as ns/op and its CPU
column as cpu-ns/op. A row BM_Parse/64 98.1 ns 98.0 ns 7000000 becomes
BenchmarkBM_Parse/64 7000000 98.1 ns/op 98 cpu-ns/op.
Details#
A line that is not a row is left out, and so are the rows Google Benchmark adds for
repetitions, whose names end in _mean, _median, _stddev or _cv: the task computes
its own statistics. Counters after the iterations are left out.
Errors#
OutOfMemoryError if the Allocator cannot supply the text.
Associated function fromCriterion#
pub effect fn fromCriterion(text: string) -> String
! OutOfMemoryError
? &mut AllocatorReturns what a Rust benchmark printed in text as result lines with 1 iteration and the
unit ns/op. Two shapes are read: Criterion's name time: [low estimate high], of
which the estimate in the middle is taken, and the test name ... bench: 1,240 ns/iter
(+/- 12) of the built-in harness and of Criterion's --output-format bencher.
Details#
Criterion writes a long name on a line of its own, before the line with time:; that
name is used. Every other line is left out. A space in a name becomes _.
Errors#
OutOfMemoryError if the Allocator cannot supply the text.
Category: Benchmarks.
Profiles: aarch64-apple-darwin, aarch64-unknown-linux-gnu, aarch64-unknown-linux-gnu-no-libc, x86_64-unknown-linux-gnu, x86_64-unknown-linux-gnu-no-libc, wasm32-unknown-unknown, wasm32-wasip1, wasm32-wasip2, x86_64-pc-windows-msvc, wasm64-unknown-unknown.