silk.mem.prefetch.

Cache prefetch hints: tell the processor that the program will soon use the memory at a pointer.

When to use#

Use Prefetch.hint or Prefetch.hintMany in a hot loop that reads memory in an order the processor cannot predict, such as the next node of a linked structure or a table entry chosen by a hash, when a measurement shows that the loop waits for memory. Issue the hint a few iterations before the access, so that the memory is in cache when the access comes.

Details#

A hint is only a request. It never reads or writes memory, never faults, and never changes what the program computes, so it is safe with any pointer, also null or released memory. The PrefetchOptions argument is static and selects the access (read or write), the locality (how long to keep the memory in cache) and the cache (data or instruction).

On x86_64 and AArch64 a hint is one prefetch instruction, such as prefetcht0 or prfm pldl1keep for a read with high locality. A target without a prefetch instruction, such as WebAssembly, ignores the hint.

Gotchas#

A hint that comes too late, or that requests memory the program does not use, costs time and cache space. Measure before and after you add one.

Examples#

Prefetch the next element while you sum an array#

silk
import silk.mem.pointer { Pointer }
import silk.mem.prefetch { Prefetch, PrefetchAccess, PrefetchCache, PrefetchLocality, PrefetchOptions }

pub fn main() -> i32 {
  let values = [3, 5, 8, 13, 21]
  let mut sum = 0
  let mut at: usize = 0
  while at < values.length {
    if at + 1 < values.length {
      Prefetch.hint<i32>(Pointer.fromRef<i32>(&values[at + 1]), PrefetchOptions {
        access: PrefetchAccess.Read,
        locality: PrefetchLocality.High,
        cache: PrefetchCache.Data,
      })
    }
    sum = sum + values[at]
    at = at + 1
  }
  if sum != 50 {
    return 1
  }
  return 0
}

Import as Prefetch with import silk.mem.prefetch { Prefetch }.

Public declarations: 5.

Prefetch#

silk
pub struct Prefetch

The owner type of the prefetch hints. It has no fields and is never constructed.

Details#

A hint never reads or writes memory, never faults, and never changes what the program computes. A target without a prefetch instruction, such as WebAssembly, ignores it.

PrefetchAccess#

silk
pub enum PrefetchAccess

The access that a prefetch hint announces.

Read#

silk
Read

The program will read the memory.

Write#

silk
Write

The program will write the memory. On some processors this fetches the cache line in a state that a store can change without a second request.

PrefetchLocality#

silk
pub enum PrefetchLocality

How long a prefetched cache line is likely to stay useful.

Details#

The levels are 0 to 3 of the llvm.prefetch locality. On x86_64, a read hint with NoReuse is prefetchnta, Low is prefetcht2, Moderate is prefetcht1 and High is prefetcht0.

NoReuse#

silk
NoReuse

The program uses the memory one time. Keep it out of the caches as much as possible.

Low#

silk
Low

The program uses the memory again soon, a little.

Moderate#

silk
Moderate

The program uses the memory again several times.

High#

silk
High

The program uses the memory again many times. Keep it in every cache level.

PrefetchCache#

silk
pub enum PrefetchCache

The cache that a prefetch hint fills.

Data#

silk
Data

The data cache, for memory that the program reads or writes.

Instruction#

silk
Instruction

The instruction cache, for machine code that the program will run.

PrefetchOptions#

silk
pub struct PrefetchOptions

The static selections of one prefetch hint.

Details#

A read of data with High locality is the common hint. It is prefetcht0 on x86_64 and prfm pldl1keep on AArch64. Hints for the instruction cache ignore access.

Field access#

silk
pub access: PrefetchAccess

Whether the program will read or write the memory.

Field locality#

silk
pub locality: PrefetchLocality

How long the memory is likely to stay useful in cache.

Field cache#

silk
pub cache: PrefetchCache

The data cache or the instruction cache.

Implementation Prefetch#

silk
impl Prefetch

Associated function hint#

silk
pub fn hint<T>(pointer: ?*const T, static options: PrefetchOptions) -> ()

Hints that the program will soon use the memory at pointer, as options selects.

Details#

It does not read or write memory and does not fault, also when pointer is null or addresses released memory, and it never changes what the program computes. options is static, so the hint is fixed at compile time. A target without a prefetch instruction, such as WebAssembly, ignores it.

Associated function hintMany#

silk
pub fn hintMany<T>(pointer: ?[*]const T, static options: PrefetchOptions) -> ()

Hints that the program will soon use the memory at the many-element pointer, as options selects.

Details#

It does not read or write memory and does not fault, also when pointer is null or addresses released memory, and it never changes what the program computes. options is static, so the hint is fixed at compile time. A target without a prefetch instruction, such as WebAssembly, ignores it.

Category: Memory.

Profiles: aarch64-apple-darwin, aarch64-unknown-linux-gnu, aarch64-unknown-linux-gnu-no-libc, x86_64-unknown-linux-gnu, x86_64-unknown-linux-gnu-no-libc, wasm32-unknown-unknown, wasm32-wasip1, wasm32-wasip2, x86_64-pc-windows-msvc, wasm64-unknown-unknown.

View the source of this page