silk.mem.prefetch.
Cache prefetch hints: tell the processor that the program will soon use the memory at a pointer.
When to use#
Use Prefetch.hint or Prefetch.hintMany in a hot loop that reads memory in an order the
processor cannot predict, such as the next node of a linked structure or a table entry chosen
by a hash, when a measurement shows that the loop waits for memory. Issue the hint a few
iterations before the access, so that the memory is in cache when the access comes.
Details#
A hint is only a request. It never reads or writes memory, never faults, and never changes what
the program computes, so it is safe with any pointer, also null or released memory. The
PrefetchOptions argument is static and selects the access (read or write), the locality
(how long to keep the memory in cache) and the cache (data or instruction).
On x86_64 and AArch64 a hint is one prefetch instruction, such as prefetcht0 or
prfm pldl1keep for a read with high locality. A target without a prefetch instruction, such as
WebAssembly, ignores the hint.
Gotchas#
A hint that comes too late, or that requests memory the program does not use, costs time and cache space. Measure before and after you add one.
Examples#
Prefetch the next element while you sum an array#
import silk.mem.pointer { Pointer }
import silk.mem.prefetch { Prefetch, PrefetchAccess, PrefetchCache, PrefetchLocality, PrefetchOptions }
pub fn main() -> i32 {
let values = [3, 5, 8, 13, 21]
let mut sum = 0
let mut at: usize = 0
while at < values.length {
if at + 1 < values.length {
Prefetch.hint<i32>(Pointer.fromRef<i32>(&values[at + 1]), PrefetchOptions {
access: PrefetchAccess.Read,
locality: PrefetchLocality.High,
cache: PrefetchCache.Data,
})
}
sum = sum + values[at]
at = at + 1
}
if sum != 50 {
return 1
}
return 0
}Import as Prefetch with import silk.mem.prefetch { Prefetch }.
Public declarations: 5.
Prefetch#
pub struct PrefetchThe owner type of the prefetch hints. It has no fields and is never constructed.
Details#
A hint never reads or writes memory, never faults, and never changes what the program computes. A target without a prefetch instruction, such as WebAssembly, ignores it.
PrefetchAccess#
pub enum PrefetchAccessThe access that a prefetch hint announces.
Read#
ReadThe program will read the memory.
Write#
WriteThe program will write the memory. On some processors this fetches the cache line in a state that a store can change without a second request.
PrefetchLocality#
pub enum PrefetchLocalityHow long a prefetched cache line is likely to stay useful.
Details#
The levels are 0 to 3 of the llvm.prefetch locality. On x86_64, a read hint with
NoReuse is prefetchnta, Low is prefetcht2, Moderate is prefetcht1 and High is
prefetcht0.
NoReuse#
NoReuseThe program uses the memory one time. Keep it out of the caches as much as possible.
Low#
LowThe program uses the memory again soon, a little.
Moderate#
ModerateThe program uses the memory again several times.
High#
HighThe program uses the memory again many times. Keep it in every cache level.
PrefetchCache#
pub enum PrefetchCacheThe cache that a prefetch hint fills.
Data#
DataThe data cache, for memory that the program reads or writes.
Instruction#
InstructionThe instruction cache, for machine code that the program will run.
PrefetchOptions#
pub struct PrefetchOptionsThe static selections of one prefetch hint.
Details#
A read of data with High locality is the common hint. It is prefetcht0 on x86_64 and
prfm pldl1keep on AArch64. Hints for the instruction cache ignore access.
Field access#
pub access: PrefetchAccessWhether the program will read or write the memory.
Field locality#
pub locality: PrefetchLocalityHow long the memory is likely to stay useful in cache.
Field cache#
pub cache: PrefetchCacheThe data cache or the instruction cache.
Implementation Prefetch#
impl PrefetchAssociated function hint#
pub fn hint<T>(pointer: ?*const T, static options: PrefetchOptions) -> ()Hints that the program will soon use the memory at pointer, as options selects.
Details#
It does not read or write memory and does not fault, also when pointer is null or addresses
released memory, and it never changes what the program computes. options is static, so the
hint is fixed at compile time. A target without a prefetch instruction, such as WebAssembly,
ignores it.
Associated function hintMany#
pub fn hintMany<T>(pointer: ?[*]const T, static options: PrefetchOptions) -> ()Hints that the program will soon use the memory at the many-element pointer, as options
selects.
Details#
It does not read or write memory and does not fault, also when pointer is null or addresses
released memory, and it never changes what the program computes. options is static, so the
hint is fixed at compile time. A target without a prefetch instruction, such as WebAssembly,
ignores it.
Category: Memory.
Profiles: aarch64-apple-darwin, aarch64-unknown-linux-gnu, aarch64-unknown-linux-gnu-no-libc, x86_64-unknown-linux-gnu, x86_64-unknown-linux-gnu-no-libc, wasm32-unknown-unknown, wasm32-wasip1, wasm32-wasip2, x86_64-pc-windows-msvc, wasm64-unknown-unknown.