silk.compress.brotli.

When to use#

Use Brotli to read .br files or HTTP bodies with Content-Encoding: br. Make a decoder with Brotli.make, and call Brotli.step until it returns Finished. Call Brotli.reset to decode another stream with the same storage. To compress, use silk.compress.brotli_encode.BrotliEncoder.

Details#

The decoder supports all of RFC 7932: compressed, uncompressed, and metadata meta-blocks, all context modes, and the static dictionary with its 121 word transforms. Brotli.make allocates all storage one time: the history window and the space for prefix code tables. Brotli.step does not allocate and keeps no borrow of your buffers, so you can move the decoder between threads between calls.

A Brotli stream ends itself. The decoder returns Finished when the last meta-block is decoded and all output is written, and it leaves the bytes after the stream unconsumed.

Errors#

• OutOfMemoryError from Brotli.make if the Allocator fails.

• BrotliError if the configuration is not valid, the data is not valid, a limit is exceeded, or a call after the final input does not give exactly the input that is not consumed yet. reason gives the cause and offset the input position.

Gotchas#

The window holds windowBytes rounded down to a power of two. A stream can declare a larger window than that and still decode, as long as no back-reference reaches further than the window holds. A failure is permanent until Brotli.reset.

Examples#

Decode a short stream#

import silk.mem.allocator { Allocator, OutOfMemoryError }
import silk.core.effect { Effect }
import silk.core.result { Result }
import silk.compress.brotli { Brotli, BrotliError, BrotliLimits, BrotliProgress, BrotliStatus }

pub effect fn main() -> i32
  ! BrotliError | OutOfMemoryError {
  let mut allocator = Allocator.systemAllocatorProvider()
  let mut decoder = run Brotli.make(BrotliLimits.make())
    |> Effect.provideMut<Allocator>(&mut allocator)

  // The reference encoder's output for "hi": one uncompressed meta-block and an empty last one.
  let input = b"\x8b\x00\x80hi\x03"
  let mut output: [u8; 16] = [0; 16]
  return match Brotli.step(&mut decoder, &input, &mut output, true) {
    Result<BrotliProgress, BrotliError>.Success { value } => {
      if value.status != BrotliStatus.Finished || value.written != 2 || output[0] != 104
        || output[1] != 105 {
        return 1
      }
      return 0
    }
    Result<BrotliProgress, BrotliError>.Failure {..} => 1
  }
}

Import as Brotli with import silk.compress.brotli { Brotli }.

Public declarations: 6.

BrotliLimits#

pub struct BrotliLimits

The resource limits for one stream. BrotliLimits.make gives defaults.

Field inputBytes#

pub inputBytes: u64

The maximum number of input bytes to consume.

Field outputBytes#

pub outputBytes: u64

The maximum number of decoded bytes.

Field windowBytes#

pub windowBytes: usize

The size of the history window that Brotli.make allocates, rounded down to a power of two and capped at 16 MiB, the largest window of the format. It must not be 0. A back-reference further than the window fails with WindowLimit.

Field workspaceBytes#

pub workspaceBytes: usize

The storage budget other than the window: decoder state, context maps, and the space for the prefix code tables of one meta-block. If it is too small for the fixed part, Brotli.make fails with WorkspaceLimit, and if a meta-block needs more table space than it leaves, the step fails with WorkspaceLimit.

Implementation BrotliLimits#

impl BrotliLimits

Associated function make#

pub fn make() -> Self

Returns default limits: 64 MiB of input, 256 MiB of output, a 16 MiB window, and a 3 MiB workspace. These decode any valid stream within the input and output limits.

BrotliStatus#

pub enum BrotliStatus

What the decoder needs after a successful step.

NeedInput#

NeedInput

All given input is consumed. Call again with more input, or with finalInput set if there is no more.

NeedOutput#

NeedOutput

The output buffer is full. Call again with an output buffer that is not empty, and the input that was not consumed.

Finished#

Finished

The stream is complete and all of its output is written. The bytes after it are not consumed.

BrotliProgress#

pub struct BrotliProgress

The result of one successful step.

Field consumed#

pub consumed: usize

The number of input bytes that this call consumed. Give only the rest of the input on the next call.

Field written#

pub written: usize

The number of output bytes that this call wrote. Only this prefix of the output changed.

Field status#

pub status: BrotliStatus

What the decoder needs next.

BrotliReason#

pub enum BrotliReason

The cause of a BrotliError.

None#

None

No failure. The decoder uses it internally. A returned error never has it.

InvalidConfiguration#

InvalidConfiguration

The window size is 0, or the storage sizes do not fit in usize.

InvalidState#

InvalidState

A call changes the final input after it was set, or uses a decoder that lost its storage.

WindowBits#

WindowBits

The stream header asks for a window size that RFC 7932 does not define.

Header#

Header

A meta-block header is not valid: a reserved bit is set, a length has an extra zero nibble or byte, or padding bits are not zero.

PrefixCode#

PrefixCode

A prefix code is not valid: a symbol is out of range or repeated, a code is incomplete or oversubscribed, or a repeat runs past the alphabet.

ContextMap#

ContextMap

A context map has a run of zeros past its end.

BlockLength#

BlockLength

The commands of a meta-block produce more bytes than its length.

Distance#

Distance

A distance is 0 or less, or too large.

Dictionary#

Dictionary

A static dictionary reference has a length with no words, or a transform that does not exist.

Truncated#

Truncated

The final input ends before the stream is complete.

InputLimit#

InputLimit

The consumed input would go over BrotliLimits.inputBytes.

OutputLimit#

OutputLimit

The decoded output would go over BrotliLimits.outputBytes.

WindowLimit#

WindowLimit

A back-reference reaches further than the window that Brotli.make allocated.

WorkspaceLimit#

WorkspaceLimit

BrotliLimits.workspaceBytes is too small for the decoder state or for the prefix code tables of a meta-block.

Implementation Debug for BrotliReason#

impl Debug for BrotliReason

Operation debug#

effect fn debug(self: &Self, formatter: &mut Formatter) -> ()
    ! WriterError
    ? &mut Writer

Writes the name of the variant, such as None.

BrotliError#

pub struct BrotliError

A decoding failure. It is permanent until Brotli.reset.

Each later call returns the same reason and offset with no progress. The output that earlier calls wrote is not valid.

Field reason#

pub reason: BrotliReason

The cause of the failure.

Field offset#

pub offset: u64

The input position where the decoder found the problem, counted from the start of the stream.

Field consumed#

pub consumed: usize

The number of input bytes that this call consumed before it failed.

Field written#

pub written: usize

The number of output bytes that this call wrote before it failed.

Implementation Debug for BrotliError#

impl Debug for BrotliError

Operation debug#

effect fn debug(self: &Self, formatter: &mut Formatter) -> ()
    ! WriterError
    ? &mut Writer

Writes the fields, as BrotliError { reason: ..., offset: ..., ... }.

Brotli#

pub struct Brotli

A Brotli decoder for one stream at a time. It owns all its storage.

When to use#

Use it to read .br files or HTTP bodies with Content-Encoding: br. Make a decoder with Brotli.make, call Brotli.step until it returns Finished, and call Brotli.reset to decode another stream.

Details#

Brotli.make allocates the window and the table space one time. Brotli.step does not allocate and keeps no borrow of your buffers, so you can move the decoder between threads between calls.

Gotchas#

A failure is permanent until Brotli.reset. Output that the stream wrote before a failure is not valid.

Implementation Brotli#

impl Brotli

Associated function make#

pub effect fn make(limits: BrotliLimits) -> Self
    ! BrotliError | OutOfMemoryError
    ? &mut Allocator

Returns a decoder that uses limits, with storage from the ambient Allocator.

It allocates the window and the workspace. Brotli.step does not allocate. Both come from Allocator.allocateZeroed, so with the system allocator a large window takes memory only as the stream fills it.

Errors#

• BrotliError with WorkspaceLimit if workspaceBytes is too small for the fixed state, or with InvalidConfiguration if windowBytes is 0. These checks come before the allocation.

• OutOfMemoryError if the Allocator fails.

Method reset#

pub fn reset(self: &mut Self) -> ()

Starts a new stream with the same limits and storage. It does not allocate. It drops the state of the current stream and a failure.

Method step#

pub fn step(self: &mut Self, input: &[u8], output: &mut [u8], finalInput: bool) -> Result<
    BrotliProgress,
    BrotliError,
  >

Decodes from input into output and returns how much it consumed and wrote.

On the next call, give only the input after the consumed prefix, and use only the written prefix of the output. Set finalInput when input holds all remaining data. After that, each call must set finalInput and give exactly the input that is not consumed yet. NeedOutput with an empty output makes no progress. After Finished, each call returns Finished and consumes nothing.

Errors#

• BrotliError if the data is not valid, a BrotliLimits value would be exceeded, the final input ends early (Truncated), or the call changes the final input (InvalidState).

Gotchas#

A failure is permanent until Brotli.reset.

Category: Data formats.

Profiles: aarch64-apple-darwin, aarch64-unknown-linux-gnu, aarch64-unknown-linux-gnu-no-libc, x86_64-unknown-linux-gnu, x86_64-unknown-linux-gnu-no-libc, wasm32-unknown-unknown, wasm32-wasip1, wasm32-wasip2, x86_64-pc-windows-msvc, wasm64-unknown-unknown.