silk.io.buffered_input.

When to use#

Use BufferedInput to parse a protocol from a ByteDuplex transport or from StandardInput. For example, a parser can look for a line end with BufferedInput.peek and then remove only that line with BufferedInput.consume. The bytes after the line stay in the buffer for the next step. In a scoped session that also writes, use silk.io.buffered_duplex.BufferedDuplex.

Details#

BufferedInput.make allocates the buffer one time. No other operation allocates. BufferedInput.fill reads from the transport until the buffer holds a minimum number of unread bytes or the transport ends. BufferedInput.peek returns the unread bytes and does not change the buffer. BufferedInput.consume removes bytes from the front.

When the transport ends, the end is recorded, and later fills do not read again. The bytes that were read before the end stay available.

Errors#

• BufferError for each failure. A transport failure makes the buffer terminal, and each later fill fails with BufferError.Terminal.

Gotchas#

A message that is longer than the capacity cannot be in the buffer at one time. Choose the capacity for the longest unit that the parser must see, such as the longest header line.

Examples#

Read one line and keep the rest#

import silk.mem.allocator { Allocator, OutOfMemoryError }
import silk.io.buffered_input { BufferError, BufferedInput }
import silk.io.byte_duplex { ByteDuplex }
import silk.collections.bytes { Bytes }
import silk.core.effect { Effect }
import silk.collections.list { List }
import silk.io.memory_byte_duplex { MemoryByteDuplex, MemoryReadEvent, MemoryWriteEvent }
import silk.system.monotonic_clock { MonotonicClock }
import silk.core.option { Option }
import silk.system.system_clock { Instant, SystemClock }

struct FixedClock {}

impl MonotonicClock for FixedClock {
  effect fn now(self: &mut Self) -> Instant {
    return SystemClock.make(0, 0)
  }

  effect fn getResolution(self: &mut Self) -> u64 {
    return 1
  }

  effect fn waitUntil(self: &mut Self, when: Instant) -> () {
    return ()
  }

  effect fn waitFor(self: &mut Self, howLong: u64) -> () {
    return ()
  }
}

effect fn transport() -> MemoryByteDuplex
  ! OutOfMemoryError
  ? &mut Allocator {
  let mut reads = List<MemoryReadEvent>.make()
  let bytes = run Bytes.copy(&b"GET\nrest")
  run List<MemoryReadEvent>.append(
    &mut reads,
    MemoryReadEvent.Data { readyAt: SystemClock.make(0, 0), bytes: move bytes }
  )
  run List<MemoryReadEvent>.append(
    &mut reads,
    MemoryReadEvent.End { readyAt: SystemClock.make(0, 0) }
  )
  return run MemoryByteDuplex.make(
    move reads,
    List<MemoryWriteEvent>.make(),
    0,
    8,
    Option<i32>.none()
  )
}

// Reads bytes up to the first newline and returns how many came before it.
effect fn firstLineLength(input: &mut BufferedInput) -> usize
  ! BufferError
  ? &mut ByteDuplex | &mut MonotonicClock {
  let filled = run BufferedInput.fill(&mut input, 4, Option<Instant>.none())
  drop filled
  let view = BufferedInput.peek(&input)
  let mut length: usize = 0
  while length < view.length && view[length] != 10 {
    length = length + 1
  }
  drop view
  run BufferedInput.consume(&mut input, length + 1)
  return length
}

pub effect fn main() -> i32
  ! BufferError | OutOfMemoryError {
  let mut allocator = Allocator.systemAllocatorProvider()
  let mut clock = FixedClock {}
  let mut duplex = run transport() |> Effect.provideMut<Allocator>(&mut allocator)
  let mut input = run BufferedInput.make(16) |> Effect.provideMut<Allocator>(&mut allocator)
  let length = run firstLineLength(&mut input) |> Effect.provideMut<ByteDuplex>(&mut duplex)
    |> Effect.provideMut<MonotonicClock>(&mut clock)
  if length != 3 || BufferedInput.unread(&input) != 4 {
    return 1
  }
  return 0
}

Import as BufferedInput with import silk.io.buffered_input { BufferedInput }.

Public declarations: 5.

MAX_CAPACITY#

pub const MAX_CAPACITY: usize

The largest capacity that a buffer can have: 1 MiB (1,048,576 bytes).

BufferError#

pub union BufferError

The error from a buffered input, output, or transfer operation.

Details#

The failures that report progress give the number of bytes that the operation completed before the failure. This tells a protocol exactly where the stream stopped.

InvalidCapacity#

InvalidCapacity

The capacity was 0 or more than MAX_CAPACITY. Nothing was allocated.

Field capacity#

pub capacity: usize

The capacity that was rejected, in bytes.

BufferTooSmall#

BufferTooSmall

A fill asked for more bytes than the buffer can hold.

Field requested#

pub requested: usize

The minimum byte count that was requested.

Field capacity#

pub capacity: usize

The capacity of the buffer, in bytes.

InvalidConsumption#

InvalidConsumption

A consume asked to remove more bytes than are unread. The buffer did not change.

Field requested#

pub requested: usize

The byte count that was requested.

Field available#

pub available: usize

The number of unread bytes.

InvalidReadCount#

InvalidReadCount

An input provider reported a byte count that is not possible: 0, or more than the space it was given.

Field count#

pub count: usize

The count that the provider reported.

Field limit#

pub limit: usize

The space that the provider was given, in bytes.

UnexpectedEnd#

UnexpectedEnd

The transport ended before an exact read, discard, or transfer completed.

Field progress#

pub progress: usize

The number of bytes that the operation completed before the end.

ReadFailed#

ReadFailed

A ByteDuplex read failed. The buffer becomes terminal.

Field progress#

pub progress: usize

The number of bytes that the operation completed before the failure.

Field error#

pub error: ByteIoError

The transport error.

InputFailed#

InputFailed

A StandardInput read failed. The buffer becomes terminal.

Field progress#

pub progress: usize

The number of bytes that the operation completed before the failure.

Field error#

pub error: StreamReadError

The standard input error.

WriteFailed#

WriteFailed

A ByteDuplex write or flush failed. The output buffer becomes terminal.

Field accepted#

pub accepted: usize

The number of bytes of the current input that the buffer accepted before the failure.

Field drained#

pub drained: usize

The number of pending bytes that the transport received during the failed operation.

Field error#

pub error: ByteIoError

The transport error.

UnknownExternalTransfer#

UnknownExternalTransfer

A Writer call failed. The Writer does not report how many bytes of that call it wrote.

Field progress#

pub progress: usize

The number of bytes from earlier calls that completed before the failed call.

Field error#

pub error: WriterError

The Writer error.

LengthOverflow#

LengthOverflow

The total length of the input slices does not fit in usize. Nothing was written.

Terminal#

Terminal

An earlier transport failure or a close made the buffer unusable.

Implementation Debug for BufferError#

impl Debug for BufferError

Operation debug#

effect fn debug(self: &Self, formatter: &mut Formatter) -> () ! WriterError ? &mut Writer

Writes the name of the variant and its fields, such as InvalidCapacity { ... }.

FillOutcome#

pub union FillOutcome

The result of a fill.

Available#

Available

The buffer holds at least the requested number of unread bytes.

Field count#

pub count: usize

The number of unread bytes. It can be more than the requested minimum.

End#

End

The transport ended. The buffer can hold fewer bytes than requested.

Field available#

pub available: usize

The number of unread bytes that are still in the buffer.

DiscardOutcome#

pub struct DiscardOutcome

The result of BufferedInput.discardAtMost.

Field count#

pub count: usize

The number of bytes that were discarded.

Field endObserved#

pub endObserved: bool

true if the operation stopped because the transport ended. false if it reached the limit.

BufferedInput#

pub struct BufferedInput

A fixed-size input buffer for parsers that must look at bytes before they use them. Make one with BufferedInput.make.

When to use#

Use it to parse a protocol from a ByteDuplex transport or from StandardInput. For example, a parser can look for a line end with BufferedInput.peek and then remove only that line with BufferedInput.consume. The bytes after the line stay in the buffer for the next step. In a scoped session that also writes, use silk.io.buffered_duplex.BufferedDuplex.

Details#

BufferedInput.make allocates the buffer one time. No other operation allocates. When the transport ends, the end is recorded, and later fills do not read again. The bytes that were read before the end stay available. A transport failure makes the buffer terminal, and each later fill fails with BufferError.Terminal.

Gotchas#

A message that is longer than the capacity cannot be in the buffer at one time. Choose the capacity for the longest unit that the parser must see, such as the longest header line.

Implementation BufferedInput#

impl BufferedInput

Associated function make#

pub effect fn make(capacity: usize) -> BufferedInput
  ! BufferError | OutOfMemoryError
  ? &mut Allocator

Returns an empty buffer that holds at most capacity bytes.

Errors#

• BufferError.InvalidCapacity if capacity is 0 or more than MAX_CAPACITY. Nothing is allocated.

• OutOfMemoryError if the Allocator cannot supply capacity bytes.

Method capacity#

pub fn capacity(self: &Self) -> usize

Returns the capacity in bytes. It does not change.

Method unread#

pub fn unread(self: &Self) -> usize

Returns the number of bytes that were read from the transport and not yet consumed.

Method isEnded#

pub fn isEnded(self: &Self) -> bool

Returns true if the transport ended. Unread bytes can still be in the buffer.

Method isTerminal#

pub fn isTerminal(self: &Self) -> bool

Returns true if a transport failure made the buffer unusable.

Method peek#

pub fn peek<'a>(self: &'a Self) -> &'a [u8]

Returns the unread bytes. It does not read from the transport and does not change the buffer.

Details#

The slice borrows the buffer. Drop the slice before you call an operation that changes the buffer.

Method consume#

pub effect fn consume(self: &mut Self, count: usize) -> () ! BufferError

Removes count unread bytes from the front of the buffer.

Errors#

• BufferError.InvalidConsumption if count is more than BufferedInput.unread. The buffer does not change.

Method fill#

pub effect fn fill(
    self: &mut Self,
    minimum: usize,
    deadline: Option<Instant>,
  ) -> FillOutcome
  ! BufferError
  ? &mut ByteDuplex | &mut MonotonicClock

Reads from the ByteDuplex transport until the buffer holds at least minimum unread bytes, or until the transport ends.

Details#

If the buffer already holds minimum bytes, it does not read. Each read asks for all free space, so the buffer can hold more than minimum bytes after the call. Unread bytes move to the front of the buffer when necessary. A minimum of 0 never reads. deadline is an absolute time on the active MonotonicClock. Option.none() means no time limit.

Errors#

• BufferError.BufferTooSmall if minimum is more than the capacity.

• BufferError.Terminal if an earlier failure made the buffer unusable.

• BufferError.ReadFailed if a read fails. The buffer becomes terminal. progress gives the bytes that this call added.

Method fillStandard#

pub effect fn fillStandard(self: &mut Self, minimum: usize) -> FillOutcome
  ! BufferError
  ? &mut StandardInput

Reads from StandardInput until the buffer holds at least minimum unread bytes, or until the transport ends.

Details#

If the buffer already holds minimum bytes, it does not read. Each read asks for all free space, so the buffer can hold more than minimum bytes after the call. Unread bytes move to the front of the buffer when necessary. A minimum of 0 never reads. There is no deadline.

Errors#

• BufferError.BufferTooSmall if minimum is more than the capacity.

• BufferError.Terminal if an earlier failure made the buffer unusable.

• BufferError.InputFailed if a read fails. The buffer becomes terminal. progress gives the bytes that this call added.

• BufferError.InvalidReadCount if the provider reports a count that is not possible. The buffer becomes terminal.

Method readSome#

pub effect fn readSome(
    self: &mut Self,
    output: &mut [u8],
    deadline: Option<Instant>,
  ) -> ReadTransfer
  ! BufferError
  ? &mut ByteDuplex | &mut MonotonicClock

Copies unread bytes into output and consumes them. Reads from the transport first if the buffer is empty.

Details#

The result is ReadTransfer.Data with a count from 1 to output.length, or ReadTransfer.End if the transport ended and the buffer is empty. An empty output returns a count of 0 and does not read. deadline is an absolute time on the active MonotonicClock. Option.none() means no time limit.

Errors#

• BufferError.Terminal if an earlier failure made the buffer unusable.

• BufferError.ReadFailed if a read fails. The buffer becomes terminal. progress gives the bytes that this call added.

Method readSomeStandard#

pub effect fn readSomeStandard(self: &mut Self, output: &mut [u8]) -> ReadTransfer
  ! BufferError
  ? &mut StandardInput

Copies unread bytes into output and consumes them. Reads from StandardInput first if the buffer is empty.

Details#

The result is ReadTransfer.Data with a count from 1 to output.length, or ReadTransfer.End if the transport ended and the buffer is empty. An empty output returns a count of 0 and does not read.

Errors#

• BufferError.Terminal if an earlier failure made the buffer unusable.

• BufferError.InputFailed if a read fails. The buffer becomes terminal. progress gives the bytes that this call added.

• BufferError.InvalidReadCount if the provider reports a count that is not possible. The buffer becomes terminal.

Method readExact#

pub effect fn readExact(
    self: &mut Self,
    output: &mut [u8],
    deadline: Option<Instant>,
  ) -> ()
  ! BufferError
  ? &mut ByteDuplex | &mut MonotonicClock

Fills all of output with the next bytes and consumes them.

Details#

deadline is an absolute time on the active MonotonicClock. Option.none() means no time limit. It applies to each read.

Errors#

• BufferError.UnexpectedEnd if the transport ends first. progress gives the number of bytes that were copied into output.

• BufferError.Terminal if an earlier failure made the buffer unusable.

• BufferError.ReadFailed if a read fails. The buffer becomes terminal. progress includes the bytes that this call copied before the failure.

Method discardAtMost#

pub effect fn discardAtMost(
    self: &mut Self,
    limit: usize,
    deadline: Option<Instant>,
  ) -> DiscardOutcome
  ! BufferError
  ? &mut ByteDuplex | &mut MonotonicClock

Consumes and drops up to limit bytes. It stops early if the transport ends.

When to use#

Use it to skip a body or a field of a known maximum length.

Details#

The result gives the number of discarded bytes and if the transport ended. deadline is an absolute time on the active MonotonicClock. Option.none() means no time limit.

Errors#

• BufferError.Terminal if an earlier failure made the buffer unusable.

• BufferError.ReadFailed if a read fails. The buffer becomes terminal. progress gives the bytes that this call added.

Method discardExact#

pub effect fn discardExact(
    self: &mut Self,
    count: usize,
    deadline: Option<Instant>,
  ) -> ()
  ! BufferError
  ? &mut ByteDuplex | &mut MonotonicClock

Consumes and drops exactly count bytes.

Details#

deadline is an absolute time on the active MonotonicClock. Option.none() means no time limit.

Errors#

• BufferError.UnexpectedEnd if the transport ends first. progress gives the number of bytes that were discarded.

• BufferError.Terminal if an earlier failure made the buffer unusable.

• BufferError.ReadFailed if a read fails. The buffer becomes terminal. progress gives the bytes that this call added.

Category: Input and output.

Profiles: aarch64-apple-darwin, aarch64-unknown-linux-gnu, aarch64-unknown-linux-gnu-no-libc, x86_64-unknown-linux-gnu, x86_64-unknown-linux-gnu-no-libc, wasm32-unknown-unknown, wasm32-wasip1, wasm32-wasip2.