silk.io.buffered_output.

When to use#

Use BufferedOutput when a program makes many small writes to a byte transport and each transport write has a cost, such as a system call. Make the buffer with BufferedOutput.make, write with BufferedOutput.writeAll, and end with BufferedOutput.finish. To give code that requires &mut Writer a buffered destination, wrap that destination, such as silk.io.writer_native.StdoutWriter, in a BufferedWriter.

Each operation has two forms. The deadline form, such as writeAll, uses a silk.io.byte_duplex.ByteDuplex transport and a deadline. The Writer form, such as writeAllWriter, uses a silk.io.writer.Writer provider and has no deadline.

Details#

When a write returns, the buffer owns the accepted bytes. The buffer sends pending bytes to the transport only when it is full or when the caller flushes it. Input that is longer than the buffer is not copied: once the pending bytes are sent, it goes to the transport directly. A flush sends all pending bytes in order, one time each.

A failed transport write or flush makes the buffer terminal. After that, each write and flush fails with BufferError.Terminal. The error reports exactly how many bytes the transport accepted before the failure.

Gotchas#

Dropping a BufferedOutput does not flush it. Pending bytes that are not flushed are lost. Call BufferedOutput.finish before the buffer goes out of scope.

Examples#

Collect two writes into one transport write#

import silk.mem.allocator { Allocator, OutOfMemoryError }
import silk.io.buffered_input { BufferError }
import silk.io.buffered_output { BufferedOutput }
import silk.core.effect { Effect }
import silk.io.writer { Writer, WriterError }

// A Writer provider that counts the calls and bytes it receives.
struct CountingWriter {
  calls: usize,
  bytes: usize,
}

impl Writer for CountingWriter {
  effect fn writeAll(self: &mut Self, val: &[u8]) -> ()
    ! WriterError
    ? &mut Writer {
    self.calls = self.calls + 1
    self.bytes = self.bytes + val.length
    return ()
  }

  effect fn flush(self: &mut Self) -> ()
    ! WriterError
    ? &mut Writer {
    return ()
  }
}

pub effect fn main() -> i32
  ! BufferError | OutOfMemoryError {
  let mut allocator = Allocator.systemAllocatorProvider()
  let mut output = run BufferedOutput.make(64) |> Effect.provideMut<Allocator>(&mut allocator)
  let mut sink = CountingWriter { calls: 0, bytes: 0 }
  run BufferedOutput.writeAllWriter(&mut output, b"hello, ") |> Effect.provideMut<Writer>(&mut sink)
  run BufferedOutput.writeAllWriter(&mut output, b"world") |> Effect.provideMut<Writer>(&mut sink)

  // Both writes fit in the buffer, so the sink has received nothing yet.
  if sink.calls != 0 {
    return 1
  }
  run BufferedOutput.flushWriter(&mut output) |> Effect.provideMut<Writer>(&mut sink)
  if sink.calls != 1 || sink.bytes != 12 {
    return 1
  }
  return 0
}

Import as BufferedOutput with import silk.io.buffered_output { BufferedOutput }.

Public declarations: 2.

BufferedOutput#

pub struct BufferedOutput

A fixed-size buffer that holds output bytes until the transport receives them.

When to use#

Use it when a program makes many small writes to a byte transport and each transport write has a cost, such as a system call. Make the buffer with BufferedOutput.make, write with BufferedOutput.writeAll, and end with BufferedOutput.finish.

Each operation has two forms. The deadline form, such as writeAll, uses a silk.io.byte_duplex.ByteDuplex transport and a deadline. The Writer form, such as writeAllWriter, uses a silk.io.writer.Writer provider and has no deadline.

Details#

The buffer sends pending bytes to the transport only when it is full or when the caller flushes it. Input that is longer than the buffer is not copied: once the pending bytes are sent, it goes to the transport directly. A failed transport write or flush makes the buffer terminal. After that, each write and flush fails with BufferError.Terminal.

The buffer has one allocation of BufferedOutput.capacity bytes. The size does not change after BufferedOutput.make. BufferedOutput.pending gives the number of bytes that the buffer holds and the transport has not yet received.

Gotchas#

Dropping the buffer does not flush it. Pending bytes that are not flushed are lost. Call BufferedOutput.finish before the buffer goes out of scope.

Implementation BufferedOutput#

impl BufferedOutput

Associated function make#

pub effect fn make(capacity: usize) -> BufferedOutput
  ! BufferError | OutOfMemoryError
  ? &mut Allocator

Returns an empty buffer that holds at most capacity bytes.

Details#

This function makes one allocation of capacity bytes from the active Allocator. The buffer does no transport operation until the first write that does not fit or the first flush.

Errors#

• BufferError.InvalidCapacity if capacity is zero or more than silk.io.buffered_input.MAX_CAPACITY (1 MiB). The function does not allocate.

• OutOfMemoryError if the active Allocator cannot supply capacity bytes.

Method capacity#

pub fn capacity(self: &Self) -> usize

Returns the maximum number of bytes that the buffer can hold. This value does not change.

Method pending#

pub fn pending(self: &Self) -> usize

Returns the number of bytes that the buffer holds and the transport has not yet received.

Method isTerminal#

pub fn isTerminal(self: &Self) -> bool

Returns true if an earlier transport failure made the buffer unusable.

Details#

A failed transport write or flush makes the buffer terminal. A terminal buffer stays terminal. Each later write of one or more bytes and each later flush fails with BufferError.Terminal.

Method writeSome#

pub effect fn writeSome(
    self: &mut Self,
    input: &[u8],
    deadline: Option<Instant>,
  ) -> usize
  ! BufferError
  ? &mut ByteDuplex | &mut MonotonicClock

Copies the longest prefix of input that fits into the buffer and returns its length.

When to use#

Use writeSome when the caller can continue with a partial write, such as a loop that also does other work. Use BufferedOutput.writeAll to write all of input.

Details#

If the buffer is full, this function first sends all pending bytes to the active ByteDuplex transport. Then it copies input bytes into the free space. If input is longer than BufferedOutput.capacity, it does not copy: it gives the pending bytes and input to ByteDuplex.writeSomeVectored, or only input to ByteDuplex.writeSome when nothing is pending, until the transport accepts some input bytes, and returns their count. For a non-empty input, the result is at least 1. For an empty input, the result is 0 and the transport is not used.

Errors#

• BufferError.Terminal if an earlier failure made the buffer terminal.

• BufferError.WriteFailed if the transport fails or deadline passes. The buffer becomes terminal. drained gives the number of pending bytes that the transport received.

Method writeAll#

pub effect fn writeAll(
    self: &mut Self,
    input: &[u8],
    deadline: Option<Instant>,
  ) -> ()
  ! BufferError
  ? &mut ByteDuplex | &mut MonotonicClock

Copies all of input into the buffer and sends pending bytes to the transport when the buffer is full.

When to use#

Use writeAll for normal output through a ByteDuplex transport. Use BufferedOutput.writeAllWriter for a Writer provider.

Details#

When this function returns, the buffer owns all of input, or the transport already has them. The transport does not necessarily have the buffered bytes yet. Call BufferedOutput.flush to send them. A part of input that is longer than the buffer goes to the transport directly, without a copy. deadline applies to each transport write that this call makes.

Errors#

• BufferError.Terminal if an earlier failure made the buffer terminal.

• BufferError.WriteFailed if the transport fails or deadline passes. The buffer becomes terminal. accepted gives the number of input bytes that the buffer accepted before the failure. drained gives the number of pending bytes that the transport received.

Method writeVecAll#

pub effect fn writeVecAll(
    self: &mut Self,
    inputs: &[Bytes],
    deadline: Option<Instant>,
  ) -> ()
  ! BufferError
  ? &mut ByteDuplex | &mut MonotonicClock

Copies each element of inputs into the buffer in order, as one BufferedOutput.writeAll call for each element.

When to use#

Use writeVecAll to write a message that is in more than one Bytes value, such as a header and a body, without first joining the values.

Details#

Before it writes, this function adds the lengths of all elements and checks the sum for overflow. deadline applies to each transport write that this call makes.

Errors#

• BufferError.LengthOverflow if the sum of the element lengths overflows usize. The function writes nothing.

• BufferError.Terminal if an earlier failure made the buffer terminal.

• BufferError.WriteFailed if the transport fails or deadline passes. The buffer becomes terminal. accepted counts the accepted bytes of all elements together, not only the bytes of the element that failed.

Method flush#

pub effect fn flush(
    self: &mut Self,
    deadline: Option<Instant>,
  ) -> ()
  ! BufferError
  ? &mut ByteDuplex | &mut MonotonicClock

Sends all pending bytes to the transport and then flushes the transport.

When to use#

Use flush at a message boundary when the other side must receive the bytes now, such as before the program waits for a response.

Details#

This function sends pending bytes in order, one time each, and then calls ByteDuplex.flush. When it returns, BufferedOutput.pending is 0. deadline applies to each transport operation that this call makes.

Errors#

• BufferError.Terminal if an earlier failure made the buffer terminal.

• BufferError.WriteFailed if a transport write or the transport flush fails, or if deadline passes. The buffer becomes terminal. drained gives the number of pending bytes that the transport received.

Method finish#

pub effect fn finish(
    self: &mut Self,
    deadline: Option<Instant>,
  ) -> ()
  ! BufferError
  ? &mut ByteDuplex | &mut MonotonicClock

Sends all pending bytes to the transport and then flushes the transport.

When to use#

Use finish as the last operation on a buffer. The name shows that the output is complete. Dropping a buffer does not flush it. BufferedOutput.flush has the same behavior.

Details#

This function sends pending bytes in order, one time each, and then calls ByteDuplex.flush. When it returns, BufferedOutput.pending is 0. deadline applies to each transport operation that this call makes.

Errors#

• BufferError.Terminal if an earlier failure made the buffer terminal.

• BufferError.WriteFailed if a transport write or the transport flush fails, or if deadline passes. The buffer becomes terminal. drained gives the number of pending bytes that the transport received.

Method writeSomeWriter#

pub effect fn writeSomeWriter(self: &mut Self, input: &[u8]) -> usize
  ! BufferError
  ? &mut Writer

Accepts input into the buffer, or sends it to the active Writer together with the pending bytes, and returns the number of accepted bytes. This function has no deadline.

When to use#

Use writeSomeWriter when the destination is a Writer, such as standard output, and the caller can continue with a partial write. Use BufferedOutput.writeAllWriter to write all of input.

Details#

If input fits in the free part of the buffer, this function copies it. If it is shorter than the whole buffer, this function sends the pending bytes in one Writer.writeAll call and then copies input. Otherwise it does not copy: it sends the pending bytes and then input in one Writer.writeVectored call, and the buffer is empty after it. If no bytes are pending, it sends input in one Writer.writeAll call. Thus the result is the length of input in every case. A provider such as silk.io.writer_native.StdoutWriter sends a vectored write in one system call.

Errors#

• BufferError.Terminal if an earlier failure made the buffer terminal.

• BufferError.UnknownExternalTransfer if the Writer fails. The buffer becomes terminal. The destination can have received some of the pending bytes and of input.

Method writeAllWriter#

pub effect fn writeAllWriter(self: &mut Self, input: &[u8]) -> ()
  ! BufferError
  ? &mut Writer

Accepts all of input. This function uses the active Writer provider when input does not fit in the buffer.

When to use#

Use writeAllWriter to buffer many small writes to a Writer, such as standard output. Use BufferedOutput.writeAll for a ByteDuplex transport with a deadline.

Details#

When this function returns, the buffer owns all of input, or the Writer already has them. The Writer does not necessarily have the buffered bytes yet. Call BufferedOutput.flushWriter to send them. An input as long as the buffer or longer goes to the Writer without a copy, after the pending bytes, in one Writer.writeVectored call.

Errors#

• BufferError.Terminal if an earlier failure made the buffer terminal.

• BufferError.UnknownExternalTransfer if the Writer fails. The buffer becomes terminal. progress gives the number of input bytes that the buffer accepted before the failure. The destination can have received some of the pending bytes and of input.

Method flushWriter#

pub effect fn flushWriter(self: &mut Self) -> () ! BufferError ? &mut Writer

Sends all pending bytes to the active Writer in one call and then calls Writer.flush.

When to use#

Use flushWriter as the last operation on a buffer that writes to a Writer, and at each point where the destination must have the bytes now.

Details#

When this function returns, BufferedOutput.pending is 0.

Errors#

• BufferError.Terminal if an earlier failure made the buffer terminal.

• BufferError.UnknownExternalTransfer if Writer.writeAll or Writer.flush fails. The buffer becomes terminal. A Writer does not report partial progress. Thus the destination can have received some, all, or none of the pending bytes.

BufferedWriter#

pub struct BufferedWriter<W>

A Writer that owns a destination Writer and collects small writes into larger ones.

When to use#

Use BufferedWriter when code that requires &mut Writer makes many small writes, such as field-by-field output, and each write to the destination has a cost, such as a system call. Wrap the destination, such as silk.io.writer_native.StdoutWriter, with BufferedWriter.make and provide the result as the Writer. Use BufferedOutput when the caller keeps the destination itself or writes to a ByteDuplex.

Details#

Writer.writeAll copies small writes into the buffer and sends them when it is full, so the destination gets one write per fill. A write as long as the buffer or longer goes to the destination after the pending bytes, without a copy. Writer.flush sends the pending bytes and then flushes the destination. A failed write or flush of the destination makes the buffer terminal, as for BufferedOutput. Each later write or flush fails with a WriterError.

Gotchas#

Dropping a BufferedWriter does not flush it. Pending bytes that are not flushed are lost. Call Writer.flush or BufferedWriter.finish before the writer goes out of scope.

Examples#

import silk.mem.allocator { Allocator, OutOfMemoryError }
import silk.io.buffered_input { BufferError }
import silk.io.buffered_output { BufferedWriter }
import silk.core.effect { Effect }
import silk.io.writer_native { StdoutWriter }
import silk.io.writer { Writer, WriterError }

pub effect fn main() -> i32
  ! BufferError | OutOfMemoryError | WriterError {
  let mut allocator = Allocator.systemAllocatorProvider()
  let mut output = run BufferedWriter<StdoutWriter>.make(StdoutWriter.make(), 4096)
    |> Effect.provideMut<Allocator>(&mut allocator)
  run Writer.writeString("one ") |> Effect.provideMut<Writer>(&mut output)
  run Writer.writeString("write\n") |> Effect.provideMut<Writer>(&mut output)
  let stdout = run BufferedWriter<StdoutWriter>.finish(move output)
  drop stdout
  return 0
}

Implementation BufferedWriter<W>#

impl<W> BufferedWriter<W>

Associated function make#

pub effect fn make<W>(destination: W, capacity: usize) -> BufferedWriter<W>
  ! BufferError | OutOfMemoryError
  ? &mut Allocator

Returns a writer that sends its bytes to destination through a buffer of capacity bytes.

Errors#

• BufferError.InvalidCapacity if capacity is zero or more than silk.io.buffered_input.MAX_CAPACITY (1 MiB).

• OutOfMemoryError if the active Allocator cannot supply capacity bytes.

Method pending#

pub fn pending<W>(self: &Self) -> usize

Returns the number of bytes that the buffer holds and the destination has not yet received.

Method finish#

pub effect fn finish<W: Writer>(self: Self) -> W ! WriterError

Sends the pending bytes, flushes the destination, and returns it.

Errors#

• WriterError if an earlier failure made the buffer terminal, or if the destination fails. The destination is dropped.

Implementation Writer for BufferedWriter<W>#

impl<W: Writer> Writer for BufferedWriter<W>

Operation writeAll#

effect fn writeAll(self: &mut Self, val: &[u8]) -> () ! WriterError ? &mut Writer

Operation flush#

effect fn flush(self: &mut Self) -> () ! WriterError ? &mut Writer

Category: Input and output.

Profiles: aarch64-apple-darwin, aarch64-unknown-linux-gnu, aarch64-unknown-linux-gnu-no-libc, x86_64-unknown-linux-gnu, x86_64-unknown-linux-gnu-no-libc, wasm32-unknown-unknown, wasm32-wasip1, wasm32-wasip2.