silk.schema.msgpack.
Read and write Schema types as MessagePack.
When to use#
Call SchemaMsgpack.decode to read a value of a silk.schema.schema.Schema type from
MessagePack bytes, SchemaMsgpack.decodeAll to report every problem in the bytes at once, and
SchemaMsgpack.encode to write a value to the ambient Writer. Use MessagePack instead of
silk.schema.json.SchemaJson when the reader is a program and size or binary data matters. Use
silk.msgpack.value.Msgpack for MessagePack whose shape is not known in advance.
Details#
A struct is a map with one text key per field, in declaration order; a union follows the same
tag rules as in JSON. An Option that is None is nil, and a None field is left out. A
silk.collections.bytes.Bytes field is MessagePack binary data, not Base64 text. Integers are
written in the shortest format and read from any width, and a float 32 is read as f64.
Decoding streams over the bytes with silk.msgpack.scanner.MsgpackScanner and builds the value
directly, with no tree in between. Each map is scanned once into an index of its entries, so
entries may come in any order, and entries the type does not name are skipped, or reported when
its spec is Spec.strict. Every problem carries the span of the bytes it is about.
Gotchas#
A map with a key that is not text cannot be read as a struct. An extension value, such as a
timestamp (type -1), has no schema type and is reported as IssueReason.InvalidType where a
field reads it; it is skipped where no field does. Nesting is limited to 64 levels.
silk.schema.report.SchemaReport shows problems as lines and columns of text, which MessagePack
does not have. Show a problem by its path and by the byte offsets of its issue's span.
Examples#
Write a struct and read it back#
import silk.collections.bytes { Bytes }
import silk.collections.slice { Slice }
import silk.core.effect { Effect }
import silk.io.writer { Writer, WriterError }
import silk.mem.allocator { Allocator, OutOfMemoryError }
import silk.schema.issue { SchemaError }
import silk.schema.msgpack { SchemaMsgpack }
import silk.schema.schema { Schema }
struct Capture {
bytes: [u8; 64],
length: usize,
}
impl Writer for Capture {
effect fn writeAll(self: &mut Self, value: &[u8]) -> () ! WriterError ? &mut Writer {
if self.length + value.length > 64 { fail Writer.failure() }
Slice.copyInto<u8>(&mut self.bytes, self.length, value)
self.length = self.length + value.length
return ()
}
effect fn flush(self: &mut Self) -> () ! WriterError ? &mut Writer { return () }
}
struct Frame {
pub id: u32,
pub payload: Bytes,
}
impl Schema for Frame {}
effect fn check() -> i32 ! SchemaError | WriterError | OutOfMemoryError ? &mut Allocator {
let frame = Frame { id: 7, payload: run Bytes.copy(b"\xff\x00") }
let mut capture = Capture { bytes: [0; 64], length: 0 }
run SchemaMsgpack.encode<Frame>(&frame) |> Effect.provideMut<Writer>(&mut capture)
let written = Slice.view<u8>(&capture.bytes, 0, capture.length)
// {"id": 7, "payload": bin(ff 00)}
if written != b"\x82\xa2id\x07\xa7payload\xc4\x02\xff\x00" {
return 1
}
let again = run SchemaMsgpack.decode<Frame>(written)
if again.id != 7 || again.payload.asSlice() != b"\xff\x00" {
return 2
}
return 0
}
pub effect fn main() -> i32 ! SchemaError | WriterError | OutOfMemoryError {
let mut allocator = Allocator.systemAllocatorProvider()
return run check() |> Effect.provideMut<Allocator>(&mut allocator)
}Import as SchemaMsgpack with import silk.schema.msgpack { SchemaMsgpack }.
Public declarations: 3.
SchemaMsgpack#
pub struct SchemaMsgpackThe owner of the MessagePack entry points. It has no fields and is never constructed.
Implementation SchemaMsgpack#
impl SchemaMsgpackAssociated function decode#
pub effect fn decode<T: Schema>(bytes: &[u8]) -> T
! SchemaError | OutOfMemoryError
? &mut AllocatorReads one value of type T from the MessagePack bytes, stopping at the first problem.
Details#
The whole input must be exactly one MessagePack value. A struct is read from a map with text
keys, in any order; integers are read from any width, and Bytes from binary data.
Errors#
SchemaErrorif the bytes are not valid MessagePack, hold more than one value, or do not hold aT. Itspathnames the part that failed, and its issue has the span of the bytes it is about, as byte offsets intobytes.OutOfMemoryErrorif theAllocatorcannot supply the value.
Associated function decodeAll#
pub effect fn decodeAll<T: Schema>(bytes: &[u8]) -> T
! SchemaErrors | OutOfMemoryError
? &mut AllocatorReads one value of type T from the MessagePack bytes, reporting every problem.
Details#
A field, element or variant that fails is recorded and skipped, and reading goes on with the
next one, so one call lists every problem, in input order: a missing key, a wrong type, a
number out of range and a failed rule alike. Bytes that are not one valid MessagePack value
are the one problem reported. At most MAX_PROBLEMS (64) are listed.
Errors#
SchemaErrorswith every problem found.OutOfMemoryErrorif theAllocatorcannot supply the value.
Associated function encode#
pub effect fn encode<T: Schema>(value: &T) -> ()
! SchemaError | WriterError | OutOfMemoryError
? &mut Writer | &mut AllocatorWrites value as MessagePack to the ambient Writer, without flushing it.
Details#
A struct is written as a map with one text key per field, in declaration order, without its
None fields. Every value takes the shortest format that holds it, and a float is always a
float 64. Nothing is buffered: each value goes to the Writer as it is reached.
Errors#
SchemaErrorif MessagePack cannot hold the value: text, bytes or a list longer than 4294967295, or a value nested more than 64 levels deep. Its path is empty.WriterErrorif theWriterfails. Bytes that it accepted before the failure stay written.OutOfMemoryErrorif a codec needs memory that theAllocatorcannot supply.
MsgpackSchemaReader#
pub struct MsgpackSchemaReader<'input>A SchemaReader over MessagePack bytes.
When to use#
SchemaMsgpack.decode and SchemaMsgpack.decodeAll use it. Use it directly with
silk.schema.schema.Schemas.read to read a value that more bytes follow, which
SchemaMsgpack.decode refuses.
Implementation MsgpackSchemaReader<'input>#
impl<'input> MsgpackSchemaReader<'input>Associated function make#
pub fn make(input: &'input [u8]) -> MsgpackSchemaReader<'input>Returns a reader at the start of input that stops at the first problem.
Associated function collectingAll#
pub fn collectingAll(input: &'input [u8]) -> MsgpackSchemaReader<'input>Returns a reader at the start of input that records every problem instead of stopping at
the first. Read them with MsgpackSchemaReader.intoErrors.
Gotchas#
A collecting read goes on after a failed part by reading past it, so input must be valid
MessagePack; SchemaMsgpack.decodeAll checks that first.
Method intoErrors#
pub fn intoErrors(self: Self) -> List<SchemaError>Returns the problems the reader recorded, in the order they are in the input.
Implementation MsgpackSchemaReader<'input>#
impl<'input> MsgpackSchemaReader<'input>Implementation SchemaReader for MsgpackSchemaReader<'input>#
impl<'input> SchemaReader for MsgpackSchemaReader<'input>Operation path#
fn path(self: &mut Self) -> &mut SchemaPathReturns the path of the part being read.
Operation report#
effect fn report(self: &mut Self, issue: Issue) -> ()
! Issue | OutOfMemoryError
? &mut AllocatorFails with the issue, or records it and returns when the reader collects every problem.
Operation collecting#
fn collecting(self: &Self) -> boolReturns whether the reader records every problem.
Operation lastSpan#
fn lastSpan(self: &Self) -> Option<Span>Returns the span of the last complete value read.
Operation lastKind#
fn lastKind(self: &Self) -> ValueKindReturns the kind of the last complete value read.
Operation mark#
effect fn mark(self: &mut Self) -> usize
! OutOfMemoryError
? &mut AllocatorRemembers the open maps and arrays and the scanner's position.
Operation recover#
effect fn recover(self: &mut Self, mark: usize, skip: bool) -> ()
! Issue | OutOfMemoryError
? &mut AllocatorReturns to the mark, closing the maps and arrays opened since, and reads past the value there
when skip is true.
Operation release#
fn release(self: &mut Self, mark: usize) -> ()Forgets the mark and every later one.
Operation boolean#
effect fn boolean(self: &mut Self) -> bool
! Issue | OutOfMemoryError
? &mut AllocatorReads true or false.
Operation signed#
effect fn signed(self: &mut Self) -> i64
! Issue | OutOfMemoryError
? &mut AllocatorReads an integer of any width as an i64. A float is not an integer, even a whole one.
Operation unsigned#
effect fn unsigned(self: &mut Self) -> u64
! Issue | OutOfMemoryError
? &mut AllocatorReads an integer of any width as a u64. A float is not an integer, even a whole one.
Operation float#
effect fn float(self: &mut Self) -> f64
! Issue | OutOfMemoryError
? &mut AllocatorReads a float, or an integer converted to the nearest f64.
Operation text#
effect fn text(self: &mut Self) -> String
! Issue | OutOfMemoryError
? &mut AllocatorReads a MessagePack string. Binary data is not text.
Operation bytes#
effect fn bytes(self: &mut Self) -> Bytes
! Issue | OutOfMemoryError
? &mut AllocatorReads MessagePack binary data. A string is not binary data, and is not read as Base64.
Operation null#
effect fn null(self: &mut Self) -> bool
! Issue | OutOfMemoryError
? &mut AllocatorReads past nil and returns true, or returns false for another value.
Operation beginList#
effect fn beginList(self: &mut Self) -> ()
! Issue | OutOfMemoryError
? &mut AllocatorReads an array header.
Operation nextElement#
effect fn nextElement(self: &mut Self) -> bool
! Issue | OutOfMemoryError
? &mut AllocatorReturns true at the next element, or closes the array and returns false after the last.
Operation beginObject#
effect fn beginObject(self: &mut Self) -> ()
! Issue | OutOfMemoryError
? &mut AllocatorReads the map's entries into an index once, so member finds each without scanning.
Errors#
IssuewithIssueReason.InvalidTypefor a value that is not a map, or for a map with a key that is not text.
Operation member#
effect fn member(self: &mut Self, key: string) -> bool
! Issue | OutOfMemoryError
? &mut AllocatorFinds the entry key in the map's index and leaves the reader at its value; the first one
when the map repeats the key. A missing key leaves the map's header as the last span.
Operation nextMember#
effect fn nextMember(self: &mut Self) -> Option<String>
! Issue | OutOfMemoryError
? &mut AllocatorReturns the key of the next entry no member or nextMember call found, and leaves the
reader at its value. Returns None after every entry has been visited.
Operation rejectUnread#
effect fn rejectUnread(self: &mut Self, expected: string<'static>) -> ()
! Issue | OutOfMemoryError
? &mut AllocatorReports every entry no member call found, at its key.
Operation endObject#
effect fn endObject(self: &mut Self) -> ()
! Issue | OutOfMemoryError
? &mut AllocatorJumps past the whole map being read.
MsgpackSchemaWriter#
pub struct MsgpackSchemaWriterA SchemaWriter that writes MessagePack to the ambient Writer.
When to use#
SchemaMsgpack.encode uses it. Use it directly to write a value inside MessagePack that the
program writes itself with silk.msgpack.output.MsgpackOutput.
Details#
It holds no bytes: every call writes straight to the Writer. An array or map header carries
the count the schema gives beginList and beginObject.
Implementation MsgpackSchemaWriter#
impl MsgpackSchemaWriterAssociated function make#
pub fn make() -> MsgpackSchemaWriterReturns a writer for one top-level value.
Implementation SchemaWriter for MsgpackSchemaWriter#
impl SchemaWriter for MsgpackSchemaWriterOperation boolean#
effect fn boolean(self: &mut Self, value: bool) -> ()
! Issue | WriterError
? &mut WriterWrites true or false in one byte.
Operation signed#
effect fn signed(self: &mut Self, value: i64) -> ()
! Issue | WriterError
? &mut WriterWrites the integer in the shortest format that holds it.
Operation unsigned#
effect fn unsigned(self: &mut Self, value: u64) -> ()
! Issue | WriterError
? &mut WriterWrites the integer in the shortest format that holds it.
Operation float#
effect fn float(self: &mut Self, value: f64) -> ()
! Issue | WriterError
? &mut WriterWrites the number as a float 64, including NaN and the infinities.
Operation text#
effect fn text(self: &mut Self, value: string) -> ()
! Issue | WriterError
? &mut WriterWrites a MessagePack string.
Errors#
IssuewithIssueReason.InvalidValueifvalueis longer than 4294967295 bytes.WriterErrorif theWriterfails.
Operation bytes#
effect fn bytes(self: &mut Self, value: &[u8]) -> ()
! Issue | WriterError | OutOfMemoryError
? &mut Writer | &mut AllocatorWrites MessagePack binary data, not Base64 text.
Errors#
IssuewithIssueReason.InvalidValueifvalueis longer than 4294967295 bytes.WriterErrorif theWriterfails.
Operation null#
effect fn null(self: &mut Self) -> ()
! Issue | WriterError
? &mut WriterWrites nil.
Operation beginList#
effect fn beginList(self: &mut Self, count: usize) -> ()
! Issue | WriterError
? &mut WriterWrites the header of an array of count elements.
Errors#
IssuewithIssueReason.InvalidValueifcountis above 4294967295, or if 64 arrays and maps are already open.WriterErrorif theWriterfails.
Operation element#
effect fn element(self: &mut Self) -> ()
! Issue | WriterError
? &mut WriterWrites nothing: MessagePack has no separator between elements.
Operation endList#
effect fn endList(self: &mut Self) -> ()
! Issue | WriterError
? &mut WriterWrites nothing: the array's header said how many elements it has.
Operation beginObject#
effect fn beginObject(self: &mut Self, count: usize) -> ()
! Issue | WriterError
? &mut WriterWrites the header of a map of count entries.
Errors#
IssuewithIssueReason.InvalidValueifcountis above 4294967295, or if 64 arrays and maps are already open.WriterErrorif theWriterfails.
Operation key#
effect fn key(self: &mut Self, name: string) -> ()
! Issue | WriterError
? &mut WriterWrites the key as a MessagePack string. Its value comes next.
Operation endObject#
effect fn endObject(self: &mut Self) -> ()
! Issue | WriterError
? &mut WriterWrites nothing: the map's header said how many entries it has.
Category: Data formats.
Profiles: aarch64-apple-darwin, aarch64-unknown-linux-gnu, aarch64-unknown-linux-gnu-no-libc, x86_64-unknown-linux-gnu, x86_64-unknown-linux-gnu-no-libc, wasm32-unknown-unknown, wasm32-wasip1, wasm32-wasip2, x86_64-pc-windows-msvc, wasm64-unknown-unknown.