silk.toml.scanner.

When to use#

Most programs use silk.toml.value, which calls this scanner for them. Use TomlScanner directly to check that a document is valid TOML without allocating, or to read tokens and their source spans, for example in an editor tool.

Details#

TomlScanner.validate checks the full syntax of TOML 1.1.0: UTF-8, keys, strings, numbers, dates and times, arrays, inline tables, and headers. It does not check that each key is defined one time, because that needs a table tree. TomlScanner.next returns one token at a time. Tokens hold a TomlSpan into the input, not a copy, and do not decode strings. The scanner does not allocate.

Errors#

The functions return a Result with a TomlError. offset is the byte position of the first problem in the input.

Gotchas#

TomlScanner.next does not check that the tokens make a valid document. Call TomlScanner.validate first. validate does not move the position of the scanner.

Examples#

Validate a document and count its keys#

import silk.core.result { Result }
import silk.toml.scanner { TomlError, TomlScanner, TomlToken }

pub fn main() -> i32 {
  let mut scanner = TomlScanner.make(b"name = \"Ada\"\nyear = 1815\n")
  match TomlScanner.validate(&mut scanner) {
    Result<bool, TomlError>.Success { value: _ } => {}
    Result<bool, TomlError>.Failure { error: _ } => {
      return 1
    }
  }
  let mut keys = 0
  let mut atLineStart = true
  while true {
    match TomlScanner.next(&mut scanner) {
      Result<TomlToken, TomlError>.Success { value } => match move value {
        TomlToken.EndOfInput => {
          break
        }
        TomlToken.Newline => {
          atLineStart = true
        }
        TomlToken.Text { span: _ } => {
          if atLineStart {
            keys = keys + 1
          }
          atLineStart = false
        }
        _ => {
          atLineStart = false
        }
      }
      Result<TomlToken, TomlError>.Failure { error: _ } => {
        return 1
      }
    }
  }
  if keys != 2 {
    return 1
  }
  return 0
}

Find the position of a syntax error#

import silk.core.result { Result }
import silk.toml.scanner { TomlError, TomlReason, TomlScanner }

pub fn main() -> i32 {
  let mut scanner = TomlScanner.make(b"port = 80x\n")
  return match TomlScanner.validate(&mut scanner) {
    Result<bool, TomlError>.Failure {
      error: TomlError { reason: TomlReason.InvalidNumber, offset }
    } => {
      if offset != 7 {
        return 1
      }
      return 0
    }
    _ => 1
  }
}

Import as TomlScanner with import silk.toml.scanner { TomlScanner }.

Public declarations: 7.

MAX_DEPTH#

pub const MAX_DEPTH: usize

The maximum number of arrays and inline tables that can be nested in one value.

TomlSpan#

pub struct TomlSpan

A range of bytes in the input: length bytes from offset.

Field offset#

pub offset: usize

The position of the first byte, from the start of the input.

Field length#

pub length: usize

The number of bytes.

Implementation Copy for TomlSpan#

impl Copy for TomlSpan

Implementation Copy for TomlError#

impl Copy for TomlError

TomlReason#

pub enum TomlReason

The cause of a TomlError.

UnexpectedByte#

UnexpectedByte

A byte that TOML does not allow at this position, such as a missing = or an unexpected bracket.

UnexpectedEnd#

UnexpectedEnd

The input ends before a string, array, inline table, or header is closed.

InvalidUtf8#

InvalidUtf8

The input is not valid UTF-8.

InvalidString#

InvalidString

A string is not closed, or holds a control character that must be escaped.

InvalidEscape#

InvalidEscape

A basic string has an escape that TOML does not define, or a \u or \U escape that is not a Unicode scalar value.

InvalidNumber#

InvalidNumber

An unquoted value is not a valid integer, float, or boolean.

InvalidDateTime#

InvalidDateTime

A date or time has a field out of range or the wrong form.

TypeMismatch#

TypeMismatch

A silk.toml.codec conversion found a value of a different kind.

NumberOutOfRange#

NumberOutOfRange

A silk.toml.codec conversion found a valid number that does not fit the target type.

DuplicateKey#

DuplicateKey

A key is defined two times in one table.

TableConflict#

TableConflict

A table definition conflicts with an earlier value or table, for example a second [a] header or a header inside an inline table. silk.toml.output also uses it for a tree that it cannot write.

DepthExceeded#

DepthExceeded

More than MAX_DEPTH arrays or inline tables are nested.

Implementation Debug for TomlReason#

impl Debug for TomlReason

Operation debug#

effect fn debug(self: &Self, formatter: &mut Formatter) -> () ! WriterError ? &mut Writer

Writes the name of the variant, such as UnexpectedByte.

TomlError#

pub struct TomlError

A TOML error: the cause and the byte position where it occurred.

Field reason#

pub reason: TomlReason

The cause of the error.

Field offset#

pub offset: usize

The byte position of the problem in the input. Errors that are not tied to a source position, such as codec errors, use 0.

Implementation Debug for TomlError#

impl Debug for TomlError

Operation debug#

effect fn debug(self: &Self, formatter: &mut Formatter) -> () ! WriterError ? &mut Writer

Writes the fields, as TomlError { reason: ..., offset: ... }.

TomlScalarKind#

pub enum TomlScalarKind

The kind of an unquoted value, from TomlScanner.scalarKind.

Integer#

Integer

A decimal integer, or a hexadecimal (0x), octal (0o), or binary (0b) integer.

Float#

Float

A float with a fraction or an exponent, or inf or nan with an optional sign.

Boolean#

Boolean

true or false.

OffsetDateTime#

OffsetDateTime

A date and time with a UTC offset, such as 1979-05-27T07:32:00Z.

LocalDateTime#

LocalDateTime

A date and time without an offset.

LocalDate#

LocalDate

A calendar date.

LocalTime#

LocalTime

A time of day.

TomlToken#

pub union TomlToken

One token of TOML source. Use TomlScanner.slice to get the bytes of a span.

Text#

Text

A bare key or an unquoted value, such as name, 42, or 1979-05-27 07:32:00.

Dots stay in the token, so a dotted bare key such as a.b and a float such as 3.14 are each one token. In a value position, use TomlScanner.scalarKind to find the kind of value.

Field span#

pub span: TomlSpan

The source bytes of the token.

String#

String

A quoted key or string. The span includes the quotes, and escapes are not decoded. Use silk.toml.text to decode it.

Field span#

pub span: TomlSpan

The source bytes of the token, with the quotes.

Field literal#

pub literal: bool

true for a literal string, which uses ' quotes and has no escapes.

Field multiline#

pub multiline: bool

true for a string with three quotes at each end.

Equals#

Equals

The = between a key and its value.

Dot#

Dot

A . at the start of a token, such as the one after the quoted part of "a".b. A dot inside a bare spelling stays in TomlToken.Text.

Comma#

Comma

The , between array elements or inline table members.

ArrayBegin#

ArrayBegin

A [. It starts an array or a header.

ArrayEnd#

ArrayEnd

A ]. It ends an array or a header.

InlineBegin#

InlineBegin

A {. It starts an inline table.

InlineEnd#

InlineEnd

A }. It ends an inline table.

Newline#

Newline

The end of a line: LF or CRLF. Comments are skipped.

EndOfInput#

EndOfInput

The end of the input. Each later call returns it again.

Implementation Copy for TomlToken#

impl Copy for TomlToken

TomlScanner#

pub struct TomlScanner<'input>

A scanner that borrows the complete source of one TOML document. It does not allocate.

When to use#

Most programs use silk.toml.value, which calls this scanner for them. Use it directly to check that a document is valid TOML without allocating, or to read tokens and their source spans, for example in an editor tool.

Details#

TomlScanner.validate checks the full syntax of TOML 1.1.0, but not that each key is defined one time. TomlScanner.next returns one token at a time. Tokens hold a TomlSpan into the input, not a copy, and do not decode strings.

Gotchas#

TomlScanner.next does not check that the tokens make a valid document. Call TomlScanner.validate first.

Implementation TomlScanner<'input>#

impl<'input> TomlScanner<'input>

Associated function make#

pub fn make(input: &'input [u8]) -> TomlScanner<'input>

Returns a scanner at the start of input.

Method validate#

pub fn validate(self: &mut TomlScanner) -> Result<bool, TomlError>

Checks the syntax of the full input as a TOML 1.1.0 document. It does not move the scanner.

Returns Success(true) for valid syntax. It does not check that each key is defined one time. silk.toml.value does that when it builds the tree.

Errors#

• TomlError for the first syntax problem in the input.

Method next#

pub fn next(self: &mut TomlScanner) -> Result<TomlToken, TomlError>

Returns the next token and moves past it. Spaces, tabs, and comments are skipped.

After the last token, it returns TomlToken.EndOfInput on each call.

Errors#

• TomlError for a string that is not closed, a bad comment, or a lone carriage return. next does not check the full syntax. Call TomlScanner.validate first.

Method scalarKind#

pub fn scalarKind(self: &TomlScanner, span: TomlSpan) -> Option<TomlScalarKind>

Returns the kind of the unquoted value in span, or None if the bytes are not a valid integer, float, boolean, date, or time.

Method slice#

pub fn slice(self: &TomlScanner<'input>, span: TomlSpan) -> &'input [u8]

Returns the source bytes of span. They borrow from the input, not from the scanner.

Method offset#

pub fn offset(self: &TomlScanner) -> usize

Returns the byte position where the next call to TomlScanner.next starts.

Category: Data formats.

Profiles: aarch64-apple-darwin, aarch64-unknown-linux-gnu, aarch64-unknown-linux-gnu-no-libc, x86_64-unknown-linux-gnu, x86_64-unknown-linux-gnu-no-libc, wasm32-unknown-unknown, wasm32-wasip1, wasm32-wasip2.