Browse the docs

silk.text.string.

UTF-8 text: owned strings, validation of bytes, search, splitting, trimming, and traversal one Unicode scalar at a time.

When to use#

Use the built-in string type for borrowed text, such as a literal or a view into a buffer. Use String when the text must own its storage, for example to build text at runtime. Use silk.collections.bytes.Bytes for bytes that are not necessarily UTF-8.

Details#

A string and a String always hold valid UTF-8.

String.find, String.split, String.trim, and String.slice search and cut text without allocation. They return views of the input, and their offsets count UTF-8 bytes.

A failed String.append or String.appendOwned leaves the string unchanged. Traversal with String.nextScalar gives Unicode scalar values and byte offsets. It does not give grapheme clusters, so one visible character can be more than one scalar.

Gotchas#

A ScalarCursor is valid only for the string where the traversal started, and only while that string does not change. Start with String.scalarCursor and advance only with String.nextCursor.

Examples#

Check bytes as UTF-8#

silk
import silk.core.result { Result }
import silk.text.string { String, InvalidUtf8 }

pub fn main() -> i32 {
  let valid = String.fromUtf8(b"Silk") |> Result<string, InvalidUtf8>.unwrapOr("")
  let rejected = String.fromUtf8(b"a\x80") |> Result<string, InvalidUtf8>.unwrapOr("")
  if String.byteLength(valid) != 4 || String.byteLength(rejected) != 0 {
    return 1
  }
  return 0
}

Count the scalars of a string#

silk
import silk.core.option { Option }
import silk.text.string { String, ScalarStep }

pub fn main() -> i32 {
  // "né" has 2 scalars in 3 bytes.
  let text = "né"
  let mut count = 0
  let mut step = String.nextScalar(text, String.scalarCursor())
  while true {
    match move step {
      Option<ScalarStep>.Some { value } => {
        count = count + 1
        step = String.nextScalar(text, String.nextCursor(move value))
      }
      Option<ScalarStep>.None => {
        break
      }
    }
  }
  if count != 2 || String.byteLength(text) != 3 {
    return 1
  }
  return 0
}

Build an owned string#

silk
import silk.mem.allocator { Allocator, OutOfMemoryError }
import silk.core.effect { Effect }
import silk.text.string { String }

pub effect fn main() -> i32
  ! OutOfMemoryError {
  let mut allocator = Allocator.systemAllocatorProvider()
  let mut text = run String.copy("hello") |> Effect.provideMut<Allocator>(&mut allocator)
  run String.append(&mut text, ", world") |> Effect.provideMut<Allocator>(&mut allocator)
  if String.view(&text) != "hello, world" {
    return 1
  }
  return 0
}

Import as String with import silk.text.string { String }.

Public declarations: 5.

String#

silk
pub struct String

Owned UTF-8 text. Dropping a String releases its storage.

Details#

Get a borrowed string view with String.view. A String is affine. Use String.copy to make an independent copy of a view. A String always holds valid UTF-8. A failed String.append or String.appendOwned leaves the string unchanged.

The capacity is the number of UTF-8 bytes that fit before the next allocation. Use String.makeWithCapacity or String.reserve to allocate before a run of appends. String.clear keeps the storage for reuse, and String.clearAndFree releases it.

Gotchas#

Traversal with String.nextScalar gives Unicode scalar values, not grapheme clusters, so one visible character can be more than one scalar.

Implementation Debug for String#

silk
impl Debug for String

Operation debug#

silk
effect fn debug(self: &Self, formatter: &mut Formatter) -> () ! WriterError ? &mut Writer

Writes the text as a quoted string, like a string.

Implementation String#

silk
impl String

Associated function fromUtf8Unchecked#

silk
pub unsafe fn fromUtf8Unchecked(values: &[u8]) -> string

Returns values as a string view, with no UTF-8 check.

When to use#

Use this function only when earlier code proves that all of values is valid UTF-8, such as a buffer that this module made. Use String.fromUtf8 in other cases.

Gotchas#

This function is unsafe. If values is not valid UTF-8, the behavior of later text operations is not defined.

Associated function fromUtf8#

silk
pub fn fromUtf8(values: &[u8]) -> Result<string, InvalidUtf8>

Checks that values is valid UTF-8 and returns it as a string view. This function does not allocate.

Details#

The view borrows values and has the same lifetime.

Errors#

  • InvalidUtf8 if values is not valid UTF-8. offset gives the position of the first invalid byte.

Associated function make#

silk
pub fn make() -> String

Returns an empty String. This function does not allocate.

Associated function makeWithCapacity#

silk
pub effect fn makeWithCapacity(capacity: usize) -> String ! OutOfMemoryError ? &mut Allocator

Returns an empty String with storage for exactly capacity UTF-8 bytes.

When to use#

Use makeWithCapacity to build text with appends when its byte length is known or can be estimated, so that the string allocates once.

Details#

The capacity counts UTF-8 bytes, not characters. A capacity of 0 does not allocate.

Errors#

  • OutOfMemoryError if the Allocator cannot supply the storage.

Examples#

silk
import silk.mem.allocator { Allocator, OutOfMemoryError }
import silk.core.effect { Effect }
import silk.text.string { String }

pub effect fn main() -> i32
  ! OutOfMemoryError {
  let mut allocator = Allocator.systemAllocatorProvider()
  let mut text = run String.makeWithCapacity(32) |> Effect.provideMut<Allocator>(&mut allocator)
  run String.append(&mut text, "hello") |> Effect.provideMut<Allocator>(&mut allocator)
  String.clear(&mut text)
  if String.ownedByteLength(&text) != 0 || String.capacity(&text) != 32 {
    return 1
  }
  return 0
}

Method capacity#

silk
pub fn capacity(self: &Self) -> usize

Returns the number of UTF-8 bytes the storage holds before the next append allocates.

Method reserve#

silk
pub effect fn reserve(self: &mut Self, additional: usize) -> () ! OutOfMemoryError ? &mut Allocator

Makes sure that at least additional more UTF-8 bytes fit without another allocation, with room to spare for later appends. The text does not change.

When to use#

Use reserve before a run of appends whose total is not known. Use String.reserveExact when the final byte length is known.

Details#

If the capacity is already sufficient, this function does nothing. Otherwise, it grows the capacity by doubling until the bytes fit.

Errors#

  • OutOfMemoryError if the byte length plus additional overflows usize, or the Allocator cannot supply the storage. The string does not change.

Method reserveExact#

silk
pub effect fn reserveExact(self: &mut Self, additional: usize) -> () ! OutOfMemoryError ? &mut Allocator

Makes sure that at least additional more UTF-8 bytes fit, with no extra capacity. The text does not change.

Details#

If the capacity is already sufficient, this function does nothing and does not shrink the storage. Otherwise, it makes one allocation for exactly the byte length plus additional.

Errors#

  • OutOfMemoryError if the byte length plus additional overflows usize, or the Allocator cannot supply the storage. The string does not change.

Method shrinkToFit#

silk
pub effect fn shrinkToFit(self: &mut Self) -> () ! OutOfMemoryError ? &mut Allocator

Releases the capacity beyond the byte length.

Details#

If the capacity equals the byte length, this function does nothing. If the string is empty, it releases the storage. Otherwise, it moves the text into storage for exactly its byte length.

Errors#

  • OutOfMemoryError if the Allocator cannot supply the new storage. The string does not change. An empty string never fails.

Method clear#

silk
pub fn clear(self: &mut Self) -> ()

Removes all text. The storage stays allocated for reuse.

Details#

Later appends up to the old capacity do not allocate. Use String.clearAndFree to also release the storage.

Method clearAndFree#

silk
pub fn clearAndFree(self: &mut Self) -> ()

Removes all text and releases the storage. The byte length and the capacity become 0.

Details#

This function does not allocate and does not need an Allocator. Use String.clear to keep the storage for reuse.

Associated function fromBytes#

silk
pub fn fromBytes(values: Bytes) -> Result<String, InvalidUtf8>

Checks that values is valid UTF-8 and returns it as a String that keeps the same storage.

When to use#

Use fromBytes to turn an owned byte buffer, such as file contents, into owned text without a copy. Use String.fromUtf8 to borrow bytes without taking ownership.

Details#

This function consumes values. On success, the String keeps the allocation and capacity of values. This function does not allocate and does not copy.

Errors#

  • InvalidUtf8 if values is not valid UTF-8. offset gives the position of the first invalid byte. The storage of values is released.

Associated function fromBytesUnchecked#

silk
pub unsafe fn fromBytesUnchecked(values: Bytes) -> String

Returns values as a String that keeps the same storage, with no UTF-8 check.

When to use#

Use this function only when earlier code proves that all bytes of values are valid UTF-8. Use String.fromBytes in other cases.

Gotchas#

This function is unsafe. If values is not valid UTF-8, the behavior of later text operations is not defined.

Method intoBytes#

silk
pub fn intoBytes(self: Self) -> Bytes

Returns the storage of the string as Bytes, with no copy.

Details#

This function consumes the string. The result keeps the allocation, the capacity, and the UTF-8 contents. After the conversion, the bytes can change to any value.

Associated function copy#

silk
pub effect fn copy(value: string) -> String ! OutOfMemoryError ? &mut Allocator

Returns a new String that holds a copy of value.

Errors#

  • OutOfMemoryError if the Allocator cannot supply the storage.

Associated function copyUtf8#

silk
pub effect fn copyUtf8(values: &[u8]) -> Result<String, InvalidUtf8> ! OutOfMemoryError ? &mut Allocator

Checks that values is valid UTF-8 and returns a new String that holds a copy.

When to use#

Use copyUtf8 when the text must live longer than the buffer of values. Use String.fromUtf8 to borrow the bytes without allocation.

Errors#

  • InvalidUtf8 as the Result failure if values is not valid UTF-8. offset gives the position of the first invalid byte. The function does not allocate.

  • OutOfMemoryError if the Allocator cannot supply the storage.

Method append#

silk
pub effect fn append(self: &mut Self, value: string) -> () ! OutOfMemoryError ? &mut Allocator

Adds a copy of value at the end of the string.

When to use#

Use append for borrowed text. Use String.appendOwned when the suffix is an owned String.

Errors#

  • OutOfMemoryError if the Allocator cannot supply the storage. The string does not change.

Method appendOwned#

silk
pub effect fn appendOwned(self: &mut Self, value: String) -> () ! OutOfMemoryError ? &mut Allocator

Adds the text of value at the end of the string and consumes value.

When to use#

Use appendOwned when the suffix is an owned String that the caller does not need again. Use String.append for borrowed text.

Errors#

  • OutOfMemoryError if the Allocator cannot supply the storage. The string does not change, and value is dropped.

Method view#

silk
pub fn view(self: &Self) -> string

Returns a string view of the text. This function does not allocate or copy.

Associated function utf8Bytes#

silk
pub fn utf8Bytes(value: string) -> &[u8]

Returns the UTF-8 bytes of value as a shared slice.

Associated function byteLength#

silk
pub fn byteLength(value: string) -> usize

Returns the number of UTF-8 bytes in value. This is not the number of characters.

Associated function isCharBoundary#

silk
pub fn isCharBoundary(value: string, offset: usize) -> bool

Returns true if offset is a valid place to cut value: the start of a scalar, or the end of the text.

When to use#

Use isCharBoundary to check an offset before you give it to String.slice, when the offset does not come from a search such as String.find.

Details#

Offsets count UTF-8 bytes. Offset 0 and offset String.byteLength(value) are always boundaries. An offset after the end gives false.

Associated function slice#

silk
pub fn slice(value: string, offset: usize, length: usize) -> string

Returns the length bytes of value that start at byte offset, as a string view. This function does not allocate or copy.

When to use#

Use slice to cut text at offsets from String.find, String.findLast, or String.scalarByteOffset. Those offsets are always at the start of a scalar.

Details#

Offsets and lengths count UTF-8 bytes, not characters. The view borrows value and has the same lifetime. slice(value, String.byteLength(value), 0) is a valid empty view.

Gotchas#

The program traps if the range does not fit in value, or if offset or offset + length falls inside a multi-byte scalar. Check an offset of unknown origin with String.isCharBoundary first.

Examples#

Take a word out of a sentence#

silk
import silk.text.string { String }

pub fn main() -> i32 {
  // "é" is two bytes, so "café" ends at byte 5.
  let text = "café au lait"
  if String.slice(text, 0, 5) != "café" || String.slice(text, 6, 2) != "au" {
    return 1
  }
  return 0
}

Associated function startsWith#

silk
pub fn startsWith(value: string, prefix: string) -> bool

Returns true if value begins with prefix. Every text begins with the empty string.

Details#

The comparison is exact, byte for byte. It does not ignore case and does not normalize Unicode, so "é" written as one scalar does not begin with "e".

Associated function endsWith#

silk
pub fn endsWith(value: string, suffix: string) -> bool

Returns true if value ends with suffix. Every text ends with the empty string.

Details#

The comparison is exact, byte for byte. It does not ignore case and does not normalize Unicode.

Associated function contains#

silk
pub fn contains(value: string, needle: string) -> bool

Returns true if needle occurs anywhere in value. Every text contains the empty string.

Details#

The comparison is exact, byte for byte. It does not ignore case and does not normalize Unicode. The search compares sixteen bytes at a time on CPUs with 128-bit vectors.

Associated function find#

silk
pub fn find(value: string, needle: string) -> Option<usize>

Returns the byte offset of the first occurrence of needle in value, or None.

Details#

The offset counts UTF-8 bytes and is always the start of a scalar, so it can go straight to String.slice. An empty needle occurs at offset 0. The comparison is exact, byte for byte, with no case folding or Unicode normalization. The search compares sixteen bytes at a time on CPUs with 128-bit vectors and does not allocate.

Examples#

Read the value of a setting#

silk
import silk.core.option { Option }
import silk.text.string { String }

pub fn main() -> i32 {
  let line = "name = Silk"
  let at = match String.find(line, " = ") {
    Option<usize>.Some { value } => value
    Option<usize>.None => { return 1 }
  }
  let start = at + 3
  let value = String.slice(line, start, String.byteLength(line) - start)
  if at != 4 || value != "Silk" {
    return 1
  }
  return 0
}

Associated function findLast#

silk
pub fn findLast(value: string, needle: string) -> Option<usize>

Returns the byte offset of the last occurrence of needle in value, or None.

Details#

The offset counts UTF-8 bytes and is always the start of a scalar, so it can go straight to String.slice. An empty needle occurs at offset String.byteLength(value). The comparison is exact, byte for byte, with no case folding or Unicode normalization. This function does not allocate.

Examples#

Take the extension of a file name#

silk
import silk.core.option { Option }
import silk.text.string { String }

pub fn main() -> i32 {
  let name = "archive.tar.gz"
  let dot = match String.findLast(name, ".") {
    Option<usize>.Some { value } => value
    Option<usize>.None => { return 1 }
  }
  let extension = String.slice(name, dot + 1, String.byteLength(name) - dot - 1)
  if extension != "gz" {
    return 1
  }
  return 0
}

Associated function split#

silk
pub fn split<'text, 'separator>(value: string<'text>, separator: string<'separator>) -> Split<'text, 'separator>

Returns a Split that gives the pieces of value between occurrences of separator, one at a time.

When to use#

Use split to read fields separated by a comma, words separated by a space, or lines separated by "\n". Read the pieces with Split.next. Use Split.remainder to take the rest of the text in one piece, for example the value after the first =.

Details#

The pieces are views of value, with its lifetime, and this function does not allocate. A separator at the start or the end of value, or two separators in a row, give an empty piece, so "a,,b" split on "," gives "a", "", and "b", and an empty value gives one empty piece. A Split is Copy, so copying it starts a second, independent reading.

Gotchas#

An empty separator matches nowhere, so the only piece is all of value. Text with Windows line endings split on "\n" keeps a "\r" at the end of each line; String.trimEnd removes it.

Examples#

Add up comma-separated numbers#

silk
import silk.text.format { ParseError }
import silk.num.i32
import silk.core.option { Option }
import silk.core.result { Result }
import silk.text.string { String, Split }

pub fn main() -> i32 {
  let mut fields = String.split("4, 5,6", ",")
  let mut total = 0
  while true {
    let field = match Split.next(&mut fields) {
      Option<string>.Some { value } => value
      Option<string>.None => { break }
    }
    total = total + (i32.parse(String.trim(field)) |> Result<i32, ParseError>.unwrapOr(0))
  }
  if total != 15 {
    return 1
  }
  return 0
}

Associated function trim#

silk
pub fn trim(value: string) -> string

Returns value without the whitespace at its start and its end. This function does not allocate or copy.

Details#

Whitespace is every scalar with the Unicode White_Space property: the ASCII space, tab, line feed, vertical tab, form feed, and carriage return, and also U+0085, the no-break space U+00A0, U+1680, U+2000 to U+200A, U+2028, U+2029, U+202F, U+205F, and U+3000. Whitespace inside the text stays. The result is a view of value with the same lifetime.

Examples#

Clean up a line of input#

silk
import silk.text.string { String }

pub fn main() -> i32 {
  if String.trim("\t  hello world \r\n") != "hello world" || String.trim("   ") != "" {
    return 1
  }
  return 0
}

Associated function trimStart#

silk
pub fn trimStart(value: string) -> string

Returns value without the whitespace at its start. This function does not allocate or copy.

Details#

Whitespace is every scalar with the Unicode White_Space property: the ASCII space, tab, line feed, vertical tab, form feed, and carriage return, and also U+0085, the no-break space U+00A0, U+1680, U+2000 to U+200A, U+2028, U+2029, U+202F, U+205F, and U+3000. The result is a view of value with the same lifetime.

Associated function trimEnd#

silk
pub fn trimEnd(value: string) -> string

Returns value without the whitespace at its end. This function does not allocate or copy.

Details#

Whitespace is every scalar with the Unicode White_Space property: the ASCII space, tab, line feed, vertical tab, form feed, and carriage return, and also U+0085, the no-break space U+00A0, U+1680, U+2000 to U+200A, U+2028, U+2029, U+202F, U+205F, and U+3000. The result is a view of value with the same lifetime.

Method ownedUtf8Bytes#

silk
pub fn ownedUtf8Bytes(self: &Self) -> &[u8]

Returns the UTF-8 bytes of the string as a shared slice.

Method ownedByteLength#

silk
pub fn ownedByteLength(self: &Self) -> usize

Returns the number of UTF-8 bytes in the string. This is not the number of characters.

Associated function scalarCursor#

silk
pub fn scalarCursor() -> ScalarCursor

Returns a cursor at the start of a string, before the first scalar.

Details#

Give the cursor to String.nextScalar to read the first scalar.

Associated function cursorByteOffset#

silk
pub fn cursorByteOffset(cursor: &ScalarCursor) -> usize

Returns the byte position of cursor in its string.

Associated function scalarValue#

silk
pub fn scalarValue(step: &ScalarStep) -> char

Returns the Unicode scalar that step read.

Associated function scalarByteOffset#

silk
pub fn scalarByteOffset(step: &ScalarStep) -> usize

Returns the byte position where the scalar of step starts.

Associated function nextCursor#

silk
pub fn nextCursor(step: ScalarStep) -> ScalarCursor

Returns the cursor after the scalar of step, and consumes step.

Details#

Give the cursor to String.nextScalar to read the next scalar.

Associated function nextScalar#

silk
pub fn nextScalar(value: string, cursor: ScalarCursor) -> Option<ScalarStep>

Reads the scalar at cursor in value. Returns None at the end of the text.

Details#

A present ScalarStep holds the scalar, its byte position, and the cursor after it. Read them with String.scalarValue, String.scalarByteOffset, and String.nextCursor. This function does not allocate.

Gotchas#

cursor must come from String.scalarCursor or String.nextCursor for the same value. A cursor from a different string gives incorrect results.

InvalidUtf8#

silk
pub struct InvalidUtf8

The error value of a UTF-8 check: the position of the first invalid byte.

Field offset#

silk
pub offset: usize

The zero-based position of the first byte that is not valid UTF-8.

Implementation Debug for InvalidUtf8#

silk
impl Debug for InvalidUtf8

Operation debug#

silk
effect fn debug(self: &Self, formatter: &mut Formatter) -> () ! WriterError ? &mut Writer

Writes the fields, as InvalidUtf8 { offset: ... }.

ScalarCursor#

silk
pub struct ScalarCursor

A position in a string for traversal one scalar at a time.

Details#

Get the first cursor from String.scalarCursor and the next one from String.nextCursor.

ScalarStep#

silk
pub struct ScalarStep

One scalar that String.nextScalar read, with its byte position and the cursor after it.

Split#

silk
pub struct Split<'text, 'separator>

The pieces of a text between occurrences of a separator, read one at a time. Make one with String.split.

Details#

Each piece is a string view of the text that was split, with its lifetime. Read the pieces with Split.next, or take everything not yet read with Split.remainder. A Split does not allocate. It is Copy, so a copy reads the remaining pieces again on its own.

Implementation Copy for Split<'text, 'separator>#

silk
impl<'text, 'separator> Copy for Split<'text, 'separator>

Implementation Split<'text, 'separator>#

silk
impl<'text, 'separator> Split<'text, 'separator>

Method next#

silk
pub fn next(self: &mut Split<'text, 'separator>) -> Option<string<'text>>

Returns the next piece of the text, or None after the last piece.

Details#

A piece is the text up to the next separator, without the separator. The last piece is the text after the last separator, which is empty when the text ends with the separator. Text with no separator gives one piece: all of the text. The piece is a view with the lifetime of the text that was split. This function does not allocate.

Examples#

Read the parts of a path#

silk
import silk.core.option { Option }
import silk.text.string { String, Split }

pub fn main() -> i32 {
  let mut parts = String.split("usr/local/bin", "/")
  let mut count = 0
  let mut last = ""
  while true {
    match Split.next(&mut parts) {
      Option<string>.Some { value } => {
        count = count + 1
        last = value
      }
      Option<string>.None => { break }
    }
  }
  if count != 3 || last != "bin" {
    return 1
  }
  return 0
}

Method remainder#

silk
pub fn remainder(self: &mut Split<'text, 'separator>) -> Option<string<'text>>

Returns the text that Split.next has not given yet, in one piece, and ends the split.

When to use#

Use remainder to split only at the first separator: call Split.next once for the part before it, then remainder for everything after it, separators included.

Details#

After remainder, Split.next returns None. After the last piece, remainder returns None. The text is a view with the lifetime of the text that was split.

Examples#

Split a setting at its first =#

silk
import silk.core.option { Option }
import silk.text.string { String, Split }

pub fn main() -> i32 {
  let mut parts = String.split("query=a=b", "=")
  let key = Split.next(&mut parts) |> Option<string>.unwrapOr("")
  let value = Split.remainder(&mut parts) |> Option<string>.unwrapOr("")
  if key != "query" || value != "a=b" {
    return 1
  }
  return 0
}

Category: Text.

Profiles: aarch64-apple-darwin, aarch64-unknown-linux-gnu, aarch64-unknown-linux-gnu-no-libc, x86_64-unknown-linux-gnu, x86_64-unknown-linux-gnu-no-libc, wasm32-unknown-unknown, wasm32-wasip1, wasm32-wasip2.

View the source of this page