silk.text.script_itemization.

When to use#

Use scripts before font shaping, which needs one script per run, or to find the scripts that a text uses.

Details#

Common and Inherited characters, such as spaces, digits and combining marks, join the run around them. Script_Extensions sets are intersected across a run, combining marks stay with their base, and a closing bracket takes the script of its opening bracket. The runs are byte ranges of the text, appended to a caller-owned list. Only ambiguous script sets and open brackets use scratch storage.

Examples#

Split mixed Latin and Greek text#

import silk.collections.list { List }
import silk.core.effect { Effect }
import silk.core.option { Option }
import silk.mem.allocator { Allocator, OutOfMemoryError }
import silk.text.script_itemization
import silk.text.script_itemization { ScriptRun }
import silk.text.unicode_script { Script }

pub effect fn main() -> i32
  ! OutOfMemoryError {
  let mut allocator = Allocator.systemAllocatorProvider()
  let mut runs = List<ScriptRun>.make()
  run script_itemization.scripts("abc \u{3b1}\u{3b2}\u{3b3}", &mut runs)
    |> Effect.provideMut<Allocator>(&mut allocator)
  let greek = Script.parse("Grek") |> Option<Script>.unwrapOr(Script.unknown())
  // The space joins the Latin run, so the Greek run starts at byte 4.
  let items = runs.asSlice()
  if items.length != 2 || items[1].start != 4 || &items[1].script != &greek {
    return 1
  }
  return 0
}

Import as script_itemization with import silk.text.script_itemization as script_itemization.

Public declarations: 2.

ScriptRun#

pub struct ScriptRun

A maximal range of a text in one script, as scripts gives it.

Field start#

pub start: usize

The byte offset where the run starts.

Field end#

pub end: usize

The byte offset where the run ends.

Field script#

pub script: Script

The script of the run. It is never Zyyy (Common) or Zinh (Inherited) unless the whole text has no other script.

Implementation Copy for ScriptRun#

impl Copy for ScriptRun

scripts#

pub effect fn scripts(text: string, output: &mut List<ScriptRun>) -> ()
  ! OutOfMemoryError
  ? &mut Allocator

Appends the script runs of text to output.

When to use#

Use scripts to split text into runs that a font shaper can shape with one script each.

Details#

The runs follow UAX #24. They are not empty, they cover the text in order without gaps, and empty text gives none. Common and Inherited characters join the run around them: leading ones take the following script, others the preceding one. The Script_Extensions sets of a run are intersected before its script is chosen; an ambiguous set prefers the primary script of its first character, then the lowest ISO 15924 tag. Combining marks stay with their base, and a closing bracket takes the script of its opening bracket. The function does not clear output, so clear it first to reuse its storage.

Errors#

• OutOfMemoryError if the Allocator cannot grow output or the scratch storage for brackets and ambiguous sets.

Gotchas#

Quotation marks whose pairing depends on the language are not treated as brackets.

Category: Text.

Profiles: aarch64-apple-darwin, aarch64-unknown-linux-gnu, aarch64-unknown-linux-gnu-no-libc, x86_64-unknown-linux-gnu, x86_64-unknown-linux-gnu-no-libc, wasm32-unknown-unknown, wasm32-wasip1, wasm32-wasip2, x86_64-pc-windows-msvc, wasm64-unknown-unknown.