silk.text.unicode_tables.

When to use#

Use silk.text.unicode in application code. Use this module only to read the raw normalization data, for example in a custom text algorithm.

Details#

The data comes from Unicode 17.0.0. The tables are generated and stored as packed byte strings. The compiler gives no special treatment to this module. A later generation can change the table layout.

Gotchas#

A lookup result of 0 means "no value". It is not the scalar U+0000. Hangul syllables decompose and compose by a formula, so they are not in these tables.

Examples#

Read the canonical decomposition of é#

import silk.text.unicode_tables { UnicodeTables }

pub fn main() -> i32 {
  // U+00E9 decomposes to U+0065 and U+0301.
  let decomposition = UnicodeTables.canonicalDecomposition(233)
  if decomposition.first != 101 || decomposition.second != 769 {
    return 1
  }
  return 0
}

Import as UnicodeTables with import silk.text.unicode_tables { UnicodeTables }.

Public declarations: 3.

UnicodeTables#

pub struct UnicodeTables

The owner type of the table lookups. It has no fields and is never constructed.

Implementation UnicodeTables#

impl UnicodeTables

Associated function dataVersion#

pub fn dataVersion() -> string<'static>

Returns the Unicode version of the table data, such as "17.0.0".

Associated function maximumDecompositionLength#

pub fn maximumDecompositionLength() -> usize

Returns the maximum number of scalars in the full canonical decomposition of one scalar.

Associated function combiningClass#

pub fn combiningClass(scalar: u32) -> u32

Returns the canonical combining class of scalar. Returns 0 for a starter or an unassigned code point.

Associated function canonicalDecomposition#

pub fn canonicalDecomposition(scalar: u32) -> Decomposition

Returns the canonical decomposition of scalar, as one or two scalars.

Gotchas#

If scalar has no decomposition in the table, both fields of the result are 0. Hangul syllables are not in the table.

Associated function canonicalComposition#

pub fn canonicalComposition(first: u32, second: u32) -> u32

Returns the primary composite of the pair first, second. Returns 0 if the pair does not compose.

Details#

Pairs that Unicode excludes from composition are not in the table.

Gotchas#

Hangul syllables are not in the table.

Decomposition#

pub struct Decomposition

The canonical decomposition of one scalar. A field of 0 means "no scalar".

Field first#

pub first: u32

The first scalar of the decomposition, or 0 if the scalar has no decomposition.

Field second#

pub second: u32

The second scalar of the decomposition, or 0 if the decomposition has one scalar.

PairRecord#

pub struct PairRecord

One table record: a scalar and its two-scalar canonical decomposition.

Field composed#

pub composed: u32

The scalar that decomposes to first and second.

Field first#

pub first: u32

The first scalar of the decomposition.

Field second#

pub second: u32

The second scalar of the decomposition.

Category: Text.

Profiles: aarch64-apple-darwin, aarch64-unknown-linux-gnu, aarch64-unknown-linux-gnu-no-libc, x86_64-unknown-linux-gnu, x86_64-unknown-linux-gnu-no-libc, wasm32-unknown-unknown, wasm32-wasip1, wasm32-wasip2.