Module ffi 

The C ABI surface of pith-text: the entry points the Python (ctypes), Node (koffi) and Go (cgo) SDKs bind through.

The suite’s FFI convention, defined by this module and mirrored by every pith-* cdylib:

• one flat set of #[unsafe(no_mangle)] pub unsafe extern "C" functions — raw pointers plus lengths, no structs across the boundary;
• every function returns a status code (see the constants below), never a Result, never a panic: a panic = "abort" cdylib must not be reachable from a foreign caller;
• an operation either hands ownership to the caller (and ships a matching _free — pith_text_free here) or writes into caller-provided out-parameters — pith_text_jaccard allocates nothing, so it ships no free;
• the unsafe allowance is confined to this module; every core module stays unsafe-free behind the crate-root #![deny].

Wire formats (the cross-SDK contract)

pith_text_fingerprint hands out the canonical fingerprint stream of crate::reference::canonical_stream:

[0..4)    word_count     u32 big-endian
[4..8)    shingle_count  u32 big-endian
[8..8+C)  canonical      the canonical UTF-8 bytes (C bytes)
[8+C..)   signature      128 u64 little-endian words (1024 bytes)

so canonical_sha256 == sha256(stream[8..8+C]) and the signature_fnv1a64 / signature_sha256 folds of reference.json are the standard folds over the 1024-byte little-endian tail. The input is UTF-8: len is a byte length, and bytes that are not valid UTF-8 are rejected (PITH_E_REJECTED), never lossily coerced. The empty input is valid and pins the sentinel stream (zero counts, 128 u64::MAX words) — it is not a refusal.

pith_text_jaccard compares two u64 word slices: the words are read one element at a time with core::ptr::read_unaligned (the SDK-side blob is a byte buffer, alignment is not guaranteed), and the estimate crosses the ABI as its f64 bit pattern through a u64 out-parameter — no float crosses the boundary, so the SDKs compare value_bits hex-exactly. A null pointer with len == 0 is the legal empty slice — the sentinel operand (reference.json’s empty-signature arm passes Vec::new(), not signature(""), whose all-u64::MAX words would score ≈ 0) — while a null pointer with len > 0 is PITH_E_INVALID.

Constants

PITH_E_INVALID

Status: a caller argument is invalid — a null pointer (or a null buffer carrying a nonzero length where a slice is read).

PITH_E_REJECTED

Status: the core pipeline refused the input (the bytes are not valid UTF-8).

PITH_OK

Status: success.

Functions

pith_text_fingerprint^⚠

Computes the full text fingerprint of a UTF-8 string into the canonical byte stream the reference.json vectors are defined over.

pith_text_free^⚠

Releases a buffer handed out by pith_text_fingerprint.

pith_text_jaccard^⚠

Estimates the Jaccard index of the two shingle sets that produced the a_words and b_words signature words.