Crate hedl_ffi

Expand description

HEDL FFI Bindings

Provides C-compatible interface for HEDL operations. All functions use C-style error handling with return codes.

§Memory Management

IMPORTANT: Memory ownership follows strict rules:

Strings returned by hedl_* functions MUST be freed with hedl_free_string
Byte arrays returned by hedl_to_parquet MUST be freed with hedl_free_bytes
Documents MUST be freed with hedl_free_document
Diagnostics MUST be freed with hedl_free_diagnostics

WARNING - Memory Safety Requirements:

The hedl_free_* functions ONLY accept pointers that were allocated by HEDL functions. Passing any of the following will cause undefined behavior:

Pointers from malloc/calloc/realloc (wrong allocator)
Stack-allocated variables
Already-freed pointers (double free)
Pointers from other libraries
NULL is safe and will be ignored

§Thread Safety

§Error Handling Thread Safety

Error messages are stored in thread-local storage, providing lock-free, wait-free error handling for multi-threaded applications.

Key Guarantees:

Each thread maintains its own independent error state
hedl_get_last_error() / hedl_get_last_error_threadsafe() returns the error for the CALLING thread only
Errors from one thread will NEVER appear in or overwrite errors in another thread
No mutexes, locks, or other synchronization primitives are required
Zero contention between threads accessing error messages
You MUST call error functions from the same thread that received the error code

Thread-Safe Functions:

hedl_get_last_error() - Get error for current thread
hedl_get_last_error_threadsafe() - Explicit thread-safe alias
hedl_clear_error_threadsafe() - Clear error for current thread

Example (Multi-threaded C with pthreads):

void* worker(void* arg) {
    const char* input = (const char*)arg;
    HedlDocument* doc = NULL;

    if (hedl_parse(input, -1, 0, &doc) != HEDL_OK) {
        // Get error for THIS thread - independent of other threads
        const char* err = hedl_get_last_error_threadsafe();
        fprintf(stderr, "Parse error: %s\n", err);
        return NULL;
    }

    // Process document...
    hedl_free_document(doc);
    return (void*)1;
}

int main() {
    pthread_t threads[8];
    const char* inputs[8] = { ... };

    // Launch threads - each with independent error state
    for (int i = 0; i < 8; i++) {
        pthread_create(&threads[i], NULL, worker, (void*)inputs[i]);
    }

    for (int i = 0; i < 8; i++) {
        pthread_join(threads[i], NULL);
    }
}

§Document Handle Thread Safety

Document handles (HedlDocument*) are NOT thread-safe by design for performance reasons. Do not share document handles between threads without external synchronization (mutexes, etc.).

Safe Pattern:

Each thread creates its own document handles
Each thread frees its own document handles
No sharing of document pointers across threads

Unsafe Pattern:

Passing a HedlDocument* to multiple threads (data race)
Accessing the same document from multiple threads (undefined behavior)

§Error Handling

All functions return error codes (HEDL_OK on success)
Use hedl_get_last_error to get the error message for the current thread

§Security

§Poison Pointers

To detect double-free and use-after-free bugs, this library uses poison pointers:

After freeing a document or diagnostics, the internal pointer is checked against a poison value
All accessor functions validate that pointers are not poisoned before use
This provides defense-in-depth against memory safety bugs

Note: Since C passes pointers by value, we cannot modify the caller’s pointer after freeing. However, we can detect if a freed pointer is passed back to us by checking for the poison value in accessor functions.

§Audit Logging

This library provides comprehensive audit logging for all FFI function calls using the tracing crate. The logging system captures:

Function entry/exit with timing information
Sanitized parameters (pointer addresses are masked for security)
Success/failure outcomes with error details
Performance metrics (call duration)
Thread context for correlation

§Configuring Logging

To enable logging, initialize a tracing subscriber in your application:

use tracing_subscriber::{fmt, EnvFilter};

// Initialize the tracing subscriber
tracing_subscriber::fmt()
    .with_env_filter(
        EnvFilter::try_from_default_env()
            .unwrap_or_else(|_| EnvFilter::new("info"))
    )
    .with_target(true)
    .with_thread_ids(true)
    .with_line_number(true)
    .init();

// Now all FFI calls will be logged

§Log Levels

ERROR: Function failures with error details
WARN: Recoverable errors or unusual conditions
INFO: Function call entry/exit with basic metrics
DEBUG: Detailed parameter information (sanitized)

§Environment Variables

Control logging via the RUST_LOG environment variable:

# Log all INFO and above
export RUST_LOG=info

# Log only FFI audit events
export RUST_LOG=hedl_ffi::audit=debug

# Log everything at DEBUG level
export RUST_LOG=debug

§Example Output

2025-01-05T10:30:45.123Z INFO hedl_ffi::audit: FFI call started function="hedl_parse" thread_id=ThreadId(1) depth=0
2025-01-05T10:30:45.125Z DEBUG hedl_ffi::audit: FFI call parameters function="hedl_parse" params=[("input_len", "1024")]
2025-01-05T10:30:45.130Z INFO hedl_ffi::audit: FFI call completed function="hedl_parse" duration_ms=7.2 status="success"

See the audit module for more details on the logging implementation.

Re-exports§

pub use async_ops::hedl_async_cancel;
pub use async_ops::hedl_async_free;
pub use async_ops::hedl_canonicalize_async;
pub use async_ops::hedl_lint_async;
pub use async_ops::hedl_parse_async;
pub use async_ops::HedlAsyncOp;
pub use async_ops::HedlCompletionCallback;
pub use async_ops::HedlCompletionCallbackFn;
pub use async_ops::hedl_to_json_async;
pub use async_ops::hedl_to_yaml_async;
pub use async_ops::hedl_to_xml_async;
pub use async_ops::hedl_to_csv_async;
pub use async_ops::hedl_to_neo4j_cypher_async;
pub use async_ops::hedl_to_toon_async;

Modules§

async_ops: Async FFI operations. Asynchronous operation support for FFI.
audit: FFI audit logging. Audit logging for FFI function calls.
reentrancy: Reentrancy guard. Reentrancy detection for FFI callback functions.

Macros§

audit_ffi_call: Helper macro to wrap an FFI function with audit logging.

Structs§

HedlDiagnostics: Opaque handle to lint diagnostics
HedlDocument: Opaque handle to a HEDL document

Constants§

HEDL_ERR_ALLOC: Memory allocation error.
HEDL_ERR_CANCELLED: Operation was cancelled.
HEDL_ERR_CANONICALIZE: Canonicalization error.
HEDL_ERR_CSV: CSV conversion error.
HEDL_ERR_INVALID_HANDLE: Invalid handle provided.
HEDL_ERR_INVALID_UTF8: Invalid UTF-8 encoding error.
HEDL_ERR_JSON: JSON conversion error.
HEDL_ERR_LINT: Lint validation error.
HEDL_ERR_NEO4J: Neo4j export error.
HEDL_ERR_NULL_PTR: Null pointer argument error.
HEDL_ERR_PARQUET: Parquet conversion error.
HEDL_ERR_PARSE: HEDL parsing error.
HEDL_ERR_QUEUE_FULL: Task queue is full.
HEDL_ERR_REENTRANT_CALL: Reentrant call detected (thread safety violation).
HEDL_ERR_TOON: TOON conversion error.
HEDL_ERR_XML: XML conversion error.
HEDL_ERR_YAML: YAML conversion error.
HEDL_OK: Success return code.

Functions§

hedl_alias_count^⚠: Get the number of aliases in a document.
hedl_canonicalize^⚠: Canonicalize a HEDL document.
hedl_canonicalize_callback^⚠: Canonicalize a HEDL document using zero-copy callback pattern.
hedl_clear_error_threadsafe: Clear the last error for the current thread.
hedl_diagnostics_count^⚠: Get the number of diagnostics.
hedl_diagnostics_get^⚠: Get a diagnostic message.
hedl_diagnostics_severity^⚠: Get a diagnostic severity (0=Hint, 1=Warning, 2=Error).
hedl_free_bytes^⚠: Free byte array allocated by HEDL functions (e.g., hedl_to_parquet).
hedl_free_diagnostics^⚠: Free a diagnostics handle.
hedl_free_document^⚠: Free a document handle.
hedl_free_string^⚠: Free a string allocated by HEDL functions.
hedl_from_json^⚠: Parse JSON into a HEDL document.
hedl_from_parquet^⚠: Parse Parquet bytes into a HEDL document.
hedl_from_toon^⚠: Parse TOON into a HEDL document.
hedl_from_xml^⚠: Parse XML into a HEDL document.
hedl_from_yaml^⚠: Parse YAML into a HEDL document.
hedl_get_last_error: Get the last error message for the current thread.
hedl_get_last_error_threadsafe: Get the last error message for the current thread (thread-safe variant).
hedl_get_version^⚠: Get the HEDL version of a parsed document.
hedl_lint^⚠: Lint a HEDL document.
hedl_parse^⚠: Parse a HEDL document from a string.
hedl_root_item_count^⚠: Get the number of root items in a document.
hedl_schema_count^⚠: Get the number of struct definitions in a document.
hedl_to_csv^⚠: Convert a HEDL document to CSV.
hedl_to_csv_callback^⚠: Convert a HEDL document to CSV using zero-copy callback pattern.
hedl_to_json^⚠: Convert a HEDL document to JSON.
hedl_to_json_callback^⚠: Convert a HEDL document to JSON using zero-copy callback pattern.
hedl_to_neo4j_cypher^⚠: Convert a HEDL document to Cypher queries for Neo4j.
hedl_to_neo4j_cypher_callback^⚠: Convert a HEDL document to Cypher queries using zero-copy callback pattern.
hedl_to_parquet^⚠: Convert a HEDL document to Parquet bytes.
hedl_to_toon^⚠: Convert a HEDL document to TOON format.
hedl_to_xml^⚠: Convert a HEDL document to XML.
hedl_to_xml_callback^⚠: Convert a HEDL document to XML using zero-copy callback pattern.
hedl_to_yaml^⚠: Convert a HEDL document to YAML.
hedl_to_yaml_callback^⚠: Convert a HEDL document to YAML using zero-copy callback pattern.
hedl_validate^⚠: Validate a HEDL document string.

Type Aliases§

HedlOutputCallback: Output callback function type for zero-copy string return.

Crate hedl_ffi

Crate hedl_ffi Copy item path

§Memory Management

§Thread Safety

§Error Handling Thread Safety

§Document Handle Thread Safety

§Error Handling

§Security

§Poison Pointers

§Audit Logging

§Configuring Logging

§Log Levels

§Environment Variables

§Example Output

Re-exports§

Modules§

Macros§

Structs§

Constants§

Functions§

Type Aliases§

Crate hedl_ffi