Skip to main content

ChunkedUtf8Validator

Trait ChunkedUtf8Validator 

Source
pub trait ChunkedUtf8Validator {
    // Required methods
    unsafe fn new() -> Self
       where Self: Sized;
    unsafe fn update_from_chunks(&mut self, input: &[u8]);
    unsafe fn finalize(
        self,
        remaining_input: Option<&[u8]>,
    ) -> Result<(), Utf8Error>;
}
Available on crate feature public_imp only.
Expand description

Like Utf8Validator this low-level API is for streaming validation of UTF-8 data.

It has additional restrictions imposed on how the input is passed in to allow validation with as little overhead as possible.

To feed it data you need to call the Self::update_from_chunks() method which takes slices which have to be a multiple of 64 bytes long. The method will panic otherwise. There is no way to find out if the input so far was valid UTF-8 during the validation. Only when the validation is completed with the Self::finalize() method the result of the validation is returned.

The Self::finalize() method can be fed the rest of the data. There is no restriction on the data passed to it.

This implementation requires CPU SIMD features specified by the module it resides in. It is undefined behavior to use it if the required CPU features are not available which is why all trait methods are unsafe.

Required Methods§

Source

unsafe fn new() -> Self
where Self: Sized,

Creates a new validator.

§Safety

This implementation requires CPU SIMD features specified by the module it resides in. It is undefined behavior to call it if the required CPU features are not available.

Source

unsafe fn update_from_chunks(&mut self, input: &[u8])

Updates the validator with input.

§Panics

If input.len() is not a multiple of 64.

§Safety

This implementation requires CPU SIMD features specified by the module it resides in. It is undefined behavior to call it if the required CPU features are not available.

Source

unsafe fn finalize( self, remaining_input: Option<&[u8]>, ) -> Result<(), Utf8Error>

Updates the validator with remaining input if any. There is no restriction on the data provided.

Finishes the validation and returns Ok(()) if the input was valid UTF-8.

§Errors

A basic::Utf8Error is returned if the input was not valid UTF-8. No further information about the location of the error is provided.

§Safety

This implementation requires CPU SIMD features specified by the module it resides in. It is undefined behavior to call it if the required CPU features are not available.

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§

Source§

impl ChunkedUtf8Validator for simdutf8::basic::imp::x86::avx512::ChunkedUtf8ValidatorImp

Available on avx512_stable and (runtime_detection, or crate feature public_imp, or target features avx512bw and avx512f and avx512vbmi2 and avx512vbmi) and (x86 or x86-64) only.
Source§

impl ChunkedUtf8Validator for simdutf8::basic::imp::x86::avx2::ChunkedUtf8ValidatorImp

Available on (crate feature public_imp, or runtime_detection and not (avx512_stable and target feature avx512bw and target feature avx512f and target feature avx512vbmi2 and target feature avx512vbmi), or target feature avx2 and non-runtime_detection and not (avx512_stable and target feature avx512bw and target feature avx512f and target feature avx512vbmi2 and target feature avx512vbmi)) and (x86 or x86-64) only.
Source§

impl ChunkedUtf8Validator for simdutf8::basic::imp::x86::sse42::ChunkedUtf8ValidatorImp

Available on (crate feature public_imp, or runtime_detection and neither avx512_stable and target feature avx512bw and target feature avx512f and target feature avx512vbmi2 and target feature avx512vbmi, nor target feature avx2 and non-avx512_stable, or target feature sse4.2 and non-runtime_detection and non-target feature avx2 and not (avx512_stable and target feature avx512bw and target feature avx512f and target feature avx512vbmi2 and target feature avx512vbmi)) and (x86 or x86-64) only.
Source§

impl ChunkedUtf8Validator for simdutf8::basic::imp::portable::simd128::ChunkedUtf8ValidatorImp

Available on crate feature portable_public_imp only.
Source§

impl ChunkedUtf8Validator for simdutf8::basic::imp::portable::simd256::ChunkedUtf8ValidatorImp

Available on crate feature portable_public_imp only.