Struct Transcoder

pub struct Transcoder { /* private fields */ }

Low level API for transcoding.

Implementations

impl Transcoder

pub fn new( src_encoding: Option<&'static Encoding>, dst_encoding: &'static Encoding, ) -> Self

Creates a new Transcoder with a source encoding and a destination encoding. The source encoding is optional but if not specifed, guess_and_transcode() must be called before transcode() is called. Default buffer size is 8K bytes.

Example

use encoding_rs;

let transcoder = transcoding_rs::Transcoder::new(
    Some(encoding_rs::UTF_16BE),
    encoding_rs::UTF_8
    );

pub fn buffer_size(self, size: usize) -> Self

Sets buffer size. The size needs to be more than 4 bytes. Otherwise, the specified value is ignored.

Example

use encoding_rs;

let transcoder = transcoding_rs::Transcoder::new(
    Some(encoding_rs::UTF_16BE),
    encoding_rs::UTF_8)
    .buffer_size(1024); // sets buffer size.

pub fn transcode( &mut self, src: &[u8], dst: &mut [u8], last: bool, ) -> (CoderResult, usize, usize, bool)

Transcodes the source encoding to the destination encoding. Main functionality and usage are the same as decode_to_* or encode_from_* methods in encoding_rs. The key difference is that this function combined decode_to_* and encode_from_* methods excluding *_without_no_replacement variants in encoding_rs. So this function can transcode any given encoding to another as long as they are supported.

Parameters

• src: The input to be encoded.
• dst: The estination buffer the output is written to.
• last: Specify true if the input has reached EOF, or otherwise false. This method can be called multiple times with this value being true, until the method returns InputEmpty. After that, no more method must not be called. Otherwise, a panic is raised.

Return values

In addition to encoded data being written to dst, the guessed encoding and some information are returned by a tuple.

• 1: InputEmpty if the input is all read, OutputFull if dst has no more available buffer to write the output.
• 3: The number of bytes read.
• 4: The number of bytes written.
• 4: Whether malformed byte sequences or unmappable characters are found while transcoding.

Example

use encoding_rs;

let mut transcoder = transcoding_rs::Transcoder::new(
    Some(encoding_rs::SHIFT_JIS),
    encoding_rs::UTF_8);
let src = b"\x83\x6E\x83\x8D\x81\x5B"; // ハロー in SHIFT_JIS
let buf = &mut [0u8; 128];
let result = transcoder.transcode(src, buf, true);
let (_, _, num_written, _) = result;

assert_eq!("ハロー".as_bytes(), &buf[..num_written]);

pub fn guess_and_transcode( &mut self, src: &[u8], dst: &mut [u8], non_ascii_to_guess: usize, non_text_threshold: u8, last: bool, ) -> (Option<&'static Encoding>, CoderResult, usize, usize, bool)

Guesses the source encoding and try to transcode input. This method should be called once before the transcode() method if Transcode is created with no source encoding provided. This method must not be called after transcode() is called.

Parameters

• src: The input to be encoded.
• dst: The estination buffer the output is written to.
• non_ascii_to_guess: The number of non-ASCII characters to be used to guess the encoding. Non-ASCII here includes non-textual characters.
• non_text_threshold: The threshold to determine the guess is failed. The value should be specified in percentage.
• last: Specify true if the input has reached EOF, or otherwise false.

Return values

In addition to encoded data being written to dst, the guessed encoding and some information are returned by a tuple.

• 1: The guessed encoding if the guess succeeds.
• 2: InputEmpty if input is all read, OutputFull if dst has no more available buffer to write the output.
• 3: The number of bytes read.
• 4: The number of bytes written.
• 5: Whether malformed byte sequences or unmappable characters are found while transcoding.

Example

use encoding_rs;

let mut transcoder = transcoding_rs::Transcoder::new(
    None, // Not provide the source encoding.
    encoding_rs::UTF_8);
let src = b"\x83\x6E\x83\x8D\x81\x5B"; // ハロー in SHIFT_JIS
let buf = &mut [0u8; 128];
let result = transcoder.guess_and_transcode(src, buf, 6, 0, true);
let (enc, _, _, num_written, _) = result;

assert_eq!(encoding_rs::SHIFT_JIS, enc.unwrap());
assert_eq!("ハロー".as_bytes(), &buf[..num_written]);

pub fn is_non_text(c: &char) -> bool

Checks if the specified character is a non-text character or not. Non-text characters here are the characters defined in the file command, plus the REPLACEMENT CHARACTER (U+FFFD).