ctranslate2-rs

This library provides Rust bindings for OpenNMT/CTranslate2. At this time, it has only been tested and confirmed to work on Intel Mac and Linux.

Compilation

On Linux, OpenBLAS is required. Please add the path to the directory containing libopenblas.a to LIBRARY_PATH environment variable.

Model Conversion for CTranslate2

To use model files with CTranslate2, they must first be converted. Below is an example of how to convert the nllb-200-distilled-600M model:

pip install ctranslate2
ct2-transformers-converter --model facebook/nllb-200-distilled-600M --output_dir nllb-200-distilled-600M --copy_files tokenizer.json

For more details, please refer to the CTranslate2's docs.

Example of text translation

The following example translates English to German and Japanese using the previously converted model nllb-200-distilled-600M.

use anyhow::Result;
use ct2rs::config::{Config, Device};
use ct2rs::{TranslationOptions, Translator};

fn main() -> Result<()> {
    let path = "/path/to/nllb-200-distilled-600M";
    let t = Translator::new(path, Device::CPU, Config::default())?;
    let res = t.translate_batch(
        vec![
            "Hello world!",
            "This library provides Rust bindings for CTranslate2.",
        ],
        vec![vec!["deu_Latn"], vec!["jpn_Jpan"]],
        &TranslationOptions {
            return_scores: true,
            ..Default::default()
        },
    )?;
    for r in res {
        println!("{}, (score: {:?})", r.0, r.1);
    }


    Ok(())
}

Output

Hallo Welt!<unk>, (score: Some(-0.5597002))
このライブラリでは,CTranslate2 の Rust バインディングが提供されています., (score: Some(-0.56321025))

Example of text generation

fn main() -> Result<()> {
    let path = "/path/to/model";
    let g = Generator::new(path, Device::CPU, Config::default())?;
    let res = g.generate_batch(
        vec!["prompt"],
        &GenerationOptions::default(),
    )?;
    for r in res {
        println!("{:?}", r.0);
    }

    Ok(())
}

License

This application is released under the MIT License. For details, see the LICENSE file.