ctranslate2-rs

This library provides Rust bindings for OpenNMT/CTranslate2. At this time, it has only been tested and confirmed to work on Intel Mac and Linux.

Compilation

On Linux, OpenBLAS is required. Please add the path to the directory containing libopenblas.a to LIBRARY_PATH environment variable.

About the Model

The model files need to be converted for CTranslate2. For instance, the following command will convert nllb-200-distilled-600M:

pip install ctranslate2
ct2-transformers-converter --model facebook/nllb-200-distilled-600M --output_dir nllb-200-distilled-600M --copy_files tokenizer.json

Please do not forget to copy tokenizer.json. For more details, please refer to the CTranslate2's docs.

Example of text translation

The following example translates English to German using the previously converted model nllb-200-distilled-600M.

use anyhow::Result;
use ct2rs::config::{Config, Device};
use ct2rs::{TranslationOptions, Translator};

fn main() -> Result<()> {
    let path = "/path/to/nllb-200-distilled-600M";
    let t = Translator::new(path, Device::CPU, Config::default())?;
    let res = t.translate_batch(
        vec![
            "Hello world!",
            "This library provides Rust bindings for CTranslate2.",
        ],
        vec![vec!["deu_Latn"], vec!["jpn_Jpan"]],
        &TranslationOptions {
            return_scores: true,
            ..Default::default()
        },
    )?;
    for r in res {
        println!("{}, (score: {:?})", r.0, r.1);
    }


    Ok(())
}

Output

Hallo Welt!<unk>, (score: Some(-0.5597002))
このライブラリでは,CTranslate2 の Rust バインディングが提供されています., (score: Some(-0.56321025))

Example of text generation

fn main() -> Result<()> {
    let path = "/path/to/model";
    let g = Generator::new(path, Device::CPU, Config::default())?;
    let res = g.generate_batch(
        vec!["prompt"],
        &GenerationOptions::default(),
    )?;
    for r in res {
        println!("{:?}", r.0);
    }

    Ok(())
}

License

This application is released under the MIT License. For details, see the LICENSE file.