Crate spring_batch_rs

Spring-Batch for Rust

🐞 A toolkit for building enterprise-grade batch applications

Spring-Batch for Rust

Spring Batch for Rust, offers a robust and flexible framework for the development of batch processing applications, addressing the challenges of handling large-scale data processing tasks efficiently and reliably. It provides developers a comprehensive toolkit for building enterprise-grade batch applications.

Features

Feature	Description
mongodb	Enable reader and writer for Mongodb database
rdbc-postgres	Enable rdbc reader and writer for Postgres database
rdbc-mysql	Enable rdbc reader and writer for Mysql and MariaDb databases
rdbc-sqlite	Enable rdbc reader and writer for Sqlite database
json	Enable json reader and writer
csv	Enable csv reader and writer
fake	Enable fake reader. Useful for generate fake dataset
logger	Enable logger writer. Useful for debugging

Roadmap

• XML reader and writer
• SQL reader and writer
• Kafka reader and writer
• Pulsar reader and writer
• Retry/Skip policies
• Save execution data in database

Getting Started

Make sure you activated the suitable features crate on Cargo.toml:

[dependencies]
spring-batch-rs = { version = "<version>", features = ["<full|json|csv|fake|logger>"] }

Then, on your main.rs:

fn main() -> Result<(), BatchError> {
    let csv = "year,make,model,description
   1948,Porsche,356,Luxury sports car
   1995,Peugeot,205,City car
   2021,Mazda,CX-30,SUV Compact
   1967,Ford,Mustang fastback 1967,American car";

    let reader = CsvItemReaderBuilder::new()
        .delimiter(b',')
        .has_headers(true)
        .from_reader(csv.as_bytes());

    let processor = UpperCaseProcessor::default();

    let writer = JsonItemWriterBuilder::new().from_path(temp_dir().join("cars.json"));

    let step: StepInstance<Car, Car> = StepBuilder::new()
        .reader(&reader) // set csv reader
        .writer(&writer) // set json writer
        .processor(&processor) // set upper case processor
        .chunk(2) // set commit interval
        .skip_limit(2) // set fault tolerance
        .build();

    let job = JobBuilder::new().start(&step).build();
    let result = job.run();

    assert!(result.is_ok());
    assert!(step.get_status() == StepStatus::Success);

    Ok(())
}

Examples

• Generate CSV file from JSON file with processor
• Generate JSON file from CSV string with fault tolerance
• Generate JSON file from fake persons
• Generate CSV file without headers from fake persons
• Insert records into Mysql database
• Log records from Postgres database
• Read records from MongoDb database
• Write records to MongoDb database

License

Licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions

Modules

• core
  Core module for batch operations
• error
  Error types for batch operations
• item
  Set of items readers / writers (for exemple: csv reader and writer)

Enums

• BatchError
  Represents the possible errors that can occur during batch processing.