Crate csv_pipeline

CSV processing library inspired by csvsc

Get started

The first thing you need is to create a Pipeline. This can be done by calling Pipeline::from_reader with a csv::Reader, or Pipeline::from_path with a path.

Once you have a pipeline, there are various methods available which let you add your desired processing steps. Check the Pipeline for more details and examples.

In the end, you may want to write the result somewhere. To do that, you can flush into a Target.

Finally, you probably want to run the pipeline. There are a few options:

• Pipeline::build gives you a PipelineIter which you can iterate through
• Pipeline::run runs through the pipeline until it finds an error, or the end
• Pipeline::collect_into_string runs the pipeline and returns the csv as a Result<String, Error>. Can be a convenient alternative to flushing to a StringTarget.

Basic Example

use csv_pipeline::{Pipeline, Transformer};

// First create a pipeline from a CSV file path
let csv = Pipeline::from_path("test/Countries.csv")
  .unwrap()
  // Add a column with values computed from a closure
  .add_col("Language", |headers, row| {
    match headers.get_field(row, "Country") {
      Some("Norway") => Ok("Norwegian".into()),
      _ => Ok("Unknown".into()),
    }
  })
  // Make the "Country" column uppercase
  .rename_col("Country", "COUNTRY")
  .map_col("COUNTRY", |id_str| Ok(id_str.to_uppercase()))
  // Collect the csv into a string
  .collect_into_string()
  .unwrap();

assert_eq!(
  csv,
  "ID,COUNTRY,Language\n\
    1,NORWAY,Norwegian\n\
    2,TUVALU,Unknown\n"
);

Transform Example

use csv_pipeline::{Pipeline, Transformer};

let source = "\
  Person,Score\n\
  A,1\n\
  A,8\n\
  B,3\n\
  B,4\n";
let reader = csv::Reader::from_reader(source.as_bytes());
let csv = Pipeline::from_reader(reader)
  .unwrap()
  .map(|_headers, row| Ok(row))
  // Transform into a new csv
  .transform_into(|| {
    vec![
      // Keep every Person
      Transformer::new("Person").keep_unique(),
      // Sum the scores into a "Total score" column
      Transformer::new("Total score").from_col("Score").sum(0),
    ]
  })
  .collect_into_string()
  .unwrap();

assert_eq!(
  csv,
  "Person,Total score\n\
    A,9\n\
    B,7\n"
);

Modules

• target

Structs

• Headers
  The headers of a CSV file
• Pipeline
  The main thing
• PipelineIter
  A pipeline you can iterate through. You can get one using Pipeline::build.
• PlError
  Error originating from the specified pipeline source index
• Target
  Helper for building a target to flush data into
• Transformer
  A struct for building a Transform, which you can use with Pipeline::transform_into.

Enums

• Error

Traits

• Transform
  For grouping and reducing rows.

Type Definitions

• Row
  Alias of csv::StringRecord
• RowResult
  Alias of Result<Row, Error>