polars-view 0.11.2

A fast and interactive viewer for CSV, JSON and Parquet data.
Documentation

PolarsView

Crates.io Documentation License Rust

A fast and interactive viewer for CSV, JSON and Parquet data, built with Polars and egui.

This project is inspired by and forked from the parqbench project.

Features

  • Fast Loading: Leverages Polars for efficient data loading and processing.
  • CSV, JSON and Parquet Support: Handles CSV, JSON and Parquet file formats.
  • Save as: Save to file and select CSV, JSON or Parquet format.
  • SQL Querying: Apply SQL queries to filter and transform data.
  • Data Filtering: Provides UI elements for setting CSV delimiters, schema inference length, and decimal precision.
  • Metadata Display: Shows file metadata (column count, row count) and schema information.
  • Sorting Capabilities: Interactive column sorting (ascending/descending).
  • Asynchronous Operations: Uses Tokio for non-blocking file loading and query execution.
  • Error Handling: Custom error enum type for robust error handling.

Building and Running

  1. Prerequisites:

    • Rust and Cargo (latest stable version recommended).

  2. Clone the Repository:

    git clone https://github.com/claudiofsr/polars-view.git
cd polars-view

  3. Build and Install:

    cargo b -r && cargo install --path=.

  4. Run:

    polars-view [path_to_file] [options]

    • Replace [path_to_file] with the actual path to your CSV, Json or Parquet file.

    • Use polars-view --help for a list of available options (delimiter, query, table name).

    • Tracing Options (for logging): To enable detailed logging, use the RUST_LOG environment variable:

      RUST_LOG=info polars-view [path_to_file] [options]  # General info logs
RUST_LOG=debug polars-view [path_to_file] [options] # More detailed logs
RUST_LOG=trace polars-view [path_to_file] [options] # Very detailed logs (for debugging)

      You can also specify the log level for specific modules:

      RUST_LOG=polars_view=debug,polars=info polars-view [path_to_file] [options]

    • Examples:

      polars-view data.parquet
polars-view --delimiter ; data.csv --query "SELECT * FROM mytable WHERE x > 10"
RUST_LOG=info polars-view data.parquet

Usage

  • Open a File:

    • Run the application with a file path as an argument.
    • Use the "File" -> "Open" menu option.
    • Drag and drop a CSV, Json or Parquet file onto the application window.

  • Filtering:

    • Use the "Query" panel to set CSV delimiter, schema inference length, and decimal places.
    • Enter SQL queries in the "SQL Query" field.
    • Click "Apply" to apply the filters.

  • Sorting:

    • Click the column headers in the table to sort by that column (ascending/descending).

  • Metadata:

    • The "Metadata" panel shows file metadata.
    • The "Schema" panel displays the data schema.

  • SQL Examples:

    The interface includes predefined SQL command for easy reference.
    These cover various common filtering and aggregation operations.
    See Polars SQL documentation.

    SELECT * FROM AllData;
SELECT * FROM AllData WHERE column_name > value;
SELECT column1, COUNT(*) FROM AllData GROUP BY column1;

Dependencies

  • Polars: High-performance DataFrame library.
  • eframe: Immediate-mode GUI library.
  • egui: Immediate-mode GUI library.
  • egui_extras: Complement egui features
  • clap: Command-line argument parser.
  • tokio: Asynchronous runtime.
  • tracing: Application-level tracing framework.
  • thiserror: Library for deriving the Error trait.
  • rfd: Native file dialogs.

Contributing

Contributions are welcome! Please open an issue or submit a pull request.

License

This project is licensed under the MIT License.