csv2arrow 0.26.1

Convert CSV files to Arrow
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92

# CSV to Arrow

[![Crates.io](https://img.shields.io/crates/v/csv2arrow.svg)](https://crates.io/crates/csv2arrow)

Convert CSV files to Apache Arrow. This package is part of [Arrow CLI tools](https://github.com/domoritz/arrow-tools).

## Installation

### Download prebuilt binaries

You can get the latest releases from https://github.com/domoritz/arrow-tools/releases.

### With Homebrew

```
brew install domoritz/homebrew-tap/csv2arrow
```

### With Cargo

```
cargo install csv2arrow
```

## With [Cargo B(inary)Install](https://github.com/cargo-bins/cargo-binstall)

To avoid re-compilation and speed up installation, you can install this tool with `cargo binstall`:

```
cargo binstall csv2arrow
```

## Usage

```
Usage: csv2arrow [OPTIONS] <CSV> [ARROW]

Arguments:
  <CSV>
          Input CSV file, stdin if not present

  [ARROW]
          Output file, stdout if not present

Options:
  -s, --schema-file <SCHEMA_FILE>
          File with Arrow schema in JSON format

  -m, --max-read-records <MAX_READ_RECORDS>
          The number of records to infer the schema from. All rows if not present. Setting max-read-records to zero will stop schema inference and all columns will be string typed

      --header <HEADER>
          Set whether the CSV file has headers

          [default: true]
          [possible values: true, false]

      --delimiter <DELIMITER>
          Set the CSV file's column delimiter as a byte character

      --escape <ESCAPE>
          Specify an escape character

      --quote <QUOTE>
          Specify a custom quote character

      --comment <COMMENT>
          Specify a comment character.

          Lines starting with this character will be ignored

      --null-regex <NULL_REGEX>
          Provide a regex to match null values

  -p, --print-schema
          Print the schema to stderr

  -n, --dry
          Only print the schema

  -h, --help
          Print help (see a summary with '-h')

  -V, --version
          Print version
```

The --schema-file option uses the same file format as --dry and --print-schema.

## Examples

For usage examples, see the [`csv2parquet` examples](https://github.com/domoritz/arrow-tools/tree/main/crates/csv2parquet#examples) which shares a similar interface.