datu 0.3.4

datu - a data file utility
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134

Feature: Head

  Scenario: Head from Parquet
    When the REPL is ran and the user types:
      ```
      read("fixtures/table.parquet") |> head(2) |> write("$TEMPDIR/head_parquet.csv")
      ```
    Then the file "$TEMPDIR/head_parquet.csv" should exist
    And that file should be a CSV file
    And the first line of that file should be: "one,two,three,four,five,__index_level_0__"
    And that file should have 3 lines
    And that file should contain "foo"
    And that file should contain "bar"

  Scenario: Head a single row
    When the REPL is ran and the user types:
      ```
      read("fixtures/table.parquet") |> head(1) |> write("$TEMPDIR/head_one.csv")
      ```
    Then the file "$TEMPDIR/head_one.csv" should exist
    And that file should be a CSV file
    And that file should have 2 lines
    And that file should contain "foo"

  Scenario: Head more rows than available
    When the REPL is ran and the user types:
      ```
      read("fixtures/table.parquet") |> head(100) |> write("$TEMPDIR/head_all.csv")
      ```
    Then the file "$TEMPDIR/head_all.csv" should exist
    And that file should be a CSV file
    And that file should have 4 lines

  Scenario: Head from Avro
    When the REPL is ran and the user types:
      ```
      read("fixtures/userdata5.avro") |> head(5) |> write("$TEMPDIR/head_avro.csv")
      ```
    Then the file "$TEMPDIR/head_avro.csv" should exist
    And that file should be a CSV file
    And the first line of that file should contain "id"
    And the first line of that file should contain "first_name"
    And that file should have 6 lines

  Scenario: Head from CSV
    When the REPL is ran and the user types:
      ```
      read("fixtures/table.csv") |> head(3) |> write("$TEMPDIR/head_csv.csv")
      ```
    Then the file "$TEMPDIR/head_csv.csv" should exist
    And that file should be a CSV file
    And the first line of that file should be: "one,two,three,four,five,__index_level_0__"
    And that file should have 4 lines
    And that file should contain "foo"
    And that file should contain "bar"

  Scenario: Head from ORC
    When the REPL is ran and the user types:
      ```
      read("fixtures/userdata.orc") |> head(3) |> write("$TEMPDIR/head_orc.csv")
      ```
    Then the file "$TEMPDIR/head_orc.csv" should exist
    And that file should be a CSV file
    And that file should have 4 lines

  Scenario: Head with select
    When the REPL is ran and the user types:
      ```
      read("fixtures/table.parquet") |> select(:two, :three) |> head(2) |> write("$TEMPDIR/head_select.csv")
      ```
    Then the file "$TEMPDIR/head_select.csv" should exist
    And that file should be a CSV file
    And the first line of that file should be: "two,three"
    And that file should have 3 lines

  Scenario: Head to JSON
    When the REPL is ran and the user types:
      ```
      read("fixtures/table.parquet") |> head(2) |> write("$TEMPDIR/head.json")
      ```
    Then the file "$TEMPDIR/head.json" should exist
    And that file should be valid JSON
    And that file should contain "foo"
    And that file should contain "bar"
    And that file should have 2 records

  Scenario: Head to YAML
    When the REPL is ran and the user types:
      ```
      read("fixtures/table.parquet") |> head(2) |> write("$TEMPDIR/head.yaml")
      ```
    Then the file "$TEMPDIR/head.yaml" should exist
    And that file should be valid YAML
    And that file should contain "two:"
    And that file should contain "foo"

  Scenario: Head to Parquet
    When the REPL is ran and the user types:
      ```
      read("fixtures/userdata5.avro") |> head(10) |> write("$TEMPDIR/head.parquet")
      ```
    Then the file "$TEMPDIR/head.parquet" should exist
    And that file should be a valid Parquet file
    And that file should have 10 records

  Scenario: Head to Avro
    When the REPL is ran and the user types:
      ```
      read("fixtures/table.parquet") |> head(2) |> write("$TEMPDIR/head.avro")
      ```
    Then the file "$TEMPDIR/head.avro" should exist
    And that file should be valid Avro
    And that file should have 2 records

  Scenario: Head with select from Avro
    When the REPL is ran and the user types:
      ```
      read("fixtures/userdata5.avro") |> select(:id, :email) |> head(3) |> write("$TEMPDIR/head_select_avro.csv")
      ```
    Then the file "$TEMPDIR/head_select_avro.csv" should exist
    And that file should be a CSV file
    And the first line of that file should be: "id,email"
    And that file should have 4 lines

  Scenario: Head with select to JSON
    When the REPL is ran and the user types:
      ```
      read("fixtures/userdata5.avro") |> select(:id, :first_name) |> head(2) |> write("$TEMPDIR/head_select.json")
      ```
    Then the file "$TEMPDIR/head_select.json" should exist
    And that file should be valid JSON
    And that file should contain "id"
    And that file should contain "first_name"
    And that file should have 2 records