anew 0.1.0

A tool for adding new lines to files, skipping duplicates and write in Rust!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141

<h1 align="center">anew</h1>
<h3 align="center">A tool for adding new lines to files written in Rust.</h3>

The tool aids in appending lines from stdin to a file, but only if they don't already appear in the file.
Outputs new lines to `stdout` too, making it a bit like a `tee -a` that removes duplicates.

## Install

Manual installation
```
git clone https://github.com/zer0yu/anew
cd anew
cargo build
```
or you can download the binary from [releases](https://github.com/zer0yu/anew/releases)

## Usage

```
❯ anew --help
A tool for adding new lines to files, skipping duplicates

Usage: anew [OPTIONS] <FILEPATH>

Arguments:
  <FILEPATH>  Destination file

Options:
  -q, --quiet-mode  Do not output new lines to stdout
  -s, --sort        Sort lines (natsort)
  -t, --trim        Trim whitespaces
  -r, --rewrite     Rewrite existing destination file to remove duplicates
  -d, --dry-run     Do not write to file, only output what would be written
  -h, --help        Print help
  -V, --version     Print version
```

## Usage Example

Here, a file called `things.txt` contains a list of numbers. `newthings.txt` contains a second
list of numbers, some of which appear in `things.txt` and some of which do not. `anew` is used
to append the latter to `things.txt`.

Usage 1: Add differences to `things.txt`
```
❯ cat things.txt
One
Zero
Two
One

❯ cat newthings.txt
Three
One
Five
Two
Four

❯ cat newthings.txt | anew things.txt
Three
Five
Four

❯ cat things.txt
One
Zero
Two
One
Three
Five
Four
```

Usage 2: Disable terminal output
```
❯ cat newthings.txt | anew things.txt -q
Three
Five
Four
``` 

Usage 3: Sorting the contents of `things.txt` after adding new differences
```
❯ cat newthings.txt | ./anew things.txt -q -s

❯ cat things.txt
Five
Four
One
Three
Two
Zero
```
PS: 
1. anew uses the fastest sorting algorithm, so don't worry about efficiency.
2. Sort mode automatically de-duplicates, so `-s` and `-r` do not need to be used at the same time.

Usage 4: De-duplication of `things.txt` after adding new differences
```
❯ cat newthings.txt | ./anew things.txt -q -r

❯ cat things.txt
One
Zero
Two
Three
Five
Four
```

## Efficiency Comparison

We use two files `newoutput.txt` and `output.txt` of size 10MB as input to the program to compare the difference in speed between tomnomnom's Go implementation, rwese's Rust implementation, and this project's Rust implementation.

This project
```
❯ time cat newoutput.txt | ./anew output.txt -q
cat newoutput.txt  0.00s user 0.02s system 1% cpu 1.398 total
./anew output.txt -q  1.46s user 0.22s system 97% cpu 1.717 total
```

anew implemented by rwese
```
❯ time cat newoutput.txt | ./anew_rwese output.txt -q
cat newoutput.txt  0.00s user 0.02s system 0% cpu 2:28.38 total
./anew output.txt -q  6.95s user 101.08s system 72% cpu 2:29.49 total
```

anew implemented by tomnomnom
```
❯ time cat newoutput.txt | ./anew_go -q output.txt
cat newoutput.txt  0.00s user 0.02s system 0% cpu 4.797 total
./anew_go -q output.txt  2.11s user 3.14s system 108% cpu 4.838 total
```

As can be seen from the above, the project has been implemented most efficiently!

## References

1. [anew@tomnomnom](https://github.com/tomnomnom/anew)
2. [anew@rwese](https://github.com/rwese/anew)