Rust_Grammar 2.0.0

A comprehensive, production-ready text analysis tool
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
# πŸš€ Text Analyzer REST API

## πŸ“‘ **API Server**

A REST API that provides comprehensive text analysis with exact position tracking.

---

## 🎯 **Quick Start**

### **1. Build the API Server**
```bash
cd text-analyzer
cargo build --release --bin api-server
```

### **2. Start the Server**
```bash
./target/release/api-server
```

**Output:**
```
πŸš€ Text Analyzer API running on http://0.0.0.0:2000
πŸ“ POST to http://0.0.0.0:2000/analyze with JSON body: {"text": "your text"}
```

### **3. Test the API**
```bash
curl -X POST http://localhost:2000/analyze \
  -H "Content-Type: application/json" \
  -d '{"text": "The report was written yesterday. This is very important and very critical."}'
```

---

## πŸ“‹ **API Endpoint**

### **POST /analyze**

Analyzes text and returns all detected issues with positions.

**URL:** `http://0.0.0.0:2000/analyze`

**Method:** `POST`

**Content-Type:** `application/json`

---

## πŸ“₯ **Request Format**

```json
{
  "text": "Your text here to analyze"
}
```

**Example:**
```json
{
  "text": "The comprehensive analysis was conducted by the team. At the end of the day, we need to synergize our efforts. This is very important."
}
```

---

## πŸ“€ **Response Format**

```json
{
  "issues": [
    {
      "Id": "auto_27_40_13_passive_was_conducted",
      "start": 27,
      "length": 13,
      "end": 40,
      "paragraphKey": "para_0",
      "string": "was conducted",
      "type": "PassiveVoice",
      "suggestions": {
        "recommendation": [
          "Consider using active voice for clarity"
        ]
      }
    },
    {
      "Id": "auto_56_77_21_cliche_at_the_end_of_the_day",
      "start": 56,
      "length": 21,
      "end": 77,
      "paragraphKey": "para_0",
      "string": "at the end of the day",
      "type": "Cliche",
      "suggestions": {
        "recommendation": [
          "Avoid clichΓ©s",
          "Use original phrasing"
        ]
      }
    },
    {
      "Id": "auto_92_101_9_jargon_synergize",
      "start": 92,
      "length": 9,
      "end": 101,
      "paragraphKey": "para_0",
      "string": "synergize",
      "type": "BusinessJargon",
      "suggestions": {
        "recommendation": [
          "Avoid corporate jargon",
          "Use plain language"
        ]
      }
    },
    {
      "Id": "auto_120_124_4_vague_very",
      "start": 120,
      "length": 4,
      "end": 124,
      "paragraphKey": "para_0",
      "string": "very",
      "type": "VagueWord",
      "suggestions": {
        "recommendation": [
          "Be more specific",
          "Use concrete language"
        ]
      }
    }
  ],
  "summary": {
    "total_issues": 15,
    "word_count": 24,
    "sentence_count": 3,
    "style_score": 65
  }
}
```

---

## 🎨 **Issue Types**

| Type | Description | Example |
|------|-------------|---------|
| **PassiveVoice** | Passive voice construction | "was written", "has been done" |
| **Grammar_*** | Grammar issues | SubjectVerbAgreement, DoubleNegative |
| **StickySentence** | High glue word percentage | Sentences with >40% glue words |
| **OverusedWord** | Words used too frequently | "analysis" (8 times, 2.1% frequency) |
| **Repetition** | Repeated phrases | "the analysis", "in order to" |
| **Cliche** | Overused expressions | "at the end of the day" |
| **VagueWord** | Imprecise language | "thing", "stuff", "very", "really" |
| **BusinessJargon** | Corporate speak | "synergize", "leverage", "circle back" |

---

## πŸ’‘ **Usage Examples**

### **Example 1: Using cURL**
```bash
curl -X POST http://localhost:2000/analyze \
  -H "Content-Type: application/json" \
  -d '{
    "text": "The report was written. It is very important."
  }' | jq '.'
```

### **Example 2: Using Python**
```python
import requests
import json

url = "http://localhost:2000/analyze"
data = {
    "text": "The comprehensive analysis was conducted yesterday. At the end of the day, we need to synergize."
}

response = requests.post(url, json=data)
result = response.json()

print(f"Total issues: {result['summary']['total_issues']}")
print(f"Style score: {result['summary']['style_score']}")

for issue in result['issues']:
    print(f"\n{issue['type']} at position {issue['start']}-{issue['end']}")
    print(f"  Text: '{issue['string']}'")
    print(f"  Suggestions: {issue['suggestions']['recommendation']}")
```

### **Example 3: Using JavaScript/Node.js**
```javascript
const axios = require('axios');

async function analyzeText(text) {
  const response = await axios.post('http://localhost:2000/analyze', {
    text: text
  });
  
  const { issues, summary } = response.data;
  
  console.log(`Total issues: ${summary.total_issues}`);
  console.log(`Style score: ${summary.style_score}`);
  
  issues.forEach(issue => {
    console.log(`\n${issue.type} at ${issue.start}-${issue.end}`);
    console.log(`  "${issue.string}"`);
    console.log(`  Suggestions: ${issue.suggestions.recommendation.join(', ')}`);
  });
}

analyzeText("The document was written very carefully.");
```

### **Example 4: Using JavaScript/Fetch (Browser)**
```javascript
async function analyzeText(text) {
  const response = await fetch('http://localhost:2000/analyze', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ text })
  });
  
  const data = await response.json();
  
  // Highlight issues in text editor
  data.issues.forEach(issue => {
    highlightText(issue.start, issue.end, issue.type);
  });
  
  return data;
}

// Usage
const result = await analyzeText(document.getElementById('editor').value);
console.log('Analysis complete:', result.summary);
```

---

## πŸ”§ **Advanced Usage**

### **Batch Processing**
```bash
#!/bin/bash
# Process multiple texts
for file in texts/*.txt; do
    echo "Analyzing $file..."
    curl -X POST http://localhost:2000/analyze \
      -H "Content-Type: application/json" \
      -d "{\"text\": \"$(cat $file)\"}" \
      > "results/$(basename $file .txt).json"
done
```

### **Integration with Text Editor**
```javascript
// VS Code Extension example
class TextAnalyzerProvider {
  async provideDiagnostics(document) {
    const text = document.getText();
    const response = await fetch('http://localhost:2000/analyze', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ text })
    });
    
    const { issues } = await response.json();
    
    return issues.map(issue => ({
      range: new vscode.Range(
        document.positionAt(issue.start),
        document.positionAt(issue.end)
      ),
      message: issue.suggestions.recommendation.join('\n'),
      severity: this.getSeverity(issue.type)
    }));
  }
}
```

---

## 🎯 **Response Fields**

### **Issue Object**
| Field | Type | Description |
|-------|------|-------------|
| `Id` | string | Unique identifier for the issue |
| `start` | number | Start position (character index) |
| `end` | number | End position (character index) |
| `length` | number | Length in characters |
| `paragraphKey` | string | Paragraph identifier |
| `string` | string | The actual text with the issue |
| `type` | string | Type of issue |
| `suggestions` | object | Recommendations for fixing |

### **Summary Object**
| Field | Type | Description |
|-------|------|-------------|
| `total_issues` | number | Total number of issues found |
| `word_count` | number | Total word count |
| `sentence_count` | number | Total sentence count |
| `style_score` | number | Overall style score (0-100) |

---

## πŸš€ **Deployment**

### **Production Mode**
```bash
# Build optimized binary
cargo build --release --bin api-server

# Run in background
nohup ./target/release/api-server > api.log 2>&1 &

# Or use systemd
sudo systemctl start text-analyzer-api
```

### **Docker Deployment**
```dockerfile
FROM rust:1.75 as builder
WORKDIR /app
COPY . .
RUN cargo build --release --bin api-server

FROM debian:bookworm-slim
COPY --from=builder /app/target/release/api-server /usr/local/bin/
EXPOSE 2000
CMD ["api-server"]
```

```bash
docker build -t text-analyzer-api .
docker run -p 2000:2000 text-analyzer-api
```

---

## πŸ”’ **Security Notes**

- The API accepts any text input up to reasonable limits
- CORS is enabled (permissive) - configure as needed for production
- No authentication by default - add if needed
- Rate limiting not implemented - add if needed
- Consider adding request size limits for production

---

## ⚑ **Performance**

- Average response time: ~50-200ms for typical documents
- Handles texts up to 100KB efficiently
- Concurrent request support via Tokio async runtime
- Memory efficient streaming processing

---

## πŸ› **Error Responses**

### **Empty Text**
```json
{
  "error": "Text cannot be empty"
}
```
**Status Code:** 400 Bad Request

### **Analysis Error**
```json
{
  "error": "Failed to analyze text: [error details]"
}
```
**Status Code:** 500 Internal Server Error

---

## πŸ“Š **Testing**

### **Health Check**
```bash
curl http://localhost:2000/analyze \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{"text": "test"}'
```

### **Load Testing**
```bash
# Using Apache Bench
ab -n 1000 -c 10 -p test.json -T application/json \
  http://localhost:2000/analyze
```

---

## πŸŽ‰ **You're All Set!**

Start the server and begin analyzing text via REST API! πŸš€βœ¨