Kelora

Scriptable log processor for the command line. Treats logs as structured events and lets you filter, transform, and analyze them using embedded Rhai scripts with 40+ built-in functions.

[!NOTE] Pre-1.0 software using AI-generated code. Validated through automated testing, not manual review. Breaking changes may occur without migration paths. Backed by 770+ tests plus cargo-audit/deny; see SECURITY.md for details.

Overview

Kelora parses log streams into structured events and runs them through a programmable pipeline powered by Rhai scripting.

• Turns lines into structured events you can access as e.field or e["field-name"].
• Embeds 40+ built-in Rhai helpers for parsing, enrichment, metrics, and windowed analysis.
• Speaks JSON, logfmt, syslog, CSV/TSV, column specs, and gzip-compressed inputs out of the box.
• Handles streaming or batch workloads with sequential and --parallel execution modes.
• Emits metrics and processing stats so you can observe pipelines while they run.

Quick Examples

# Parse embedded formats - extract logfmt from within syslog messages
kelora -f syslog examples/simple_syslog.log \
  --exec 'if e.msg.contains("=") { e += e.msg.parse_logfmt() }' \
  --keys timestamp,host,user,action,detail,message \
  -F json

# Keep full stacktraces together with case-insensitive search
kelora examples/multiline_stacktrace.log \
  --multiline timestamp \
  --filter 'e.line.lower().contains("valueerror")' \
  --before-context 1 --after-context 1

# Extract container prefixes, track log volume by source
kelora examples/prefix_docker.log --extract-prefix container \
  --exec 'e.level = e.line.between("[", "]")' \
  --metrics \
  --exec 'track_count(e.container); track_count(e.level)' \
  --keys container,level,line \
  -F csv

# Parse JWT tokens, mask IPs for privacy-safe log sharing
kelora -j examples/security_audit.jsonl \
  --exec 'if e.has_field("token") {
            let jwt = e.token.parse_jwt();
            e.role = jwt.get_path("claims.role", "guest")
          }' \
  --exec 'e.ip = e.ip.mask_ip(2)' \
  --keys timestamp,event,role,ip \
  -F json

More quick commands to copy-paste:

• Stream-level error watch: tail -f examples/simple_json.jsonl | kelora -j --levels warn,error --exec 'track_count(e.service)' --metrics
• Fan out nested arrays: kelora -j examples/json_arrays.jsonl --exec 'emit_each(e.get_path(\"users\", []))' --keys id,name,score
• Visual level distribution: kelora -f logfmt examples/simple_logfmt.log -F levelmap

[!TIP] These examples use files in examples/ — see examples/README.md for the full catalog. For a complete walkthrough with annotated output, visit the Quickstart Guide.

Installation

Download from GitHub Releases (macOS, Linux, Windows) or:

cargo install kelora

Documentation

Full documentation is available at kelora.dev. Some quick links:

• Quickstart — 5-minute tour with annotated output
• How-To Guides — solutions for common tasks
• Tutorials — step-by-step guides for building custom pipelines
• Concepts — deep dive into the streaming pipeline
• Reference — CLI flags, Rhai functions, formats, and configuration

Highlights

Kelora is built for streaming-first log analysis with a programmable Rhai core.

• Streaming pipeline — Parse, filter, transform, and output logs in one flow. Read the Pipeline Model for a stage-by-stage breakdown.
• Built-in Rhai toolbox — 100+ helpers for enrichment, parsing, time-window analysis, and metrics. Scan the Functions Reference for the full catalog.
• Format flexibility — JSON, logfmt, syslog, Apache/Nginx combined, CSV/TSV, column specs, and gzip. See Input Formats.
• Powerful filtering — Chain --filter, --level, --since/--until, and context flags to zero in on events. Walkthroughs in Filtering How-To.
• Span aggregations — --span forms count- or time-based spans and triggers --span-close hooks for per-span summaries. See CLI Reference for usage patterns.
• Observability built in — --metrics, --stats, and window helpers expose throughput and aggregations for live pipelines. Learn more in Metrics & Telemetry.
• Parallel or streaming — Stay sequential for tailing or enable --parallel for archive crunching. Tuning guidance in Parallel Processing.

Input → Parse → --exec → --filter → --exec → --filter → ... → Output
  ↓       ↓         ↓         ↓         ↓         ↓              ↓
Files   JSON   transform  narrow   enrich    narrow        logfmt
stdin   syslog                                              JSON
.gz     custom                                              CSV

Works Well With

Kelora focuses on normalising noisy logs into structured data. Pipe or export Kelora's output to complementary tools for deeper analysis:

• jq — process Kelora's JSON output for complex transformations, filtering, or reformatting
• lnav — explore Kelora's output in an interactive TUI with live filtering, histograms, and ad-hoc SQL queries
• qsv — analyze Kelora's CSV output with statistical operations, joins, and aggregations
• SQLite/DuckDB — load Kelora's CSV/JSON output into a database for SQL queries and reporting
• miller — transform Kelora's CSV output for reshaping, aggregating, and format conversion

For performance comparisons and guidance on choosing between these tools for different tasks, see the Performance Comparisons guide.

License

Kelora is open source software licensed under the MIT License.