RookDb SQL Compiler (lexical_parser)

Lightweight SQL lexer + parser in Rust that converts SQL queries into an Abstract Syntax Tree (AST).

📖 Overview

This project is a small SQL front-end designed for learning, experimentation, and compiler-course style work.

It helps you:

• tokenize SQL input (lexical analysis),
• parse supported SQL grammar (syntactic analysis),
• inspect AST output in debug and readable formats.

Problem solved: it provides a clean way to understand how SQL text becomes structured syntax trees.

Who it is for:

• students learning compiler construction,
• beginners learning Rust parsing patterns,
• developers prototyping SQL tooling.

✨ Features

• SQL tokenization using sqlparser tokenizer + custom token classification.
• Recursive-descent parser with expression precedence handling.
• AST generation for common statement families.
• Supports SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, ALTER.
• Join support: INNER, LEFT, RIGHT, FULL, CROSS.
• Multiple runtime input modes:
  • command argument SQL,
  • --stdin piped SQL,
  • interactive prompt,
  • demo-query fallback.
• Separate integration test suites for lexer and parser.

🛠 Tech Stack

📦 Prerequisites / Required Installations

System requirements

• Windows, Linux, or macOS
• Terminal/command prompt

Required software

• Rust toolchain (recommended stable)
• Cargo (bundled with Rust)

Install Rust (if needed)

# Linux/macOS
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Windows
# Install from: https://rustup.rs/

Verify installation:

rustc --version
cargo --version

Assumption: Rust stable 1.75+ is sufficient (exact minimum version is not pinned in this repo).

🚀 Installation

1. Clone the repository:

git clone <your-repo-url>
cd RookDb_IS/Code/compiler

2. Build once to fetch dependencies:

cargo build

3. (Optional) Run tests to verify setup:

cargo test

Environment variables

No .env file is required.

Assumption: Default logger configuration is sufficient for local usage.

▶️ How to Run

Development run

cargo run

Behavior with no arguments:

1. prompts for one SQL query,
2. if you press Enter without input, it runs built-in demo queries.

Run a single SQL query (argument mode)

cargo run -- "SELECT id, name FROM users WHERE age > 18;"

Run from stdin (recommended for pipes)

echo "SELECT name FROM users;" | cargo run -- --stdin

Production-style run

cargo run --release -- "SELECT * FROM users;"

⚙️ Configuration

This project currently has minimal runtime configuration.

No custom config file is required.

🧠 How It Works (Architecture / Code Flow)

High-level flow:

1. Input collection (src/main.rs)

• read SQL from argument, --stdin, interactive input, or demo list.

2. Lexical analysis (src/lib.rs)

• LexicalParser tokenizes raw SQL and maps tokens to TokenType.

3. Syntactic analysis (src/parser.rs)

• SyntacticParser consumes tokens and builds AST nodes.

4. AST output (src/main.rs)

• prints both debug AST and display-formatted AST.

Key modules:

• src/lib.rs: exported lexer/token API + module exports.
• src/parser.rs: recursive-descent SQL parser.
• src/ast.rs: AST types for statements/expressions/clauses.
• src/main.rs: CLI runtime entry point.

📁 Project Structure

compiler/
├─ Cargo.toml
├─ README.md
├─ src/
│  ├─ main.rs        # CLI/runtime entry
│  ├─ lib.rs         # Public lexer/token API
│  ├─ parser.rs      # Syntactic parser
│  └─ ast.rs         # AST definitions
├─ tests/
│  ├─ lexer_tests.rs
│  └─ parser_tests.rs
└─ target/           # Build artifacts (generated)

🧪 Testing

Test framework: Rust built-in test framework (cargo test).

Run all tests:

cargo test

Run with strict linting:

cargo clippy --all-targets -- -D warnings

🚧 Known Limitations / Future Improvements

• Supports a practical SQL subset, not full SQL standard compliance.
• Error diagnostics can be improved (friendlier messages, richer context).
• No benchmark/performance profiling pipeline yet.
• No CI workflow file included yet.
• No fuzz/property-based parser robustness tests yet.