# Contributing to GrumpyDB
## Prerequisites
- **Rust** (edition 2024) — install via [rustup](https://rustup.rs/)
- `cargo`, `clippy`, `rustfmt` (included with rustup)
## Commands
```bash
cargo build --workspace # Build all crates
cargo test --workspace # All tests (~515 across all workspace crates)
cargo test --lib # Unit tests only (current crate)
cargo test --test '*' # Integration tests only
cargo clippy --workspace --all-targets -- -D warnings # Lint (strict, zero warnings)
cargo fmt --all -- --check # Check formatting
cargo doc --workspace --no-deps # Generate docs
# Optional cloud SDK feature flags (used by snapshot/restore — see below).
cargo build --workspace --features grumpydb-server/cloud-aws
cargo build --workspace --features grumpydb-server/cloud-azure
cargo build --workspace --features grumpydb-server/cloud-aws,grumpydb-server/cloud-azure
# Benchmarks (criterion). HTML reports land under target/criterion/report/.
cargo bench # All benches: engine + protocol
cargo bench --bench engine # Engine-only benches
cargo bench --bench protocol # Protocol-only benches
cargo bench -- --quick # Quick smoke run (used by CI's bench-smoke)
# Fuzzing (requires `cargo install cargo-fuzz` and a nightly toolchain).
# The fuzz/ crate is intentionally excluded from the workspace.
cd fuzz && cargo +nightly fuzz run parse_command
cd fuzz && cargo +nightly fuzz run value_codec_roundtrip
cd fuzz && cargo +nightly fuzz run wal_record_decode
cd fuzz && cargo +nightly fuzz run response_serialize
# Run a specific binary
# Note: the server requires --bootstrap-password on the FIRST start
# (or the env variable GRUMPYDB_BOOTSTRAP_PASSWORD). Subsequent starts on
# the same data directory do not need it.
cargo run -p grumpydb-server -- --no-tls --data ./data \
--bootstrap-password "dev-only-password"
# Optional server flags: --log-format json|text (default: json, or text on a TTY).
# RUST_LOG is honored, e.g. RUST_LOG=grumpydb_server=debug.
cargo run -p grumpy-repl
cargo run --example taskman -- help
# Snapshot / restore subcommands (Phase 38). Local destination always
# available; s3:// requires the cloud-aws feature, az:// requires
# cloud-azure. The server must be stopped (snapshot/restore operate on
# files directly in v5).
cargo run -p grumpydb-server -- snapshot --data ./data ./backup.tar.gz
cargo run -p grumpydb-server -- restore --data ./data ./backup.tar.gz
cargo run -p grumpydb-server --features cloud-aws -- \
snapshot --data ./data s3://my-bucket/grumpydb/2026-04-28.tar.gz
cargo run -p grumpydb-server --features cloud-azure -- \
restore --data ./data az://my-container/grumpydb/2026-04-28.tar.gz --force
# Docker stack (Phase 37): server + Prometheus + Grafana.
# Requires GRUMPYDB_BOOTSTRAP_PASSWORD set in .env (copy from .env.example).
docker compose up -d # full stack
docker compose up -d server # server only
docker compose --profile repl run --rm repl \
--host server --tenant _system --user admin \
--password "$(grep ^GRUMPYDB_BOOTSTRAP_PASSWORD .env | cut -d= -f2-)"
# 3-node cluster smoke test (v5 demo compose).
# Defaults to GRUMPYDB_BOOTSTRAP_PASSWORD=admin if unset.
scripts/smoke_cluster.sh
GRUMPYDB_BOOTSTRAP_PASSWORD=monsecret scripts/smoke_cluster.sh --keep-up
# TypeScript driver (drivers/typescript/)
cd drivers/typescript && npm ci && npm run lint && npm test && npm run build
```
## Project structure
```
grumpydb/ # workspace root
├── Cargo.toml # workspace members
│
├── .github/workflows/ # CI: fmt, clippy, test (stable + 1.85 MSRV), docs, audit, typescript-driver
│
├── src/ # grumpydb crate (storage engine library)
│ ├── error.rs # GrumpyError enum, Result<T> alias
│ ├── naming.rs # Name validation: [a-z0-9_]{1,64}, reserved: _default, _system
│ ├── lib.rs # Public API, re-exports
│ ├── engine.rs # GrumpyDb — thin wrapper over Collection + WAL
│ ├── database/ # Database — multi-collection management with shared WAL
│ ├── collection/ # Collection — unit of document storage, raw CRUD, compact
│ ├── index/ # Secondary indexes: encoding, SecondaryIndex, IndexDefinition
│ ├── page/ # Page storage (8 KiB), slotted layout, overflow, free-list
│ ├── btree/ # B+Tree (fixed UUID + variable-length keys)
│ ├── document/ # Value enum + binary codec
│ ├── wal/ # Write-Ahead Log: record, writer, recovery
│ ├── buffer/ # LRU buffer pool with dirty tracking
│ ├── concurrency/ # SharedDb / SharedDatabase / SharedServer (SWMR)
│ └── server/ # GrumpyServer + Client (multi-tenant)
│
├── grumpydb-protocol/ # RESP-like wire protocol crate
│ └── src/{lib,command,response,parser}.rs
│
├── grumpydb-server/ # TCP/TLS server binary + library
│ └── src/
│ ├── lib.rs main.rs config.rs
│ ├── limits.rs # governor token bucket + login backoff + conn caps
│ ├── http.rs # /healthz /readyz /metrics (hyper 1.x, port 6381)
│ ├── snapshot.rs # tar.gz snapshot/restore (local + s3:// + az://)
│ ├── auth/{user,role,jwt,store}.rs # argon2, JWT HS256, AuthStore
│ ├── session/mod.rs # SessionContext + RBAC enforcer
│ └── tcp/{listener,handler}.rs # tokio + tokio-rustls
│
├── grumpydb-client/ # Async Rust client driver
│ └── src/{lib,connection,error}.rs
│
├── grumpy-repl/ # Interactive REPL binary (dual-mode: embedded + TCP)
│ └── src/{main,repl,parser,filter,json_parser,tcp_backend}.rs
│
├── grumpydb-testing/ # Internal test harness crate (publish = false).
│ └── src/{lib,server}.rs # TestServer: spawns the real server binary,
│ # random port + tempdir + auto-kill on Drop;
│ # exposes crash() (SIGKILL) and restart()
│ # for crash-recovery tests.
│
├── drivers/typescript/ # @grumpydb/client npm package (Node ≥ 18)
│ └── src/{index,client,database,connection,protocol,auth,types,errors}.ts
│
├── examples/
│ └── taskman/ # Demo task manager (embedded engine)
│
├── benches/ # Criterion benchmarks (cargo bench)
│ ├── engine.rs # 8 benches: insert (small/medium/4 KB), get,
│ │ # scan, index exact/range
│ └── protocol.rs # 3 benches: parse simple cmd, parse 1 KB INSERT,
│ # serialize 100-bulk array
│
├── fuzz/ # cargo-fuzz targets (excluded from workspace)
│ ├── Cargo.toml
│ ├── corpus/<target>/ # Seed corpora
│ └── fuzz_targets/
│ ├── parse_command.rs # Protocol parser
│ ├── value_codec_roundtrip.rs # Document codec encode/decode stability
│ ├── wal_record_decode.rs # WAL record decoder
│ └── response_serialize.rs # Protocol response serializer
│
├── tests/ # Integration tests
│ ├── crud_test.rs # Engine CRUD
│ ├── stress_test.rs # Concurrency stress
│ ├── server_e2e.rs # TCP end-to-end (8 tests, uses TestServer)
│ ├── server_concurrency.rs # 50 concurrent clients × 100 ops
│ ├── server_auth.rs # Expired/tampered tokens, role enforcement, login rate limit
│ ├── server_http.rs # /healthz /readyz /metrics e2e (Phase 36)
│ ├── snapshot_e2e.rs # snapshot → restore round-trip via TestServer (Phase 38)
│ └── crash_recovery.rs # 6 crash-and-restart scenarios
│
├── Dockerfile.server # Multi-stage: rust:1.88 builder → distroless runtime (~30 MB)
├── Dockerfile.repl # Same builder, ships grumpy-repl
├── Dockerfile.publish-ci # CI bash image used for crates.io publication
├── docker-compose.yml # server + prometheus + grafana + repl (profile-gated)
├── docker/
│ ├── prometheus.yml # scrape config for server:6381
│ └── grafana/provisioning/datasources/prometheus.yml
├── .env.example # GRUMPYDB_BOOTSTRAP_PASSWORD template
├── .dockerignore # Excludes heavy/non-runtime paths (target/, docs/, tests/, examples/, ...); keeps benches/ for workspace Docker builds
│
└── docs/
├── ARCHITECTURE.md
├── IMPLEMENTATION_PLAN.md # phases 1–8: storage engine
├── IMPLEMENTATION_PLAN_V2.md # phases 9–15: server + concurrency + shell
├── IMPLEMENTATION_PLAN_V3.md # phases 16–23: protocol + auth + TCP + drivers
└── IMPLEMENTATION_PLAN_V4.md # phases 24–43: hardening, observability, RS256/JWKS, replication, MVCC
```
## Code conventions
### Naming
- `snake_case` for functions and variables
- `CamelCase` for types (structs, enums, traits)
- `UPPER_SNAKE_CASE` for constants
- Test functions: `test_<module>_<expected_behavior>`
### Visibility
- `pub(crate)` by default
- `pub` only for the public API in `lib.rs`
### Error handling
- Use `thiserror` for error definitions
- All functions return `Result<T, GrumpyError>`
- **No `unwrap()`, `expect()`, or `panic!` in `src/`** — enforced by the
crate-level lint
`#![cfg_attr(not(test), warn(clippy::unwrap_used, clippy::panic, clippy::expect_used))]`
in `src/lib.rs`. Use `?` propagation; for "shouldn't happen" cases return
`GrumpyError::Corruption(...)`.
- The lint is allowed inside `#[cfg(test)]` modules.
### Serialization
- All binary formats use **little-endian** byte order
- No `unsafe` code unless documented and justified
### Tests — mandatory
Every `.rs` file must have a `#[cfg(test)] mod tests` block.
Each feature needs at minimum:
- Happy path
- Edge cases (page full, overflow, B+Tree split)
- Error cases (I/O failure, missing key, corruption)
Use `tempfile::TempDir` for any test involving disk I/O.
## Workflow
1. Read the relevant skill file in `.claude/skills/` before coding
2. Implement the feature
3. Write unit tests in the same file
4. Run: `cargo test --workspace && cargo clippy --workspace --all-targets -- -D warnings && cargo fmt --all -- --check`
5. Write integration tests in `tests/` if the feature spans multiple modules
6. Update documentation (the docs-agent handles this automatically)
Every push to `master` and every PR is validated by GitHub Actions
(`.github/workflows/ci.yml`): `fmt`, `clippy --all-targets -D warnings`,
`test` (matrix: stable + 1.85 MSRV), `docs`, `audit`. A red CI blocks merge.
## Dependencies
| `uuid` | UUID v4 key generation |
| `thiserror` | Error type definitions |
| `crc32fast` | CRC32 checksums for pages and WAL |
| `parking_lot` | Fast RwLock/Mutex for SWMR concurrency |
| `tempfile` | Temporary directories for tests |
| `rand` | Random data generation for tests |
| `criterion` | Benchmark harness (`benches/engine.rs`, `benches/protocol.rs`) |
| `rustyline` | Line editing for grumpy-repl (binary crate) |
| `serde_json` | JSON serialization for grumpy-repl (binary crate) |
| `futures` | `FutureExt::catch_unwind` for panic isolation in `grumpydb-server` |
| `tracing` | Structured logging spans + events in `grumpydb-server` |
| `tracing-subscriber` (`env-filter`, `json`) | JSON log subscriber, `RUST_LOG` env-filter |
| `governor` | Token-bucket rate limiter (per-IP) in `grumpydb-server::limits` |
| `nonzero_ext` | `nonzero!` macro feeding `governor`'s `NonZeroU32` quotas |
| `hyper` (1.x) | Tiny HTTP server for `/healthz`, `/readyz`, `/metrics` (port 6381) |
| `hyper-util` | `TokioExecutor` glue for `hyper 1.x` server runtime |
| `metrics` | Counter / gauge / histogram facade used throughout `grumpydb-server` |
| `metrics-exporter-prometheus` | Prometheus exposition format renderer for `/metrics` |
| `tar` | Tar archive read/write for snapshot/restore tooling |
| `flate2` | Gzip codec wrapping the tar streams |
| `sha2` | SHA-256 per-file checksums in `snapshot.json` |
| `hex` | Hex encoding of the SHA-256 digests in the manifest |
| `aws-sdk-s3` *(optional, feature `cloud-aws`)* | S3 backend for snapshot/restore |
| `aws-config` *(optional, feature `cloud-aws`)* | Standard AWS credential chain |
| `azure_storage` *(optional, feature `cloud-azure`)* | Azure Blob storage core types |
| `azure_storage_blobs` *(optional, feature `cloud-azure`)* | Azure Blob backend for snapshot/restore |
| `azure_identity` *(optional, feature `cloud-azure`)* | `DefaultAzureCredential` chain |
### Workspace crates
| `grumpydb` | Core storage engine (library) |
| `grumpydb-protocol` | RESP-like wire protocol (commands, responses, parser) |
| `grumpydb-server` | TCP+TLS server binary (tokio, JWT auth, RBAC, structured tracing) |
| `grumpydb-client` | Async Rust client driver |
| `grumpy-repl` | Interactive REPL shell binary (embedded + TCP) |
| `grumpydb-testing` | Internal test harness (`TestServer`); `publish = false`, never released |
## Contribution licensing
Unless you explicitly state otherwise, any contribution intentionally
submitted for inclusion in GrumpyDB by you shall be dual-licensed as
`MIT OR Apache-2.0` (as defined in `LICENSE`, `LICENSE-MIT`, and
`LICENSE-APACHE`), without additional terms or conditions.