LocalGPT

Build explorable 3D worlds with natural language — geometry, materials, lighting, audio, and behaviors. Open source, runs locally.

Install

# World Building
cargo install localgpt-gen

# AI Assistant (chat, memory, daemon)
cargo install localgpt

Gen Mode (World Building)

localgpt-gen is a standalone binary for AI-driven 3D world creation with the Bevy game engine.

# Start interactive mode
localgpt-gen

# Start with an initial prompt
localgpt-gen "Create a desert scene with pyramids and a UFO hovering above"

# Load an existing scene
localgpt-gen --scene ./world.glb

# Verbose logging
localgpt-gen --verbose

Features

• Parametric shapes — box, sphere, cylinder, capsule, plane, torus, pyramid, tetrahedron, icosahedron, wedge
• PBR materials — color, metalness, roughness, emissive, alpha, double-sided
• Lighting — point, spot, directional lights with color and intensity
• Behaviors — orbit, spin, bob, look_at, pulse, path_follow, bounce
• Audio — ambient sounds (wind, rain, forest, ocean, cave) and spatial emitters
• Export — glTF/GLB, HTML (browser-viewable), screenshots
• World skills — save/load complete worlds as reusable skills

Headless Mode & Experiment Queue

Queue world experiments and generate without a window — overnight batch runs, CI pipelines, or scripted variations:

# Generate a single world (no window)
localgpt-gen headless --prompt "Build a cozy cabin in a snowy forest"

# With style hint
localgpt-gen headless --prompt "Village marketplace" --style "Studio Ghibli"

The memory system learns your creative style across sessions — palettes, lighting preferences, entity templates — and applies them automatically in future generations.

Full docs: Headless Mode & Experiment Queue

MCP Server

Use Gen from any MCP-compatible tool (Claude CLI, Codex CLI, Gemini CLI, VS Code, Zed, Cursor):

localgpt-gen mcp-server

Add to your .mcp.json:

{
  "mcpServers": {
    "localgpt-gen": {
      "command": "localgpt-gen",
      "args": ["mcp-server"]
    }
  }
}

When using Gen interactively with a CLI backend, use --connect to route tool calls to your existing window instead of spawning a new one. See CLI Mode (MCP Relay).

Full docs: LocalGPT Gen | MCP Server

Built something cool? Share on Discord or YouTube!

AI Assistant

localgpt is a local-first AI assistant with persistent memory, autonomous tasks, and multiple interfaces.

# Interactive chat
localgpt chat

# Single question
localgpt ask "What is the meaning of life?"

# Run as daemon with HTTP API and web UI
localgpt daemon start

Why LocalGPT?

• Single binary — no Node.js, Docker, or Python required
• Local device focused — runs entirely on your machine, your data stays yours
• Persistent memory — markdown-based knowledge store with full-text and semantic search
• Hybrid web search — native provider search passthrough plus client-side fallback
• Autonomous heartbeat — delegate tasks and let it work in the background
• Multiple interfaces — CLI, web UI, desktop GUI, Telegram bot
• Defense-in-depth security — signed policy files, kernel-enforced sandbox, prompt injection defenses
• Multiple LLM providers — Anthropic, OpenAI, xAI, Ollama, GLM, Vertex AI, CLI providers

How It Works

LocalGPT uses XDG-compliant directories for config/data/state/cache. Run localgpt paths to see resolved paths.

Workspace memory layout:

<workspace>/
├── MEMORY.md     # Long-term knowledge (auto-loaded each session)
├── HEARTBEAT.md  # Autonomous task queue
├── SOUL.md       # Personality and behavioral guidance
└── knowledge/    # Structured knowledge bank

Files are indexed with SQLite FTS5 for keyword search and sqlite-vec for semantic search with local embeddings.

Configuration

Stored at <config_dir>/config.toml:

[agent]
default_model = "claude-cli/opus"

[providers.anthropic]
api_key = "${ANTHROPIC_API_KEY}"

[heartbeat]
enabled = true
interval = "30m"

[telegram]
enabled = true
api_token = "${TELEGRAM_BOT_TOKEN}"

Full config reference: website/docs/configuration.md

Security

• Kernel-enforced sandbox — Landlock/seccomp on Linux, Seatbelt on macOS
• Signed policy files — HMAC-SHA256 signed LocalGPT.md with tamper detection
• Prompt injection defenses — marker stripping, pattern detection, content boundaries
• Audit chain — hash-chained security event log

Security docs: website/docs/sandbox.md | website/docs/localgpt.md

HTTP API

Full API reference: website/docs/http-api.md

CLI Commands

localgpt chat                   # Interactive chat
localgpt ask "question"         # Single question
localgpt daemon start           # Start daemon
localgpt memory search "query"  # Search memory
localgpt config show            # Show config
localgpt paths                  # Show resolved paths

Full CLI reference: website/docs/cli-commands.md

Blog

• Explorable World as Agent Skill
• Why I Built LocalGPT in 4 Nights

Built With

Rust, Tokio, Axum, Bevy, SQLite (FTS5 + sqlite-vec), fastembed, eframe

Contributors

License

Apache-2.0