# Patto Note - LLM Agent Guide
## Project Overview
**Patto Note** is a plain-text note-taking format and ecosystem inspired by Cosense (formerly Scrapbox), designed for quick note-taking, task management, and outlining. It provides a Language Server Protocol (LSP) implementation, real-time preview, and editor integration.
**Repository**: https://github.com/ompugao/patto
**Language**: Rust (backend/LSP), TypeScript/JavaScript (VSCode extension, web preview), Lua/VimScript (Vim integration)
**Version**: 0.1.21
## Core Concepts
### 1. File Format (.pn files)
- **Line-oriented**: Every newline creates a new line
- **Indentation**: Leading hard tabs (`\t`) create hierarchical structure
- **Links**: WikiLinks `[note name]` or `[note name#anchor]` for cross-referencing
- **Properties**: Line properties like `{@anchor name}` or `{@task status=todo due=2024-12-31}`
- **Decorations**: Bold `[* text]`, italic `[/ text]`, code blocks, math expressions
- **Task Management**: Tasks with deadlines using `!YYYY-MM-DD` syntax
### 2. Architecture Components
#### Rust Backend (~5,400 LOC)
Located in `src/` and `src/bin/`:
1. **Parser** (`parser.rs`, `patto.pest`)
- Uses Pest parser generator for line-level parsing
- Grammar defined in `patto.pest`
- Produces AST (Abstract Syntax Tree) with `AstNode` structure
- Handles WikiLinks, anchors, tasks, code blocks, math, images, URLs
- Error handling with `ParserError` types
2. **Repository** (`repository.rs`)
- Manages workspace of `.pn` files
- Maintains document graph using `gdsl` crate for link relationships
- Tracks file metadata (modified time, created time, link count)
- Asynchronous workspace scanning with progress notifications
- Broadcasts change notifications via `tokio::sync::broadcast`
- Provides backlinks and 2-hop link computation
- Caches ASTs and document content using `DashMap`
3. **Line Tracker** (`line_tracker.rs`)
- Assigns stable IDs to lines based on content hashing
- Enables tracking of lines across document edits
- Uses content-based and position-based ID assignment
- Critical for real-time preview synchronization
4. **Renderer** (`renderer.rs`)
- Converts AST to HTML or Markdown
- `HtmlRenderer`: Produces HTML with classes for styling
- `MarkdownRenderer`: Exports to standard Markdown format
- Handles task checkboxes, code highlighting, math rendering
5. **LSP Server** (`bin/patto-lsp.rs`)
- Implements Language Server Protocol using `tower-lsp`
- Features:
- Go-to-definition for WikiLinks
- Find references (backlinks)
- Completion for note names and anchors
- Diagnostics for parse errors
- Custom commands: `aggregate_tasks`, `retrieve_two_hop_notes`, `scan_workspace`
- UTF-16 position encoding for compatibility
6. **Preview Server** (`bin/patto-preview.rs`)
- Axum-based web server for real-time preview
- WebSocket for live updates
- Serves Next.js static site
- Endpoints: file list, file content, backlinks, 2-hop links
- Auto-reloads on file changes using `notify` crate
7. **Utilities**
- `patto-html-renderer`: CLI tool to convert .pn to HTML
- `patto-markdown-renderer`: CLI tool to convert .pn to Markdown
- `utils.rs`: Helpers for YouTube embeds, Twitter embeds, Gyazo images
#### TypeScript/JavaScript
1. **VSCode Extension** (`client/src/extension.ts`)
- Language client for patto-lsp
- Task aggregation tree view
- Commands: `patto.tasks` to show workspace tasks
- Activates on `.pn` files
2. **Web Preview** (`patto-preview-next/`)
- Next.js React application
- Components:
- `Sidebar`: File list with sorting (modified, created, link count)
- `Preview`: Rendered HTML display with anchor navigation
- WebSocket integration for real-time updates
- Client-side routing for note navigation
- Features: backlinks panel, 2-hop links, search/filter
#### Vim Integration
1. **Neovim Lua** (`lua/patto.lua`)
- LSP configuration using `vim.lsp`
- Auto-launches preview server on file open
- Commands:
- `:LspPattoTasks` - Aggregate tasks in location list
- `:LspPattoTwoHopLinks` - Show 2-hop link graph
- `:LspPattoScanWorkspace` - Force workspace rescan
- Port finding logic for preview server
- Browser auto-launch (Linux/macOS/WSL)
2. **Vim-lsp Support** (`autoload/patto.vim`, `settings/vimlsp.vim`)
- Integration with `vim-lsp` plugin
- Syntax highlighting defined in `syntax/patto.vim`
- File type detection in `ftdetect/patto.vim`
## Key Data Structures
### AstNode
```rust
pub struct AstNode {
kind: AstNodeKind,
location: Location,
contents: Vec<AstNode>, // inline content
children: Vec<AstNode>, // nested children
stable_id: Option<i64>, // for line tracking
}
```
### AstNodeKind Variants
- `Dummy`: Root node
- `Line`: Represents a line with properties
- `WikiLink`: Link to another note `[link#anchor]`
- `UrlLink`, `MailLink`, `LocalFileLink`: External links
- `CodeBlock`, `MathBlock`, `QuoteBlock`, `TableBlock`: Block elements
- `CodeInline`, `MathInline`: Inline elements
- `RawText`, `DecoratedText`: Text content
### Properties
- `Anchor { name }`: Named anchor for linking
- `Task { status, due }`: Task with status (Todo/Doing/Done) and optional deadline
### Repository Message Types
- `FileChanged`, `FileAdded`, `FileRemoved`: File system events
- `BackLinksChanged`, `TwoHopLinksChanged`: Graph updates
- `ScanStarted`, `ScanProgress`, `ScanCompleted`: Workspace scanning
## Build System
### Rust
- **Cargo.toml**: Multi-binary project
- **Dependencies**: tower-lsp, pest, tokio, axum, notify, dashmap, gdsl
- **Binaries**: `patto-lsp`, `patto-preview`, `patto-html-renderer`, `patto-markdown-renderer`
- **Build script**: `build.rs` for embedding assets
### TypeScript/JavaScript
- **package.json**: VSCode extension + preview app
- **Webpack**: Bundles extension
- **Next.js**: Static site generation for preview
## Development Workflow
### Testing
```bash
cargo build --release
cargo test
```
### Running LSP Server
```bash
cargo run --bin patto-lsp
```
### Running Preview Server
```bash
cargo run --bin patto-preview -- /path/to/notes --port 3000
```
### Building VSCode Extension
```bash
npm run build
npm run package
```
## Important Files for Modifications
### Adding Grammar Features
1. **src/patto.pest**: Define new syntax rules
2. **src/parser.rs**: Parse new rules into AST nodes
3. **src/renderer.rs**: Render new elements to HTML/Markdown
### Adding LSP Features
1. **src/bin/patto-lsp.rs**: Implement LSP methods
2. **src/repository.rs**: Update graph/cache logic if needed
### Adding Preview Features
1. **patto-preview-next/src/**: React components
2. **src/bin/patto-preview.rs**: WebSocket message types, API endpoints
### Adding Vim Features
1. **lua/patto.lua**: Neovim commands and keybindings
2. **autoload/patto.vim**: Vim-lsp integration
## Common Patterns
### Parsing Flow
1. Text → `parse_text()` → AST + Diagnostics
2. Each line parsed independently via Pest grammar
3. Indentation handled manually to build tree structure
4. WikiLinks extracted and added to document graph
### Update Flow (Preview)
1. File system change detected by `notify`
2. Repository re-parses file → broadcasts `FileChanged`
3. WebSocket sends update to preview client
4. React component re-renders with new HTML
### Link Resolution
1. WikiLink `[note name]` → `note name.pn` in workspace
2. Link graph: `PathBuf` → `Vec<PathBuf>` (forward links)
3. Backlinks computed by inverting graph
4. 2-hop links: backlinks of backlinks
## Configuration
### LSP Capabilities
- `textDocument/definition`: Jump to WikiLink target
- `textDocument/references`: Find backlinks
- `textDocument/completion`: Note/anchor completion
- `textDocument/diagnostic`: Parse error reporting
- `workspace/executeCommand`: Custom commands
### Preview Server
- Port: Default 3000 (configurable)
- WebSocket: `/ws` endpoint
- Static files: Embedded Next.js build
- File watching: Recursive on workspace root
## Testing Strategy
- **Parser tests**: Ensure grammar correctly parses all syntax
- **Renderer tests**: Verify HTML/Markdown output
- **LSP tests**: Test definition, references, completion
- **Integration tests**: Full workflow with example notes
## Performance Considerations
- **Lazy workspace scanning**: Background task on startup
- **DashMap**: Concurrent cache access without locks
- **Line tracking**: Fast content-based hashing for stable IDs
- **Graph caching**: Avoid re-parsing for link queries
- **WebSocket**: Efficient real-time updates vs polling
## Extension Points
1. **New syntax**: Add to `patto.pest` + parser + renderer
2. **New LSP commands**: Add to `patto-lsp.rs` + Vim/VSCode integrations
3. **New preview features**: Add WebSocket message types + React components
4. **Export formats**: New renderer implementations (PDF, etc.)
5. **Link types**: Extend `AstNodeKind` for new link semantics
## Dependencies of Note
- **tower-lsp**: LSP framework for Rust
- **pest**: Parser generator with PEG grammar
- **axum**: Web framework for preview server
- **gdsl**: Graph data structure for link relationships
- **dashmap**: Concurrent hashmap for caching
- **notify**: File system watching
- **ropey**: Rope data structure for text editing
## Common Tasks for LLM Agents
### Adding a New Syntax Element
1. Define grammar rule in `src/patto.pest`
2. Add variant to `AstNodeKind` in `src/parser.rs`
3. Parse the element in parser logic
4. Add rendering logic in `src/renderer.rs` for HTML and Markdown
5. Update syntax highlighting if needed
### Adding a New LSP Feature
1. Implement handler in `src/bin/patto-lsp.rs`
2. Update repository queries if needed (`src/repository.rs`)
3. Add Vim/Neovim command in `lua/patto.lua`
4. Add VSCode command in `client/src/extension.ts`
### Fixing Parser Issues
1. Check grammar in `src/patto.pest`
2. Test with failing input using parser tests
3. Adjust rule precedence or structure
4. Verify AST structure matches expectations
### Debugging Preview Issues
1. Check WebSocket message flow in browser DevTools
2. Verify file watching in `src/bin/patto-preview.rs`
3. Check React component rendering in `patto-preview-next/src/`
4. Ensure line IDs are stable (check `line_tracker.rs`)
## File Structure Summary
```
patto/
├── src/ # Rust library code
│ ├── parser.rs # AST and parsing logic
│ ├── repository.rs # Workspace and graph management
│ ├── renderer.rs # HTML/Markdown rendering
│ ├── line_tracker.rs # Stable line ID tracking
│ ├── patto.pest # Grammar definition
│ └── bin/ # Executable binaries
│ ├── patto-lsp.rs # LSP server
│ ├── patto-preview.rs # Preview web server
│ ├── patto-html-renderer.rs
│ └── patto-markdown-renderer.rs
├── client/ # VSCode extension (TypeScript)
│ └── src/extension.ts
├── patto-preview-next/ # Web preview (Next.js/React)
│ └── src/
│ ├── app/ # Pages and components
│ └── lib/ # WebSocket and routing
├── lua/patto.lua # Neovim LSP configuration
├── autoload/patto.vim # Vim plugin autoload
├── plugin/patto.vim # Vim plugin entry
├── syntax/patto.vim # Vim syntax highlighting
└── ftdetect/patto.vim # Vim filetype detection
```
## Notes for LLM Agents
- **Preserve compatibility**: Changes to parser affect all renderers and LSP
- **UTF-16 encoding**: LSP uses UTF-16 positions, use `str_indices` crate helpers
- **Async operations**: Repository scanning is async to avoid blocking LSP
- **Line stability**: `stable_id` critical for preview scroll sync
- **Graph consistency**: Keep `document_graph` and `link_graph` in sync
- **Error handling**: Parser errors become LSP diagnostics
- **Testing**: Add tests for any parser/renderer changes
- **Documentation**: Update README.md for user-facing changes