lazylog-framework

A powerful, extensible framework for building terminal-based log viewers with vim-like navigation and real-time monitoring capabilities.

Features

• Provider-based architecture - Pluggable log sources
• Vim-like navigation - j/k, gg/G, Ctrl+d/u, and more
• Real-time streaming - Monitor logs as they arrive
• Filtering - Dynamic log filtering with / search
• Detail levels - Control information density (0-4)
• Mouse support - Click and scroll
• Modern UI - Clean interface with tailwind-inspired colors
• Clipboard integration - Yank logs with y
• Memory-efficient - Ring buffer prevents unbounded growth

Installation

[dependencies]
lazylog-framework = "0.3"

Quick Start

use lazylog_framework::{LogProvider, LogParser, LogItem, start_with_provider};
use anyhow::Result;
use std::sync::Arc;

// 1. implement LogProvider for your log source
struct MyLogProvider;

impl LogProvider for MyLogProvider {
    fn start(&mut self) -> Result<()> {
        // setup resources (open files, connect to streams, etc.)
        Ok(())
    }

    fn stop(&mut self) -> Result<()> {
        // cleanup resources
        Ok(())
    }

    fn poll_logs(&mut self) -> Result<Vec<String>> {
        // return raw log strings since last poll (non-blocking)
        Ok(vec!["2025-01-15 10:30:00 INFO Application started".to_string()])
    }
}

// 2. implement LogParser to format your logs
struct MyLogParser;

impl LogParser for MyLogParser {
    fn parse(&self, raw_log: &str) -> Option<LogItem> {
        // parse raw string into LogItem
        Some(LogItem::new(raw_log.to_string(), raw_log.to_string()))
    }

    fn format_preview(&self, item: &LogItem, detail_level: u8) -> String {
        // format log for display at given detail level
        match detail_level {
            0 => item.content.clone(),
            _ => format!("[{}] {}", item.time, item.content),
        }
    }

    fn get_searchable_text(&self, item: &LogItem, _detail_level: u8) -> String {
        // return text that should be searchable
        item.content.clone()
    }
}

// 3. run the application
fn main() -> Result<()> {
    use ratatui::{Terminal, backend::CrosstermBackend};
    use std::io;

    let mut terminal = Terminal::new(CrosstermBackend::new(io::stdout()))?;
    let provider = MyLogProvider;
    let parser = Arc::new(MyLogParser);

    lazylog_framework::start_with_provider(&mut terminal, provider, parser)?;
    Ok(())
}

Architecture

The framework uses a two-trait system that separates log acquisition from parsing:

┌──────────────┐   poll_logs()      ┌─────────────┐
│ LogProvider  │ ─────────────────> │ Vec<String> │ (raw logs)
└──────────────┘                    └──────┬──────┘
                                           │
                                           │ parse()
                                           │
┌──────────────┐   format_preview()  ┌────▼───────┐
│  LogParser   │ <──────────────────│  LogItem   │
└──────────────┘                    └────────────┘
         │                                  │
         └──────────────┬───────────────────┘
                        │
                        ↓
              ┌──────────────────┐
              │   Ring Buffer    │
              │   (16K capacity) │
              └─────────┬────────┘
                        │
                        ↓
              ┌──────────────────┐
              │ lazylog-framework│
              │  - Rendering     │
              │  - Navigation    │
              │  - Filtering     │
              │  - UI management │
              └─────────┬────────┘
                        │
                        ↓
                  ┌──────────┐
                  │ Terminal │
                  └──────────┘

Core Concepts

LogProvider Trait

Provides raw log data from any source (files, network, APIs, etc.):

pub trait LogProvider: Send {
    /// initialize resources (called once at startup)
    fn start(&mut self) -> Result<()>;

    /// cleanup resources (called once at shutdown)
    fn stop(&mut self) -> Result<()>;

    /// poll for new logs (non-blocking, returns raw strings)
    fn poll_logs(&mut self) -> Result<Vec<String>>;
}

Key points:

• poll_logs() must be non-blocking - return empty vec if no logs available
• Returns raw strings, not parsed LogItems
• Called repeatedly at configured interval (default: 100ms)

LogParser Trait

Parses raw strings and formats them for display:

pub trait LogParser: Send + Sync {
    /// parse raw log string into structured LogItem (return None to filter)
    fn parse(&self, raw_log: &str) -> Option<LogItem>;

    /// format log for display at given detail level (0-4)
    fn format_preview(&self, item: &LogItem, detail_level: u8) -> String;

    /// extract searchable text for filtering
    fn get_searchable_text(&self, item: &LogItem, detail_level: u8) -> String;

    /// format for clipboard (optional, has default)
    fn make_yank_content(&self, item: &LogItem) -> String { /* default impl */ }

    /// max detail level supported (optional, default: 4)
    fn max_detail_level(&self) -> u8 { 4 }
}

LogItem Structure

Structured representation of a log entry:

pub struct LogItem {
    pub id: Uuid,                       // auto-generated unique ID
    pub time: String,                   // timestamp (auto-generated or custom)
    pub content: String,                // parsed log message
    pub raw_content: String,            // original log line
    pub metadata: HashMap<String, String>, // extensible key-value storage
}

Use the builder pattern to add metadata:

let log = LogItem::new(
    "Application started".to_string(),
    "2025-01-15 10:30:00 INFO main.rs Application started".to_string(),
)
.with_metadata("level", "INFO")
.with_metadata("module", "main")
.with_metadata("severity", "1");

Configuration

Customize behavior with AppDesc:

use std::time::Duration;
use lazylog_framework::{AppDesc, start_with_desc};
use std::sync::Arc;

let parser = Arc::new(MyParser);
let mut desc = AppDesc::new(parser);

desc.poll_interval = Duration::from_millis(50);  // poll every 50ms
desc.ring_buffer_size = 32768;                   // 32K log capacity
desc.show_debug_logs = true;                     // show debug panel

start_with_desc(&mut terminal, provider, desc)?;

Keybindings

Detail Levels

Control how much information is displayed (0-4):

Your parser defines what each level shows via format_preview(). Common convention:

Users can adjust levels with +/- keys to progressively reveal more information.

Examples

See the examples/ directory for complete working implementations:

Simple Example

Generate dummy logs to demonstrate basic usage:

cargo run --example simple

File Tailing Example

Tail a log file (like tail -f):

cargo run --example file -- /path/to/logfile.log

# or generate test logs:
while true; do echo "$(date) Test log"; sleep 1; done > /tmp/test.log
cargo run --example file -- /tmp/test.log

Structured JSON Example

Parse JSON logs with metadata and detail levels:

cargo run --example structured

Advanced Examples

File-based Provider with Tailing

use std::fs::File;
use std::io::{BufRead, BufReader, Seek, SeekFrom};

struct FileProvider {
    reader: Option<BufReader<File>>,
    path: String,
}

impl LogProvider for FileProvider {
    fn start(&mut self) -> Result<()> {
        let file = File::open(&self.path)?;
        let mut reader = BufReader::new(file);
        reader.seek(SeekFrom::End(0))?; // start at end (tail mode)
        self.reader = Some(reader);
        Ok(())
    }

    fn stop(&mut self) -> Result<()> {
        self.reader = None;
        Ok(())
    }

    fn poll_logs(&mut self) -> Result<Vec<String>> {
        let mut logs = Vec::new();
        if let Some(reader) = &mut self.reader {
            let mut line = String::new();
            while reader.read_line(&mut line)? > 0 {
                if !line.trim().is_empty() {
                    logs.push(line.trim().to_string());
                }
                line.clear();
            }
        }
        Ok(logs)
    }
}

JSON Parser with Metadata

struct JsonParser;

impl LogParser for JsonParser {
    fn parse(&self, raw_log: &str) -> Option<LogItem> {
        let json: serde_json::Value = serde_json::from_str(raw_log).ok()?;

        Some(LogItem::new(
            json["message"].as_str()?.to_string(),
            raw_log.to_string(),
        )
        .with_metadata("level", json["level"].as_str().unwrap_or("INFO"))
        .with_metadata("module", json["module"].as_str().unwrap_or("")))
    }

    fn format_preview(&self, item: &LogItem, level: u8) -> String {
        match level {
            0 => item.content.clone(),
            1 => format!("[{}] {}", item.time, item.content),
            2 => format!("[{}] [{}] {}",
                item.time,
                item.get_metadata("level").unwrap_or(""),
                item.content),
            _ => format!("[{}] [{}] [{}] {}",
                item.time,
                item.get_metadata("level").unwrap_or(""),
                item.get_metadata("module").unwrap_or(""),
                item.content),
        }
    }

    fn get_searchable_text(&self, item: &LogItem, level: u8) -> String {
        if level >= 2 {
            format!("{} {} {}",
                item.get_metadata("level").unwrap_or(""),
                item.get_metadata("module").unwrap_or(""),
                item.content)
        } else {
            item.content.clone()
        }
    }
}

UDP Syslog Receiver

use std::net::UdpSocket;

struct SyslogProvider {
    socket: UdpSocket,
}

impl SyslogProvider {
    fn new(addr: &str) -> Result<Self> {
        let socket = UdpSocket::bind(addr)?;
        socket.set_nonblocking(true)?; // critical for non-blocking poll
        Ok(Self { socket })
    }
}

impl LogProvider for SyslogProvider {
    fn start(&mut self) -> Result<()> { Ok(()) }
    fn stop(&mut self) -> Result<()> { Ok(()) }

    fn poll_logs(&mut self) -> Result<Vec<String>> {
        let mut logs = Vec::new();
        let mut buf = [0u8; 65536];

        loop {
            match self.socket.recv(&mut buf) {
                Ok(size) => {
                    let msg = String::from_utf8_lossy(&buf[..size]);
                    logs.push(msg.to_string());
                }
                Err(e) if e.kind() == std::io::ErrorKind::WouldBlock => break,
                Err(e) => return Err(e.into()),
            }
        }

        Ok(logs)
    }
}

Use Cases

• Application logs - Tail local log files
• Container logs - Monitor Docker/Kubernetes logs
• Syslog viewer - UDP/TCP syslog receiver
• Database logs - Stream from PostgreSQL, MongoDB, etc.
• API logs - Fetch from logging services (Elasticsearch, Loki, etc.)
• SSH logs - Remote log tailing via SSH
• Multi-source aggregator - Combine multiple log sources
• Device logs - Monitor mobile devices (iOS, Android)
• Custom protocols - Any streaming log source

Performance

• Memory-efficient: Ring buffer prevents unbounded growth (default: 16K items)
• Lock-free: Uses ringbuf crate for zero-allocation producer/consumer
• Lazy rendering: Only visible logs are formatted and drawn
• Parallel filtering: Uses rayon for fast regex filtering on large log sets
• Non-blocking: Provider runs in background thread, UI stays responsive

Requirements

• Rust 1.70+
• Terminal with ANSI color support
• For best experience: 256 colors, mouse support

Testing

The framework includes doctests for all public APIs. Run them with:

cargo test --doc

Documentation

Full API documentation is available at docs.rs/lazylog-framework.

Generate local documentation:

cargo doc --open

License

MIT OR Apache-2.0

Contributing

Contributions welcome! Please:

1. Check existing issues or create one
2. Fork and create a feature branch
3. Add tests for new functionality
4. Ensure cargo test and cargo clippy pass
5. Submit a pull request

Credits

Built with:

• ratatui - Terminal UI framework
• crossterm - Terminal manipulation
• ringbuf - Lock-free ring buffer
• regex - Regular expressions
• rayon - Parallel filtering

Related Projects

• lazylog - Reference implementation with file/iOS/Android providers