turbovault-export

Data export functionality for vault analysis in multiple formats.

This crate provides comprehensive export capabilities for Obsidian vault data, enabling downstream analysis, reporting, and integration with external tools. It supports multiple output formats and export types for different use cases.

Architecture Overview

┌─────────────────────────────────────────────────────────────┐
│                    ExportEngine                             │
│                                                             │
│  ┌─────────────────────────────────────────────────────────┐ │
│  │              Format Handlers                           │ │
│  │  - JSON Exporter                                        │ │
│  │  - CSV Exporter                                         │ │
│  │  - Markdown Exporter                                    │ │
│  │  - XML Exporter                                         │ │
│  └─────────────────────────────────────────────────────────┘ │
│                                                             │
│  ┌─────────────────────────────────────────────────────────┐ │
│  │              Export Types                              │ │
│  │  - Health Report                                       │ │
│  │  - Broken Links Report                                 │ │
│  │  - Vault Statistics                                   │ │
│  │  - Analysis Report                                     │ │
│  └─────────────────────────────────────────────────────────┘ │
│                                                             │
│  ┌─────────────────────────────────────────────────────────┐ │
│  │              Data Sources                              │ │
│  │  - VaultManager                                        │ │
│  │  - ObsidianVaultGraph                                  │ │
│  │  - SearchEngine                                        │ │
│  └─────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘

Design Philosophy:

Format Agnostic: Supports multiple output formats
Data Rich: Exports comprehensive vault information
Performance: Efficient streaming for large vaults
Extensible: Easy to add new formats and export types
Thread Safe: Concurrent export operations

Supported Formats

JSON Export

Structured data export for programmatic consumption:

use TurboVault_export::{ExportEngine, ExportFormat, ExportType};

let engine = ExportEngine::new(vault_manager, graph, search_engine)?;

// Export health report as JSON
let json_data = engine.export(ExportType::HealthReport, ExportFormat::Json)?;
// Returns: serde_json::Value with structured health data

// Export vault statistics as JSON
let stats_data = engine.export(ExportType::VaultStatistics, ExportFormat::Json)?;
// Returns: VaultStatistics struct serialized to JSON

JSON Structure Example:

{
  "export_type": "health_report",
  "timestamp": "2024-01-15T10:30:00Z",
  "vault_path": "/path/to/vault",
  "health_score": 85,
  "issues": [
    {
      "type": "orphaned_files",
      "count": 25,
      "files": ["Temp/ScratchNote.md", "Temp/OldDraft.md"]
    },
    {
      "type": "broken_links",
      "count": 5,
      "links": [
        {
          "file": "Index.md",
          "line": 15,
          "target": "NonExistentNote.md",
          "link_type": "wikilink"
        }
      ]
    }
  ],
  "recommendations": [
    "Consider linking orphaned files to main content",
    "Fix broken links to improve vault connectivity"
  ]
}

CSV Export

Tabular data export for spreadsheet analysis:

// Export broken links as CSV
let csv_data = engine.export(ExportType::BrokenLinksReport, ExportFormat::Csv)?;
// Returns: String with CSV data

// Export vault statistics as CSV
let stats_csv = engine.export(ExportType::VaultStatistics, ExportFormat::Csv)?;
// Returns: String with CSV data

CSV Structure Example:

file_path,line_number,target,link_type,severity
Index.md,15,NonExistentNote.md,wikilink,high
Projects/ProjectA.md,8,MissingImage.png,embed,medium
Notes/NoteB.md,22,OldNote.md,wikilink,low

Markdown Export

Human-readable reports in markdown format:

// Export analysis report as markdown
let markdown_data = engine.export(ExportType::AnalysisReport, ExportFormat::Markdown)?;
// Returns: String with markdown content

Markdown Structure Example:

# Vault Analysis Report

**Generated:** 2024-01-15T10:30:00Z  
**Vault:** /path/to/vault  
**Health Score:** 85/100

## Overview

This vault contains 1,250 files with 3,400 links. The overall health score is 85/100.

## Issues Found

### Orphaned Files (25 files)
- `Temp/ScratchNote.md`
- `Temp/OldDraft.md`
- `Notes/UnlinkedNote.md`

### Broken Links (5 links)
- `Index.md:15` → `NonExistentNote.md` (wikilink)
- `Projects/ProjectA.md:8` → `MissingImage.png` (embed)

## Recommendations

1. Consider linking orphaned files to main content
2. Fix broken links to improve vault connectivity
3. Review hub notes for better organization

Export Types

Health Report

Comprehensive vault health analysis:

use TurboVault_export::{ExportEngine, ExportType, ExportFormat};

let engine = ExportEngine::new(vault_manager, graph, search_engine)?;

// Export health report
let health_report = engine.export(ExportType::HealthReport, ExportFormat::Json)?;

// Health report includes:
// - Overall health score (0-100)
// - Detailed issue breakdown
// - Orphaned files list
// - Broken links with locations
// - Cycle detection results
// - Hub note analysis
// - Recommendations for improvement

Health Report Data:

Overall Score: 0-100 health rating
Issues: Categorized problems (orphans, broken links, cycles)
Statistics: File counts, link counts, connectivity metrics
Recommendations: Actionable improvement suggestions

Broken Links Report

Detailed analysis of broken links:

// Export broken links report
let broken_links = engine.export(ExportType::BrokenLinksReport, ExportFormat::Csv)?;

// Broken links report includes:
// - File path and line number
// - Target that doesn't exist
// - Link type (wikilink, embed, reference)
// - Severity level (high, medium, low)
// - Suggested fixes

Broken Links Data:

Location: File path and line number
Target: Missing file or resource
Type: Wikilink, embed, or reference
Severity: Impact level for prioritization
Context: Surrounding content for context

Vault Statistics

Comprehensive vault metrics and analytics:

// Export vault statistics
let stats = engine.export(ExportType::VaultStatistics, ExportFormat::Json)?;

// Vault statistics include:
// - File counts by type
// - Link counts by type
// - Graph metrics (density, connectivity)
// - Content analysis (word counts, tag usage)
// - Growth trends over time
// - Performance metrics

Statistics Data:

File Metrics: Total files, by type, by folder
Link Metrics: Total links, by type, connectivity
Graph Metrics: Density, components, cycles
Content Metrics: Word counts, tag usage, frontmatter
Performance: Search times, graph build times

Analysis Report

Detailed vault analysis with insights:

// Export analysis report
let analysis = engine.export(ExportType::AnalysisReport, ExportFormat::Markdown)?;

// Analysis report includes:
// - Executive summary
// - Detailed findings
// - Trend analysis
// - Comparative metrics
// - Actionable recommendations
// - Future outlook

Analysis Data:

Summary: High-level findings and trends
Findings: Detailed analysis results
Trends: Changes over time
Comparisons: Benchmarks and standards
Recommendations: Specific action items
Outlook: Future considerations

Usage Examples

Example 1: Health Dashboard Export

use TurboVault_export::{ExportEngine, ExportType, ExportFormat};

let engine = ExportEngine::new(vault_manager, graph, search_engine)?;

// Export health report for dashboard
let health_data = engine.export(ExportType::HealthReport, ExportFormat::Json)?;

// Save to file
std::fs::write("health_report.json", health_data.to_string())?;

// Also export as CSV for spreadsheet analysis
let health_csv = engine.export(ExportType::HealthReport, ExportFormat::Csv)?;
std::fs::write("health_report.csv", health_csv)?;

println!("Health reports exported successfully");

Example 2: Broken Links Analysis

// Export broken links for manual review
let broken_links = engine.export(ExportType::BrokenLinksReport, ExportFormat::Csv)?;

// Parse CSV for processing
let mut reader = csv::Reader::from_reader(broken_links.as_bytes());
for result in reader.records() {
    let record = result?;
    let file_path = &record[0];
    let line_number = record[1].parse::<u32>()?;
    let target = &record[2];
    let link_type = &record[3];
    let severity = &record[4];
    
    println!("{}:{} → {} ({}) - {}", file_path, line_number, target, link_type, severity);
}

Example 3: Vault Statistics Export

// Export comprehensive statistics
let stats = engine.export(ExportType::VaultStatistics, ExportFormat::Json)?;

// Parse and analyze statistics
let stats_data: serde_json::Value = serde_json::from_str(&stats.to_string())?;

let total_files = stats_data["file_counts"]["total"].as_u64().unwrap();
let total_links = stats_data["link_counts"]["total"].as_u64().unwrap();
let health_score = stats_data["health_score"].as_u64().unwrap();

println!("Vault Statistics:");
println!("  Files: {}", total_files);
println!("  Links: {}", total_links);
println!("  Health: {}/100", health_score);

// Export as markdown for documentation
let stats_md = engine.export(ExportType::VaultStatistics, ExportFormat::Markdown)?;
std::fs::write("vault_stats.md", stats_md)?;

Example 4: Automated Reporting

use chrono::Utc;

// Generate daily health report
let engine = ExportEngine::new(vault_manager, graph, search_engine)?;

let timestamp = Utc::now().format("%Y-%m-%d").to_string();
let report_name = format!("health_report_{}.json", timestamp);

// Export health report
let health_data = engine.export(ExportType::HealthReport, ExportFormat::Json)?;
std::fs::write(&report_name, health_data.to_string())?;

// Export analysis report
let analysis_data = engine.export(ExportType::AnalysisReport, ExportFormat::Markdown)?;
let analysis_name = format!("analysis_report_{}.md", timestamp);
std::fs::write(&analysis_name, analysis_data)?;

println!("Daily reports generated: {} and {}", report_name, analysis_name);

Integration Points

With turbovault-vault

// Vault manager provides file data for export
let vault_manager = VaultManager::new(config)?;
let engine = ExportEngine::new(vault_manager, graph, search_engine)?;

With turbovault-graph

// Graph provides link analysis data
let graph = ObsidianVaultGraph::from_files(files)?;
let engine = ExportEngine::new(vault_manager, graph, search_engine)?;

With turbovault-server

// MCP server provides export tools
let export_tools = ExportTools::new(vault_manager, graph, search_engine)?;
let result = export_tools.export_health_report(ExportFormat::Json)?;

With turbovault-tools

// Tools layer orchestrates export operations
let tools = MCPTools::new(vault_manager, graph, search_engine)?;
let export_result = tools.export_data(ExportType::HealthReport, ExportFormat::Json)?;

Performance Characteristics

Memory Usage

Streaming Export: Processes data in chunks to minimize memory usage
Format Buffers: Small buffers for output formatting
Data Structures: Efficient serialization with minimal overhead
Total: ~1MB for typical vault exports

Export Performance

JSON Export: ~50ms for 1k files
CSV Export: ~30ms for 1k files
Markdown Export: ~100ms for 1k files
Large Vaults: Scales linearly with file count

Thread Safety

Concurrent Exports: Multiple export operations can run simultaneously
Read-Only Operations: No shared mutable state
Format Handlers: Stateless and thread-safe
Data Sources: Read-only access to vault data

Development

Running Tests

# All tests
cargo test

# Specific test categories
cargo test --test json_export
cargo test --test csv_export
cargo test --test markdown_export

# With output
cargo test -- --nocapture

Adding New Formats

Define Format Handler: Implement FormatHandler trait
Add Format Enum: Add to ExportFormat enum
Implement Serialization: Add format-specific serialization
Add Tests: Comprehensive test coverage
Update Documentation: Add usage examples

Adding New Export Types

Define Export Type: Add to ExportType enum
Implement Data Collection: Gather required data from sources
Add Format Support: Implement for all supported formats
Add Tests: Test with sample data
Update Documentation: Add usage examples

Dependencies

turbovault-core: Core data models and error types
turbovault-vault: Vault management and file operations
turbovault-graph: Link graph analysis
serde: Serialization framework
serde_json: JSON serialization
csv: CSV serialization
chrono: Timestamp handling

Design Decisions

Why Multiple Formats?

Different Use Cases: JSON for programs, CSV for spreadsheets, Markdown for humans
Integration: Easy integration with external tools
Flexibility: Users can choose appropriate format
Standards: Common formats for data exchange

Why Streaming Export?

Memory Efficiency: Handle large vaults without memory issues
Performance: Faster processing for large datasets
Scalability: Linear scaling with vault size
Reliability: Less likely to fail on large exports

Why Read-Only Operations?

Thread Safety: No synchronization needed
Performance: No locking overhead
Reliability: No risk of data corruption
Simplicity: Easier to reason about and test

License

See workspace license.

turbovault-export 0.1.1