title: Diaryx Core Library

author: adammharris

audience:

• public

part_of: ../../README.md

Diaryx Core Library

This is the diaryx_core library! It contains shared code for the Diaryx clients.

Async-first Architecture

This library uses an async-first design. All core modules (Workspace, Validator, Exporter, Searcher, Publisher) use the AsyncFileSystem trait for filesystem operations.

For CLI/native code: Wrap a sync filesystem with SyncToAsyncFs and use futures_lite::future::block_on():

use diaryx_core::fs::{RealFileSystem, SyncToAsyncFs};
use diaryx_core::workspace::Workspace;

let fs = SyncToAsyncFs::new(RealFileSystem);
let workspace = Workspace::new(fs);

// Use block_on for sync contexts
let tree = futures_lite::future::block_on(
    workspace.build_tree(Path::new("README.md"))
);

For WASM: Implement AsyncFileSystem directly using JS promises/IndexedDB.

Quick overview

diaryx_core
└── src
    ├── backup.rs ("Backup" is making a ZIP file of all the markdown files)
    ├── config.rs (configuration for the core to share)
    ├── diaryx.rs (Central data structure used)
    ├── entry (Functionality to manipulate entries)
    │   ├── helpers.rs
    │   └── mod.rs
    ├── error.rs (Shared error types)
    ├── export.rs (Like backup, but filtering by "audience" trait)
    ├── frontmatter.rs (Operations to read and manipulate frontmatter in markdown files)
    ├── fs (Filesystem abstraction)
    │   ├── async_fs.rs (Async filesystem trait and SyncToAsyncFs adapter)
    │   ├── memory.rs (In-memory filesystem, used by WASM/web client)
    │   ├── mod.rs
    │   └── native.rs (Actual filesystem [std::fs] used by Tauri/CLI)
    ├── lib.rs
    ├── publish (Uses comrak to export to HTML)
    │   ├── mod.rs
    │   └── types.rs
    ├── search.rs (Searching by frontmatter or content)
    ├── template.rs (Templating functionality, mostly for daily files)
    ├── test_utils.rs (Feature-gated unit test utility functions)
    ├── utils
    │   ├── date.rs (chrono for date and time manipulation)
    │   ├── mod.rs
    │   └── path.rs (finding relative paths, etc.)
    ├── validate.rs (Validating and fixing incorrectly organized workspaces)
    └── workspace (organizing collections of markdown files as "workspaces")
        ├── mod.rs
        └── types.rs

Provided functionality

Managing frontmatter

Full key-value operations for managing frontmatter properties:

• set_frontmatter_property
• get_frontmatter_property
• rename_frontmatter_property
• remove_frontmatter_property
• get_all_frontmatter

Also, sorting frontmatter properties:

• sort_frontmatter
• sort_alphabetically
• sort_by_pattern

Managing file content

Operations for managing content of markdown files separate from frontmatter:

• set_content
• get_content
• append_content
• clear_content

Search

Search frontmatter or content separately:

• SearchQuery::content
• SearchQuery::frontmatter

Export

use diaryx_core::export::{ExportOptions, ExportPlan, Exporter};
use diaryx_core::fs::{RealFileSystem, SyncToAsyncFs};
use std::path::Path;

let workspace_root = Path::new("./workspace");
let audience = "public";
let destination = Path::new("./export");
let fs = SyncToAsyncFs::new(RealFileSystem);
let exporter = Exporter::new(fs);

// Use futures_lite::future::block_on for sync contexts
let plan = futures_lite::future::block_on(
    exporter.plan_export(&workspace_root, audience, destination)
).unwrap();

let force = false;
let keep_audience = false;
let options = ExportOptions {
    force,
    keep_audience,
};

let result = futures_lite::future::block_on(
    exporter.execute_export(&plan, &options)
);

match result {
  Ok(stats) => {
    println!("✓ {}", stats);
    println!("  Exported to: {}", destination.display());
  }
  Err(e) => {
    eprintln!("✗ Export failed: {}", e);
  }
}

Validation

The validate module provides functionality to check workspace link integrity and automatically fix issues.

Validator

The Validator struct checks part_of and contents references within a workspace:

use diaryx_core::validate::Validator;
use diaryx_core::fs::{RealFileSystem, SyncToAsyncFs};
use std::path::Path;

let fs = SyncToAsyncFs::new(RealFileSystem);
let validator = Validator::new(fs);

// Validate entire workspace starting from root index
let root_path = Path::new("./workspace/README.md");
let result = futures_lite::future::block_on(
    validator.validate_workspace(&root_path)
).unwrap();

// Or validate a single file
let file_path = Path::new("./workspace/notes/my-note.md");
let result = futures_lite::future::block_on(
    validator.validate_file(&file_path)
).unwrap();

if result.is_ok() {
    println!("✓ Validation passed ({} files checked)", result.files_checked);
} else {
    println!("Found {} errors and {} warnings",
             result.errors.len(),
             result.warnings.len());
}

Validation Errors

• BrokenPartOf - A file's part_of points to a non-existent file
• BrokenContentsRef - An index's contents references a non-existent file
• BrokenAttachment - A file's attachments references a non-existent file

Validation Warnings

• OrphanFile - A markdown file not referenced by any index
• UnlinkedEntry - A file/directory not in the contents hierarchy
• UnlistedFile - A markdown file in a directory but not in the index's contents
• CircularReference - Circular reference detected in workspace hierarchy
• NonPortablePath - A path contains absolute paths or ./.. components
• MultipleIndexes - Multiple index files in the same directory
• OrphanBinaryFile - A binary file not referenced by any attachments
• MissingPartOf - A non-index file has no part_of property

ValidationFixer

The ValidationFixer struct provides methods to automatically fix validation issues:

use diaryx_core::validate::{Validator, ValidationFixer};
use diaryx_core::fs::{RealFileSystem, SyncToAsyncFs};
use std::path::Path;

let fs = SyncToAsyncFs::new(RealFileSystem);
let validator = Validator::new(fs.clone());
let fixer = ValidationFixer::new(fs);

// Validate workspace
let root_path = Path::new("./workspace/README.md");
let result = futures_lite::future::block_on(
    validator.validate_workspace(&root_path)
).unwrap();

// Fix all issues at once
let (error_fixes, warning_fixes) = futures_lite::future::block_on(
    fixer.fix_all(&result)
);

for fix in error_fixes.iter().chain(warning_fixes.iter()) {
    if fix.success {
        println!("✓ {}", fix.message);
    } else {
        println!("✗ {}", fix.message);
    }
}

// Or fix individual issues (all methods are async)
futures_lite::future::block_on(async {
    fixer.fix_broken_part_of(Path::new("./file.md")).await;
    fixer.fix_broken_contents_ref(Path::new("./index.md"), "missing.md").await;
    fixer.fix_unlisted_file(Path::new("./index.md"), Path::new("./new-file.md")).await;
    fixer.fix_missing_part_of(Path::new("./orphan.md"), Path::new("./index.md")).await;
});

Publish

Templates

Workspaces

Date parsing

Shared errors

Configuration

Filesystem abstraction

The fs module provides filesystem abstraction through two traits: FileSystem (synchronous) and AsyncFileSystem (asynchronous).

Note: As of the async-first refactor, all core modules (Workspace, Validator, Exporter, Searcher, Publisher) use AsyncFileSystem. For synchronous contexts (CLI, tests), wrap a sync filesystem with SyncToAsyncFs and use futures_lite::future::block_on().

FileSystem trait

The synchronous FileSystem trait provides basic implementations:

• RealFileSystem - Native filesystem using std::fs (not available on WASM)
• InMemoryFileSystem - In-memory implementation, useful for WASM and testing

use diaryx_core::fs::{FileSystem, InMemoryFileSystem};
use std::path::Path;

// Create an in-memory filesystem
let fs = InMemoryFileSystem::new();

// Write a file (sync)
fs.write_file(Path::new("workspace/README.md"), "# Hello").unwrap();

// Read it back
let content = fs.read_to_string(Path::new("workspace/README.md")).unwrap();
assert_eq!(content, "# Hello");

AsyncFileSystem trait (Primary API)

The AsyncFileSystem trait is the primary API for all core modules:

• WASM environments where JavaScript APIs (like IndexedDB) are async
• Native code using async runtimes like tokio
• All workspace operations (Workspace, Validator, Exporter, etc.)

use diaryx_core::fs::{AsyncFileSystem, InMemoryFileSystem, SyncToAsyncFs};
use diaryx_core::workspace::Workspace;
use std::path::Path;

// Wrap a sync filesystem for use with async APIs
let sync_fs = InMemoryFileSystem::new();
let async_fs = SyncToAsyncFs::new(sync_fs);

// Use with Workspace (async)
let workspace = Workspace::new(async_fs);

// For sync contexts, use block_on
let tree = futures_lite::future::block_on(
    workspace.build_tree(Path::new("README.md"))
);

SyncToAsyncFs adapter

The SyncToAsyncFs struct wraps any synchronous FileSystem implementation to provide an AsyncFileSystem interface. This is the recommended way to use the async-first API in synchronous contexts:

use diaryx_core::fs::{InMemoryFileSystem, SyncToAsyncFs, RealFileSystem};
use diaryx_core::workspace::Workspace;

// For native code
let fs = SyncToAsyncFs::new(RealFileSystem);
let workspace = Workspace::new(fs);

// For tests/WASM
let fs = SyncToAsyncFs::new(InMemoryFileSystem::new());
let workspace = Workspace::new(fs);

// Access the inner sync filesystem if needed
// let inner = async_fs.inner();