peisar 1.0.0 - Docs.rs

Peisar parses CommonMark Markdown with GitHub Flavored Markdown (GFM), Kramdown block attributes, YAML front matter, source spans, AST visitors, and custom parser hooks.

Features

CommonMark blocks and inlines, including HTML and reference links
GFM tables, strikethrough, task lists, and autolinks
Kramdown block attributes such as {:#intro .lead key="value"}
YAML front matter, returned separately from rendered Markdown
Source spans on every AST node
AST visitors and during-parse extension hooks
A native N-API Node.js addon with generated TypeScript declarations
A Rust library (rlib) with public module paths for the parser, renderer, and high-level document APIs

Node.js

Install the package:

npm i peisar

Next.js

Example App : https://github.com/phothinmg/peisar/tree/main/examples/next

Demo Web APP : https://peisar-next-example.vercel.app/

Next.js provides a built-in configuration option called serverExternalPackages.This opts specific dependencies out of Server Component bundling and allows the server to use a native Node.js require to load them directly from node_modules.

Add peisar to serverExternalPackages in your configuration file:

For next.config.ts (TypeScript)

import type { NextConfig } from 'next'

const nextConfig: NextConfig = {
  serverExternalPackages: ["peisar"],
}

export default nextConfig

For next.config.js or next.config.mjs (JavaScript)

/** @type {import('next').NextConfig} */
const nextConfig = {
  serverExternalPackages: ["peisar"],
}

module.exports = nextConfig // or export default nextConfig if using .mjs

Vite

Example App : https://github.com/phothinmg/peisar/tree/main/examples/vite

Demo Web APP : https://peisar-vite-example.vercel.app/

Vite browser applications must use Peisar's WebAssembly package rather than the native peisar package:

npm i @peisar/peisar-wasm32-wasi

Import it directly in client-side code:

import { Peisar } from "@peisar/peisar-wasm32-wasi";

const document = new Peisar("# Hello from Vite", { fragment: true });
console.log(document.html);

The WebAssembly package uses threads and requires cross-origin isolation. Add the required headers and exclude the package from Vite's dependency optimizer so its internal worker is handled correctly:

// vite.config.ts
import { defineConfig } from "vite";
import react from "@vitejs/plugin-react";

export default defineConfig({
  server: {
    headers: {
      "Cross-Origin-Opener-Policy": "same-origin",
      "Cross-Origin-Embedder-Policy": "require-corp",
    },
  },
  plugins: [
    react(),
    {
      name: "configure-preview-response-headers",
      configurePreviewServer(server) {
        server.middlewares.use((_req, res, next) => {
          res.setHeader("Cross-Origin-Opener-Policy", "same-origin");
          res.setHeader("Cross-Origin-Embedder-Policy", "require-corp");
          next();
        });
      },
    },
  ],
  optimizeDeps: {
    exclude: ["@peisar/peisar-wasm32-wasi"],
  },
});

After changing this configuration, clear Vite's optimized-dependency cache and restart the development server:

rm -rf node_modules/.vite
npm run dev

For production builds, configure the same headers in the server or hosting platform that serves the app. Vite's server and preview headers apply only to local development and vite preview. Verify the deployed app reports globalThis.crossOriginIsolated === true. With Cross-Origin-Embedder-Policy: require-corp, cross-origin assets must also allow embedding through CORS or Cross-Origin-Resource-Policy.

WebAssembly (WASM) environments

The main package loads a native binary matching your platform, selected by npm through optionalDependencies. Hosts that cannot load native .node binaries — WebAssembly-restricted environments such as some edge runtimes — fall back to the WebAssembly build, @peisar/peisar-wasm32-wasi.

That package is deliberately not installed by default: npm would download the .wasm binary for every consumer, including the ones whose native binary loads. Install it explicitly when you need the WebAssembly build:

npm i peisar @peisar/peisar-wasm32-wasi

The generated loader resolves the WebAssembly binding automatically whenever no native binary can be loaded. To control the preference, set an environment variable:

NAPI_RS_FORCE_WASI=true — prefer the WebAssembly build and keep the native binding as a lazy fallback
NAPI_RS_FORCE_WASI=error — require the WebAssembly build and fail when it is missing

Use

Create a Peisar document, then read its AST, rendered HTML, front matter, or serialized AST:

const { Peisar } = require("peisar");

const document = new Peisar("# Hello **world**", { fragment: true });

console.log(document.html); // <h1>Hello <strong>world</strong></h1>\n
console.log(document.ast); // { nodeType: "root", children: [...], ... }
console.log(document.astJson); // JSON string
console.log(document.frontmatter); // object, or null

ast and astJson serialize the same tree differently: ast (the real JavaScript object) uses PascalCase type values (Heading, Paragraph, Link) and a nodeType: "root" root, while astJson mirrors the Rust serialization with snake_case types (heading, paragraph, link) and a type: "root" root.

Options

All options are optional. GFM and Kramdown are enabled by default; html returns a complete document unless fragment is true.

const document = new Peisar("# Title", {
  gfm: false,
  kramdown: false,
  fileName: "guide.md",
  fragment: true,
  charset: true,
  viewport: true,
  title: "Guide",
  bodyClass: "docs",
  style: "h1 { color: rebeccapurple; }",
});

charset, viewport, title, bodyClass, and style affect full-document output only.

Front matter

Leading YAML front matter is removed before Markdown parsing and exposed as a JavaScript object:

const document = new Peisar(`---
title: Hello
tags:
  - docs
---

# Hello`, { fragment: true });

console.log(document.frontmatter); // { title: "Hello", tags: ["docs"] }
console.log(document.html); // <h1>Hello</h1>\n

AST visitors

Register visitors with useVisitor. Each callback receives a one-item tuple, may mutate the supplied node, and can return a control object. Return { recurse: true } to visit children; replacements are not visited again. An omitted return value keeps the node without recursing.

const document = new Peisar("# Hello", { fragment: true });

document.useVisitor({
  visitBlock([block]) {
    if (block.type === "Heading") {
      return { replaceWith: [{ ...block, level: 2 }] };
    }
    return { recurse: true };
  },
});

console.log(document.html); // <h2>Hello</h2>\n

The control fields are insertBefore, insertAfter, replaceWith, remove, and recurse. The same shape applies to visitInline.

Custom parser hooks

Use useParser for syntax that must be recognized while parsing. Block hooks receive { line, lineIndex, lines }; inline hooks receive { rest, index }. Hooks run before built-in matchers, in registration order. Return undefined to decline a position.

const document = new Peisar("See [[Some Page]].", { fragment: true });
const zero = { line: 0, column: 0, offset: 0 };

document.useParser({
  parseInline([{ rest }]) {
    if (!rest.startsWith("[[")) return;
    const close = rest.indexOf("]]");
    if (close === -1) return;
    const target = rest.slice(2, close);
    return {
      inline: {
        type: "Link",
        text: [{ type: "Text", value: target, pos: { start: zero, end: zero } }],
        url: `/wiki/${target.replaceAll(" ", "_")}`,
        autolink: false,
        pos: { start: zero, end: zero },
      },
      consumed: close + 2,
    };
  },
});

console.log(document.html); // <p>See <a href="/wiki/Some_Page">Some Page</a>.</p>\n

parseBlock must consume at least one line. parseInline must consume at least one character; a missing or zero consumed value declines the result. Peisar recomputes the source span of a hook-returned node, so a complete placeholder pos is sufficient.

File cache

PeisarCache keeps a site's Markdown and asset files in a size-bounded LRU in memory, flushes content into binary shards under .peisar-cache (with a map.json search index), derives Liquid-ready page and asset objects, and resolves hosting baseUrl for deployments like GitHub Pages or Vercel. This is the engine behind peisar-ssg's dev server and build pipeline.

const { PeisarCache } = require("peisar");

// new PeisarCache(entryDir, assetsDir?, options?)
const cache = new PeisarCache("contents", "public", {
  markdown: { fragment: true }, // PeisarOptions for every page render
  hosting: { provider: "GithubPages", repo: "my-repo" },
  outDir: ".peisar",
});

cache.markdownFiles(); // absolute paths of cached markdown files
cache.assetFiles(); // absolute paths of cached assets
cache.listFiles(); // everything cached (memory + disk archive)

const raw = cache.getText(cache.markdownFiles()[0]); // cached file text, or null
const bytes = cache.getBinary(cache.assetFiles()[0]); // cached bytes, or null

// Liquid template shapes: every page's layout/title/tags/html/… flattened
const pages = cache.pages(); // PageObject[] sorted by slug
const page = cache.getPage(pages[0].inputFilePath); // single page object
cache.assets(); // AssetsObject[] (fileType: "Image" | "Js" | "Css" | …)

// Disk archive: binary shards + map.json (path → location + frontmatter)
cache.archived(); // paths flushed to the archive tier
cache.mapJson(); // the map.json document as a JSON string
cache.flush(); // write pending archive entries now

// Hosting-aware URLs for GitHub Pages/Vercel/Netlify/Cloudflare/custom
cache.baseUrl(); // e.g. "/my-repo/"
cache.resolveUrl("docs/intro"); // "/my-repo/docs/intro"
cache.siteUrl(); // e.g. "https://user.github.io", or null

// Custom markdown rendering: register JS visitors/parsers like on `Peisar`
cache.useVisitor({ visitBlock([block]) { return { recurse: true }; } });
cache.useParser({ parseBlock([{ line }]) { /* … */ } });

cache.startWatchingJs(); // keep the cache in sync with the filesystem
// every change is logged to stderr, e.g. `created docs/intro.md`
const id = cache.onChange((event) => {
  // event: { path, kind: "create" | "modify" | "remove" | "other", isMarkdown }
});
cache.offChange(id); // unsubscribe
cache.dispose(); // stop the watcher and release worker threads

Paths are absolute on both sides: markdownFiles() returns absolute paths, so consumers pass them straight back to getText() / getBinary(). pages() / assets() derive template objects on demand — pass { archive: { shardBytes, flushIntervalMs } } to control when content is written to disk, and { memory: { byteBudget, entryBudget } } (via PeisarCache.withConfig(config)) to bound the hot set; entries evicted from memory stay readable through the archive. Always call dispose() when done so the watcher and archive threads do not keep the Node process alive.

Rust

Add the crate to a Rust project:

[dependencies]
peisar = "1.0.0"

The public Rust API is module-based. Import the parser from peisar::markdown::ast, the renderer from peisar::markdown::html, the high-level document wrapper from peisar::markdown::peisar, and the file cache from peisar::cache.

Parse and render

use peisar::markdown::ast::{AstOptions, Document};
use peisar::markdown::ast::tokens::token::Block;
use peisar::markdown::html::{RenderOptions, render_document_html};

let document = Document::parse("# Hello **world**\n", &AstOptions::default(), None);
assert!(matches!(document.children[0], Block::Heading { level: 1, .. }));

let html = render_document_html(
    &document,
    Some(RenderOptions {
        fragment: true,
        ..Default::default()
    }),
);
assert_eq!(html, "<h1>Hello <strong>world</strong></h1>\n");

For the Node-like parse-and-render wrapper:

use peisar::markdown::config::PeisarOptions;
use peisar::markdown::peisar::Peisar;

let mut document = Peisar::new(
    "# Hello".into(),
    Some(PeisarOptions {
        fragment: Some(true),
        ..Default::default()
    }),
);
assert_eq!(document.html(), "<h1>Hello</h1>\n");

Front matter and visitors

use peisar::frontmatter::frontmatter;

let parsed = frontmatter("---\ntitle: Hello\n---\n\n# Hello")?;
assert_eq!(parsed.yaml_data().unwrap()["title"], "Hello");
assert_eq!(parsed.pure_markdown_content(), "# Hello");

parse_markdown_frontmatter returns a ParseResult with yaml_data(), pure_markdown_content(), and into_parts(). A front-matter block must open with --- on the first non-whitespace line and close with a line starting ---; an unterminated block is treated as ordinary Markdown body.

For post-parse traversal, implement peisar::markdown::ast::AstVisitor and call peisar::markdown::ast::visitor::visit_document_mut. The visitor controls are VisitControl and InlineVisitControl from that same visitor module.

AstParser and ParseHooks are publicly exported from peisar::markdown::ast for integration with Peisar internals, but the hook-enabled parser entry point is not yet public to external Rust consumers. Use useParser from Node.js for registered custom syntax today.

File cache

The JavaScript PeisarCache class has a Rust equivalent, peisar::cache::PeisarCache. It loads every Markdown file under an entry directory (plus optional assets) into a size-bounded LRU in memory, flushes cold content into binary shards under .peisar-cache (with a map.json search index) on a background worker, and keeps everything in sync with recursive file-watching once start_watching() is called.

use peisar::cache::{CachedContent, PeisarCache, PeisarCacheConfig, MemoryConfig};

// Markdown from "contents", assets from "public", a 16 MiB hot set.
let mut cache = PeisarCache::with_config(PeisarCacheConfig {
    entry_dir: "contents".into(),
    assets_dir: Some("public".into()),
    memory: Some(MemoryConfig { byte_budget: Some(16 * 1024 * 1024), ..Default::default() }),
    ..Default::default()
})?;

// Every cached Markdown file: absolute path -> raw text.
for (path, text) in cache.all() {
    println!("{} ({} bytes)", path.display(), text.len());
}

// Single entries keep their variant: Markdown is `Text`, binary assets
// are `Binary`. Reads hit memory first and fall through to the disk
// archive for entries the LRU evicted.
let index = std::env::current_dir()?.join("contents").join("index.md");
match cache.get(&index) {
    Some(CachedContent::Text(md)) => println!("{md}"),
    Some(CachedContent::Binary(bytes)) => println!("{} bytes", bytes.len()),
    None => println!("not cached"),
}

// Liquid-ready page objects: layout/title/tags/html flattened per page.
for page in cache.page_objects() {
    println!("{} -> {}", page.slug, page.out_file_path);
}

// Keep memory, derived objects, and the archive in sync with the filesystem.
// Every change is logged to stderr: `created docs/intro.md` (paths relative
// to the project root), colored on terminals.
cache.start_watching()?;
// Stop watching and flush pending archive writes — the graceful shutdown
// to wire to Ctrl+C in long-running processes. Dropping the cache does the
// same cleanup automatically; Rust consumers need no explicit dispose.
cache.stop_watching();

The cache is keyed by absolute paths, so keys from all() can be passed straight back to get(). all() returns only text entries; use get() to read binary assets. Construction enqueues the snapshot to the archive worker (no blocking disk writes) and appends .peisar-cache to .gitignore when one exists; call cache.flush() to force shards to disk immediately.

Development

npm install
npm run js:test
npm run build
cargo test

More detail in CONTRIBUTING

AI integration

Peisar supplies Agent Skills for AI coding tools:

npx skills add phothinmg/peisar

peisar-parse-markdown: consume the Node.js and Rust Markdown APIs
peisar-ast-visitors: make post-parse AST transformations
peisar-parser-hooks: add syntax recognized during parsing
peisar-development: modify and validate the codebase

Project structure

src/
├── frontmatter/             YAML front-matter parser
├── cache/                   Markdown/asset file cache (`PeisarCache`)
├── markdown/
│   ├── ast/                 AST nodes, parsers, visitors, and hooks
│   ├── config/              combined parser and renderer options
│   ├── html/                HTML renderer
│   └── peisar/              high-level N-API `Peisar` class
└── lib.rs                   public module declarations

LICENSE

Security policy: SECURITY.md