Crate marty_plugin_protocol 

Marty Plugin Protocol

This crate provides everything needed to develop plugins for the Marty monorepo management tool. Plugins are compiled as dynamic libraries (.so, .dylib, .dll) and implement the MartyPlugin and WorkspaceProvider traits to add support for different project types and languages.

Overview

Marty plugins enable automatic discovery and management of projects within a monorepo. Each plugin can:

• Detect project files using glob patterns
• Parse project metadata to extract names and dependencies
• Identify workspace dependencies between projects for proper task ordering
• Provide configuration options for customizing plugin behavior

Quick Start Guide

1. Create a New Plugin Crate

# Cargo.toml
[package]
name = "marty-plugin-myframework"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["cdylib"]  # Required for dynamic library

[dependencies]
marty_plugin_protocol = "0.2"
serde_json = "1.0"

2. Implement Your Plugin

use marty_plugin_protocol::{
    dylib::export_plugin, InferredProject, MartyPlugin, Workspace, WorkspaceProvider,
};
use serde_json::{json, Value as JsonValue};
use std::path::Path;

/// Main plugin struct
pub struct MyFrameworkPlugin;

impl MyFrameworkPlugin {
    pub const fn new() -> Self {
        Self
    }
}

impl Default for MyFrameworkPlugin {
    fn default() -> Self {
        Self::new()
    }
}

/// Workspace provider for detecting projects
pub struct MyFrameworkWorkspaceProvider;

impl WorkspaceProvider for MyFrameworkWorkspaceProvider {
    fn include_path_globs(&self) -> Vec<String> {
        vec!["**/my-framework.json".to_string()]
    }

    fn exclude_path_globs(&self) -> Vec<String> {
        vec![
            "**/node_modules/**".to_string(),
            "**/target/**".to_string(),
            "**/.git/**".to_string(),
        ]
    }

    fn on_file_found(&self, workspace: &Workspace, path: &Path) -> Option<InferredProject> {
        // Only process our config files
        if path.file_name()?.to_str()? != "my-framework.json" {
            return None;
        }

        let project_dir = path.parent()?.to_path_buf();
        let name = project_dir.file_name()?.to_str()?.to_string();
         
        // Parse the config file to extract workspace dependencies
        let workspace_dependencies = self.parse_dependencies(path, workspace);

        Some(InferredProject {
            name,
            project_dir,
            discovered_by: "my-framework".to_string(),
            workspace_dependencies,
        })
    }
}

impl MyFrameworkWorkspaceProvider {
    fn parse_dependencies(&self, _path: &Path, _workspace: &Workspace) -> Vec<String> {
        // Implementation details for parsing local workspace dependencies
        Vec::new()
    }
}

impl MartyPlugin for MyFrameworkPlugin {
    fn name(&self) -> &str {
        "My Framework Plugin"
    }

    fn key(&self) -> &str {
        "my-framework"
    }

    fn workspace_provider(&self) -> &dyn WorkspaceProvider {
        &MyFrameworkWorkspaceProvider
    }

    fn configuration_options(&self) -> Option<JsonValue> {
        Some(json!({
            "type": "object",
            "properties": {
                "framework_version": {
                    "type": "string",
                    "description": "Framework version to target",
                    "default": "latest"
                }
            },
            "additionalProperties": false
        }))
    }
}

// Export the plugin - this creates the C ABI interface
export_plugin!(MyFrameworkPlugin);

3. Build Your Plugin

cargo build --lib --release

The resulting dynamic library will be in target/release/ with the appropriate extension for your platform (.so on Linux, .dylib on macOS, .dll on Windows).

Core Concepts

Plugin Discovery Process

1. Scanning: Marty walks the workspace using your include_path_globs() patterns
2. Filtering: Files matching exclude_path_globs() are skipped
3. Detection: on_file_found() is called for each matching file
4. Project Creation: If a project is detected, an InferredProject is returned

Workspace Dependencies (Critical Concept)

⚠️ Important: workspace_dependencies represents dependencies between projects within the same workspace, NOT external packages. This is used for:

• Task Ordering: Determining the correct order to build/test projects
• Dependency Resolution: Understanding which projects depend on each other
• Change Impact: Knowing which projects are affected by changes

Examples of Workspace Dependencies:

// ✅ CORRECT: References to other projects in the workspace
vec![
    "shared-utils".to_string(),     // Another project in the workspace
    "common-types".to_string(),     // Shared library project
    "data-models".to_string(),      // Internal dependency
]

// ❌ INCORRECT: External packages (don't include these)
vec![
    "serde".to_string(),           // External crate from crates.io
    "tokio".to_string(),           // External crate
    "@types/node".to_string(),     // External npm package
    "lodash".to_string(),          // External npm package
]

How to Detect Workspace Dependencies:

Different project types have different ways to reference workspace projects:

Rust (Cargo.toml):

[dependencies]
shared-utils = { path = "../shared-utils" }    # ✅ Workspace dependency
serde = "1.0"                                  # ❌ External dependency (ignore)

JavaScript/TypeScript (package.json):

{
  "dependencies": {
    "@myworkspace/shared": "workspace:*",       // ✅ Workspace dependency
    "@myworkspace/utils": "file:../utils",     // ✅ Workspace dependency  
    "lodash": "^4.17.21"                       // ❌ External dependency (ignore)
  }
}

Python (pyproject.toml):

[build-system]
dependencies = [
    "shared-lib @ file:../shared-lib",         # ✅ Workspace dependency
    "requests>=2.25.0",                        # ❌ External dependency (ignore)
]

Configuration Schema

Plugins can define JSON Schema configuration options that users can set in their Marty configuration. The schema should be complete and descriptive:

Some(json!({
    "type": "object",
    "properties": {
        "build_command": {
            "type": "string",
            "description": "Command to build projects",
            "default": "build",
            "examples": ["build", "compile", "make"]
        },
        "target_directory": {
            "type": "string",
            "description": "Directory for build outputs relative to project root",
            "default": "dist"
        },
        "enable_optimization": {
            "type": "boolean",
            "description": "Enable build optimizations in release mode",
            "default": true
        },
        "supported_versions": {
            "type": "array",
            "description": "List of supported framework versions",
            "items": { "type": "string" },
            "default": ["1.0", "2.0"]
        },
        "exclude_patterns": {
            "type": "array",
            "description": "Additional glob patterns to exclude during scanning",
            "items": { "type": "string" },
            "default": []
        }
    },
    "additionalProperties": false,
    "required": []  // Specify required properties if any
}))

Advanced Topics

Error Handling

Plugins should be resilient to malformed or missing files. The on_file_found method should return None for files it can’t process, allowing other plugins to handle them.

Performance Considerations

• Minimize I/O: Only read files you need to process
• Fast Rejection: Use filename checks before parsing file contents
• Efficient Parsing: Use streaming parsers for large files
• Cache Results: Consider caching parsed data if files are accessed multiple times

Testing Your Plugin

Unit Testing: Test your workspace provider logic in isolation:

#[cfg(test)]
mod tests {
    use super::*;
    use std::path::PathBuf;
    use std::fs;
    use tempfile::TempDir;

    #[test]
    fn test_project_detection() {
        let provider = MyFrameworkWorkspaceProvider;
        let workspace = Workspace {
            root: PathBuf::from("/workspace"),
            projects: vec![],
            inferred_projects: vec![],
        };
         
        let path = PathBuf::from("/workspace/project/my-framework.json");
        let result = provider.on_file_found(&workspace, &path);
         
        assert!(result.is_some());
        let project = result.unwrap();
        assert_eq!(project.name, "project");
        assert_eq!(project.discovered_by, "my-framework");
        assert!(project.workspace_dependencies.is_empty());
    }

    #[test]
    fn test_workspace_dependencies() {
        // Create a temporary workspace with test files
        let temp_dir = TempDir::new().unwrap();
        let workspace_root = temp_dir.path();
         
        // Create a config file with dependencies
        let project_dir = workspace_root.join("my-project");
        fs::create_dir_all(&project_dir).unwrap();
         
        let config_file = project_dir.join("my-framework.json");
        fs::write(&config_file, r#"
        {
            "name": "my-project",
            "dependencies": {
                "shared-utils": "workspace:*",
                "lodash": "^4.0.0"
            }
        }"#).unwrap();

        let workspace = Workspace {
            root: workspace_root.to_path_buf(),
            projects: vec![],
            inferred_projects: vec![
                InferredProject {
                    name: "shared-utils".to_string(),
                    project_dir: workspace_root.join("shared-utils"),
                    discovered_by: "my-framework".to_string(),
                    workspace_dependencies: vec![],
                }
            ],
        };

        let provider = MyFrameworkWorkspaceProvider;
        let result = provider.on_file_found(&workspace, &config_file).unwrap();
         
        // Should detect workspace dependency but not external dependency
        assert_eq!(result.workspace_dependencies, vec!["shared-utils"]);
    }

    #[test]
    fn test_include_patterns() {
        let provider = MyFrameworkWorkspaceProvider;
        let patterns = provider.include_path_globs();
         
        assert!(!patterns.is_empty());
        assert!(patterns.contains(&"**/my-framework.json".to_string()));
    }

    #[test]
    fn test_exclude_patterns() {
        let provider = MyFrameworkWorkspaceProvider;
        let patterns = provider.exclude_path_globs();
         
        // Should exclude common build/cache directories
        assert!(patterns.iter().any(|p| p.contains("node_modules")));
    }
}

Integration Testing: Test with real project files:

#[cfg(test)]
mod integration_tests {
    use super::*;
    use std::process::Command;

    #[test]
    fn test_real_project_files() {
        // Test with actual config files from your examples
        let test_data_dir = std::path::Path::new("tests/fixtures");
         
        for entry in std::fs::read_dir(test_data_dir).unwrap() {
            let path = entry.unwrap().path();
            if path.extension().and_then(|s| s.to_str()) == Some("json") {
                let provider = MyFrameworkWorkspaceProvider;
                let workspace = Workspace {
                    root: test_data_dir.to_path_buf(),
                    projects: vec![],
                    inferred_projects: vec![],
                };
                 
                // Should not panic on any real config file
                let _ = provider.on_file_found(&workspace, &path);
            }
        }
    }

    #[test]
    fn test_plugin_loading() {
        // Build the plugin and test that it can be loaded
        let output = Command::new("cargo")
            .args(&["build", "--lib", "--release"])
            .output()
            .expect("Failed to build plugin");
         
        assert!(output.status.success(), "Plugin failed to build");
         
        // Verify the dynamic library was created
        let lib_path = std::path::Path::new("target/release")
            .join(if cfg!(target_os = "linux") { "libmarty_plugin_my_framework.so" }
                  else if cfg!(target_os = "macos") { "libmarty_plugin_my_framework.dylib" }
                  else { "marty_plugin_my_framework.dll" });
         
        assert!(lib_path.exists(), "Dynamic library not found");
    }
}

Migration from WASM Plugins

If you’re migrating from the old WASM-based plugin system:

Key Changes

Build Target:

# OLD: WASM target
# cargo build --target wasm32-wasip1

# NEW: Native dynamic library
[lib]
crate-type = ["cdylib"]

Export Method:

// OLD: Manual WASM exports
#[no_mangle]
pub extern "C" fn _start() { /* ... */ }

// NEW: Simple macro
use marty_plugin_protocol::dylib::export_plugin;
struct MyPlugin;
export_plugin!(MyPlugin);

Performance:

• ✅ Faster: Native code execution vs WASM interpretation
• ✅ Simpler: No WASM runtime setup or sandboxing
• ✅ Better debugging: Native debugging tools work
• ✅ Easier deployment: Standard shared libraries

Dependencies:

# OLD: Limited to WASM-compatible crates
[dependencies]
serde = { version = "1.0", default-features = false }

# NEW: Full crate ecosystem available
[dependencies]
serde_json = "1.0"
regex = "1.0"
# Any crate works!

Migration Steps

1. Update Cargo.toml:

   • Change crate-type to ["cdylib"]
   • Remove WASM-specific profile settings
   • Add any previously unavailable dependencies

2. Update plugin code:

   • Remove manual WASM export functions
   • Add export_plugin!(YourPlugin) at the end of lib.rs
   • Ensure your plugin has a new() constructor

3. Update build process:

   • Remove WASM build commands
   • Use cargo build --lib --release instead
   • Look for .so/.dylib/.dll files in target/release/

4. Test the migration:

   • Copy the dynamic library to Marty’s plugin directory
   • Verify plugin loading and project detection
   • Check that performance has improved

Troubleshooting Guide

Common Build Issues

“crate-type must be cdylib”

# Add this to your Cargo.toml
[lib]
crate-type = ["cdylib"]

“export_plugin macro not found”

// Make sure you import the macro
use marty_plugin_protocol::dylib::export_plugin;

“Plugin struct must have new() method”

struct MyPlugin;

impl MyPlugin {
    pub const fn new() -> Self {
        Self
    }
}

Runtime Issues

Plugin not detected by Marty:

• Check that the plugin file is in the correct location (~/.marty/plugins/)
• Verify the file has the correct extension (.so, .dylib, or .dll)
• Ensure the plugin exports the required C symbols
• Check Marty logs for loading errors

Projects not being discovered:

• Verify your include_path_globs() patterns match your target files
• Check that your patterns don’t conflict with exclude_path_globs()
• Test your on_file_found() logic with sample files
• Ensure you’re returning Some(InferredProject) for valid projects

Workspace dependencies not working:

• Confirm you’re only including internal workspace projects, not external packages
• Check that dependency names match the target projects’ names exactly
• Verify the target projects exist in the workspace
• Review your dependency parsing logic for correctness

Performance Issues

Slow workspace scanning:

• Make include patterns as specific as possible
• Add exclude patterns for large directories (node_modules, target, etc.)
• Avoid reading file contents unless necessary
• Use filename checks before parsing

Memory usage:

• Don’t cache large amounts of data in plugin structs
• Let Marty handle string cleanup via plugin_cleanup_string()
• Avoid creating unnecessary allocations in hot paths

Development Tips

Testing strategies:

• Create unit tests for your workspace provider logic
• Use integration tests with real project files
• Test edge cases (malformed files, missing fields, etc.)
• Verify plugin loading with cargo build --lib --release

Debugging techniques:

• Use eprintln!() for debug output (visible in Marty logs)
• Test workspace providers independently before plugin export
• Use cargo expand to inspect macro-generated code
• Enable Marty debug logging to see plugin interactions

Data Structures Reference

• MartyPlugin - Main plugin interface with metadata and configuration
• WorkspaceProvider - Plugin workspace scanning and detection logic
• PluginKey - Non-whitespace identifier for plugins
• Workspace - Workspace context passed to plugins during discovery
• Project - Represents a project with an explicit marty.yml config file
• InferredProject - Represents a project discovered by a plugin
• InferredProjectMessage - Serializable version for plugin communication
• export_plugin! - Macro to export your plugin with C ABI interface

Modules

dylib

Dynamic library plugin interface

Macros

export_plugin

Macro to export your plugin with a C ABI interface for dynamic library loading.

Structs

InferredProject

Represents a project automatically discovered by a plugin without explicit Marty configuration.

InferredProjectMessage

Serializable message format for WASM plugin communication

PluginKey

A plugin key that ensures no whitespace characters

Project

Represents a project with an explicit marty.yml or marty.yaml configuration file.

Workspace

A workspace containing all discovered projects and their metadata.

Traits

MartyPlugin

The main plugin interface that defines plugin metadata and behavior.

WorkspaceProvider

The main interface that all workspace provider plugins must implement.