briefcase-python 2.1.3

Python bindings for Briefcase AI
Documentation

Briefcase AI Python SDK

Python SDK for AI observability, replay, and decision tracking. This package provides high-performance Rust-powered tools for monitoring and debugging AI applications in Python.

Crates.io PyPI

Overview

This Python SDK provides a complete toolkit for AI observability, decision tracking, and replay functionality. Built on a high-performance Rust core with Python bindings via PyO3, it offers the speed of Rust with the convenience of Python.

Features

  • High Performance - Rust core with minimal Python overhead
  • Memory Efficient - Zero-copy operations where possible
  • Type Safe - Full Python type hints and error handling
  • Async Support - Async/await compatible operations

Installation

Install the Python SDK using pip:

pip install briefcase-ai

Quick Start

import briefcase_ai
import openai

# Initialize the SDK
briefcase_ai.init()

# Track an OpenAI API call
def track_openai_call(user_message: str) -> str:
    # Create decision snapshot
    decision = briefcase_ai.DecisionSnapshot("openai_chat")

    # Add input
    decision.add_input(briefcase_ai.Input("user_message", user_message, "string"))
    decision.add_input(briefcase_ai.Input("model", "gpt-3.5-turbo", "string"))

    # Make API call (your existing code)
    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": user_message}]
    )

    ai_response = response.choices[0].message.content

    # Add output with confidence
    decision.add_output(
        briefcase_ai.Output("ai_response", ai_response, "string")
        .with_confidence(0.95)
    )
    decision.with_execution_time(response.usage.total_tokens * 0.1)  # Estimate

    # Save for analysis
    storage = briefcase_ai.SqliteBackend.file("ai_decisions.db")
    decision_id = storage.save_decision(decision)

    return ai_response

# Use it
result = track_openai_call("Explain quantum computing")
print(f"AI Response: {result}")

Advanced Usage Examples

1. Model Performance Monitoring

import briefcase_ai
from datetime import datetime

def monitor_model_performance():
    storage = briefcase_ai.SqliteBackend.file("model_performance.db")

    # Track multiple model variants
    models = ["gpt-3.5-turbo", "gpt-4", "claude-3"]

    for model in models:
        decision = briefcase_ai.DecisionSnapshot(f"comparison_test")
        decision.add_input(briefcase_ai.Input("prompt", "Write a poem", "string"))
        decision.add_input(briefcase_ai.Input("model", model, "string"))
        decision.add_tag("experiment", "model_comparison")
        decision.add_tag("timestamp", str(datetime.now()))

        # Your model call here...
        # decision.add_output(...)

        storage.save_decision(decision)

2. Cost Tracking for AI Operations

import briefcase_ai

def track_ai_costs():
    # Initialize cost calculator
    cost_calc = briefcase_ai.CostCalculator()

    # Track costs for different models
    models_usage = [
        ("gpt-4", 1500, 800),  # model, input_tokens, output_tokens
        ("gpt-3.5-turbo", 2000, 1200),
        ("claude-3", 1800, 900)
    ]

    total_cost = 0
    for model, input_tokens, output_tokens in models_usage:
        cost_estimate = cost_calc.estimate_cost(model, input_tokens, output_tokens)
        total_cost += cost_estimate.total_cost

        print(f"{model}: ${cost_estimate.total_cost:.4f}")

    print(f"Total estimated cost: ${total_cost:.4f}")

3. Drift Detection

import briefcase_ai

def detect_model_drift():
    # Monitor for changes in model outputs
    drift_calc = briefcase_ai.DriftCalculator()

    # Sample outputs from the same prompt over time
    outputs = [
        "The sky is blue because of light scattering",
        "The sky appears blue due to Rayleigh scattering",
        "Blue sky results from atmospheric light scattering",
        "The sky looks green"  # Anomaly!
    ]

    metrics = drift_calc.calculate_drift(outputs)
    print(f"Consistency score: {metrics.consistency_score:.2f}")

    if metrics.consistency_score < 0.8:
        print("⚠️ Potential model drift detected!")

4. Data Sanitization

import briefcase_ai

def sanitize_sensitive_data():
    # Remove sensitive information before logging
    sanitizer = briefcase_ai.Sanitizer()

    user_input = "My email is john@example.com and SSN is 123-45-6789"

    # Sanitize the input
    result = sanitizer.sanitize(user_input)
    print(f"Original: {user_input}")
    print(f"Sanitized: {result.sanitized}")
    print(f"Patterns found: {result.patterns_found}")

    # Use sanitized version in your AI tracking
    decision = briefcase_ai.DecisionSnapshot("user_query")
    decision.add_input(briefcase_ai.Input("query", result.sanitized, "string"))

5. Replay and Debugging

import briefcase_ai

def replay_ai_decision():
    storage = briefcase_ai.SqliteBackend.file("ai_decisions.db")

    # Load a previous decision
    decision_id = "decision-123"
    original_decision = storage.load_decision(decision_id)

    if original_decision:
        # Create replay engine
        replay_engine = briefcase_ai.ReplayEngine()

        # Replay the decision with same inputs
        replay_result = replay_engine.replay_decision(original_decision)

        if replay_result.is_deterministic:
            print("✅ Decision is reproducible")
        else:
            print("❌ Non-deterministic behavior detected")
            print(f"Differences: {replay_result.differences}")

For Rust Developers

This crate is primarily used for building the Python extension. To use Briefcase AI in Rust, use the core library directly:

[dependencies]
briefcase-core = "2.0.4"

Building

This crate requires:

  • Rust 1.70+
  • Python 3.10+
  • PyO3 dependencies

Build the Python extension:

cd crates/python
maturin build --release

Development

For development with maturin:

cd crates/python
maturin develop

Then test in Python:

import briefcase_ai
print(briefcase_ai.__version__)

Dependencies

License

MIT License - see LICENSE file for details.

Related Packages