Crate torsh_hub 

ToRSh Hub - Enterprise Model Hub and Management Platform

torsh-hub provides a comprehensive model hub and management platform for ToRSh, similar to PyTorch Hub and Hugging Face Hub, with enterprise-grade features.

Features

Core Functionality

• Model Registry: Centralized model discovery and version management
• Model Download: Advanced parallel downloading with mirrors and CDN support
• Model Loading: Support for ONNX, TensorFlow, and native ToRSh models
• Model Hub Integration: Seamless integration with Hugging Face Hub

Enterprise Features

• Access Control: Fine-grained RBAC and permission management
• Private Repositories: Secure private model storage for organizations
• Audit Logging: Comprehensive audit trails for compliance
• SLA Management: Service level agreements and performance monitoring

Community Platform

• Model Ratings: Community-driven model quality assessment
• Discussions: Collaborative discussions on models and techniques
• Challenges: ML challenges and competitions
• Contributions: Track and recognize community contributions

Advanced Capabilities

• Fine-tuning: Built-in fine-tuning with early stopping and checkpointing
• Profiling: Comprehensive model performance profiling
• Debugging: Advanced debugging tools with interactive sessions
• Analytics: Real-time analytics and usage tracking
• Visualization: Performance visualization and dashboard generation
• Security: Model sandboxing and security scanning

SciRS2 POLICY Compliance

This crate strictly follows the SciRS2 POLICY:

• All array operations use scirs2_core::ndarray::*
• All random operations use scirs2_core::random::*
• NO direct external dependencies (ndarray, rand, etc.)

Quick Start

use torsh_hub::registry::{ModelRegistry, SearchQuery, ModelCategory};

// Initialize the registry
let mut registry = ModelRegistry::new("./models")?;

// Search for models
let mut query = SearchQuery::default();
query.category = Some(ModelCategory::Vision);
let results = registry.search(&query);

// Load a model with load_onnx_model function
// let model = torsh_hub::load_onnx_model("model.onnx", None)?;

// Use pre-built architecture components from models module
// use torsh_hub::models::vision::ResNet;
// let resnet = ResNet::resnet18(1000)?;

Module Organization

• models: Pre-built model architectures (BERT, GPT, ResNet, ViT, CLIP, etc.)
• registry: Model registry and discovery
• download: Advanced download management with mirrors and CDN
• onnx: ONNX model loading and conversion
• huggingface: Hugging Face Hub integration
• fine_tuning: Model fine-tuning utilities
• profiling: Performance profiling and analysis
• debugging: Interactive debugging tools
• security: Model security and sandboxing
• enterprise: Enterprise features (RBAC, audit, SLA)
• community: Community platform (ratings, discussions, challenges)
• analytics: Analytics and recommendation engine
• visualization: Performance visualization
• utils: Utility functions for common tasks
• cli: Command-line interface

Re-exports

pub use access_control::AccessToken;

pub use access_control::PermissionChecker;

pub use access_control::RateLimit;

pub use access_control::TokenManager;

pub use access_control::TokenScope;

pub use access_control::TokenStats;

pub use analytics::ABTestingFramework;

pub use analytics::AnalyticsManager;

pub use analytics::AnalyticsReport;

pub use analytics::ExportFormat;

pub use analytics::ModelUsageStats;

pub use analytics::PerformanceProfiler;

pub use analytics::RealTimeMetrics;

pub use analytics::RecommendationEngine;

pub use analytics::UserAnalytics;

pub use bandwidth::format_bytes;

pub use bandwidth::format_duration;

pub use bandwidth::AdaptiveBandwidthLimiter;

pub use bandwidth::BandwidthLimiter;

pub use bandwidth::BandwidthMonitor;

pub use bandwidth::BandwidthStats;

pub use cache::compress_file;

pub use cache::decompress_file;

pub use cache::CacheCleanupResult;

pub use cache::CacheManager;

pub use cache::CacheStats;

pub use cache::CacheValidationResult;

pub use cache::CompressionResult;

pub use cache::FileCompressionStats;

pub use cli::run_cli;

pub use cli::Cli;

pub use cli::CliApp;

pub use cli::CliConfig;

pub use cli::Commands;

pub use community::Badge;

pub use community::Challenge;

pub use community::ChallengeId;

pub use community::ChallengeParticipant;

pub use community::ChallengeStatus;

pub use community::ChallengeSubmission;

pub use community::ChallengeType;

pub use community::Comment;

pub use community::CommunityManager;

pub use community::Contribution;

pub use community::ContributionStatus;

pub use community::ContributionType;

pub use community::Discussion;

pub use community::DiscussionCategory;

pub use community::DiscussionId;

pub use community::DiscussionStatus;

pub use community::EvaluationCriteria;

pub use community::MetricType;

pub use community::ModelId;

pub use community::ModelRating;

pub use community::ModelRatingStats;

pub use community::RatingCategory;

pub use community::UserId;

pub use community::UserProfile;

pub use debugging::ActivationAnalyzer;

pub use debugging::ActivationPattern;

pub use debugging::Anomaly;

pub use debugging::AnomalyType;

pub use debugging::DebugAction;

pub use debugging::DebugCommand;

pub use debugging::DebugConfig;

pub use debugging::DebugHook;

pub use debugging::DebugReport;

pub use debugging::DebugSession;

pub use debugging::GradientDebugger;

pub use debugging::GradientInfo;

pub use debugging::HookType;

pub use debugging::InteractiveDebugState;

pub use debugging::ModelDebugger;

pub use debugging::Severity;

pub use debugging::TensorInspector;

pub use debugging::TensorSnapshot;

pub use debugging::TensorStatistics;

pub use debugging::TriggerCondition;

pub use download::create_regional_cdn_config;

pub use download::create_regional_mirror_config;

pub use download::download_file_parallel;

pub use download::download_file_streaming;

pub use download::download_files_parallel;

pub use download::download_with_default_cdn;

pub use download::download_with_default_mirrors;

pub use download::validate_url;

pub use download::validate_urls;

pub use download::CdnConfig;

pub use download::CdnEndpoint;

pub use download::CdnManager;

pub use download::CdnStatistics;

pub use download::FailoverStrategy;

pub use download::MirrorAttempt;

pub use download::MirrorBenchmarkResult;

pub use download::MirrorCapacity;

pub use download::MirrorConfig;

pub use download::MirrorDownloadResult;

pub use download::MirrorLocation;

pub use download::MirrorManager;

pub use download::MirrorSelectionStrategy;

pub use download::MirrorServer;

pub use download::MirrorStatistics;

pub use download::MirrorWeights;

pub use download::ParallelDownloadConfig;

pub use enterprise::Action;

pub use enterprise::AuditAction;

pub use enterprise::AuditLogEntry;

pub use enterprise::ComplianceLabel;

pub use enterprise::ComplianceReport;

pub use enterprise::DataClassification;

pub use enterprise::EnterpriseManager;

pub use enterprise::OrganizationId;

pub use enterprise::Permission;

pub use enterprise::PermissionId;

pub use enterprise::PermissionScope;

pub use enterprise::PrivateRepository;

pub use enterprise::RepositoryAccessControl;

pub use enterprise::RepositoryVisibility;

pub use enterprise::ResourceType;

pub use enterprise::Role;

pub use enterprise::RoleId;

pub use enterprise::ServiceLevelAgreement;

pub use enterprise::ServiceTier;

pub use enterprise::SlaPerformanceReport;

pub use enterprise::UserRoleAssignment;

pub use fine_tuning::CheckpointManager;

pub use fine_tuning::EarlyStoppingConfig;

pub use fine_tuning::FineTuner;

pub use fine_tuning::FineTuningConfig;

pub use fine_tuning::FineTuningFactory;

pub use fine_tuning::FineTuningStrategy;

pub use fine_tuning::TrainingHistory;

pub use fine_tuning::TrainingMetrics;

pub use huggingface::HfModelConfig;

pub use huggingface::HfModelInfo;

pub use huggingface::HfSearchParams;

pub use huggingface::HfToTorshConverter;

pub use huggingface::HuggingFaceHub;

pub use metadata::ExtendedMetadata;

pub use metadata::FileMetadata;

pub use metadata::MetadataManager;

pub use metadata::MetadataSearchCriteria;

pub use metadata::PerformanceMetrics;

pub use metadata::QualityScores;

pub use metadata::UsageStatistics;

pub use model_info::ModelCard;

pub use model_info::ModelCardBuilder;

pub use model_info::ModelCardManager;

pub use model_info::ModelCardRenderer;

pub use model_info::ModelInfo;

pub use model_info::Version;

pub use model_info::VersionHistory;

pub use model_ops::compare_models;

pub use model_ops::create_model_ensemble;

pub use model_ops::load_model_auto;

pub use model_ops::ComparisonOptions;

pub use model_ops::ConversionMetadata;

pub use model_ops::EnsembleConfig;

pub use model_ops::ModelDiff;

pub use model_ops::QuantizationStats;

pub use model_ops::ShapeDifference;

pub use model_ops::ValueDifference;

pub use model_ops::VotingStrategy;

pub use models::audio;

pub use models::multimodal;

pub use models::nlp;

pub use models::nlp_pretrained;

pub use models::rl;

pub use models::vision;

pub use models::vision_pretrained;

pub use models::ActorCritic;

pub use models::BasicBlock;

pub use models::BertEmbeddings;

pub use models::BertEncoder;

pub use models::EfficientNet;

pub use models::GPTDecoder;

pub use models::GPTEmbeddings;

pub use models::MultiHeadAttention;

pub use models::PPOAgent;

pub use models::ResNet;

pub use models::TransformerBlock;

pub use models::VisionTransformer;

pub use models::DQN;

pub use onnx::InputShape;

pub use onnx::OnnxConfig;

pub use onnx::OnnxLoader;

pub use onnx::OnnxModel;

pub use onnx::OnnxModelMetadata;

pub use onnx::OnnxToTorshWrapper;

pub use onnx::OutputShape;

pub use profiling::ExecutionContext;

pub use profiling::ExecutionMode;

pub use profiling::LayerProfile;

pub use profiling::MemoryAnalysis;

pub use profiling::MemorySnapshot;

pub use profiling::ModelProfiler;

pub use profiling::OperationAnalysis;

pub use profiling::OperationTrace;

pub use profiling::OptimizationRecommendation;

pub use profiling::PerformanceBottleneck;

pub use profiling::PerformanceCounters;

pub use profiling::PerformanceSummary;

pub use profiling::ProfilerConfig;

pub use profiling::ProfilingResult;

pub use profiling::ProfilingSession;

pub use profiling::ResourceUtilizationSummary;

pub use profiling::TensorInfo;

pub use registry::HardwareFilter;

pub use registry::ModelCategory;

pub use registry::ModelRegistry;

pub use registry::ModelStatus;

pub use registry::RegistryAPI;

pub use registry::RegistryEntry;

pub use registry::SearchQuery;

pub use retry::retry_with_backoff;

pub use retry::retry_with_backoff_async;

pub use retry::retry_with_policy;

pub use retry::CircuitBreaker;

pub use retry::CircuitState;

pub use retry::DefaultRetryPolicy;

pub use retry::RetryConfig;

pub use retry::RetryPolicy;

pub use retry::RetryStats;

pub use security::calculate_file_hash;

pub use security::sandbox_model;

pub use security::scan_model_vulnerabilities;

pub use security::validate_model_source;

pub use security::validate_signature_age;

pub use security::verify_file_integrity;

pub use security::KeyPair;

pub use security::ModelSandbox;

pub use security::ModelSignature;

pub use security::ResourceUsage;

pub use security::RiskLevel;

pub use security::SandboxConfig;

pub use security::SandboxedModel;

pub use security::ScanMetadata;

pub use security::SecurityConfig;

pub use security::SecurityManager;

pub use security::Severity as SecuritySeverity;

pub use security::SignatureAlgorithm;

pub use security::Vulnerability;

pub use security::VulnerabilityScanResult;

pub use security::VulnerabilityScanner;

pub use security::VulnerabilityType;

pub use upload::batch_publish_models;

pub use upload::upload_model;

pub use upload::upload_model_with_versioning;

pub use upload::validate_version_change;

pub use upload::PublishResult;

pub use upload::PublishStrategy;

pub use upload::UploadConfig;

pub use upload::VersionChangeInfo;

pub use upload::VersionValidationRules;

pub use utils::cleanup_old_cache;

pub use utils::compare_versions;

pub use utils::estimate_parameters_from_size;

pub use utils::extract_extension;

pub use utils::format_parameter_count;

pub use utils::format_size;

pub use utils::get_model_cache_dir;

pub use utils::get_temp_dir;

pub use utils::is_safe_path;

pub use utils::is_supported_model_format;

pub use utils::parse_repo_string;

pub use utils::sanitize_model_name;

pub use utils::validate_semver;

pub use visualization::ChartData;

pub use visualization::ChartType;

pub use visualization::DashboardTemplate;

pub use visualization::PerformanceVisualization;

pub use visualization::TrainingVisualization;

pub use visualization::UsageVisualization;

pub use visualization::VisualizationConfig;

pub use visualization::VisualizationEngine;

Modules

access_control

Access Control and Token Management

analytics

Model analytics and usage tracking system

bandwidth

Bandwidth Management and Rate Limiting

cache

Cache management for ToRSh Hub

cli

Command-line interface for torsh-hub

community

Community features for ToRSh Hub

debugging

Model debugging tools for ToRSh Hub

download

Download functionality for ToRSh Hub

enterprise

Enterprise features for ToRSh Hub

export

Model Export Utilities

fine_tuning

Model fine-tuning framework for ToRSh Hub

huggingface

HuggingFace Hub integration for ToRSh

metadata

Metadata management system for model information

model_info

Model information and metadata

model_ops

Advanced model operations and utilities

models

Model architectures for torsh-hub

onnx

ONNX model support for ToRSh Hub

prelude

Prelude module for convenient imports

profiling

Model Profiling and Performance Analysis

quantization

Model Quantization and Compression

registry

Model Registry - Centralized Model Discovery and Management

retry

Advanced retry logic with exponential backoff and jitter

security

Security functionality for ToRSh Hub

sources

Pre-configured model sources

upload

Model uploading functionality with versioning support

utils

Utility Functions for ToRSh Hub

visualization

Visualization tools for model analytics and profiling

Structs

HubConfig

Hub configuration

Constants

VERSION

VERSION_MAJOR

VERSION_MINOR

VERSION_PATCH

Functions

apply_device_mapping

Apply device mapping to state dict

auth_status

Get current authentication status

download_repo

Download repository

get_dir

Get the current hub directory

help

Get help/docstring for a model

is_authenticated

Check if authenticated

list

List available models in a repository

load

Load a model from ToRSh Hub

load_onnx_model

Load an ONNX model from file

load_onnx_model_from_bytes

Load an ONNX model from bytes

load_onnx_model_from_hub

Load an ONNX model from ToRSh Hub

load_onnx_model_from_url

Download and load an ONNX model from URL

load_state_dict_from_url

Load model state dict from URL

parse_repo_info

Parse repository information

remove_auth_token

Remove authentication token

set_auth_token

Set authentication token

set_dir

Set the hub directory

validate_auth_token

Validate authentication token

Type Aliases

StateDict