Module transfer 

Transfer Learning module for cross-project knowledge sharing.

This module provides infrastructure for transfer learning, enabling knowledge sharing across related tasks (e.g., transpiler ecosystems like depyler, ruchy, bashrs).

Architecture

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Python    │     │    Ruby     │     │    Bash     │
│   Source    │     │   Source    │     │   Source    │
└──────┬──────┘     └──────┬──────┘     └──────┬──────┘
       │                   │                   │
       ▼                   ▼                   ▼
┌─────────────────────────────────────────────────────┐
│          Shared Error Embedding Space               │
│     (E0308, E0277, E0425, E0599, ... )             │
└─────────────────────────────────────────────────────┘
       │                   │                   │
       ▼                   ▼                   ▼
┌──────────────┐   ┌──────────────┐   ┌──────────────┐
│  depyler     │   │   ruchy      │   │   bashrs     │
│  Oracle      │   │   Oracle     │   │   Oracle     │
└──────────────┘   └──────────────┘   └──────────────┘

Example

use aprender::transfer::{TransferEncoder, MultiTaskHead};
use aprender::nn::{Linear, Module};

// Create shared encoder
let encoder = SharedEncoder::new(512, 256);

// Wrap for transfer learning
let mut transfer_encoder = TransferableEncoder::new(encoder);

// Pre-train on large dataset, then freeze
transfer_encoder.freeze_base();

// Create task-specific heads
let mut multi_task = MultiTaskHead::new(transfer_encoder);
multi_task.add_task("depyler", 128);
multi_task.add_task("ruchy", 128);

References

• Yosinski, J., et al. (2014). How transferable are features in deep neural networks? NeurIPS.
• Hu, E. J., et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685.

Structs

AttentionTransfer

Attention Transfer (Zagoruyko & Komodakis, 2017).

DomainAdapter

Domain adaptation for aligning source and target distributions.

FeatureDistillation

Feature Distillation for matching intermediate representations.

KnowledgeDistillation

Knowledge Distillation (Hinton et al., 2015).

LoRAAdapter

LoRA adapter weights for a single layer.

LoRAConfig

LoRA (Low-Rank Adaptation) configuration.

MatchingNetwork

Matching Networks for few-shot learning (Vinyals et al., 2016).

MultiTaskHead

Multi-task learning head with shared encoder.

OnlineDistillation

Online Distillation / Deep Mutual Learning (Zhang et al., 2018).

ProgressiveDistillation

Progressive Distillation (Salimans & Ho, 2022).

PrototypicalNetwork

Prototypical Networks for few-shot learning (Snell et al., 2017).

SelfDistillation

Self-Distillation (Zhang et al., 2019).

TransferableEncoder

A wrapper that adds transfer learning capabilities to any Module.

Enums

DistanceMetric

FeatureLossType

Traits

TransferEncoder

Trait for encoders that support transfer learning operations.