magma-code 0.1.3

# Magma Ingest CLI Tool

A Rust-based CLI tool for ingesting code files into the Magma scanner system. This tool automatically collects source code files from your repository, respects ignore patterns, and packages them for analysis.

## Features

- **Simple CLI Interface**: Easy-to-use command with minimal required parameters
- **Git Integration**: Automatically extracts commit hash, branch name, and author information
- **Smart File Filtering**: Uses both default patterns and custom `.magmaignore` files to exclude unwanted files
- **Efficient Packaging**: Creates compressed zip files with metadata for optimal transfer
- **CI/CD Ready**: Designed for seamless integration into continuous integration pipelines

## Installation

### Via Cargo (Recommended)
```bash
cargo install magma-code
```

### Direct Binary Download
#### **Linux x86_64**

```bash
# Download and install
curl -L -o magma-ingest https://github.com/bluemagma-compliance/magma-ingest/releases/latest/download/magma-ingest-linux-x86_64
chmod +x magma-ingest
sudo mv magma-ingest /usr/local/bin/

# Verify installation
magma-ingest --version
```

#### **macOS Intel**

```bash
# Download and install
curl -L -o magma-ingest https://github.com/bluemagma-compliance/magma-ingest/releases/latest/download/magma-ingest-macos-x86_64
chmod +x magma-ingest
sudo mv magma-ingest /usr/local/bin/

# Verify installation
magma-ingest --version
```

#### **macOS ARM (Apple Silicon)**

```bash
# Download and install
curl -L -o magma-ingest https://github.com/bluemagma-compliance/magma-ingest/releases/latest/download/magma-ingest-macos-aarch64
chmod +x magma-ingest
sudo mv magma-ingest /usr/local/bin/

# Verify installation
magma-ingest --version
```

#### **Windows PowerShell (Recommended)**

```powershell
# Download and setup (run PowerShell as Administrator)
cd $env:USERPROFILE\Downloads
Invoke-WebRequest -Uri "https://github.com/bluemagma-compliance/magma-ingest/releases/latest/download/magma-ingest-windows-x86_64.exe" -OutFile "magma-ingest.exe"

# Create tools directory and move binary
mkdir C:\tools -Force
Move-Item "magma-ingest.exe" "C:\tools\magma-ingest.exe"

# Add to system PATH permanently
$currentPath = [Environment]::GetEnvironmentVariable("PATH", "Machine")
if ($currentPath -notlike "*C:\tools*") {
    [Environment]::SetEnvironmentVariable("PATH", "$currentPath;C:\tools", "Machine")
    Write-Host "✅ Added C:\tools to system PATH"
}

# Add to current session
$env:PATH += ";C:\tools"

# Verify installation
magma-ingest --version
```

#### **Windows Git Bash**

```bash
# Download
cd ~/Downloads
curl -L -o magma-ingest.exe https://github.com/bluemagma-compliance/magma-ingest/releases/latest/download/magma-ingest-windows-x86_64.exe

# Setup directory (run in PowerShell as Administrator)
# powershell -Command "mkdir C:\tools -Force; Move-Item '$env:USERPROFILE\Downloads\magma-ingest.exe' 'C:\tools\magma-ingest.exe'; \$env:PATH += ';C:\tools'"

# Add to Git Bash PATH permanently
echo 'export PATH="$PATH:/c/tools"' >> ~/.bashrc
source ~/.bashrc

# Verify installation (restart Git Bash first)
magma-ingest --version
```



### Verify Installation

```bash
magma-ingest --version
```

### Troubleshooting PATH Issues

#### **"Command not found" or "magma-ingest is not recognized"**

**Check if binary exists:**
```bash
# Linux/macOS
ls -la /usr/local/bin/magma-ingest
ls -la ~/.local/bin/magma-ingest

# Windows
dir C:\tools\magma-ingest.exe
```

**Check current PATH:**
```bash
# Linux/macOS/Git Bash
echo $PATH

# Windows PowerShell
$env:PATH -split ';'
```

**Manually run binary:**
```bash
# Linux/macOS
/usr/local/bin/magma-ingest --version
~/.local/bin/magma-ingest --version

# Windows
C:\tools\magma-ingest.exe --version
```

**Restart terminal/shell:**
- Close and reopen your terminal
- PATH changes often require a restart to take effect

#### **Permission Issues (Linux/macOS)**
```bash
# Make sure binary is executable
chmod +x /usr/local/bin/magma-ingest
# or
chmod +x ~/.local/bin/magma-ingest
```

#### **Windows PATH Not Working**
```powershell
# Check if C:\tools is in PATH
$env:PATH -split ';' | Select-String "tools"

# Manually add for current session
$env:PATH += ";C:\tools"

# Restart PowerShell and try again
```



## Usage

### Basic Usage

```bash
magma-ingest --org <organization-id> --key <api-key> --repo-name <repository-name>
```

### Advanced Usage

```bash
magma-ingest \
  --org my-org-123 \
  --key sk-1234567890abcdef \
  --repo-name my-awesome-project \
  --root-path backend \
  --ignore-file custom-ignore.txt \
  --verbose
```

### Parameters

- `--org` (required): Organization ID for the Magma scanner
- `--key` (required): API key for authentication
- `--repo-name` (required): Repository name for identification
- `--root-path` (optional): Root path to scan - use '/' for current directory or specify subdirectory
- `--ignore-file` (optional): Custom ignore file path (defaults to `.magmaignore`)
- `--verbose` (optional): Enable verbose logging

### Configuration

The API base URL is configured in `config.yaml` (embedded in the binary) and defaults to `http://localhost`. The tool automatically:
- Scans the current working directory
- Creates temporary zip files in the system temp directory
- Cleans up temporary files after upload

## Ignore Patterns

The tool uses a `.magmaignore` file to specify which files and directories to exclude from scanning. The syntax is similar to `.gitignore` files and supports glob patterns.

### Default Ignore Patterns

The tool automatically ignores common files and directories:

- Version control directories (`.git`, `.svn`, `.hg`)
- Dependency directories (`node_modules`, `target`, `build`, `__pycache__`)
- IDE files (`.vscode`, `.idea`)
- Binary files (`*.exe`, `*.dll`, `*.so`)
- Archive files (`*.zip`, `*.tar.gz`)
- Media files (`*.jpg`, `*.png`, `*.mp4`)
- Log files (`*.log`)

### Custom Ignore File

Create a `.magmaignore` file in your repository root to specify additional patterns:

```
# Custom ignores
config/secrets.json
*.env
temp/**

# Large data files
data/**
*.csv

# Documentation
docs/**
*.md
```

## Output

The tool performs the following actions:

1. **Collects Files**: Gathers all non-ignored files from your repository
2. **Creates Package**: Generates a zip file with source files and metadata
3. **Uploads to API**: Sends the package to your Magma ingest endpoint via HTTP POST
4. **Cleans Up**: Optionally removes the local zip file after successful upload (use `--keep-zip` to retain)

### API Request Format

The tool sends a multipart/form-data POST request to `{api-base-url}/ingest/ingest/` with:

- `org_id`: Organization identifier
- `repo_name`: Repository name
- `commit_hash`: Git commit hash
- `branch_name`: Git branch name
- `root_path`: Repository root path
- `file`: Zip file containing source code and metadata

### Metadata Format

```json
{
  "version": "1.0",
  "timestamp": "2024-01-15T10:30:00Z",
  "git": {
    "branch": "main",
    "commit_hash": "abc123...",
    "commit_message": "Add new feature",
    "author_name": "John Doe",
    "author_email": "john@example.com"
  },
  "scan_info": {
    "file_count": 150,
    "tool_version": "0.1.0",
    "tool_name": "magma-ingest"
  }
}
```

## CI/CD Integration

### GitHub Actions

```yaml
name: Code Analysis
on: [push, pull_request]

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Install Rust
        uses: actions-rs/toolchain@v1
        with:
          toolchain: stable

      - name: Install magma-ingest
        run: cargo install --git <repository-url>

      - name: Run Magma Ingest
        run: magma-ingest --org ${{ secrets.MAGMA_ORG_ID }} --key ${{ secrets.MAGMA_API_KEY }}
```

### GitLab CI

```yaml
stages:
  - scan

magma_scan:
  stage: scan
  image: rust:latest
  script:
    - cargo install --git <repository-url>
    - magma-ingest --org $MAGMA_ORG_ID --key $MAGMA_API_KEY
  variables:
    MAGMA_ORG_ID: $MAGMA_ORG_ID
    MAGMA_API_KEY: $MAGMA_API_KEY
```

## Error Handling

The tool provides clear error messages for common issues:

- **No Git Repository**: Ensure you're running the tool in a Git repository
- **No Files to Ingest**: Check your ignore patterns if no files are being collected
- **Invalid Patterns**: Verify your `.magmaignore` file syntax
- **Permission Issues**: Ensure the tool has read access to files and write access to output directory

## Logging

Use the `--verbose` flag to enable detailed logging:

```bash
magma-ingest --org my-org --key my-key --verbose
```

This will show:
- Files being processed
- Ignore patterns being applied
- Git information extraction
- Zip file creation progress

## Project Structure

```
Magma-yeeter/
├── src/
│   ├── main.rs          # Main application entry point
│   ├── cli.rs           # Command-line argument parsing
│   ├── git.rs           # Git repository integration
│   ├── ignore.rs        # File ignore pattern management
│   ├── zipper.rs        # File compression and packaging
│   └── error.rs         # Error handling and types
├── examples/
│   ├── github-actions.yml  # GitHub Actions workflow example
│   └── gitlab-ci.yml       # GitLab CI configuration example
├── .magmaignore         # Default ignore patterns
├── Cargo.toml           # Rust project configuration
└── README.md            # This file
```

## Development

### Building

```bash
# Debug build
cargo build

# Release build
cargo build --release
```

### Testing

```bash
cargo test
```

### Running

```bash
# Development
cargo run -- --org test-org --key test-key

# Release binary
./target/release/magma-ingest --org test-org --key test-key
```



## License

MIT License - see LICENSE file for details.