wme-cli

A command-line interface (CLI) tool for interacting with the Wikimedia Enterprise API.

Features

• Authentication: Login, token refresh, password management
• Metadata: Browse project codes, languages, projects, and namespaces
• Articles: Fetch articles and structured article data
• Snapshots: List, inspect, and download snapshots and chunks
• Realtime: Stream live updates and manage batches
• Multiple Output Formats: JSON, NDJSON, and ASCII tables

Installation

From Source

git clone https://github.com/anomalyco/wme-sdk.git
cd wme-sdk/crates/wme-cli
cargo build --release

The binary will be available at target/release/wme-cli.

Configuration

The CLI stores configuration and tokens in ~/.wme/config.json.

You can also specify a custom config path with --config or the WME_CONFIG environment variable.

Authentication

Before using the API, you need to authenticate:

# Interactive login (will prompt for credentials)
wme auth login

# Or provide credentials via flags
wme auth login --username myuser --password mypass

# Or via environment variables
export WME_USERNAME=myuser
export WME_PASSWORD=mypass
wme auth login

Token Management

# Refresh your access token
wme auth refresh

# Revoke and clear tokens
wme auth revoke

# Change password
wme auth change-password --previous-password oldpass --proposed-password newpass

Usage Examples

Global Options

All commands support these global flags:

--token <TOKEN>        # Bearer token override (env: WME_TOKEN)
--output <FORMAT>      # Output format: json, ndjson, table (default: json)
--fields <FIELDS>      # Comma-separated fields to include
--filter <FILTER>      # Field filter as field=value (repeatable)
--limit <N>            # Max results to return
--config <PATH>        # Path to config file (env: WME_CONFIG)
-v, --verbose          # Enable verbose logging

Metadata Commands

# List all project codes
wme meta project-codes

# Get specific project code info
wme meta project-codes-get enwiki

# List all languages
wme meta languages

# Get specific language info
wme meta languages-get en

# List all projects
wme meta projects

# Get specific project info
wme meta projects-get enwiki

# List all namespaces
wme meta namespaces

# Get specific namespace info
wme meta namespaces-get 0

Article Commands

# Get a single article
wme article get "Main_Page"

# Get with specific fields
wme article get "Main_Page" --fields "name,abstract,url"

# Get structured article data (BETA)
wme article structured "Main_Page"

Snapshot Commands

# List available snapshots
wme snapshot list

# Get snapshot metadata
wme snapshot info <identifier>

# Download a snapshot (chunked, with automatic resume support)
wme snapshot download <identifier> --output-file snapshot.tar.gz

# Download a specific byte range (no resume support)
wme snapshot download <identifier> --range "0-1000000" --output-file snapshot.tar.gz

# Resume interrupted download (automatic - just rerun the same command)
wme snapshot download <identifier> --output-file snapshot.tar.gz

# List chunks for a snapshot
wme snapshot chunks list <snapshot_id>

# Get chunk metadata
wme snapshot chunks info <snapshot_id> <chunk_id>

# Download a chunk
wme snapshot chunks download <snapshot_id> <chunk_id> --output-file chunk.tar.gz

# List structured snapshots (BETA)
wme snapshot structured list

# Get structured snapshot metadata
wme snapshot structured info <identifier>

# Download structured snapshot
wme snapshot structured download <identifier> --output-file structured.tar.gz

Download Resume Feature

When downloading large snapshots in chunked mode, the CLI automatically supports resuming interrupted downloads:

• How it works: A .progress file is created next to the output file tracking completed chunks
• On interruption: Simply re-run the same download command - completed chunks are skipped
• On completion: The .progress file is automatically deleted
• Range downloads: The --range flag disables chunked mode and resume support (single HTTP request)

Example workflow:

# Start download (might timeout after chunk 45 of 94)
wme snapshot download enwiki_namespace_0 --output-file enwiki.tar.gz
# Output: Chunk 45/94 | 2.3 GB written | enwiki_chunk_45 ...

# Resume (continues from chunk 46)
wme snapshot download enwiki_namespace_0 --output-file enwiki.tar.gz
# Output: Resuming download: enwiki_namespace_0 (45 chunks already done)
# Output: Skipping 45/94 already-completed chunks
# Output: Chunk 46/94 | 2.3 GB written | enwiki_chunk_46 ...

Realtime Commands

# Connect to realtime stream
wme realtime stream

# Stream with specific partitions
wme realtime stream --parts 0 --parts 1

# Resume from timestamp
wme realtime stream --since "2024-01-01T00:00:00Z"

# List batches for a date/hour
wme realtime batches list 2024-01-01 12

# Get batch metadata
wme realtime batches info 2024-01-01 12 <identifier>

# Download a batch
wme realtime batches download 2024-01-01 12 <identifier> --output-file batch.tar.gz

Output Formats

JSON (default)

Pretty-printed JSON output:

wme meta languages --output json

NDJSON

Newline-delimited JSON for streaming processing:

wme meta languages --output ndjson

Table

ASCII table format (useful for list commands):

wme meta languages --output table

Quiet

Suppress all output except errors:

wme meta languages --output quiet

Filtering and Field Selection

# Filter results
wme snapshot list --filter "project=enwiki"

# Select specific fields
wme article get "Main_Page" --fields "name,abstract,url"

# Limit results
wme meta project-codes --limit 10

Environment Variables

Shell Completion

Generate shell completion scripts:

# Bash
wme --generate bash > /usr/share/bash-completion/completions/wme

# Zsh
wme --generate zsh > /usr/share/zsh/site-functions/_wme

# Fish
wme --generate fish > /usr/share/fish/completions/wme.fish

Development

Running Tests

cargo test -p wme-cli

Code Quality

# Format code
cargo fmt -p wme-cli

# Run linter
cargo clippy -p wme-cli --all-targets --all-features

# Run doctests
cargo test -p wme-cli --doc

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contributing

Contributions are welcome! Please read our Contributing Guide for details on our code of conduct and the process for submitting pull requests.