Crate nntp_rs 

nntp-rs

High-performance async NNTP client library for Rust with compression and connection pooling.

Scope: This library is designed for reading and downloading from Usenet/NNTP servers (client-to-server). Server-to-server peering and article posting are out of scope.

Features

• Async/await - Built on Tokio for high-performance async I/O
• TLS/SSL - Secure connections via rustls (implicit TLS on port 563)
• Compression - RFC 8054 COMPRESS DEFLATE + XFEATURE COMPRESS GZIP with automatic fallback (50-80% bandwidth reduction)
• Connection pooling - bb8-based pool with configurable size
• Retry logic - Exponential backoff with jitter to prevent thundering herd
• Binary support - yEnc decoding, NZB parsing, PAR2 verification
• Zero unsafe code - Pure safe Rust
• Production-hardened - Strict clippy linting, comprehensive error handling, well-structured modules

Code Quality

This library maintains high standards for production readiness:

• Zero unsafe code - All safe Rust, verified with #![forbid(unsafe_code)]
• Strict linting - Enforced clippy rules including #![deny(clippy::unwrap_used)] for production code
• Comprehensive error handling - All production code uses Result types with descriptive errors via thiserror
• Modular architecture - Large modules split into focused submodules for maintainability
  • src/client/ - 13 focused modules (connection, I/O, auth, commands)
  • src/article/ - Parsing, building, and validation separated
  • src/par2/ - Parsing and verification logic isolated
• Excellent test coverage - 1,400+ tests across all modules
• Well-documented - 2,900+ doc comments covering all public APIs

Installation

Add to your Cargo.toml:

[dependencies]
nntp-rs = "0.1"

Quick Start

Single Connection

use nntp_rs::{NntpClient, ServerConfig};
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = ServerConfig::tls("news.example.com", "username", "password");

    let mut client = NntpClient::connect(Arc::new(config)).await?;
    client.authenticate().await?;

    // Enable compression (optional, but recommended)
    client.try_enable_compression().await?;

    // Select a newsgroup
    let info = client.select_group("alt.test").await?;
    println!("Group has {} articles ({}-{})", info.count, info.first, info.last);

    // Fetch article overview data
    let entries = client.fetch_xover(&format!("{}-{}", info.last - 10, info.last)).await?;
    for entry in entries {
        println!("{}: {}", entry.article_number, entry.subject);
    }

    Ok(())
}

Connection Pool (Recommended for high throughput)

use nntp_rs::{NntpPool, ServerConfig};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = ServerConfig::tls("news.example.com", "username", "password");

    // Create pool with 10 connections
    let pool = NntpPool::new(config, 10).await?;

    // Get connection from pool (automatically authenticated with compression)
    let mut conn = pool.get().await?;

    let info = conn.select_group("alt.test").await?;
    println!("Group has {} articles", info.count);

    // Connection returns to pool when dropped
    Ok(())
}

Compression

This library supports two compression modes with automatic fallback:

1. RFC 8054 COMPRESS DEFLATE - Full bidirectional session compression (best)
2. XFEATURE COMPRESS GZIP - Headers-only compression (wider compatibility)

Compression is automatically negotiated when using the connection pool, or can be manually enabled:

let enabled = client.try_enable_compression().await?;
if enabled {
    let (compressed, decompressed) = client.get_bandwidth_stats();
    println!("Saved {} bytes", decompressed - compressed);
}

TLS/Security

nntp-rs uses modern TLS with strong security defaults:

• TLS 1.3 preferred, TLS 1.2 supported (TLS 1.0/1.1 disabled per RFC 8996)
• Strong cipher suites with forward secrecy (ECDHE) and authenticated encryption (GCM/Poly1305)
• Certificate validation using Mozilla’s CA bundle (webpki-roots)
• Hostname verification with SNI support

Configuration

// Secure connection (recommended)
let config = ServerConfig::tls("news.example.com", "username", "password");

// Plaintext connection (not recommended)
let config = ServerConfig::plain("news.example.com", "username", "password");

Article Parsing (RFC 5536)

nntp-rs provides comprehensive article parsing with validation and international character support:

Features

• Header parsing - All RFC 5536 required and optional headers
• Validation - Message-ID, newsgroup names, and date format validation
• RFC 2047 encoded words - Automatic decoding of international characters in headers (UTF-8, ISO-8859-1, Windows-1252)
• MIME detection - Content-Type parsing and multipart detection
• Path parsing - Extract server routing information

Example

use nntp_rs::{Article, ValidationConfig};

// Parse an article
let article = Article::parse(article_text)?;

// Access headers
println!("From: {}", article.headers.from);
println!("Subject: {}", article.headers.subject);  // RFC 2047 encoded words automatically decoded
println!("Message-ID: {}", article.headers.message_id);

// Validate article format
let config = ValidationConfig::strict();
article.headers.validate(&config)?;

// Check MIME type
if article.is_mime() {
    println!("Content-Type: {:?}", article.content_type());
    println!("Charset: {:?}", article.charset());
}

// Parse path
let servers = article.headers.parse_path();
println!("Article routed through {} servers", servers.len());

RFC Compliance

RFC	Title	Status	Test Coverage
RFC 3977	NNTP Core Protocol	Reader commands	~600 tests
RFC 4642	TLS with NNTP	Implicit TLS only	Verified
RFC 4643	Authentication	USER/PASS + SASL PLAIN	~100 tests
RFC 5536	Netnews Article Format	Complete	~156 tests
RFC 8054	Compression	Complete	~30 tests
RFC 8143	TLS Best Practices	Compliant	Verified
RFC 7525	BCP 195 TLS	Compliant	Verified
RFC 8996	Deprecate TLS 1.0/1.1	Compliant	Verified
RFC 4644	Streaming Feeds	Out of scope	-

Total test coverage: ~1,400 tests (>95% real behavioral tests)

Note: RFC 4644 (streaming feeds) is for server-to-server peering and is out of scope for this client library.

What’s Tested

• Core NNTP commands: GROUP, ARTICLE, HEAD, BODY, STAT, XOVER/OVER
• Authentication flows: AUTHINFO USER/PASS, SASL PLAIN
• Compression: COMPRESS DEFLATE, XFEATURE COMPRESS GZIP
• Response parsing and multi-line handling
• Connection pooling and retry logic
• Article format parsing (RFC 5536):
  • Header parsing and validation (Message-ID, newsgroup names, dates)
  • RFC 2047 encoded words (international characters in headers)
  • MIME detection and Content-Type parsing
  • Path header parsing
• yEnc decoding with CRC32 verification
• NZB parsing and segment ordering
• PAR2 file parsing and checksum extraction

Current Limitations

Out of Scope (by design)

This library focuses on client-to-server reading/downloading. The following server-to-server and posting features are intentionally not implemented:

• POST/IHAVE - Article posting (not a posting client)
• RFC 4644 - Streaming feeds (CHECK/TAKETHIS) for server-to-server peering
• yEnc encoding - Only decoding; not designed for uploading binaries

Not Yet Implemented

• STARTTLS - Only implicit TLS (port 563) is supported; STARTTLS upgrade not implemented
• RFC 6048 - Extended LIST commands
• PAR2 repair - Only verification; Reed-Solomon recovery not implemented
• Multi-server failover - Single server only
• Rate limiting - No bandwidth throttling
• Header caching - No persistent cache

API Reference

NntpClient

• connect(config) - Connect to NNTP server
• authenticate() - Authenticate with username/password
• authenticate_sasl(mechanism) - SASL authentication
• try_enable_compression() - Enable compression (returns true if successful)
• select_group(name) - Select a newsgroup
• fetch_article(id) - Fetch full article
• fetch_head(id) - Fetch article headers only
• fetch_body(id) - Fetch article body only
• fetch_xover(range) - Fetch article overview data
• quit() - Close connection gracefully

NntpPool

• new(config, max_size) - Create pool with default retry config
• with_retry_config(config, max_size, retry_config) - Create pool with custom retry
• get() - Get connection with automatic retry
• get_no_retry() - Get connection without retry
• state() - Get pool statistics

License

MIT License - see LICENSE file for details.

Re-exports

pub use article::Article;

pub use article::ArticleBuilder;

pub use article::ControlMessage;

pub use article::Headers;

pub use article::parse_article;

pub use article::parse_headers;

pub use assembler::ArticleAssembler;

pub use assembler::PartInfo;

pub use assembler::PartStatus;

pub use cache::HeaderCache;

pub use cache::LruHeaderCache;

pub use commands::ArticleInfo;

pub use commands::DistributionInfo;

pub use commands::GroupInfo;

pub use commands::HdrEntry;

pub use commands::ModeratorInfo;

pub use commands::XoverEntry;

pub use nzb::Nzb;

pub use nzb::NzbFile;

pub use nzb::NzbSegment;

pub use nzb::parse_nzb;

pub use par2::CreatorPacket;

pub use par2::FileDescriptionPacket;

pub use par2::FileStatus;

pub use par2::FileVerification;

pub use par2::IfscPacket;

pub use par2::MainPacket;

pub use par2::PacketHeader;

pub use par2::PacketType;

pub use par2::Par2File;

pub use par2::Par2Set;

pub use par2::RecoverySlicePacket;

pub use ratelimit::BandwidthLimiter;

pub use ratelimit::ConnectionLimiter;

pub use ratelimit::ConnectionPermit;

pub use sasl::SaslMechanism;

pub use sasl::SaslPlain;

pub use sasl::decode_sasl_data;

pub use sasl::encode_sasl_data;

pub use segments::FetchConfig;

pub use segments::FetchProgress;

pub use segments::SegmentFetchResult;

pub use segments::SegmentFetcher;

pub use segments::SegmentStatus;

pub use servers::FailoverStrategy;

pub use servers::GroupStats;

pub use servers::ServerGroup;

pub use servers::ServerStats;

pub use validation::ValidationConfig;

pub use validation::parse_date;

pub use validation::validate_date;

pub use validation::validate_message_id;

pub use validation::validate_newsgroup_name;

pub use yenc::YencDecoded;

pub use yenc::YencEnd;

pub use yenc::YencHeader;

pub use yenc::YencMultipartAssembler;

pub use yenc::YencPart;

pub use yenc::decode as yenc_decode;

pub use yenc::decode as yenc_decode;

pub use yenc::encode as yenc_encode;

pub use yenc::encode as yenc_encode;

Modules

article

RFC 5536 Article Format RFC 5536 Article Format

assembler

Article assembler for binary downloads Article assembler for binary downloads

cache

Header caching for NNTP client Header caching for NNTP client

codes

NNTP response codes from RFC 3977, RFC 4643, RFC 4644, RFC 6048, and RFC 8054

commands

NNTP command builders and response parsers NNTP command builders and response parsers

encoded_words

RFC 2047 Encoded Words support for international headers RFC 2047 Encoded Words Support

nzb

NZB file format parser NZB file format parser and generator

par2

PAR2 file format parser for error correction PAR2 (Parity Archive 2) file format parsing and verification

ratelimit

Rate limiting for bandwidth and connection management Rate limiting for bandwidth and connection management

sasl

SASL authentication framework (RFC 4643) SASL (Simple Authentication and Security Layer) support for NNTP

segments

Segment fetcher for Usenet binary downloads Segment fetcher for Usenet binary downloads

servers

Multi-server support with automatic failover Multi-server support for NNTP clients

validation

RFC 5536 Article validation utilities RFC 5536 Article Validation

yenc

yEnc binary encoding/decoding for Usenet yEnc binary encoding/decoding for Usenet

Structs

Capabilities

Represents the capabilities supported by an NNTP server

NntpBinaryResponse

NNTP binary response optimized for article fetching

NntpClient

Async NNTP client with TLS and compression support

NntpPool

NNTP connection pool with retry support

NntpResponse

NNTP response with status code, message, and optional multi-line body

RetryConfig

Configuration for connection retry behavior

ServerConfig

NNTP server configuration

Enums

NntpError

NNTP protocol and connection errors

Type Aliases

Result

Result type alias using NntpError