Module io

Module io 

Source
Expand description

Packet I/O abstractions.

This module provides traits and implementations for reading packets from various sources (files, memory-mapped files, network streams, etc.)

§Design

The module uses generics with associated types for zero-vtable hot path:

  • PacketSource trait with associated Reader type
  • PacketReader trait for sequential reading
  • Type erasure happens only at DataFusion boundaries

§Available Sources

  • FilePacketSource - Standard buffered file I/O (works with all file types)
  • MmapPacketSource - Memory-mapped I/O for PCAP/PCAPNG files (requires mmap feature)
  • [CloudPacketSource] - Cloud storage I/O for S3, GCS, Azure (requires cloud feature)

§Compression Support

Both sources support transparent decompression of compressed files. Supported formats (via feature flags):

  • Gzip (.gz) - always enabled
  • Zstd (.zst) - compress-zstd feature
  • LZ4 (.lz4) - compress-lz4 feature
  • Bzip2 (.bz2) - compress-bzip2 feature
  • XZ (.xz) - compress-xz feature

Structs§

FilePacketReader
Sequential packet reader for PCAP files.
FilePacketSource
Packet source backed by a PCAP/PCAPNG file.
GenericPcapReader
Generic PCAP/PCAPNG reader over any Read source.
MmapPacketReader
Memory-mapped packet reader.
MmapPacketSource
Memory-mapped packet source.
MmapSlice
A wrapper that allows sharing mmap data with Cursor. Implements AsRef<u8> so it can be used as Cursor’s inner type.
PacketPosition
Position within a packet source (for seeking/checkpointing).
PacketRange
Range of packets for partitioning.
PacketRef
Borrowed packet reference - zero-copy view into pcap_parser buffer.
PacketSourceMetadata
Metadata about a packet source.
RawPacket
Raw packet data from a reader.

Enums§

Compression
Detected compression format.
DecompressReader
Unified decompression reader that wraps various decompression formats.
PcapFormat
Format of the PCAP file.

Traits§

PacketReader
Sequential reader of packets from a source.
PacketSource
Source of packet data. Creates readers and computes partitions.

Functions§

decompress_header
Decompress a header buffer in memory.

Type Aliases§

AnyDecoder
Type alias for mmap-based decompression.
FileDecoder
Type alias for file-based decompression.