Crate sim_codec_robots 

RFC 9309 robots.txt records and deterministic path matching, without HTTP policy.

Structs

Group

A group of user-agent products and rules.

RedirectMetadata

Redirect observations supplied by a transport owner, retained only as metadata.

RobotsDoc

Parsed robots data.

RobotsError

Parse failure.

RobotsLimits

Parse limits. The default input ceiling honors the RFC floor.

Rule

One path rule.

Enums

RuleKind

Rule disposition.

Constants

CODEC_SYMBOL

Stable runtime codec symbol.

MEDIA_TYPES

Stable accepted media-type alias.

RFC_MINIMUM_BYTES

RFC 9309 requires parsers to accept at least this many octets.

Statics

RECIPES

Embedded pure recipes.

Functions

parse_robots

Parse robots bytes as inert records. HTTP status and redirect policy remain outside this crate.