sbircio 0.1.0

A lightweight, Unicode-aware BufRead wrapper with peek-ahead for building parsers and lexers
Documentation
  • Coverage
  • 88%
    22 out of 25 items documented10 out of 19 items with examples
  • Size
  • Source code size: 42.2 kB This is the summed size of all the files inside the crates.io package for this release.
  • Documentation size: 296.0 kB This is the summed size of all files generated by rustdoc for all configured targets
  • Ø build duration
  • this release: 8s Average build duration of successful builds.
  • all releases: 8s Average build duration of successful builds in releases after 2024-10-23.
  • Links
  • Repository
  • crates.io
  • Dependencies
  • Versions
  • Owners
  • marc0x71

sbircio

A lightweight BufRead wrapper that adds Unicode-aware peek-ahead and character-level scanning, designed for building parsers and lexers in Rust.

No complex combinators, no macros — just a thin layer over any BufRead that lets you look ahead without consuming, match literals, and scan with predicates.

Features

  • Peek ahead — inspect the next N Unicode characters without consuming them
  • Exact matching — consume and verify string literals or single characters
  • Predicate scanning — consume_while / consume_until with any Fn(char) -> bool
  • Full UTF-8 support — handles multi-byte characters and partial reads correctly
  • Zero-copy cache — peeked characters are kept in an internal buffer and reused across calls; no redundant I/O
  • Works with anything — files, stdin, in-memory buffers: any BufRead source
  • Minimal footprint — a single struct, a single error type, one optional dependency

Installation

[dependencies]
sbircio = "0.1.0"

To track the latest unreleased changes instead:

[dependencies]
sbircio = { git = "https://codeberg.org/marc0x71/sbircio" }

To pin to a specific commit or tag:

sbircio = { git = "https://codeberg.org/marc0x71/sbircio", tag = "v0.1.0" }

Quick start

use sbircio::Input;
use std::io::Cursor;

fn main() -> Result<(), sbircio::InputError> {
    let source = Cursor::new("42 + hello");
    let mut input = Input::new(source);

    // Read a number token
    let num = input.consume_while(|c| c.is_ascii_digit())?;
    input.skip_whitespace()?;

    // Verify an operator
    input.expect("+")?;
    input.skip_whitespace()?;

    // Read an identifier
    let word = input.consume_while(|c| c.is_alphabetic())?;

    println!("{num} + {word}"); // 42 + hello
    Ok(())
}

API reference

Input<R>

pub struct Input<R> { /* private fields */ }

Wraps any BufRead source and provides Unicode-aware peek-ahead and character-level scanning. Peeked characters are cached internally and reused across calls without triggering additional reads.


Constructor

pub fn new(rdr: R) -> Self

Creates a new Input wrapping the given reader. The internal cache starts empty; no reads are performed until the first peek or consume call.


Peeking

pub fn peek(&mut self) -> Result<Option<char>, InputError>

Returns the next Unicode character without consuming it.
Returns Ok(None) at end of input.


pub fn peek_n(&mut self, required: usize) -> Result<Option<&str>, InputError>

Returns the next required Unicode characters as a &str without consuming them.
Returns Ok(None) if fewer than required characters remain in the stream.
The returned slice is valid until the next consume call. Subsequent calls with the same or a smaller required reuse the internal cache without additional I/O.


Consuming

pub fn consume_char(&mut self) -> Result<Option<char>, InputError>

Consumes and returns the next Unicode character.
Returns Ok(None) at end of input. The byte offset returned by position() advances by the UTF-8 byte length of the consumed character.


pub fn consume_while<P>(&mut self, predicate: P) -> Result<String, InputError>
where
    P: Fn(char) -> bool,

Consumes and returns consecutive characters for which predicate returns true.
Stops at the first character for which predicate returns false, leaving it available for subsequent reads. Returns an empty String if the very first character does not match.


pub fn consume_until<P>(&mut self, predicate: P) -> Result<String, InputError>
where
    P: Fn(char) -> bool,

Consumes and returns characters until predicate returns true.
The first character for which predicate returns true is not consumed and remains available for subsequent reads. Returns an empty String if the very first character matches immediately.


pub fn skip_whitespace(&mut self) -> Result<(), InputError>

Consumes and discards all leading whitespace characters (as defined by char::is_whitespace). Equivalent to consume_while(char::is_whitespace).map(|_| ()).


Matching

pub fn expect(&mut self, expected: &str) -> Result<(), InputError>

Verifies that the next characters exactly match expected and consumes them.
On failure (mismatch or end of input), returns InputError::UnsatisfiedExpectation and leaves the input unchanged.


pub fn expect_char(&mut self, expected: char) -> Result<(), InputError>

Verifies that the next character matches expected and consumes it.
On failure, returns InputError::UnsatisfiedExpectation and leaves the input unchanged.


State

pub fn is_eof(&mut self) -> Result<bool, InputError>

Returns true if the end of input has been reached, false otherwise.
Internally performs a peek(), so the first unread character remains available.


pub fn position(&self) -> usize

Returns the current byte offset from the beginning of the stream.
The offset is measured in bytes, not Unicode characters: multi-byte characters such as 'è' (2 bytes) or '€' (3 bytes) advance the position accordingly.


pub fn coordinates(&self) -> Coordinates

Returns the current position in the input as line and column numbers. Both line and column are 1-based: the first character of the stream is at line 1, column 1. The column advances by one for each Unicode character consumed; a newline resets the column to 1 and increments the line counter.


Coordinates

#[derive(Debug, Clone, Copy, PartialEq)]
pub struct Coordinates { /* fields: line, column */ }

Represents a position in the input stream as a (line, column) pair, both 1-based. Returned by Input::coordinates().

pub fn new(line: usize, column: usize) -> Self
pub fn line(&self) -> usize
pub fn column(&self) -> usize

InputError

#[derive(Error, Debug, PartialEq)]
pub enum InputError {
    /// The byte sequence read is not valid UTF-8.
    InvalidString(Utf8Error),

    /// The stream ended in the middle of a multi-byte character.
    UnexpectedEof,

    /// The next characters did not match the expected value.
    UnsatisfiedExpectation {
        expected: String,
        found: String,
    },

    /// An underlying I/O error occurred.
    IO(std::io::Error),
}

InputError implements std::error::Error and converts automatically from std::io::Error and std::str::Utf8Error via From.

Note on PartialEq: two IO variants are considered equal if they have the same ErrorKind, since std::io::Error does not itself implement PartialEq.

Examples

Building a lexer

sbircio is a low-level primitive: it reads characters, the lexer gives them meaning. Input knows nothing about your Token type — that separation is intentional.

use sbircio::{Input, InputError};
use std::io::BufRead;

#[derive(Debug, PartialEq)]
enum Token {
    Number(u64),
    Ident(String),
    Add,
    Sub,
    Mul,
    Div,
}

fn next_token<R: BufRead>(input: &mut Input<R>) -> Result<Option<Token>, InputError> {
    input.skip_whitespace()?;

    match input.peek()? {
        None => Ok(None),

        Some('+') => { input.consume_char()?; Ok(Some(Token::Add)) }
        Some('-') => { input.consume_char()?; Ok(Some(Token::Sub)) }
        Some('*') => { input.consume_char()?; Ok(Some(Token::Mul)) }
        Some('/') => { input.consume_char()?; Ok(Some(Token::Div)) }

        Some(c) if c.is_ascii_digit() => {
            let s = input.consume_while(|c| c.is_ascii_digit())?;
            Ok(Some(Token::Number(s.parse().unwrap())))
        }
        Some(c) if c.is_alphabetic() || c == '_' => {
            let s = input.consume_while(|c| c.is_alphanumeric() || c == '_')?;
            Ok(Some(Token::Ident(s)))
        }

        Some(c) => Err(InputError::UnsatisfiedExpectation {
            expected: "a valid token".to_string(),
            found: c.to_string(),
        }),
    }
}

fn main() -> Result<(), InputError> {
    let mut input = Input::new(std::io::Cursor::new("x1 + 42 * y"));
    while let Some(tok) = next_token(&mut input)? {
        println!("{tok:?}");
    }
    // Ident("x1")  Add  Number(42)  Mul  Ident("y")
    Ok(())
}

License

Released under the MIT License.


This README was drafted with the assistance of an AI (Claude by Anthropic) and reviewed by the author.