format_bytes/

lib.rs

/*!
This crate exposes procedural macros that allow you to format bytestrings.
For more background on why you would want to do that,
[read this article](https://octobus.net/blog/2020-06-05-not-everything-is-utf8.html).

## Usage

Add this to your `Cargo.toml`:

```toml
[dependencies]
format-bytes = "0.4"
```

Requires Rust 1.85 or later. See the [changelog](https://foss.heptapod.net/octobus/rust/format-bytes/-/blob/branch/default/CHANGELOG.md).

Then use the macro like so:

```rust
use format_bytes::format_bytes;

fn main() {
    assert_eq!(
        format_bytes!(b"look at those {} bytes", &[0u8, 1, 2]),
        b"look at those \x00\x01\x02 bytes"
    );

    let path = b"caf\xe9";
    assert_eq!(
        format_bytes!("{{{path}}} is {len} bytes long", len = path.len()),
        b"{caf\xe9} is 4 bytes long"
    );
}
```

See more examples of how it works on the documentation of
[`format_bytes!` itself][`format_bytes!`].

Arguments can be of any type that implements [`DisplayBytes`]: bytestrings,
integers and floats, [`Utf8`] for [`std::fmt::Display`] types, [`join()`] for
sequences, and references and smart pointers to those.

### Writing to a stream: `write_bytes!`

[`write_bytes!`] takes a `&mut` to the stream to write to, followed by the same
arguments as [`format_bytes!`]:

```rust
use format_bytes::write_bytes;
use std::io;

fn write_entry(out: &mut dyn io::Write, path: &[u8]) -> io::Result<()> {
    write_bytes!(out, b"{}: {} bytes\n", path, path.len())
}

fn main() {
    let mut out = vec![];
    write_entry(&mut out, b"caf\xe9").unwrap();
    assert_eq!(out, b"caf\xe9: 4 bytes\n");
}
```

The stream can be any [`std::io::Write`], or any [`Sink`].

### Formatting later: `message!`

[`message!`] takes the same arguments as [`format_bytes!`], but returns a
[`Message`] that keeps the format string and the arguments, to be written
later, for example by an error handler. Arguments are moved into the message,
so they must be `'static`, [`Send`] and [`Sync`].

```rust
use format_bytes::{Message, format_bytes, message};

fn unknown_revision(rev: u32) -> Message {
    message!("unknown revision {rev}")
}

fn main() {
    let msg = unknown_revision(42);
    assert_eq!(format_bytes!("abort: {msg}"), b"abort: unknown revision 42");
    assert_eq!(msg.template().source, "unknown revision {rev}");
}
```

A [`Message`] is itself a [`DisplayBytes`] argument. [`Message::write_to`]
writes it to a [`Sink`], and [`Message::template`] gives the format string,
parsed at compile time.

### Custom output: `Sink`

A [`Sink`] receives the format string separately from each argument and its
name ([`ArgName`]). Every [`std::io::Write`] is a [`Sink`] that writes both
as-is. Other sinks can, for example, tell user data apart from the message
around it:

```rust
use format_bytes::{ArgName, DisplayBytes, Sink, write_bytes};
use std::io;

/// Quotes arguments
struct Quoting(Vec<u8>);

impl Sink for Quoting {
    fn literal(&mut self, text: &str) -> io::Result<()> {
        self.0.literal(text)
    }

    fn arg(&mut self, _: ArgName, value: &dyn DisplayBytes) -> io::Result<()> {
        write_bytes!(&mut self.0, "'{}'", value)
    }
}

fn main() {
    let mut sink = Quoting(vec![]);
    write_bytes!(&mut sink, "cannot open {path}", path = b"caf\xe9").unwrap();
    assert_eq!(sink.0, b"cannot open 'caf\xe9'");
}
```

A sink that cares about types for structured output, can call
[`DisplayBytes::visit`] with a [`Visit`], which implements methods for
integers, UTF-8 text, bytes, a [`join()`] sequence or a nested [`Message`].

## Missing features

* [`format!`]-like explicit indices (`{0}`) and format specs (`{:?}`, padding):
if the need manifests itself, they might appear
* ``#[no_std]`` support


## Why not 1.0?

Not until the macro gets a bit of mileage (it
will be used in [Mercurial](https://www.mercurial-scm.org)).
*/

use std::fmt;
use std::io;

/// Creates a `Vec<u8>` using interpolation of runtime expressions.
///
/// The first argument `format_bytes!` receives is a format string.
/// This must be a string literal, or a bytestring literal holding valid UTF-8.
/// The power of the formatting string
/// is in the `{}`s contained.
///
/// Literal braces are written `{{` and `}}`.
///
/// Additional arguments passed to `format_bytes!` replace the `{}`s
/// within the formatting string in the order given. Named arguments, given
/// after positional ones as `name = value`, replace each `{name}`. A `{name}`
/// with no such argument uses the variable `name` from the caller's scope.
/// Each argument is evaluated once.
///
/// These additional arguments may have any type that implements
/// the [`DisplayBytes`] trait.
///
/// # Examples
///
/// ```
/// use format_bytes::format_bytes;
///
/// assert_eq!(format_bytes!(b""), b"");
/// assert_eq!(format_bytes!(b"here"), b"here");
/// assert_eq!(format_bytes!("é{}", b"\xe9"), b"\xc3\xa9\xe9");
/// assert_eq!(format_bytes!(b"this {{ escapes }}"), b"this { escapes }");
/// assert_eq!(format_bytes!(b"also this {{}}"), b"also this {}");
/// assert_eq!(format_bytes!(b"this works {{{}}}", b"a"), b"this works {a}");
/// assert_eq!(
///     format_bytes!(b"look at those {} bytes", &[0u8, 1, 2]),
///     b"look at those \x00\x01\x02 bytes"
/// );
///
/// let bytes = vec![0u8, 1, 2];
///
/// assert_eq!(
///     format_bytes!(b"look at those {} bytes", bytes),
///     b"look at those \x00\x01\x02 bytes"
/// );
/// assert_eq!(
///     format_bytes!(b"{}.{}.{}.{}", 1_i32, 2_u8, 3_f32, &4),
///     b"1.2.3.4"
/// );
/// assert_eq!(
///     format_bytes!(b"what about this very very long message {}?", "here".as_bytes()),
///     b"what about this very very long message here?".to_vec()
/// );
/// assert_eq!(format_bytes!(b"{}", std::borrow::Cow::Borrowed("cow".as_bytes())), b"cow");
/// ```
#[macro_export]
macro_rules! format_bytes {
    ($($args: tt)*) => {{
        let mut vec = Vec::<u8>::new();
        $crate::write_bytes!(&mut vec, $($args)*)
            // Never panics since `impl std::fmt::Write for Vec<u8>` never errors:
            .unwrap();
        vec
    }}
}

#[doc(hidden)]
pub use format_bytes_macros::{_message, _write_bytes};

/// Like [`format_bytes!`], but writes to a stream given as an additional first argument.
///
/// The stream is a `&mut` reference to a [`Sink`], such as any [`std::io::Write`].
/// The macro returns [`std::io::Result<()>`](std::io::Result).
///
/// # Examples
///
/// ```
/// use format_bytes::write_bytes;
///
/// const BUFFER_LEN: usize = 20;
/// let mut buffer = [0_u8; BUFFER_LEN];
/// let mut slice = &mut buffer[..];
///
/// write_bytes!(&mut slice, b"{}", 3.14).unwrap();
///
/// // `impl std::io::Write for &mut [u8]` reassigns the slice to the unwritten remainder:
/// let written = BUFFER_LEN - slice.len();
/// assert_eq!(&buffer[..written], b"3.14");
/// ```
#[macro_export]
macro_rules! write_bytes {
    ($($args: tt)*) => {
        $crate::_write_bytes!([$crate] $($args)*)
    };
}

/// Let types decide how to format themselves for presentation to users in a byte-stream output.
///
/// Similar to `std::fmt::Display`, but the output stream is bytes instead of Unicode.
///
/// When output is presented to users, it is decoded with an unspecified character encoding
/// that is presumed to be ASCII-compatible.
///
/// Implementers should return any error from `output` (e.g. with the `?` operator),
/// and not emit other errors.
///
/// # Example
///
/// A typical `impl` for a struct with multiple fields might use the `write_bytes` macro:
///
/// ```
/// use format_bytes::{DisplayBytes, write_bytes};
///
/// struct Point2D { x: f32, y: f32 }
///
/// impl DisplayBytes for Point2D {
///     fn display_bytes(
///         &self,
///         out: &mut dyn std::io::Write,
///     ) -> std::io::Result<()> {
///         write_bytes!(out, b"x = {}, y = {}", self.x, self.y)
///     }
/// }
/// ```
pub trait DisplayBytes {
    fn display_bytes(&self, output: &mut dyn io::Write) -> io::Result<()>;

    /// Describes the value with its type, for sinks that care about types,
    /// such as structured output.
    ///
    /// Defaults to the output of `display_bytes`, as bytes.
    fn visit(&self, visitor: &mut dyn Visit) -> io::Result<()> {
        let mut bytes = vec![];
        self.display_bytes(&mut bytes)?;
        visitor.bytes(&bytes)
    }
}

/// Receives a value described by [`DisplayBytes::visit`].
///
/// Only `bytes` and `str` are required: other methods default to them, the
/// way `display_bytes` would write the value.
pub trait Visit {
    /// Bytes in an unknown encoding
    fn bytes(&mut self, value: &[u8]) -> io::Result<()>;

    /// Text known to be valid UTF-8
    fn str(&mut self, value: &str) -> io::Result<()>;

    fn int(&mut self, value: i128) -> io::Result<()> {
        self.str(&value.to_string())
    }

    fn uint(&mut self, value: u128) -> io::Result<()> {
        self.str(&value.to_string())
    }

    /// Items of [`join`]. Defaults to them joined, as bytes.
    fn seq(
        &mut self,
        items: &[&dyn DisplayBytes],
        separator: &dyn DisplayBytes,
    ) -> io::Result<()> {
        let mut bytes = vec![];
        join(items, separator).display_bytes(&mut bytes)?;
        self.bytes(&bytes)
    }

    /// A message within another, e.g. a cause within its context. Defaults to
    /// it written, as bytes: a [`Sink`] would rather write it through itself,
    /// with [`Message::write_to`].
    fn message(&mut self, message: &Message) -> io::Result<()> {
        let mut bytes = vec![];
        message.display_bytes(&mut bytes)?;
        self.bytes(&bytes)
    }
}

/// How a format string refers to an argument.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ArgName {
    /// The n-th positional argument
    Index(usize),
    /// A named argument, or a variable captured from scope
    Name(&'static str),
}

/// Destination of formatting, which keeps the format string's text apart from
/// its arguments.
///
/// Every [`std::io::Write`] is a `Sink` that writes both as-is.
pub trait Sink {
    /// Text from the format string: written by the programmer, valid UTF-8
    fn literal(&mut self, text: &str) -> io::Result<()>;

    /// A value interpolated into the format string
    fn arg(
        &mut self,
        name: ArgName,
        value: &dyn DisplayBytes,
    ) -> io::Result<()>;
}

impl<W: io::Write + ?Sized> Sink for W {
    fn literal(&mut self, text: &str) -> io::Result<()> {
        self.write_all(text.as_bytes())
    }

    fn arg(&mut self, _: ArgName, value: &dyn DisplayBytes) -> io::Result<()> {
        // `&mut Self` is `Sized`, unlike `Self`
        value.display_bytes(&mut &mut *self)
    }
}

/// Creates a [`Message`]: like [`format_bytes!`], but the arguments are kept,
/// for a [`Sink`] to write later.
///
/// Arguments are moved into the message, so they must be `'static`, `Send`
/// and `Sync`.
///
/// # Example
///
/// ```
/// use format_bytes::message;
///
/// let rev = 42;
/// let msg = message!("unknown revision {rev} in {path}", path = b"caf\xe9".to_vec());
///
/// let mut out = vec![];
/// msg.write_to(&mut out).unwrap();
/// assert_eq!(out, b"unknown revision 42 in caf\xe9");
/// assert_eq!(msg.template().source, "unknown revision {rev} in {path}");
/// ```
#[macro_export]
macro_rules! message {
    ($($args: tt)*) => {
        $crate::_message!([$crate] $($args)*)
    };
}

/// A format string, parsed at compile time.
#[derive(Debug)]
pub struct Template {
    /// The format string as written
    pub source: &'static str,
    pub pieces: &'static [Piece],
    /// Names of the arguments, in order
    pub args: &'static [ArgName],
}

/// A part of a [`Template`].
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Piece {
    /// Text, with `{{` and `}}` unescaped
    Literal(&'static str),
    /// Index of an argument
    Arg(usize),
}

/// A format string with its arguments, made by [`message!`].
pub struct Message {
    template: &'static Template,
    args: Vec<Box<dyn DisplayBytes + Send + Sync>>,
}

impl Message {
    #[doc(hidden)]
    pub fn new(
        template: &'static Template,
        args: Vec<Box<dyn DisplayBytes + Send + Sync>>,
    ) -> Self {
        Self { template, args }
    }

    pub fn template(&self) -> &'static Template {
        self.template
    }

    pub fn arg(&self, index: usize) -> &dyn DisplayBytes {
        &*self.args[index]
    }

    /// Writes the message as [`write_bytes!`] would.
    pub fn write_to(&self, sink: &mut dyn Sink) -> io::Result<()> {
        for piece in self.template.pieces {
            match *piece {
                Piece::Literal(text) => sink.literal(text)?,
                Piece::Arg(index) => {
                    sink.arg(self.template.args[index], self.arg(index))?
                }
            }
        }
        Ok(())
    }
}

impl DisplayBytes for Message {
    fn display_bytes(&self, mut output: &mut dyn io::Write) -> io::Result<()> {
        self.write_to(&mut output)
    }

    fn visit(&self, visitor: &mut dyn Visit) -> io::Result<()> {
        visitor.message(self)
    }
}

impl fmt::Debug for Message {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        let mut bytes = vec![];
        self.write_to(&mut bytes).map_err(|_| fmt::Error)?;
        f.debug_tuple("Message")
            .field(&String::from_utf8_lossy(&bytes))
            .finish()
    }
}

macro_rules! impl_through_deref {
    // Macro hygiene requires the `$Inner` ident to be an input
    // so it matches corresponding idents in `$Wrapper` types:
    ($Inner: ident => $( $Wrapper: ty, )*) => {
        $(
            /// Forward to the inner type.
            impl<$Inner: ?Sized + DisplayBytes> DisplayBytes for $Wrapper {
                #[inline]
                fn display_bytes(&self, output: &mut dyn io::Write) -> io::Result<()> {
                    (**self).display_bytes(output)
                }

                #[inline]
                fn visit(&self, visitor: &mut dyn Visit) -> io::Result<()> {
                    (**self).visit(visitor)
                }
            }
        )*
    };
}

impl_through_deref! {
    Inner =>
    &'_ Inner,
    &'_ mut Inner,
    Box<Inner>,
    std::rc::Rc<Inner>,
    std::sync::Arc<Inner>,
}

/// Forward to the inner type.
impl<Inner> DisplayBytes for std::borrow::Cow<'_, Inner>
where
    Inner: ?Sized + ToOwned + DisplayBytes,
{
    #[inline]
    fn display_bytes(&self, output: &mut dyn io::Write) -> io::Result<()> {
        (**self).display_bytes(output)
    }

    #[inline]
    fn visit(&self, visitor: &mut dyn Visit) -> io::Result<()> {
        (**self).visit(visitor)
    }
}

macro_rules! impl_for_byte_string {
    ($($Ty: ty),+) => {
        $(
            /// Byte strings are "formatted" as-is.
            impl DisplayBytes for $Ty {
                #[inline]
                fn display_bytes(&self, output: &mut dyn io::Write) -> io::Result<()> {
                    output.write_all(self)
                }
            }
        )+
    };
}

impl_for_byte_string!([u8], Vec<u8>);

macro_rules! impl_for_arrays {
    ($( $LEN: expr )+) => {
        impl_for_byte_string! {
            $( [u8; $LEN] ),+
        }
    };
}

impl_for_arrays! {
    0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
    33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
    49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64
}

/// Adaptor for types that implement `std::fmt::Display`. The Unicode output is encoded as UTF-8.
///
/// # Example
///
/// ```rust
/// use format_bytes::{format_bytes, Utf8};
///
/// assert_eq!(format_bytes!(b"{}", Utf8("è_é")), b"\xc3\xa8_\xc3\xa9");
/// ```
pub struct Utf8<Inner>(pub Inner);

impl<Inner: fmt::Display> DisplayBytes for Utf8<Inner> {
    fn display_bytes(&self, output: &mut dyn io::Write) -> io::Result<()> {
        // Calling `Display::fmt` requires a `Formatter` which we can’t create directly,
        // so let’s go through `write!` with an adaptor type.
        struct Adapter<'a> {
            output: &'a mut dyn io::Write,
            result: io::Result<()>,
        }

        impl fmt::Write for Adapter<'_> {
            fn write_str(&mut self, s: &str) -> fmt::Result {
                if self.result.is_err() {
                    return Err(fmt::Error);
                }
                let utf8 = s.as_bytes();
                match self.output.write_all(utf8) {
                    Ok(()) => Ok(()),
                    Err(error) => {
                        // `fmt::Error` cannot carry any data so stash the error
                        self.result = Err(error);
                        Err(fmt::Error)
                    }
                }
            }
        }

        let mut adapter = Adapter {
            output,
            result: Ok(()),
        };
        {
            // `write!` requires this import: https://github.com/rust-lang/rust/issues/21826
            use fmt::Write;
            write!(adapter, "{}", self.0)
                // Recover stashed error
                .map_err(|fmt::Error| adapter.result.unwrap_err())
        }
    }

    fn visit(&self, visitor: &mut dyn Visit) -> io::Result<()> {
        visitor.str(&self.0.to_string())
    }
}

macro_rules! impl_ascii_only {
    ($( $Ty: ident )* => |$value: ident, $visitor: ident| $visit: expr) => {
        $(
            /// Format to ASCII bytes with `std::fmt::Display`.
            ///
            /// The `Display` impl for this type only emits ASCII characters,
            /// so it’s less useful than in the general case
            /// to make users explicitly opt-in to UTF-8 encoding.
            impl DisplayBytes for $Ty {
                #[inline]
                fn display_bytes(&self, output: &mut dyn io::Write) -> io::Result<()> {
                    Utf8(self).display_bytes(output)
                }

                #[inline]
                fn visit(&self, $visitor: &mut dyn Visit) -> io::Result<()> {
                    let $value = *self;
                    $visit
                }
            }
        )*
    };
}

impl_ascii_only! {
    u8 u16 u32 u64 u128 usize => |value, visitor| visitor.uint(value as u128)
}

impl_ascii_only! {
    i8 i16 i32 i64 i128 isize => |value, visitor| visitor.int(value as i128)
}

impl_ascii_only! {
    f32 f64 => |value, visitor| visitor.str(&value.to_string())
}

/// Format a sequence of values with the given separator repeated
/// between any two consecutive values, but not at the start or end of the sequence.
///
/// It is visited with [`Visit::seq`].
///
/// # Example
///
/// ```
/// use format_bytes::{format_bytes, join};
///
/// let formatted = format_bytes!(b"Got {}.", join(&[4, 3, 2], b" and "));
/// assert_eq!(formatted, b"Got 4 and 3 and 2.");
/// ```
pub fn join(
    iterable: impl IntoIterator<Item = impl DisplayBytes>,
    separator: impl DisplayBytes,
) -> impl DisplayBytes {
    Join {
        items: iterable.into_iter().collect::<Vec<_>>(),
        separator,
    }
}

struct Join<T, S> {
    items: Vec<T>,
    separator: S,
}

impl<T: DisplayBytes, S: DisplayBytes> DisplayBytes for Join<T, S> {
    fn display_bytes(&self, output: &mut dyn io::Write) -> io::Result<()> {
        for (i, item) in self.items.iter().enumerate() {
            if i > 0 {
                self.separator.display_bytes(output)?;
            }
            item.display_bytes(output)?;
        }
        Ok(())
    }

    fn visit(&self, visitor: &mut dyn Visit) -> io::Result<()> {
        let items: Vec<&dyn DisplayBytes> =
            self.items.iter().map(|item| item as _).collect();
        visitor.seq(&items, &self.separator)
    }
}