Module file_rotation::asynchronous

Write output to a file and rotate the files when limits have been exceeded.

Defines a simple [tokio::io::Write] object that you can plug into your writers as middleware.

Rotating by Lines

We can rotate log files by using the amount of lines as a limit.

use file_rotation::{FileRotate, RotationMode};
use tokio::{fs, io::AsyncWriteExt};

tokio_test::block_on(async {
  // Create a directory to store our logs, this is not strictly needed but shows how we can
  // arbitrary paths.
  fs::create_dir("target/async-my-log-directory-lines").await.unwrap();

  // Create a new log writer. The first argument is anything resembling a path. The
  // basename is used for naming the log files.
  //
  // Here we choose to limit logs by 10 lines, and have at most 2 rotated log files. This
  // makes the total amount of log files 4, since the original file is present as well as
  // file 0.
  let mut log = FileRotate::new("target/async-my-log-directory-lines/my-log-file", RotationMode::Lines(3), 2).await.unwrap();

  // Write a bunch of lines
  log.write("Line 1: Hello World!\n".as_bytes()).await;
  for idx in 2..11 {
    log.write(format!("Line {}\n", idx).as_bytes()).await;
  }

  assert_eq!("Line 10\n", fs::read_to_string("target/async-my-log-directory-lines/my-log-file").await.unwrap());

  assert_eq!("Line 1: Hello World!\nLine 2\nLine 3\n", fs::read_to_string("target/async-my-log-directory-lines/my-log-file.0").await.unwrap());
  assert_eq!("Line 4\nLine 5\nLine 6\n", fs::read_to_string("target/async-my-log-directory-lines/my-log-file.1").await.unwrap());
  //assert_eq!("Line 7\nLine 8\nLine 9\n", fs::read_to_string("target/async-my-log-directory-lines/my-log-file.2").await.unwrap());

  fs::remove_dir_all("target/async-my-log-directory-lines").await;
});

Rotating by Bytes surpassing a threshold, but without splitting a buffer mid-way#

We can rotate log files but never splitting a buffer half-way. This means a single buffer may end up surpassing the number of expected bytes in a file, but that entire buffer will be written. When the file surpasses the number of bytes to rotate, it'll be rotated for the next buffer.

When lines are written in a single buffer as demonstrated below, this ensures the logs contain complete lines which do not split across files.

use file_rotation::{FileRotate, RotationMode};
use tokio::{fs, io::AsyncWriteExt};

tokio_test::block_on(async {

  // Create a directory to store our logs, this is not strictly needed but shows how we can
  // arbitrary paths.
  fs::create_dir("target/async-my-log-directory-lines").await.unwrap();

  // Create a new log writer. The first argument is anything resembling a path. The
  // basename is used for naming the log files.
  //
  // Here we choose to limit logs by 10 lines, and have at most 2 rotated log files. This
  // makes the total amount of log files 4, since the original file is present as well as
  // file 0.
  let mut log = FileRotate::new("target/async-my-log-directory-lines/my-log-file", RotationMode::BytesSurpassed(2), 2).await.unwrap();

  // Write a bunch of lines
  log.write("Line 1: Hello World!\n".as_bytes()).await;
  for idx in 2..11 {
    log.write(format!("Line {}", idx).as_bytes()).await;
  }

  // the latest file is empty - since the previous file surpassed bytes and was rotated out
  assert_eq!("", fs::read_to_string("target/async-my-log-directory-lines/my-log-file").await.unwrap());

  assert_eq!("Line 10", fs::read_to_string("target/async-my-log-directory-lines/my-log-file.0").await.unwrap());
  assert_eq!("Line 8", fs::read_to_string("target/async-my-log-directory-lines/my-log-file.1").await.unwrap());
  assert_eq!("Line 9", fs::read_to_string("target/async-my-log-directory-lines/my-log-file.2").await.unwrap());

  fs::remove_dir_all("target/async-my-log-directory-lines").await;
});

Rotating by Bytes

Another method of rotation is by bytes instead of lines.

use file_rotation::{FileRotate, RotationMode};
use tokio::{fs, io::AsyncWriteExt};

tokio_test::block_on(async {
  fs::create_dir("target/async-my-log-directory-bytes").await;

  let mut log = FileRotate::new("target/async-my-log-directory-bytes/my-log-file", RotationMode::Bytes(5), 2).await.unwrap();

  log.write("Test file\n".as_bytes()).await;

  assert_eq!("Test ", fs::read_to_string("target/async-my-log-directory-bytes/my-log-file.0").await.unwrap());
  assert_eq!("file\n", fs::read_to_string("target/async-my-log-directory-bytes/my-log-file").await.unwrap());

  fs::remove_dir_all("target/async-my-log-directory-bytes").await;
});

Rotation Method

The rotation method used is to always write to the base path, and then move the file to a new location when the limit is exceeded. The moving occurs in the sequence 0, 1, 2, n, 0, 1, 2...

Here's an example with 1 byte limits:

use file_rotation::{FileRotate, RotationMode};
use tokio::{fs, io::AsyncWriteExt};

tokio_test::block_on(async {
  fs::create_dir("target/async-my-log-directory-small").await;

  let mut log = FileRotate::new("target/async-my-log-directory-small/my-log-file", RotationMode::Bytes(1), 3).await.unwrap();

  log.write("A".as_bytes()).await;
  assert_eq!("A", fs::read_to_string("target/async-my-log-directory-small/my-log-file").await.unwrap());

  log.write("B".as_bytes()).await;
  assert_eq!("A", fs::read_to_string("target/async-my-log-directory-small/my-log-file.0").await.unwrap());
  assert_eq!("B", fs::read_to_string("target/async-my-log-directory-small/my-log-file").await.unwrap());

  log.write("C".as_bytes()).await;
  assert_eq!("A", fs::read_to_string("target/async-my-log-directory-small/my-log-file.0").await.unwrap());
  assert_eq!("B", fs::read_to_string("target/async-my-log-directory-small/my-log-file.1").await.unwrap());
  assert_eq!("C", fs::read_to_string("target/async-my-log-directory-small/my-log-file").await.unwrap());

  log.write("D".as_bytes()).await;
  assert_eq!("A", fs::read_to_string("target/async-my-log-directory-small/my-log-file.0").await.unwrap());
  assert_eq!("B", fs::read_to_string("target/async-my-log-directory-small/my-log-file.1").await.unwrap());
  assert_eq!("C", fs::read_to_string("target/async-my-log-directory-small/my-log-file.2").await.unwrap());
  assert_eq!("D", fs::read_to_string("target/async-my-log-directory-small/my-log-file").await.unwrap());

  log.write("E".as_bytes()).await;
  assert_eq!("A", fs::read_to_string("target/async-my-log-directory-small/my-log-file.0").await.unwrap());
  assert_eq!("B", fs::read_to_string("target/async-my-log-directory-small/my-log-file.1").await.unwrap());
  assert_eq!("C", fs::read_to_string("target/async-my-log-directory-small/my-log-file.2").await.unwrap());
  assert_eq!("D", fs::read_to_string("target/async-my-log-directory-small/my-log-file.3").await.unwrap());
  assert_eq!("E", fs::read_to_string("target/async-my-log-directory-small/my-log-file").await.unwrap());


  // Here we overwrite the 0 file since we're out of log files, restarting the sequencing
  log.write("F".as_bytes()).await;
  assert_eq!("E", fs::read_to_string("target/async-my-log-directory-small/my-log-file.0").await.unwrap());
  assert_eq!("B", fs::read_to_string("target/async-my-log-directory-small/my-log-file.1").await.unwrap());
  assert_eq!("C", fs::read_to_string("target/async-my-log-directory-small/my-log-file.2").await.unwrap());
  assert_eq!("D", fs::read_to_string("target/async-my-log-directory-small/my-log-file.3").await.unwrap());
  assert_eq!("F", fs::read_to_string("target/async-my-log-directory-small/my-log-file").await.unwrap());

  fs::remove_dir_all("target/async-my-log-directory-small").await;
});

Filesystem Errors

If the directory containing the logs is deleted or somehow made inaccessible then the rotator will simply continue operating without fault. When a rotation occurs, it attempts to open a file in the directory. If it can, it will just continue logging. If it can't then the written date is sent to the void.

This logger never panics.

Structs

FileRotate

The main writer used for rotating logs.

Enums

RotateState
RotationMode	Condition on which a file is rotated.