Crate file_rotate

Write output to a file and rotate the files when limits have been exceeded.

Defines a simple std::io::Write object that you can plug into your writers as middleware.

Content limit

ContentLimit specifies at what point a log file has to be rotated.

Rotating by Lines

We can rotate log files with the amount of lines as a limit, by using ContentLimit::Lines.

use file_rotate::{FileRotate, ContentLimit, suffix::AppendCount, compression::Compression};
use std::{fs, io::Write};

// Create a new log writer. The first argument is anything resembling a path. The
// basename is used for naming the log files.
//
// Here we choose to limit logs by 10 lines, and have at most 2 rotated log files. This
// makes the total amount of log files 3, since the original file is present as well.

let log_path = directory.join("my-log-file");

let mut log = FileRotate::new(
    log_path.clone(),
    AppendCount::new(2),
    ContentLimit::Lines(3),
    Compression::None,
    None,
);

// Write a bunch of lines
writeln!(log, "Line 1: Hello World!");
for idx in 2..11 {
    writeln!(log, "Line {}", idx);
}

assert_eq!("Line 10\n", fs::read_to_string(&log_path).unwrap());

assert_eq!("Line 4\nLine 5\nLine 6\n", fs::read_to_string(&directory.join("my-log-file.2")).unwrap());
assert_eq!("Line 7\nLine 8\nLine 9\n", fs::read_to_string(&directory.join("my-log-file.1")).unwrap());

Rotating by Bytes

Another method of rotation is by bytes instead of lines, with ContentLimit::Bytes.

use file_rotate::{FileRotate, ContentLimit, suffix::AppendCount, compression::Compression};
use std::{fs, io::Write};

let log_path = directory.join("my-log-file");

let mut log = FileRotate::new(
    "target/my-log-directory-bytes/my-log-file",
    AppendCount::new(2),
    ContentLimit::Bytes(5),
    Compression::None,
    None,
);

writeln!(log, "Test file");

assert_eq!("Test ", fs::read_to_string(&log.log_paths()[0]).unwrap());
assert_eq!("file\n", fs::read_to_string("target/my-log-directory-bytes/my-log-file").unwrap());

fs::remove_dir_all("target/my-log-directory-bytes");

Rotation Method

Two rotation methods are provided, but any behaviour can be implemented with the SuffixScheme trait.

Basic count

With AppendCount, when the limit is reached in the main log file, the file is moved with suffix .1, and subsequently numbered files are moved in a cascade.

Here’s an example with 1 byte limits:

use file_rotate::{FileRotate, ContentLimit, suffix::AppendCount, compression::Compression};
use std::{fs, io::Write};

let log_path = directory.join("my-log-file");

let mut log = FileRotate::new(
    log_path.clone(),
    AppendCount::new(3),
    ContentLimit::Bytes(1),
    Compression::None,
    None,
);

write!(log, "A");
assert_eq!("A", fs::read_to_string(&log_path).unwrap());

write!(log, "B");
assert_eq!("A", fs::read_to_string(directory.join("my-log-file.1")).unwrap());
assert_eq!("B", fs::read_to_string(&log_path).unwrap());

write!(log, "C");
assert_eq!("A", fs::read_to_string(directory.join("my-log-file.2")).unwrap());
assert_eq!("B", fs::read_to_string(directory.join("my-log-file.1")).unwrap());
assert_eq!("C", fs::read_to_string(&log_path).unwrap());

write!(log, "D");
assert_eq!("A", fs::read_to_string(directory.join("my-log-file.3")).unwrap());
assert_eq!("B", fs::read_to_string(directory.join("my-log-file.2")).unwrap());
assert_eq!("C", fs::read_to_string(directory.join("my-log-file.1")).unwrap());
assert_eq!("D", fs::read_to_string(&log_path).unwrap());

write!(log, "E");
assert_eq!("B", fs::read_to_string(directory.join("my-log-file.3")).unwrap());
assert_eq!("C", fs::read_to_string(directory.join("my-log-file.2")).unwrap());
assert_eq!("D", fs::read_to_string(directory.join("my-log-file.1")).unwrap());
assert_eq!("E", fs::read_to_string(&log_path).unwrap());

Timestamp suffix

With AppendTimestamp, when the limit is reached in the main log file, the file is moved with suffix equal to the current timestamp (with the specified or a default format). If the destination file name already exists, .1 (and up) is appended.

Note that this works somewhat different to AppendCount because of lexical ordering concerns: Higher numbers mean more recent logs, whereas AppendCount works in the opposite way. The reason for this is to keep the lexical ordering of log names consistent: Higher lexical value means more recent. This is of course all assuming that the format start with the year (or most significant component).

With this suffix scheme, you can also decide whether to delete old files based on the age of their timestamp (FileLimit::Age), or just maximum number of files (FileLimit::MaxFiles).

use file_rotate::{FileRotate, ContentLimit, suffix::{AppendTimestamp, FileLimit},
compression::Compression};
use std::{fs, io::Write};

let log_path = directory.join("my-log-file");

let mut log = FileRotate::new(
    log_path.clone(),
    AppendTimestamp::default(FileLimit::MaxFiles(2)),
    ContentLimit::Bytes(1),
    Compression::None,
    None,
);

write!(log, "A");
assert_eq!("A", fs::read_to_string(&log_path).unwrap());

write!(log, "B");
assert_eq!("A", fs::read_to_string(&log.log_paths()[0]).unwrap());
assert_eq!("B", fs::read_to_string(&log_path).unwrap());

write!(log, "C");
assert_eq!("A", fs::read_to_string(&log.log_paths()[0]).unwrap());
assert_eq!("B", fs::read_to_string(&log.log_paths()[1]).unwrap());
assert_eq!("C", fs::read_to_string(&log_path).unwrap());

write!(log, "D");
assert_eq!("B", fs::read_to_string(&log.log_paths()[0]).unwrap());
assert_eq!("C", fs::read_to_string(&log.log_paths()[1]).unwrap());
assert_eq!("D", fs::read_to_string(&log_path).unwrap());

If you use timestamps as suffix, you can also configure files to be removed as they reach a certain age. For example:

use file_rotate::suffix::{AppendTimestamp, FileLimit};
AppendTimestamp::default(FileLimit::Age(chrono::Duration::weeks(1)));

Compression

Select a Compression mode to make the file rotater compress old files using flate2. Compressed files get an additional suffix .gz after the main suffix.

Compression example

If we run this:

use file_rotate::{compression::*, suffix::*, *};
use std::io::Write;

let mut log = FileRotate::new(
    "./log",
    AppendTimestamp::default(FileLimit::MaxFiles(4)),
    ContentLimit::Bytes(1),
    Compression::OnRotate(2),
    None,
);

for i in 0..6 {
    write!(log, "{}", i).unwrap();
    std::thread::sleep(std::time::Duration::from_secs(1));
}

The following files will be created:

log  log.20220112T112415.gz  log.20220112T112416.gz  log.20220112T112417  log.20220112T112418

And we can assemble all the available log data with:

$ gunzip -c log.20220112T112415.gz  ; gunzip -c log.20220112T112416.gz ; cat log.20220112T112417 log.20220112T112418 log
12345

Get structured list of log files

We can programmatically get the list of log files. The following code scans the current directory and recognizes log files based on their file name:

println!(
    "{:#?}",
    AppendTimestamp::default(FileLimit::MaxFiles(4)).scan_suffixes(Path::new("./log"))
);

SuffixScheme::scan_suffixes also takes into account the possibility of the extra .gz suffix, and interprets it correctly as compression. The output:

{
    SuffixInfo {
        suffix: TimestampSuffix {
            timestamp: "20220112T112418",
            number: None,
        },
        compressed: false,
    },
    SuffixInfo {
        suffix: TimestampSuffix {
            timestamp: "20220112T112417",
            number: None,
        },
        compressed: false,
    },
    SuffixInfo {
        suffix: TimestampSuffix {
            timestamp: "20220112T112416",
            number: None,
        },
        compressed: true,
    },
    SuffixInfo {
        suffix: TimestampSuffix {
            timestamp: "20220112T112415",
            number: None,
        },
        compressed: true,
    },
}

This information can be used by for example a program to assemble log history.

Filesystem Errors

If the directory containing the logs is deleted or somehow made inaccessible then the rotator will simply continue operating without fault. When a rotation occurs, it attempts to open a file in the directory. If it can, it will just continue logging. If it can’t then the written data is sent to the void.

Modules

compression

Compression - configuration and implementation

suffix

Suffix schemes determine the suffix of rotated files

Structs

FileRotate

The main writer used for rotating logs.

SuffixInfo

Used mostly internally. Info about suffix + compressed state.

Enums

ContentLimit

When to move files: Condition on which a file is rotated.

TimeFrequency

At which frequency to rotate the file.