Crate turnstiles

Library which defines a struct implementing the io::Write trait which will allows file rotation, if applicable, when a file write is done.

Warning

This is currently in active development and may change/break often. Every effort will be taken to ensure that breaking changes that occur are reflected in a change of at least the minor version of the package, both in terms of the API and the generation of log files. Versions prior to 0.2.0 were so riddled with bugs I'm amazed I managed to put my pants on on those days I was writing it.

How it works

Rotation works by keeping track of the ‘active’ file, the one currently being written to, which upon rotation is renamed to include the next log file index. For example when there is only one log file it will be test.log.ACTIVE, which when rotated will get renamed to test.log.1 and the test.log.ACTIVE will represent a new file being written to. Originally no file renaming was done to keep the surface area with the filesystem as small as possible, however this has a few disadvantages and this active-file-approach (courtesy of flex-logger) was seen as a good compromise. The downside is that the file extension now superifically looks different, but it does mean all logs can be found by simply searching for test.log*.

Log suffix numbers will increase with age, so the first of the rotated logs will be test.log.1, second will be test.log.2 etc until N-1 after which it will be test.log.ACTIVE, the current one.

Warning

Little to no protection is given defend against the file indices being modified during the operation of whatever code is using this logger: when `write` is called it does not currently refresh the internal index which tracks the suffix integer, this is only done when the logger is created. Changing this is noted as future work.

Error handling

Not all internal errors are handled the same way. For example, if during the process of checking if rotation is required an error occurs, the default is to print a warning to stdout and not rotate. In contrast to this, if an error occurs during the actual rotation procedure, this error is bubbled up through error handling eventually returning as a std::io::Error to the caller. However probable future state will outsource all error handling logic to the caller of this library rather than making assumptions.

Examples

Rotate when a log file exceeds a certain filesize

use std::{io::Write, thread::sleep, time::Duration};
use turnstiles::{RotatingFile, RotationCondition, PruneCondition};
use tempdir::TempDir; // Subcrate provided for testing
let dir = TempDir::new();

let path = &vec![dir.path.clone(), "test.log".to_string()].join("/");
let data: Vec<u8> = vec![0; 500_000];
// The `false` here is to do with require_newline and is only needed for async loggers
let mut file = RotatingFile::new(path, RotationCondition::SizeMB(1), PruneCondition::None, false)
                .unwrap();

// Write 500k to file creating test.log
file.write(&data).unwrap();
assert!(file.index() == 0);

// Write another 500kb so test.log is 1mb
file.write_all(&data).unwrap();
assert!(file.index() == 0);

// The check for rotation is done _before_ writing, so we don't rotate, and then write 500kb
// so this file is ~1.5mb now, still the same file
file.write_all(&data).unwrap();
assert!(file.index() == 0);


// Now we check if we need to rotate before writing, and it's 1.5mb > the rotation option so
// we make a new file and  write to that
file.write_all(&data).unwrap();
assert!(file.index() == 1);

// Now have test_ACTIVE.log and test.log.1

Rotate when a log file is too old (based on filesystem metadata timestamps)

use std::{io::Write, thread::sleep, time::Duration};
use turnstiles::{RotatingFile, RotationCondition, PruneCondition};
use tempdir::TempDir; // Subcrate provided for testing
let dir = TempDir::new();
let path = &vec![dir.path.clone(), "test.log".to_string()].join("/");

let max_log_age = Duration::from_millis(100);
let data: Vec<u8> = vec![0; 1_000_000];
let mut file =
    RotatingFile::new(path, RotationCondition::Duration(max_log_age), PruneCondition::None, false)
        .unwrap();

assert!(file.index() == 0);
file.write_all(&data).unwrap();
assert!(file.index() == 0);
file.write_all(&data).unwrap();
assert!(file.index() == 0);
sleep(Duration::from_millis(200));

// Rotation only happens when we call .write() so index remains unchanged after this duration
// even though it exceeds that given in the RotationCondition
assert!(file.index() == 0);
// Bit touch and go but assuming two writes of 1mb bytes doesn't take 100ms!
file.write_all(&data).unwrap();
assert!(file.index() == 1);
file.write_all(&data).unwrap();
assert!(file.index() == 1);

// Will now have test_ACTIVE.log and test.log.1

Prune old logs to avoid filling up the disk

use std::{io::Write, path::Path};
use tempdir::TempDir;
use turnstiles::{PruneCondition, RotatingFile, RotationCondition}; // Subcrate provided for testing
let dir = TempDir::new();
let path = &vec![dir.path.clone(), "test.log".to_string()].join("/");
let data: Vec<u8> = vec![0; 990_000];
let mut file = RotatingFile::new(
    path,
    RotationCondition::SizeMB(1),
    PruneCondition::MaxFiles(3),
    false,
)
.unwrap();

// Generate > 3
// (this will generate 10 files because we're only writing 990kb and rotating on 1mb)
for _ in 0..20 {
    file.write_all(&data).unwrap();
}

// Should now only have the active file and two files with the highest index
// (which will be 8 and 9 in this case)
for i in 1..4 {
    let path = &format!("{}/test.log.{}", &dir.path, i);
    let file = Path::new(path);
    if i < 8 {
        assert!(!file.is_file());
    } else {
        assert!(file.is_file());
    }
}

Structs

RotatingFile

Struct masquerades as a file handle and is written to by whatever you like

Enums

PruneCondition

Enum for possible file prune options.

RotationCondition

Enum for possible file rotation options.