Crate pinenut_log
source ·Expand description
An extremely high performance logging system for clients (iOS, Android, Desktop), written in Rust.
§Compression
Pinenut supports streaming log compression, it uses the Zstandard (aka zstd)
, a
high performance compression algorithm that has a good balance between
compression rate and speed.
§Encryption
Pinenut uses the AES 128
algorithm for symmetric encryption during logging. To
prevent embedding the symmetric key directly into the code, Pinenut uses ECDH
for key negotiation (RSA is not used because its key are too long). When
initializing the Logger, there is no need to provide the symmetric encryption
key, instead the ECDH public key should be passed.
Pinenut uses secp256r1
elliptic curve for ECDH. You can generate the secret and
public keys for encryption yourself, or use Pinenut’s built-in command line tool:
pinenut-cli
.
§Buffering
In order to minimize IO frequency, Pinenut buffers the log data before writing to
the file. Client programs may exit unexpectedly (e.g., crash), Pinenut uses
mmap
as buffer support, so that if the program unexpectedly exits, the OS can
still help to persist the buffered data. The next time the Logger is initialized,
the buffered data is automatically read and written back to the log file.
In addition, Pinenut implements a double-buffering
system to improve buffer
read/write performance and prevent asynchronous IOs from affecting logging of the
current thread.
§Extraction
With Pinenut, we don’t need to retrieve all the log files in the directory to extract logs, it provides convenient extraction capabilities and supports extraction in time ranges with minute granularity.
§Parsing
The content of Pinenut log files is a special binary sequence after encoding, compression and encryption, and we can parse the log files using the parsing capabilities provided by Pinenut.
§Usage
Pinenut’s APIs are generally similar regardless of the language used.
§Logger Initialization
Pinenut uses a Logger
instance for logging. Before we initialize the Logger, we
need to pass in the logger identifier and the path to the directory where the log
files are stored to construct the Domain
structure.
We can customize the Logger by explicitly specifying Config
, see the API
documentation for details.
let domain = Domain::new("MyApp".into(), "/path/to/dir".into());
let config = Config::new().key_str(Some("Public Key Base64")).compression_level(10);
let logger = Logger::new(domain, config);
§Logging
Just construct the Record
and call the log
method.
Records can be constructed in Rust
via the Builder pattern:
// Builds `Meta` & `Record`.
let meta = Meta::builder().level(Level::Info).build();
let record = Record::builder().meta(meta).content("Hello World").build();
logger.log(&record);
// Flushes any buffered records asynchronously.
logger.flush();
See the API documentation for details.
§Extraction
Just call the extract
method to extract the logs for the specified time range
(with minute granularity) and write them to the destination file.
let domain = Domain::new("MyApp".into(), "/path/to/dir".into());
let now = chrono::Utc::now();
let range = now.sub(Duration::from_secs(1800))..=now;
if let Err(err) = pinenut_log::extract(domain, range, "/path/to/destination") {
println!("Error: {err}");
}
Note: The content of the extracted file is still a binary sequence that has been encoded, compressed, and encrypted. We need to parse it to see the log text content that is easy to read.
§Parsing
You can use the parse
function for log parsing, and you can specify the
format of the log parsed text. See the API documentation for details.
// Specifies the `DefaultFormater` as the log formatter.
if let Err(err) = pinenut_log::parse_to_file(&path, &output, secret_key, DefaultFormatter) {
println!("Error: {err}");
}
Or use the built-in command line tool pinenut-cli
:
$ pinenut-cli parse ./my_log.pine \
--output ./plain.log \
--secret-key XXXXXXXXXXX
§Keys Generation
Before initializing the Logger or parsing the logs, you need to have the public and secret keys ready (The public key is used to initialize the Logger and the secret key is used to parse the logs).
You can use pinenut-cli
to generate this pair of keys:
$ pinenut-cli gen-keys
Re-exports§
pub use compress::CompressionError;
pub use compress::DecompressionError;
pub use encrypt::DecryptionError;
pub use encrypt::EncryptionError;
pub use encrypt::EncryptionKey;
pub use encrypt::PublicKey;
pub use encrypt::SecretKey;
pub use encrypt::PUBLIC_KEY_LEN;
pub use codec::DecodingError;
pub use codec::EncodingError;
pub use chunk::Error as ChunkError;
pub use runloop::Error as RunloopError;
pub use record::*;
Modules§
- The
Chunk
structure. - Encoding & Decoding.
- Compression & Decompression.
- Encryption & Decryption.
- The
Pinenut
log record. - An event loop implementation.
Structs§
- Configuration of a logger instance.
- The default formatter provides simple log formatting.
- Represents the domain to which the logger belongs, the logs will be organized by domain.
- The
Pinenut
logger.
Enums§
- Errors that can be occurred during the log extraction process (
extract
). - The error type for
Logger
. - Errors that can be occurred during the log parsing process (
parse
). - Represents the dimension of datetime, used for log rotation.
Constants§
- The default buffer length (320 KB) for Pinenut.
- The extension of the Pinenut log file.
- The current format version of the Pinenut log structure.
- The extension of the Pinenut mmap buffer file.
- The extension of the Pinenut plain log file.
Traits§
- Represents a formatter that formats log records into readable text.
- Represents a tracker used to track errors occurred from the logger operations.
Functions§
- Extracts the logs for the specified time range and writes them to the destination file.
- Parses the compressed and encrypted binary log file into multiple log records and calls them back one by one.
- Parses the compressed and encrypted binary log file into readable text file.
Type Aliases§
- Trait object type for
Track
.