Temporal Segment Log (TSL)

Abstract

Temporal Segment Log (TSL) is a hybrid data structure designed for high-throughput, time-ordered event streams. It combines append-only segmented storage with a temporal index to achieve efficient ingestion and range queries. TSL targets workloads where strict temporal ordering, high write throughput, and efficient retention are critical.

1. Motivation

Modern systems such as:

observability pipelines
financial tick processing
AI data ingestion
distributed logging systems

require handling large volumes of timestamped data.

Existing structures exhibit trade-offs:

Structure	Limitation
Log (append-only)	inefficient range queries
B-Tree	slower writes
LSM Tree	compaction overhead
HashMap	no ordering

TSL is designed to balance these trade-offs.

2. Design Overview

TSL consists of three primary components:

Segments
- fixed-capacity append-only blocks
- contiguous memory layout
Temporal Index
- maps timestamps → segment identifiers
- implemented using BTreeMap
Active Segment
- current write target
- not indexed until rotation

3. Data Model

Each event is defined as:

Event = (timestamp: u64, payload: Vec)

Invariant

timestamp_i <= timestamp_j for i < j

4. Operations

Operation	Description	Complexity
append	insert event	O(1)
range_query	retrieve events in [t1, t2]	O(log n + k)
latest	last N events	O(1)
segment rotation	create new segment	O(1)

5. Architecture

flowchart TB

    subgraph IndexLayer
        I["Temporal Index\nBTreeMap<u64, SegmentID>"]
    end

    subgraph StorageLayer
        S0["Segment 0\n[events...]"]
        SN["Segment N\n[events...]"]
    end

    subgraph WriteLayer
        A["Active Segment\n[recent events]"]
    end

    I --> S0
    I --> SN

    S0 --> A
    SN --> A

6. Implementation

TSL is implemented in Rust with:

Vec<Event> for storage
BTreeMap for indexing
Arc<Mutex<Segment>> for concurrency

Safety Guarantees

memory safety (Rust ownership model)
thread safety (Mutex)
deterministic behavior

7. Performance Evaluation

Benchmarks performed using criterion.

Results (v0.1.0)

Operation	Performance
Ingest (1M events)	~300 ms
Throughput	~3.3M events/sec
Range Query	~3.5 ms
Mixed Workload	~110 ms

Observations

ingestion scales linearly
query performance proportional to result size
stable under concurrent access

8. Limitations

Current design limitations:

active segment requires linear scan
Mutex introduces contention
no disk persistence
no distributed coordination

9. Roadmap

v0.2.0 (Planned)

lock-free append path
segment-level indexing
memory-mapped storage (mmap)
async API (tokio)
distributed sharding

10. Installation

Rust

cargo add tsl

use tsl::{TSL, Event};

let mut tsl = TSL::new(100);
tsl.append(Event::new(1, vec![1]));

Python

pip install tsl

import tsl

t = tsl.TSL(100)
t.append(1, b"data")
print(t.latest(1))

11. Example

let mut tsl = TSL::new(1000);

for i in 0..1000 {
tsl.append(Event::new(i, vec![1]));
}

let result = tsl.range_query(100, 200);

12. Reproducibility

To reproduce benchmarks:

cargo bench

Environment:

Rust stable
Criterion benchmark framework

13. Use Cases

real-time analytics pipelines
event sourcing systems
time-series ingestion
streaming AI preprocessing

14. License

MIT License

15. Author

Vishwanath M M

16. Citation (Draft)

Temporal Segment Log (TSL), 2026.
High-throughput time-ordered data structure.

temporal_segment_log 0.1.2