Crate auditor

source ·
Expand description

§Tutorial

This section walks you through several basic usecases of the Auditor software.

Auditor is designed around so-called records, which are the unit of accountable resources. Records are created and pushed to Auditor, which stores them in a database. These records can then be requested again from Auditor to take an action based on the information stored in the records.

A record consists of a unique identifier and meta information that provides some context (associated site, group, user, …). Furthermore, a record also contains an arbitrary number of components that are to be accounted for (CPU, RAM, Disk, …) and the amount of each of these components. The components can optionally be enhanced with scores, which are floating point values that put components of the same kind, but different performance in relation to each other.

§Creating a Record

use auditor::domain::{Component, RecordAdd, Score};
use chrono::{DateTime, TimeZone, Utc};
use std::collections::HashMap;

// Define unique identifier
let record_id = "record-1"; // Must be unique for all records in Auditor!

// Time when the resource became available
let start_time: DateTime<Utc> = Utc.with_ymd_and_hms(2023, 1, 1, 0, 0, 0).unwrap();

// Create a component (10 CPU cores)
// and attache a score (HEPSPEC06) to it
let component_cpu = Component::new("CPU", 10)?
    .with_score(Score::new("HEPSPEC06", 9.2)?);

// Create a second component (32 GB memory)
let component_mem = Component::new("MEM", 32)?;

// Store components in a vector
let components = vec![component_cpu, component_mem];

// Create meta information
let mut meta = HashMap::new();
meta.insert("site_id", vec!["site1"]);
meta.insert("features", vec!["ssd", "gpu"]);

let record = RecordAdd::new(record_id, meta, components, start_time)?;

§Connecting to Auditor

The AuditorClientBuilder is used to build an AuditorClient object that can be used for interacting with Auditor.

use auditor::client::AuditorClientBuilder;

let client = AuditorClientBuilder::new()
    .address(&"localhost", 8000)
    .timeout(20)
    .build()?;

§Pushing one record to Auditor

Assuming that a record and a client were already created, the record can be pushed to Auditor with

client.add(&record).await?;

§Pushing multiple records to AuditorClientBuilder

Assuming that list of records and a client were already created, the records can be pushed to Auditor with_ymd_and_hms

client.bulk_insert(&records).await?;

§Updating records in Auditor

Auditor accepts incomplete records. In particular, the stop time can be missing. These records can be updated at a later time, by adding the same record which includes a stop time. Note that the record_id must match the one already in the database! Fields other than the stop time cannot be updated.

use auditor::domain::RecordUpdate;
use chrono::{DateTime, TimeZone, Utc};

let stop_time: DateTime<Utc> = Utc.with_ymd_and_hms(2023, 1, 1, 12, 0, 0).unwrap();
let record = RecordUpdate::new("record-1", HashMap::new(), vec![], stop_time)?;

client.update(&record).await?;

§Receiving all records from Auditor

The complete set of records can be retrieved from Auditor with the get() method:

let records = client.get().await?;

§Receiving all records started/stopped since a given timestamp

(Deprecated: Use the advanced_query function instead)

Instead of retrieving all records, the query can be limited to records that have been started or stopped since a given timestamp:

use chrono::{DateTime, TimeZone, Utc};

let since: DateTime<Utc> = Utc.with_ymd_and_hms(2023, 1, 1, 12, 0, 0).unwrap();

let records_started_since = client.get_started_since(&since).await?;
let records_stopped_since = client.get_stopped_since(&since).await?;

§Advanced Query

Records can be queried using fields and operators.

§Template Query

The table shows the fields and the corresponding operators available for each field with which a query can be built.

GET /records?<field>[<operator>]=<value>
§Operators
  • gt (greater than)
  • gte (greater than or equal to)
  • lt (less than)
  • lte (less than or equal to)
  • equals (equal to)
§Meta Operators
  • c (contains)
  • dnc (does not contains)
§SortBy Operators
  • asc (ascending order)
  • desc (descending order)
§SortBy Column names

You can specify the column on which the sorting must happen The following columns are supported for sortby option

  • start_time
  • stop_time
  • runtime
  • record_id
FieldDescriptionOperatorsExamples (query representation)
record_idRetrieve the exact record using record_idrecord_id-<record_id>
start_timeStart time of the event (DateTime<Utc>)gt, gte, lt, ltestart_time[gt]=<timestamp>
stop_timeStop time of the event (DateTime<Utc>)gt, gte, lt, ltestop_time[gt]=<timestamp>
runtimeRuntime of the event (in seconds)gt, gte, lt, lteruntime[gt]=<u64>
metaMeta information (<meta_key>, MetaOperator(<meta_value>))d, dncmeta[<meta_key>][c]=<meta_value>
componentComponent identifier (<component_name>, Operator(<component_amount>))gt, gte, lt, lte, equalscomponent[<component_name>][gt]=<amount>
sort_bySort query results (SortBy(<column_name>))asc, descsort_by[desc]=<column_name>
limitlimit query records (number)limit=5000

Meta field can be used to query records by specifying the meta key and MetaOperator must be used to specify meta values. The MetaOperator must be used to specify whether the value is contained or does not contained for the specific Metakey.

Component field can be used to query records by specifying the component name (CPU) and [‘Operator’] must be used to specify the amount.

To query records based on a range, specify the field with two operators Either with gt or gte and lt or lte.

For example, to query a records with start_time ranging between two timestamps:

GET records?start_time[gt]=timestamp1&start_time[lt]=timestamp2

§QueryBuilder

Below are the examples to query records using QueryBuilder methods. It helps to build query string which can be passed as an argument to advanced_query function to get the records.

§Example 1:

Constructs an empty QueryBuilder to query all records

let records = QueryBuilder::new()
                .get(client)
                .await?;

The query string would look like

GET records

§Example 2:

Constructs a QueryBuilder with a start time operator that specifies a range from datetime_utc_gte to datetime_utc_lte.

let datetime_utc_gte = Utc.with_ymd_and_hms(2022, 8, 3, 9, 47, 0).unwrap();
let datetime_utc_lte = Utc.with_ymd_and_hms(2022, 8, 4, 9, 47, 0).unwrap();

let records = QueryBuilder::new()
    .with_start_time(
       Operator::default()
           .gte(datetime_utc_gte.into())
           .lte(datetime_utc_lte.into()),
   )
   .get(client)
   .await?;

The query string would look like:

GET records?start_time[lte]=datetime_utc_lte&start_time[gte]=datetime_utc_gte

§Example 3:

Constructs a QueryBuilder with start time, stop time, and runtime operators, specifying ranges for each.

use chrono::{Utc, TimeZone};

let datetime_utc_gte = Utc.with_ymd_and_hms(2022, 8, 3, 9, 47, 0).unwrap();
let datetime_utc_lte = Utc.with_ymd_and_hms(2022, 8, 4, 9, 47, 0).unwrap();
let runtime_gte: u64 = 100000;
let runtime_lte: u64 = 200000;

let records = QueryBuilder::new()
    .with_start_time(
        Operator::default()
            .gte(datetime_utc_gte.into())
            .lte(datetime_utc_lte.into()),
    )
    .with_stop_time(
        Operator::default()
            .gte(datetime_utc_gte.into())
            .lte(datetime_utc_lte.into()),
    )
    .with_runtime(
        Operator::default()
            .gte(runtime_gte.into())
            .lte(runtime_lte.into()),
    )
    .get(client)
    .await?;

The query string would look like

GET
records?start_time[gte]=datetime_utc_gte&start_time[lte]=datetime_utc_lte&stop_time[gte]=datetime_utc_gte&stop_time[lte]=datetime_utc_lte&runtime[gte]=runtime_gte&runtime[lte]=runtime_lte

§Example 4:

Constructs a QueryBuilder with a meta query specifying “site_id” should contain “site1” value and a start time operator specifying a maximum datetime.

ClientError};
use chrono::{Utc, TimeZone};

let datetime_utc_lte = Utc.with_ymd_and_hms(2022, 8, 4, 9, 47, 0).unwrap();
let records = QueryBuilder::new()
    .with_meta_query(
        MetaQuery::new().meta_operator(
            "site_id".to_string(),
            MetaOperator::default().contains("site1".to_string()),
        )
    )
    .with_start_time(
        Operator::default().lte(datetime_utc_lte.into())
    )
    .get(client)
    .await?;

The query string would look like:

GET records?meta[site_id][c]=site1&start_time[lte]=datetime_utc_lte

§Example 5:

Constructs a QueryBuilder with a component query specifying an equality condition for the “CPU” field.

use auditor::client::{QueryBuilder, Operator, ComponentQuery, AuditorClientBuilder, ClientError};

let count: u8 = 10; // querying records whose cpu amount is 10
let records = QueryBuilder::new()
    .with_component_query(
        ComponentQuery::new().component_operator(
            "CPU".to_string(),
            Operator::default().equals(count.into()),
        )
    )
    .get(client)
    .await?;

The query string would look like

GET records?component[CPU][equals]=count

//! ### Example 6:

Constructs a QueryBuilder which sorts the record in descending order by stop_time and limits the query results by 500 records

use auditor::client::{QueryBuilder, AuditorClientBuilder, ClientError, SortBy};

let number: u64 = 500;
let records = QueryBuilder::new()
    .sort_by(SortBy::new().descending("stop_time".to_string()))
    .limit(number)
    .get(client)
    .await?;

The query string would look like

GET records?sort_by[desc]=stop_time&limit=number

§Example 7:

Constructs a QueryBuilder to retrieve one record using record id

use auditor::client::{QueryBuilder, AuditorClientBuilder, ClientError};

let record_id = "record-1".to_string();
let records = client.get_single_record(record_id).await?;

The query string would look like

GET record/record-1

§Warning

equals operator is only available for querying components. It cannot be used for time based queries

If the query is directly appended to the URL, please make sure that the datetime value is urlencoded

§Checking the health of Auditor

The health of Auditor can be checked with

if client.health_check().await {
    println!(":)");
} else {
    println!(":(");
}

Modules§

Macros§