coremlit 0.3.0

Safe, synchronous CoreML runtime for macOS (CPU/GPU/Neural Engine) with opt-in on-device multimodal pipelines: speech (Whisper STT, forced alignment, speaker diarization, Silero VAD), AudioSet sound-event tagging, and audio/text/image embeddings (CLAP, granite, SigLIP)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71

//! What a `prewarm` did: per function it warmed, the load and the first
//! prediction, which carries Core ML's device specialization when it is cold.

use core::time::Duration;

use super::error::Function;

/// One warmed function: its load and its first, throwaway prediction.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub struct Warmed {
  function: Option<Function>,
  load: Duration,
  first_prediction: Duration,
}

impl Warmed {
  /// The record of `function` (`None` for the audio bundle's one function).
  #[inline]
  pub(crate) const fn new(function: Option<Function>, load: Duration, first: Duration) -> Self {
    Self {
      function,
      load,
      first_prediction: first,
    }
  }

  /// The function, or `None` for a one-function bundle.
  #[inline]
  pub const fn function(&self) -> Option<Function> {
    self.function
  }

  /// How long Core ML took to load it.
  #[inline]
  pub const fn load(&self) -> Duration {
    self.load
  }

  /// How long its first prediction took: the device specialization, when it
  /// was cold, and one inference.
  #[inline]
  pub const fn first_prediction(&self) -> Duration {
    self.first_prediction
  }
}

/// What a `prewarm` did, in the order it warmed the functions.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct PrewarmReport {
  warmed: Vec<Warmed>,
}

impl PrewarmReport {
  /// The record of every function warmed, in order.
  #[inline]
  pub fn warmed(&self) -> &[Warmed] {
    &self.warmed
  }

  /// Appends one function's record.
  #[inline]
  pub(crate) fn push(&mut self, warmed: Warmed) {
    self.warmed.push(warmed);
  }

  /// Appends another report's records, in their order.
  #[inline]
  pub(crate) fn extend(&mut self, other: Self) {
    self.warmed.extend(other.warmed);
  }
}