pub enum BindingConstraint {
FrameCap,
ContextCeiling,
TextHeuristic,
}Expand description
Which limit determined the predicted frame count.
Reported because the two have different remedies: a caller hitting the frame cap should raise it or chunk the text, while one hitting the context ceiling must shorten the prompt.
Variants§
FrameCap
max_new_tokens bound the generation. The usual case.
ContextCeiling
The context ceiling bound it — the prompt is long enough to crowd out generation.
TextHeuristic
The text-derived EOS backstop bound it (prompt_tokens * 4 + 64 frames).
The sampled EOS is a stochastic stop (README: “EOS stop timing is sampling-dependent”),
so a bare ftts say without FTTS_MAX_FRAMES needs a cap proportional to the text
rather than the flat 8,192-frame (≈11 minute) policy default. Setting FTTS_MAX_FRAMES
disables this backstop: an explicit cap is obeyed exactly.
Implementations§
Trait Implementations§
Source§impl Clone for BindingConstraint
impl Clone for BindingConstraint
Source§fn clone(&self) -> BindingConstraint
fn clone(&self) -> BindingConstraint
Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source. Read moreimpl Copy for BindingConstraint
Source§impl Debug for BindingConstraint
impl Debug for BindingConstraint
impl Eq for BindingConstraint
Source§impl PartialEq for BindingConstraint
impl PartialEq for BindingConstraint
impl StructuralPartialEq for BindingConstraint
Auto Trait Implementations§
impl Freeze for BindingConstraint
impl RefUnwindSafe for BindingConstraint
impl Send for BindingConstraint
impl Sync for BindingConstraint
impl Unpin for BindingConstraint
impl UnsafeUnpin for BindingConstraint
impl UnwindSafe for BindingConstraint
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
Source§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
Compare self to
key and return true if they are equal.Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, _span: NoopSpan) -> Self
fn instrument(self, _span: NoopSpan) -> Self
Instruments this future with a span (no-op when disabled).
Source§fn in_current_span(self) -> Self
fn in_current_span(self) -> Self
Instruments this future with the current span (no-op when disabled).