pub struct RenderedPrompt { /* private fields */ }Expand description
A rendered prompt plus its optional tokenization boundaries.
The prompt owns its segment text while dynamo-tokenizers borrows that text
during encoding. Keeping the types separate preserves the tokenizer crate’s
published zero-copy EncodeSegment<'_> API.
Implementations§
Source§impl RenderedPrompt
impl RenderedPrompt
pub fn text(text: String) -> Self
pub fn segmented(segments: Vec<RenderedSegment>) -> Self
Sourcepub fn segmented_with_pending(
segments: Vec<RenderedSegment>,
pending_segments: usize,
) -> Self
pub fn segmented_with_pending( segments: Vec<RenderedSegment>, pending_segments: usize, ) -> Self
Build a segmented prompt whose last pending_segments segments are the
assistant generation stub (see RenderedPrompt::pending_segments).
§Panics
Panics if pending_segments exceeds the number of segments; a renderer
that claims more pending segments than it produced is a bug.
pub fn as_str(&self) -> &str
pub fn segments(&self) -> Option<&[RenderedSegment]>
pub fn encode_segments(&self) -> Option<Vec<EncodeSegment<'_>>>
Sourcepub fn pending_segments(&self) -> usize
pub fn pending_segments(&self) -> usize
Number of trailing segments that form the assistant generation stub.
Zero when the prompt is unsegmented, when no generation prompt was added, or when the renderer has no stub convention to report.
Sourcepub fn pending_encode_segments(&self) -> Option<Vec<EncodeSegment<'_>>>
pub fn pending_encode_segments(&self) -> Option<Vec<EncodeSegment<'_>>>
The trailing generation-stub segments, or None for unsegmented prompts.
Callers that report usage under the reference API’s convention encode these and subtract their token count from the physical prompt length.
pub fn into_text(self) -> String
Trait Implementations§
Source§impl Clone for RenderedPrompt
impl Clone for RenderedPrompt
Source§impl Debug for RenderedPrompt
impl Debug for RenderedPrompt
impl Eq for RenderedPrompt
Source§impl PartialEq for RenderedPrompt
impl PartialEq for RenderedPrompt
impl StructuralPartialEq for RenderedPrompt
Auto Trait Implementations§
impl Freeze for RenderedPrompt
impl RefUnwindSafe for RenderedPrompt
impl Send for RenderedPrompt
impl Sync for RenderedPrompt
impl Unpin for RenderedPrompt
impl UnsafeUnpin for RenderedPrompt
impl UnwindSafe for RenderedPrompt
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
Source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
Source§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
key and return true if they are equal.Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§fn in_current_span(self) -> Instrumented<Self> ⓘ
fn in_current_span(self) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more