Skip to main content

ramvamp_server/
toolcall.rs

1//! Recovering structured `tool_calls` from the reply text the model wrote.
2//!
3//! The template tells the model to answer with
4//!
5//! ```text
6//! <tool_call>
7//! {"name": <function-name>, "arguments": <args-json-object>}
8//! </tool_call>
9//! ```
10//!
11//! which arrives here as ordinary generated text. A client cannot act on that;
12//! it acts on `choices[0].message.tool_calls`. This module is the translation,
13//! and it is pure: text in, [`Extracted`] out, no clock, no randomness, no I/O.
14//! That is what lets every rule below be pinned by a unit test.
15//!
16//! # Finding the block boundaries
17//!
18//! The obvious implementation — `find("<tool_call>")`, then
19//! `find("</tool_call>")` — has a bug that shows up the first time a model
20//! quotes the protocol back at you:
21//!
22//! ```text
23//! <tool_call>
24//! {"name": "reply", "arguments": {"text": "wrap it in </tool_call> tags"}}
25//! </tool_call>
26//! ```
27//!
28//! The naive scan closes the block inside the JSON string and produces a
29//! truncated, unparseable body. [`find_close`] therefore walks the bytes
30//! tracking whether it is inside a JSON string literal (honouring `\`
31//! escapes) and only accepts a `</tool_call>` found *outside* one.
32//!
33//! That tracking is itself a risk: a body that is not JSON at all can carry an
34//! unbalanced `"`, which would make the scan believe the rest of the reply is
35//! one long string and lose a terminator that is plainly there. So the
36//! string-aware scan falls back to a plain search when it reaches the end
37//! without a match. The fallback means the careful scan can only ever find
38//! *more* blocks than the naive one, never fewer.
39//!
40//! Locating `name` and `arguments` inside the body uses the same idea one
41//! level down: a member's value is a *span*, delimited by matching braces or
42//! brackets with strings skipped, so `{"a": {"b": 1}}` and `{"a": "} }"}` both
43//! end where they should. A naive `find('}')` ends both in the wrong place.
44//!
45//! # Bytes are preserved, never re-serialized
46//!
47//! [`ParsedCall::arguments`] is the raw substring the model emitted. It is not
48//! parsed into a `serde_json::Value` and printed back, for two reasons. The
49//! client echoes `arguments` verbatim on the next turn and
50//! [`crate::prompt`] splices that string straight into the prompt as
51//! `ToolArguments::Raw`, so preserving the bytes makes the round trip exact —
52//! the model sees on turn *n+1* the characters it wrote on turn *n*.
53//! Re-serializing would also quietly normalise key order and spacing, and
54//! would turn the everyday case of a model emitting slightly invalid JSON into
55//! a hard failure.
56//!
57//! # Malformed blocks are recovered, not dropped
58//!
59//! Models get this JSON wrong often enough that dropping a malformed block
60//! would be a routine loss of the turn. If a `name` can be recovered the call
61//! is emitted with whatever the `arguments` span held, raw and unchecked;
62//! `FunctionCall::arguments` is a `String` precisely so the client can be
63//! handed exactly what the model said and fail on its own terms. Only a block
64//! with no recoverable name falls back to being ordinary content — and then
65//! the whole block, markers included, is preserved there, so nothing the model
66//! wrote is ever discarded.
67//!
68//! An unterminated `<tool_call>` is content for the same reason: it is what a
69//! reply that ran out of `max_tokens` mid-block looks like, and no call can be
70//! honestly claimed from half a block.
71//!
72//! # Streaming, where the whole reply is not in hand
73//!
74//! [`extract`] needs the finished text. A stream does not have it: the decision
75//! to send a token as content has to be made before the token after it exists,
76//! and once a `<` has gone out as content it cannot be recalled. Scanning the
77//! text as it accumulates would also mean handling a marker split across two
78//! deltas, since nothing about a byte stream promises `</tool_call>` arrives
79//! whole.
80//!
81//! It does arrive whole, and that is the mechanism [`CallStream`] is built on.
82//! Both markers are *added tokens* in the Qwen3 tokenizer —
83//! [`OPEN_TAG_ID`] and [`CLOSE_TAG_ID`] — so each is a single id, and the
84//! decoder does not skip them. Detection is therefore an integer compare
85//! against the id the model sampled, made at the one layer that still has the
86//! id, and a marker cannot be split because a token cannot be. What is left is
87//! bookkeeping: buffer between the two ids, hand the body to the same
88//! [`parse_block`] the buffered path uses, and give the call the same id
89//! [`wire_call`] would have given it there.
90
91use crate::request::{FunctionCall, ToolCall};
92use crate::response::COMPLETION_ID_PREFIX;
93
94/// The marker that opens a call block.
95pub const OPEN_TAG: &str = "<tool_call>";
96
97/// The marker that closes one.
98pub const CLOSE_TAG: &str = "</tool_call>";
99
100/// The single token id [`OPEN_TAG`] encodes to.
101///
102/// An `added_tokens` entry of the Qwen3 tokenizer, `special: false`, so the
103/// streaming decoder emits it as one token carrying the literal text. Pinned as
104/// a constant rather than looked up because the streaming sink has no
105/// tokenizer: it sees ids and text, and this crate is meant to be testable
106/// without an install on disk.
107pub const OPEN_TAG_ID: u32 = 151_657;
108
109/// The single token id [`CLOSE_TAG`] encodes to. See [`OPEN_TAG_ID`].
110pub const CLOSE_TAG_ID: u32 = 151_658;
111
112/// The `type` of every call: OpenAI has defined exactly one.
113pub const CALL_KIND: &str = "function";
114
115/// The prefix of a generated call id.
116pub const CALL_ID_PREFIX: &str = "call_";
117
118/// What a block that named no arguments contributes.
119///
120/// The empty object rather than the empty string: a client's first move is
121/// `json.loads(arguments)`, and `""` breaks it where `{}` is the truth — a
122/// call with no arguments.
123const NO_ARGUMENTS: &str = "{}";
124
125/// One call recovered from a reply.
126#[derive(Debug, Clone, PartialEq, Eq)]
127pub struct ParsedCall {
128    /// The function name, unescaped.
129    pub name: String,
130    /// The arguments, as the exact bytes the model emitted.
131    pub arguments: String,
132}
133
134/// A reply split into what the client should show and what it should run.
135#[derive(Debug, Clone, Default, PartialEq, Eq)]
136pub struct Extracted {
137    /// Everything outside the blocks that became calls.
138    ///
139    /// When no call was extracted this is the reply unchanged, byte for byte;
140    /// a plain answer must not be altered by a parser that found nothing to
141    /// do. When calls *were* extracted the surrounding prose is joined and
142    /// trimmed at the ends, since the whitespace that framed a removed block
143    /// is framing, not text.
144    pub content: String,
145    /// The calls, in the order the model asked for them.
146    pub calls: Vec<ParsedCall>,
147}
148
149impl Extracted {
150    /// The reply as pure content, with nothing parsed out of it.
151    ///
152    /// For a request that declared no tools. The markers only appear because
153    /// the template's tools branch instructed the model to produce them, so
154    /// without tools they are ordinary text: a conversation *about* tool
155    /// calling would otherwise have its example swallowed into a structured
156    /// call the caller never enabled.
157    pub fn none(reply: &str) -> Self {
158        Extracted {
159            content: reply.to_owned(),
160            calls: Vec::new(),
161        }
162    }
163}
164
165/// Split `reply` into content and the calls it asked for.
166///
167/// Never fails and never panics: every malformed shape has a defined outcome,
168/// documented on [the module](self).
169pub fn extract(reply: &str) -> Extracted {
170    let mut content = String::new();
171    let mut calls = Vec::new();
172    let mut cursor = 0;
173
174    while let Some(open) = reply.get(cursor..).and_then(|rest| rest.find(OPEN_TAG)) {
175        let open = cursor + open;
176        let body_start = open + OPEN_TAG.len();
177        // No terminator: this block never finished, so it is text. Leaving
178        // `cursor` where it is puts the opener and everything after it into
179        // content below.
180        let Some(close) = find_close(reply, body_start) else {
181            break;
182        };
183        let body = reply.get(body_start..close).unwrap_or_default();
184        match parse_block(body) {
185            Some(call) => {
186                content.push_str(reply.get(cursor..open).unwrap_or_default());
187                calls.push(call);
188            }
189            // Not a call after all: keep the block verbatim, markers included.
190            None => content.push_str(
191                reply
192                    .get(cursor..close + CLOSE_TAG.len())
193                    .unwrap_or_default(),
194            ),
195        }
196        cursor = close + CLOSE_TAG.len();
197    }
198    content.push_str(reply.get(cursor..).unwrap_or_default());
199
200    if !calls.is_empty() {
201        content = content.trim().to_owned();
202    }
203    Extracted { content, calls }
204}
205
206/// The id a call is given, derived from the completion's own id.
207///
208/// Deterministic on purpose. The client sends this back as `tool_call_id`, so
209/// it has to be stable within a response, and a test that cannot predict an id
210/// cannot pin the round trip — the same reason `id` and `created` are
211/// parameters everywhere else in this crate rather than reads of a clock.
212/// Uniqueness comes from the completion id, which already cannot repeat.
213pub fn call_id(completion_id: &str, index: usize) -> String {
214    let suffix = completion_id
215        .strip_prefix(COMPLETION_ID_PREFIX)
216        .unwrap_or(completion_id);
217    format!("{CALL_ID_PREFIX}{suffix}_{index}")
218}
219
220/// Give one parsed call its wire shape and its id, at `index`.
221///
222/// The single place a [`ParsedCall`] becomes a [`ToolCall`], which is what
223/// makes the buffered and streaming paths agree by construction rather than by
224/// two implementations that happen to match today: the buffered path indexes a
225/// finished list, the streaming one counts calls as they close, and both arrive
226/// here.
227pub fn wire_call(completion_id: &str, index: usize, call: &ParsedCall) -> ToolCall {
228    ToolCall {
229        id: call_id(completion_id, index),
230        kind: CALL_KIND.to_owned(),
231        function: FunctionCall {
232            name: call.name.clone(),
233            arguments: call.arguments.clone(),
234        },
235    }
236}
237
238/// Give the parsed calls their wire shape and their ids.
239pub fn wire_calls(completion_id: &str, calls: &[ParsedCall]) -> Vec<ToolCall> {
240    calls
241        .iter()
242        .enumerate()
243        .map(|(index, call)| wire_call(completion_id, index, call))
244        .collect()
245}
246
247/// What the sink should do with the token it just fed to a [`CallStream`].
248#[derive(Debug, Clone, PartialEq, Eq)]
249pub enum Step {
250    /// Send nothing: the token is inside a block that has not closed.
251    Nothing,
252    /// Send this as a content delta. Not always the token's own text — a block
253    /// that closed without naming a function comes back here whole, markers
254    /// included, exactly as the buffered path keeps it.
255    Content(String),
256    /// A block closed and named a function.
257    Call {
258        /// Its position in this response's `tool_calls`, which is the `index`
259        /// the delta must carry: it is how a client knows which call a
260        /// fragment belongs to.
261        index: usize,
262        /// The call.
263        call: ParsedCall,
264    },
265}
266
267/// [`extract`]'s rules, applied one token at a time.
268///
269/// Fed every generated token in order, it answers what to put on the wire for
270/// it. The outcome matches what [`extract`] would have produced from the
271/// concatenated reply, with one unavoidable difference: [`extract`] trims the
272/// prose that framed a removed block, and a stream has already sent that
273/// whitespace by the time the block it framed appears.
274///
275/// Pure and clock-free like the rest of the module, so the state machine is
276/// pinned by unit tests rather than by a live model.
277///
278/// The other difference is [`find_close`]'s string awareness, which has no
279/// streaming equivalent and does not need one. A `</tool_call>` written *inside*
280/// an argument string reaches [`extract`] as twelve characters of a JSON
281/// literal; it reaches here as [`CLOSE_TAG_ID`], the id the model chose to
282/// sample, and an id is not text. Closing there is the honest reading of what
283/// the model emitted, and it is also the only reading available: the fallback
284/// that makes the buffered scan safe — search the *rest of the reply* — is a
285/// rest that does not exist yet.
286#[derive(Debug, Clone)]
287pub struct CallStream {
288    /// Whether calls are recovered at all.
289    ///
290    /// False for a request that declared no tools, and then every token is
291    /// content. The model only writes these markers because the tools branch of
292    /// the template told it to, so without tools they are ordinary text a user
293    /// asked for — and swallowing them would delete the answer. The same gate
294    /// the buffered path applies, for the same reason.
295    enabled: bool,
296    /// The body of the block currently open, markers excluded, or `None`
297    /// outside one.
298    open: Option<String>,
299    /// Calls emitted so far, which is both the next `index` and the answer to
300    /// "was this turn a tool call".
301    emitted: usize,
302}
303
304impl CallStream {
305    /// A stream that recovers calls, or one that passes everything through.
306    pub fn new(enabled: bool) -> Self {
307        CallStream {
308            enabled,
309            open: None,
310            emitted: 0,
311        }
312    }
313
314    /// How many calls have been emitted.
315    ///
316    /// Non-zero is what makes `finish_reason` `tool_calls`: the model stopped
317    /// because it wanted a tool, whatever token ended the reply.
318    pub fn emitted(&self) -> usize {
319        self.emitted
320    }
321
322    /// Feed one generated token and its id.
323    pub fn push(&mut self, id: u32, text: &str) -> Step {
324        if !self.enabled {
325            return Step::Content(text.to_owned());
326        }
327        match id {
328            // A second opener inside an open block is body text, matching
329            // `extract`, which looks for the terminator and not for another
330            // opener.
331            OPEN_TAG_ID if self.open.is_none() => {
332                self.open = Some(String::new());
333                Step::Nothing
334            }
335            // A terminator with nothing open closes nothing; it is the stray
336            // marker case, and it is content.
337            CLOSE_TAG_ID if self.open.is_some() => self.close(),
338            _ => match &mut self.open {
339                Some(body) => {
340                    body.push_str(text);
341                    Step::Nothing
342                }
343                None => Step::Content(text.to_owned()),
344            },
345        }
346    }
347
348    /// Generation ended: give back the text of a block that never closed.
349    ///
350    /// Half a block is not a call — it is what a reply that ran out of
351    /// `max_tokens` mid-block looks like — but it *is* text the model wrote,
352    /// and nothing the model wrote is dropped. Idempotent.
353    pub fn flush(&mut self) -> Option<String> {
354        self.open.take().map(|body| format!("{OPEN_TAG}{body}"))
355    }
356
357    /// Resolve the block that just closed.
358    fn close(&mut self) -> Step {
359        let body = self.open.take().unwrap_or_default();
360        match parse_block(&body) {
361            Some(call) => {
362                let index = self.emitted;
363                self.emitted += 1;
364                Step::Call { index, call }
365            }
366            // No recoverable name, so it was never a call: the block survives
367            // as text, markers and all.
368            None => Step::Content(format!("{OPEN_TAG}{body}{CLOSE_TAG}")),
369        }
370    }
371}
372
373/// Find the `</tool_call>` that closes the block whose body starts at `from`.
374///
375/// String-aware, then naive: see [the module](self) for why it is both.
376fn find_close(reply: &str, from: usize) -> Option<usize> {
377    let bytes = reply.as_bytes();
378    let mut i = from;
379    while i < bytes.len() {
380        match bytes[i] {
381            // A string literal is skipped whole, so a `</tool_call>` written
382            // *inside* one closes nothing.
383            b'"' => i = end_of_string(bytes, i),
384            b'<' if bytes[i..].starts_with(CLOSE_TAG.as_bytes()) => return Some(i),
385            _ => i += 1,
386        }
387    }
388    // An unbalanced quote in a body that was never JSON must not hide a
389    // terminator that is right there in the text.
390    reply
391        .get(from..)
392        .and_then(|rest| rest.find(CLOSE_TAG))
393        .map(|at| from + at)
394}
395
396/// Recover a call from one block body, or decide it is not a call.
397fn parse_block(body: &str) -> Option<ParsedCall> {
398    let open = body.find('{')?;
399    let mut name = None;
400    let mut arguments = None;
401    for (key, value) in members(body, open) {
402        match key.as_str() {
403            // First wins: a duplicated key is malformed, and the first is what
404            // a JSON reader that stops at the first match would have seen.
405            "name" if name.is_none() => name = Some(value),
406            "arguments" if arguments.is_none() => arguments = Some(value),
407            _ => {}
408        }
409    }
410    let name = unquote(name?);
411    if name.is_empty() {
412        return None;
413    }
414    Some(ParsedCall {
415        name,
416        arguments: arguments.map_or_else(|| NO_ARGUMENTS.to_owned(), str::to_owned),
417    })
418}
419
420/// The top-level members of the object starting at `open`, as (key, raw value).
421///
422/// Deliberately not a JSON parser: it recovers what it can and stops at the
423/// first thing it does not understand, keeping every member it read up to
424/// there. A strict parser would return nothing for a body with a trailing
425/// comma, which is exactly the body this has to survive.
426fn members(text: &str, open: usize) -> Vec<(String, &str)> {
427    let bytes = text.as_bytes();
428    let mut out = Vec::new();
429    let mut i = open + 1;
430    loop {
431        i = skip_space(bytes, i);
432        // A key is quoted; anything else ends the walk, `}` included.
433        if bytes.get(i) != Some(&b'"') {
434            return out;
435        }
436        let key_end = end_of_string(bytes, i);
437        let key = unquote(text.get(i..key_end).unwrap_or_default());
438        i = skip_space(bytes, key_end);
439        if bytes.get(i) != Some(&b':') {
440            return out;
441        }
442        i = skip_space(bytes, i + 1);
443        if i >= bytes.len() {
444            return out;
445        }
446        let value_end = end_of_value(bytes, i);
447        out.push((key, text.get(i..value_end).unwrap_or_default()));
448        i = skip_space(bytes, value_end);
449        if bytes.get(i) != Some(&b',') {
450            return out;
451        }
452        i += 1;
453    }
454}
455
456/// The index just past the value starting at `start`.
457fn end_of_value(bytes: &[u8], start: usize) -> usize {
458    match bytes.get(start) {
459        Some(b'"') => end_of_string(bytes, start),
460        Some(b'{') => end_of_nested(bytes, start, b'{', b'}'),
461        Some(b'[') => end_of_nested(bytes, start, b'[', b']'),
462        // A bare token: a number, `true`, `false`, `null`, or whatever the
463        // model put there instead. It ends at the first delimiter.
464        _ => {
465            let mut i = start;
466            while i < bytes.len() && !matches!(bytes[i], b',' | b'}' | b']') && !is_space(bytes[i])
467            {
468                i += 1;
469            }
470            i
471        }
472    }
473}
474
475/// The index just past the balanced `open`/`close` pair starting at `start`.
476///
477/// Strings are skipped whole, so a brace inside one does not count — the
478/// difference between reading `{"path": "a}b"}` and reading half of it.
479/// An unbalanced value ends at the end of the input, which keeps a truncated
480/// block's arguments recoverable instead of empty.
481fn end_of_nested(bytes: &[u8], start: usize, open: u8, close: u8) -> usize {
482    let mut depth = 0usize;
483    let mut i = start;
484    while i < bytes.len() {
485        let byte = bytes[i];
486        if byte == b'"' {
487            i = end_of_string(bytes, i);
488            continue;
489        }
490        if byte == open {
491            depth += 1;
492        } else if byte == close {
493            depth = depth.saturating_sub(1);
494            if depth == 0 {
495                return i + 1;
496            }
497        }
498        i += 1;
499    }
500    bytes.len()
501}
502
503/// The index just past the string literal starting at `start`, which must be
504/// its opening quote. An unterminated literal ends at the end of the input.
505fn end_of_string(bytes: &[u8], start: usize) -> usize {
506    let mut i = start + 1;
507    while i < bytes.len() {
508        match bytes[i] {
509            // A `\` consumes the next byte, so `\"` does not end the string.
510            // Landing mid-character is harmless: this scan only ever compares
511            // against ASCII, and every index it hands out is a boundary.
512            b'\\' => i += 2,
513            b'"' => return i + 1,
514            _ => i += 1,
515        }
516    }
517    bytes.len()
518}
519
520/// JSON whitespace.
521fn is_space(byte: u8) -> bool {
522    matches!(byte, b' ' | b'\t' | b'\n' | b'\r')
523}
524
525fn skip_space(bytes: &[u8], mut i: usize) -> usize {
526    while i < bytes.len() && is_space(bytes[i]) {
527        i += 1;
528    }
529    i
530}
531
532/// The text of a quoted span, with JSON escapes resolved where they can be.
533///
534/// `serde_json` does the unescaping when the span is a well-formed literal, so
535/// a `r` in a name is honoured. When it is not — an unterminated or badly
536/// escaped literal — the quotes come off by hand rather than the value being
537/// lost, which is the module's rule everywhere: recover, do not drop.
538fn unquote(raw: &str) -> String {
539    if let Ok(text) = serde_json::from_str::<String>(raw) {
540        return text;
541    }
542    raw.strip_prefix('"')
543        .map_or(raw, |rest| rest.strip_suffix('"').unwrap_or(rest))
544        .to_owned()
545}
546
547#[cfg(test)]
548mod tests {
549    use super::*;
550
551    /// The block the template asks for, around whatever `arguments` says.
552    fn block(arguments: &str) -> String {
553        format!("<tool_call>\n{{\"name\": \"read\", \"arguments\": {arguments}}}\n</tool_call>")
554    }
555
556    #[test]
557    fn a_reply_with_no_calls_is_returned_byte_for_byte() {
558        for reply in [
559            "",
560            "4",
561            "  Sure — the answer is 4.\n\n",
562            "a < b and c > d",
563            "here is a tag: <tool>",
564        ] {
565            let out = extract(reply);
566            assert!(out.calls.is_empty(), "{reply:?} produced {:?}", out.calls);
567            assert_eq!(out.content, reply, "a plain reply must not be rewritten");
568        }
569    }
570
571    #[test]
572    fn one_call_becomes_one_parsed_call_and_empty_content() {
573        let out = extract(&block(r#"{"path": "a.txt"}"#));
574        assert_eq!(out.content, "");
575        assert_eq!(
576            out.calls,
577            vec![ParsedCall {
578                name: "read".to_owned(),
579                arguments: r#"{"path": "a.txt"}"#.to_owned(),
580            }]
581        );
582    }
583
584    #[test]
585    fn two_calls_in_one_turn_are_both_extracted_in_order() {
586        let reply = format!(
587            "{}\n{}",
588            block(r#"{"path": "a"}"#),
589            block(r#"{"path": "b"}"#)
590        );
591        let out = extract(&reply);
592        assert_eq!(out.calls.len(), 2);
593        assert_eq!(out.calls[0].arguments, r#"{"path": "a"}"#);
594        assert_eq!(out.calls[1].arguments, r#"{"path": "b"}"#);
595        assert_eq!(out.content, "");
596    }
597
598    #[test]
599    fn prose_before_between_and_after_survives() {
600        let reply = format!(
601            "I will read both.\n{}\nand the other one\n{}\nThen I will compare them.",
602            block(r#"{"path": "a"}"#),
603            block(r#"{"path": "b"}"#)
604        );
605        let out = extract(&reply);
606        assert_eq!(out.calls.len(), 2);
607        assert_eq!(
608            out.content,
609            "I will read both.\n\nand the other one\n\nThen I will compare them."
610        );
611    }
612
613    #[test]
614    fn an_empty_arguments_object_is_preserved_as_written() {
615        let out = extract(&block("{}"));
616        assert_eq!(out.calls[0].arguments, "{}");
617    }
618
619    /// The absent-`arguments` case is the one place a value is invented, and
620    /// `{}` is invented rather than `""` because the client parses it.
621    #[test]
622    fn a_block_with_no_arguments_key_gets_an_empty_object() {
623        let out = extract("<tool_call>\n{\"name\": \"now\"}\n</tool_call>");
624        assert_eq!(
625            out.calls,
626            vec![ParsedCall {
627                name: "now".to_owned(),
628                arguments: "{}".to_owned(),
629            }]
630        );
631    }
632
633    /// Models emit invalid JSON here routinely. A trailing comma must cost the
634    /// client a parse error it can see, not a silently dropped turn.
635    #[test]
636    fn malformed_json_with_a_recoverable_name_still_becomes_a_call() {
637        let out = extract(&block(r#"{"path": "a.txt",}"#));
638        assert_eq!(out.calls[0].name, "read");
639        assert_eq!(
640            out.calls[0].arguments, r#"{"path": "a.txt",}"#,
641            "the model's bytes reach the client unaltered, invalid or not"
642        );
643        assert_eq!(out.content, "");
644    }
645
646    /// A block truncated inside `arguments` still names a function, so it is
647    /// still a call — with everything that was written of its arguments.
648    #[test]
649    fn a_block_missing_its_closing_brace_keeps_what_arguments_had() {
650        let out = extract(
651            "<tool_call>\n{\"name\": \"read\", \"arguments\": {\"path\": \"a\"\n</tool_call>",
652        );
653        assert_eq!(out.calls[0].name, "read");
654        assert_eq!(out.calls[0].arguments, "{\"path\": \"a\"\n");
655    }
656
657    #[test]
658    fn a_block_with_no_recoverable_name_is_content_markers_and_all() {
659        for body in [
660            "{\"arguments\": {\"path\": \"a\"}}",
661            "{\"name\": \"\"}",
662            "just some words",
663            "",
664        ] {
665            let reply = format!("<tool_call>{body}</tool_call>");
666            let out = extract(&reply);
667            assert!(out.calls.is_empty(), "{body:?} became a call");
668            assert_eq!(out.content, reply, "the block must survive as text");
669        }
670    }
671
672    /// What a reply that hit `max_tokens` mid-block looks like.
673    #[test]
674    fn an_unterminated_block_is_content() {
675        let reply = "Reading it.\n<tool_call>\n{\"name\": \"read\", \"argum";
676        let out = extract(reply);
677        assert!(out.calls.is_empty());
678        assert_eq!(out.content, reply);
679    }
680
681    #[test]
682    fn a_stray_closing_marker_is_content() {
683        let reply = "the tag is </tool_call>, as shown";
684        let out = extract(reply);
685        assert!(out.calls.is_empty());
686        assert_eq!(out.content, reply);
687    }
688
689    /// A naive `find('}')` ends the arguments at the inner object.
690    #[test]
691    fn nested_braces_inside_arguments_are_balanced_not_guessed() {
692        let arguments =
693            r#"{"filter": {"path": "a.txt", "range": {"from": 1, "to": 9}}, "raw": true}"#;
694        let out = extract(&block(arguments));
695        assert_eq!(out.calls[0].arguments, arguments);
696    }
697
698    /// A brace inside a string is text, not structure.
699    #[test]
700    fn a_brace_inside_a_string_argument_does_not_end_the_value() {
701        let arguments = r#"{"pattern": "^\\{.*\\}$", "note": "a } on its own"}"#;
702        let out = extract(&block(arguments));
703        assert_eq!(out.calls[0].arguments, arguments);
704    }
705
706    /// The bug this parser exists to avoid: a terminator quoted inside the
707    /// arguments must not close the block.
708    #[test]
709    fn a_closing_marker_inside_a_string_argument_does_not_close_the_block() {
710        let arguments = r#"{"text": "wrap it in </tool_call> tags"}"#;
711        let out = extract(&block(arguments));
712        assert_eq!(out.calls.len(), 1, "the block closed too early");
713        assert_eq!(out.calls[0].arguments, arguments);
714        assert_eq!(out.content, "");
715    }
716
717    /// The same trap one escape deeper: the quote before the marker is
718    /// escaped, so the string has not ended there.
719    #[test]
720    fn an_escaped_quote_does_not_end_the_string_scan() {
721        let arguments = r#"{"text": "he said \"</tool_call>\" out loud"}"#;
722        let out = extract(&block(arguments));
723        assert_eq!(out.calls.len(), 1);
724        assert_eq!(out.calls[0].arguments, arguments);
725    }
726
727    /// And the reason the string-aware scan has a fallback: an unbalanced
728    /// quote in a body that was never JSON must not swallow the rest of the
729    /// reply.
730    #[test]
731    fn an_unbalanced_quote_does_not_hide_the_terminator() {
732        let reply =
733            "<tool_call>\n{\"name\": \"read\", \"arguments\": {\"path: \"a}\n</tool_call>\nDone.";
734        let out = extract(reply);
735        assert_eq!(out.calls.len(), 1, "the terminator was lost");
736        assert_eq!(out.calls[0].name, "read");
737        assert_eq!(out.content, "Done.");
738    }
739
740    #[test]
741    fn an_escaped_name_is_unescaped_and_its_arguments_are_not() {
742        let reply = r#"<tool_call>{"name": "read", "arguments": {"q": "a\nb"}}</tool_call>"#;
743        let out = extract(reply);
744        assert_eq!(out.calls[0].name, "read");
745        assert_eq!(
746            out.calls[0].arguments, r#"{"q": "a\nb"}"#,
747            "arguments are bytes, not a decoded value"
748        );
749    }
750
751    /// Multibyte text anywhere near the scanner must not split a character.
752    #[test]
753    fn non_ascii_text_around_and_inside_a_block_is_safe() {
754        let reply = format!(
755            "Je vais lire → {}\n…voilà",
756            block(r#"{"path": "café.txt", "note": "naïve"}"#)
757        );
758        let out = extract(&reply);
759        assert_eq!(
760            out.calls[0].arguments,
761            r#"{"path": "café.txt", "note": "naïve"}"#
762        );
763        // Only the ends are trimmed: the space that preceded the block is
764        // prose the model wrote between two words.
765        assert_eq!(out.content, "Je vais lire → \n…voilà");
766    }
767
768    /// The whitespace and key order the model chose reach the client, because
769    /// the client sends them back and they are re-rendered into the prompt.
770    #[test]
771    fn argument_bytes_are_preserved_exactly() {
772        let arguments = "{ \"b\":2,\n  \"a\" : 1 }";
773        let out = extract(&block(arguments));
774        assert_eq!(out.calls[0].arguments, arguments);
775    }
776
777    #[test]
778    fn ids_are_derived_from_the_completion_id_and_the_index() {
779        let calls = vec![
780            ParsedCall {
781                name: "read".to_owned(),
782                arguments: "{}".to_owned(),
783            },
784            ParsedCall {
785                name: "write".to_owned(),
786                arguments: r#"{"path":"a"}"#.to_owned(),
787            },
788        ];
789        let wire = wire_calls("chatcmpl-18f3a0001", &calls);
790        assert_eq!(wire[0].id, "call_18f3a0001_0");
791        assert_eq!(wire[1].id, "call_18f3a0001_1");
792        assert!(wire.iter().all(|call| call.kind == "function"));
793        assert_eq!(wire[1].function.name, "write");
794        assert_eq!(wire[1].function.arguments, r#"{"path":"a"}"#);
795        // Same input, same ids: the round trip is assertable.
796        assert_eq!(wire_calls("chatcmpl-18f3a0001", &calls), wire);
797        // An id without the usual prefix still yields a usable one.
798        assert_eq!(call_id("x", 3), "call_x_3");
799    }
800
801    // ---- the streaming half ----
802
803    /// The id of an ordinary token. Any value that is not a marker will do:
804    /// the state machine only ever compares against the two markers.
805    const TEXT: u32 = 7;
806
807    /// Feed a token sequence through a [`CallStream`] and collect what it would
808    /// have put on the wire: the concatenated content and the calls with their
809    /// indices, including whatever the final flush produced.
810    fn drive(enabled: bool, tokens: &[(u32, &str)]) -> (String, Vec<(usize, ParsedCall)>) {
811        let mut stream = CallStream::new(enabled);
812        let mut content = String::new();
813        let mut calls = Vec::new();
814        for (id, text) in tokens {
815            match stream.push(*id, text) {
816                Step::Nothing => {}
817                Step::Content(text) => content.push_str(&text),
818                Step::Call { index, call } => calls.push((index, call)),
819            }
820        }
821        if let Some(rest) = stream.flush() {
822            content.push_str(&rest);
823        }
824        // Flushing twice must not duplicate the tail.
825        assert_eq!(stream.flush(), None, "flush is not idempotent");
826        assert_eq!(stream.emitted(), calls.len());
827        (content, calls)
828    }
829
830    /// The markers as the tokens they actually are, around one body token.
831    fn streamed(body: &str) -> Vec<(u32, &str)> {
832        vec![
833            (OPEN_TAG_ID, OPEN_TAG),
834            (TEXT, body),
835            (CLOSE_TAG_ID, CLOSE_TAG),
836        ]
837    }
838
839    /// The property the streaming path exists to preserve: the same reply, one
840    /// token at a time, recovers the same call the buffered path recovers.
841    #[test]
842    fn a_streamed_block_recovers_what_the_buffered_path_recovers() {
843        let body = "\n{\"name\": \"glob\", \"arguments\": {\"pattern\": \"*/.rs\"}}\n";
844        let (content, calls) = drive(true, &streamed(body));
845        assert_eq!(content, "");
846        assert_eq!(calls.len(), 1);
847        assert_eq!(calls[0].0, 0);
848        assert_eq!(
849            calls[0].1,
850            extract(&format!("{OPEN_TAG}{body}{CLOSE_TAG}")).calls[0]
851        );
852        assert_eq!(calls[0].1.name, "glob");
853        assert_eq!(calls[0].1.arguments, r#"{"pattern": "*/.rs"}"#);
854    }
855
856    /// A body arrives as many tokens, because a body is many tokens. Only the
857    /// markers are single ids.
858    #[test]
859    fn a_body_split_across_tokens_is_reassembled_whole() {
860        let mut tokens = vec![(OPEN_TAG_ID, OPEN_TAG)];
861        for piece in [
862            "\n{\"na",
863            "me\": \"re",
864            "ad\", \"argu",
865            "ments\": {\"path\": \"a\"}}\n",
866        ] {
867            tokens.push((TEXT, piece));
868        }
869        tokens.push((CLOSE_TAG_ID, CLOSE_TAG));
870        let (content, calls) = drive(true, &tokens);
871        assert_eq!(content, "");
872        assert_eq!(calls[0].1.name, "read");
873        assert_eq!(calls[0].1.arguments, r#"{"path": "a"}"#);
874    }
875
876    #[test]
877    fn prose_before_between_and_after_streams_as_content() {
878        let mut tokens = vec![(TEXT, "I will "), (TEXT, "read both.\n")];
879        tokens.extend(streamed(r#"{"name": "read", "arguments": {"path": "a"}}"#));
880        tokens.push((TEXT, "\nand the other\n"));
881        tokens.extend(streamed(r#"{"name": "read", "arguments": {"path": "b"}}"#));
882        tokens.push((TEXT, "\nDone."));
883
884        let (content, calls) = drive(true, &tokens);
885        assert_eq!(content, "I will read both.\n\nand the other\n\nDone.");
886        assert_eq!(calls.len(), 2);
887        // The index is how a client tells two calls apart, so it counts.
888        assert_eq!(calls[0].0, 0);
889        assert_eq!(calls[1].0, 1);
890        assert_eq!(calls[0].1.arguments, r#"{"path": "a"}"#);
891        assert_eq!(calls[1].1.arguments, r#"{"path": "b"}"#);
892    }
893
894    /// What a reply that hit `max_tokens` mid-block looks like: no call, and
895    /// not one byte of what the model wrote is lost.
896    #[test]
897    fn an_unterminated_streamed_block_flushes_as_content() {
898        let tokens = [
899            (TEXT, "Reading it.\n"),
900            (OPEN_TAG_ID, OPEN_TAG),
901            (TEXT, "\n{\"name\": \"read\", \"argum"),
902        ];
903        let (content, calls) = drive(true, &tokens);
904        assert!(calls.is_empty());
905        assert_eq!(
906            content,
907            "Reading it.\n<tool_call>\n{\"name\": \"read\", \"argum"
908        );
909    }
910
911    /// Without tools the markers are ordinary text: the request never enabled
912    /// calling, so swallowing them would delete the user's answer.
913    #[test]
914    fn a_streamed_block_without_tools_is_ordinary_text() {
915        let body = r#"{"name": "read", "arguments": {}}"#;
916        let (content, calls) = drive(false, &streamed(body));
917        assert!(calls.is_empty());
918        assert_eq!(content, format!("{OPEN_TAG}{body}{CLOSE_TAG}"));
919    }
920
921    /// A closed block that names no function was never a call, and the whole
922    /// block — markers included — reaches the client as text.
923    #[test]
924    fn a_streamed_block_with_no_recoverable_name_is_content_markers_and_all() {
925        for body in ["{\"arguments\": {}}", "{\"name\": \"\"}", "words", ""] {
926            let (content, calls) = drive(true, &streamed(body));
927            assert!(calls.is_empty(), "{body:?} became a call");
928            assert_eq!(content, format!("{OPEN_TAG}{body}{CLOSE_TAG}"));
929        }
930    }
931
932    #[test]
933    fn a_stray_streamed_terminator_is_content() {
934        let tokens = [
935            (TEXT, "the tag is "),
936            (CLOSE_TAG_ID, CLOSE_TAG),
937            (TEXT, ", as shown"),
938        ];
939        let (content, calls) = drive(true, &tokens);
940        assert!(calls.is_empty());
941        assert_eq!(content, "the tag is </tool_call>, as shown");
942    }
943
944    /// A second opener inside an open block is body text, exactly as the
945    /// buffered scan treats it.
946    #[test]
947    fn a_nested_opener_is_body_text() {
948        let tokens = [
949            (OPEN_TAG_ID, OPEN_TAG),
950            (TEXT, "{\"name\": \"echo\", \"arguments\": {\"t\": \""),
951            (OPEN_TAG_ID, OPEN_TAG),
952            (TEXT, "\"}}"),
953            (CLOSE_TAG_ID, CLOSE_TAG),
954        ];
955        let (content, calls) = drive(true, &tokens);
956        assert_eq!(content, "");
957        assert_eq!(calls.len(), 1);
958        assert_eq!(calls[0].1.arguments, "{\"t\": \"<tool_call>\"}");
959    }
960
961    /// Models get this JSON wrong routinely. The call still goes out, with
962    /// whatever the `arguments` span held.
963    #[test]
964    fn malformed_json_in_a_streamed_block_still_yields_a_call() {
965        let (content, calls) = drive(
966            true,
967            &streamed(r#"{"name": "read", "arguments": {"p": 1,}}"#),
968        );
969        assert_eq!(content, "");
970        assert_eq!(calls[0].1.name, "read");
971        assert_eq!(calls[0].1.arguments, r#"{"p": 1,}"#);
972    }
973
974    /// The streaming and buffered paths mint the same ids for the same
975    /// completion, because there is one function that mints them.
976    #[test]
977    fn streamed_call_ids_match_the_buffered_paths() {
978        let body = r#"{"name": "read", "arguments": {}}"#;
979        let mut tokens = streamed(body);
980        tokens.extend(streamed(body));
981        let (_, calls) = drive(true, &tokens);
982
983        let parsed: Vec<ParsedCall> = calls.iter().map(|(_, call)| call.clone()).collect();
984        let buffered = wire_calls("chatcmpl-18f3a0001", &parsed);
985        let streamed: Vec<ToolCall> = calls
986            .iter()
987            .map(|(index, call)| wire_call("chatcmpl-18f3a0001", *index, call))
988            .collect();
989        assert_eq!(streamed, buffered);
990        assert_eq!(streamed[0].id, "call_18f3a0001_0");
991        assert_eq!(streamed[1].id, "call_18f3a0001_1");
992    }
993
994    /// The two constants are the mechanism. If they ever drift from the
995    /// tokenizer's `added_tokens`, streaming silently emits raw markup again.
996    #[test]
997    fn the_marker_ids_are_the_qwen3_added_tokens() {
998        assert_eq!(OPEN_TAG_ID, 151_657);
999        assert_eq!(CLOSE_TAG_ID, 151_658);
1000        assert_ne!(OPEN_TAG_ID, CLOSE_TAG_ID);
1001    }
1002
1003    /// The extracted call, re-rendered on the next turn, is the text the model
1004    /// wrote. That is the whole point of keeping the bytes: the renderer emits
1005    /// `<tool_call>\n{"name": "<name>", "arguments": <raw>}\n</tool_call>`,
1006    /// splicing a `ToolArguments::Raw` verbatim, so a preserved span rebuilds
1007    /// the block character for character.
1008    #[test]
1009    fn a_call_round_trips_back_into_the_block_the_model_emitted() {
1010        let reply = block(r#"{"path": "a.txt"}"#);
1011        let out = extract(&reply);
1012        let wire = wire_calls("chatcmpl-1", &out.calls);
1013        let rendered = format!(
1014            "<tool_call>\n{{\"name\": \"{}\", \"arguments\": {}}}\n</tool_call>",
1015            wire[0].function.name, wire[0].function.arguments
1016        );
1017        assert_eq!(rendered, reply);
1018    }
1019}