pub const REQUEST_CAP: usize = _ /* 262_144usize */;Expand description
The most a request may be, in bytes: the conversation, the system prompt and the tools together. Four bytes to a token is a good rough count of plain text, so this is about sixty five thousand tokens of input, which fits a hundred and twenty eight thousand token context with this crate’s reply still allowed. A real conversation is far below it, so a normal day is sent byte for byte as it always was; a morning of work in a real folder is not, and without a limit that size is paid for again on every single turn, and a provider that refuses a request too long says so only in an error in the corner.