car-inference 0.60.0

Local model inference for CAR — Candle backend with Qwen3 models
Documentation
1
{"id":"chatcmpl-9cd55f96","object":"chat.completion","created":1793231179,"model":"mlx-community/gemma-4-E2B-it-qat-4bit","choices":[{"index":0,"message":{"role":"assistant","content":"The software requires a long runtime for proper operation. Users often notice the lengthy runtime during complex tasks. This application demands a substantial runtime for optimal performance. We observed a shorter runtime in testing scenarios."},"finish_reason":"stop"}],"usage":{"prompt_tokens":52,"completion_tokens":40,"total_tokens":92},"generation_metadata":null}