modelc 0.1.8

Rust CLI that compiles LLM weights (GGUF, Safetensors, ONNX, PyTorch) into a single .modelc artifact and serves a local OpenAI-compatible inference API with Metal GPU and CPU SIMD acceleration.
Documentation
1
2
3
4
5
6
7
it is a rust base cli that help to package model file into a single file
the outcome should be a single file that contains all the model data and can be loaded by the cli
the outcome should be optimize on size and footprint and performance
the outcome should be compatible with different model formats
the outcome should be similar usage as ollama/vllm/sglang
the outcome should be able to run on different platforms (macos, windows, linux)
the outcome should be able to use apple silicon m series chip for acceleration