tsift-local-model 0.1.78

# tsift-local-model

Local model profile, GPU probe, and lifecycle contracts for tsift Knowledge
Graph extraction.

This crate owns provider-neutral local model substrate types used by future
`tsift-kg`, `tsift-memgraphrag`, and `tsift-agent-doc` integration. It provides
RTX 5090-aware model ranking, best-effort `nvidia-smi` probing, and status
reports without binding callers to Ollama, llama.cpp, or vLLM.

Lifecycle support is provider-neutral:

- `LocalModelLease` records the selected profile, lease mode, pre-load GPU
  probe, provider endpoint or worker pid, idle TTL, and unload strategy.
- `build_unload_actions` describes provider-native cleanup hooks such as
  llama.cpp router unload, Ollama `keep_alive: 0` or `ollama stop`, vLLM sleep,
  and process-exit fallback.
- `evaluate_vram_cleanup` compares pre-load and post-unload GPU probes and
  reports cleanup failure when VRAM stays above the baseline tolerance without
  a known non-tsift GPU process accounting for the increase.

## Provider endpoint configuration (#portconf)

`build_unload_actions` resolves the unload/sleep endpoint with this precedence
(highest first):

1. Per-call explicit override (`--provider-endpoint URL` on
   `tsift local-model unload|swap`).
2. Strategy-specific env var:
   - `TSIFT_LLAMA_CPP_ENDPOINT` (default `http://127.0.0.1:8080/models/unload`)
   - `TSIFT_OLLAMA_ENDPOINT` (default `http://127.0.0.1:11434/api/generate`)
   - `TSIFT_VLLM_ENDPOINT` (default `http://127.0.0.1:8000/sleep`)
3. Compile-time default (the values above).

Blank or whitespace-only env values fall back to the compile-time default so an
empty `TSIFT_LLAMA_CPP_ENDPOINT=` cannot break a run.

**WordPress / 8080 conflict recipe.** llama-server defaults to port 8080, which
collides with a local WordPress instance on the same port. To move tsift's
unload calls to a different port:

```sh
# Start llama-server on an alternate port
llama-server --port 8081 --model Qwen3-32B-GGUF ...

# Point tsift at the new endpoint for the whole shell
export TSIFT_LLAMA_CPP_ENDPOINT="http://127.0.0.1:8081/models/unload"

# Verify the resolved endpoint
tsift local-model unload --profile qwen3-32b-q4 --no-probe \
  --pre-used-mib 200 --post-used-mib 200 --json | \
  python3 -c "import json,sys; print(json.load(sys.stdin)['lease']['unload_actions'][0]['endpoint'])"
# -> http://127.0.0.1:8081/models/unload
```

A single call can still override the env value with
`--provider-endpoint URL`. The same recipe works for Ollama (`TSIFT_OLLAMA_ENDPOINT`)
and vLLM (`TSIFT_VLLM_ENDPOINT`).

## Cooperative GPU lease registry (#gctrl1)

Single-machine cooperative (no daemon) registry that tracks who currently
holds a GPU-bound local model profile. Producers check the file before
probing the GPU, prune stale leases, and either acquire the slot or report a
conflict with the live holder.

- File location: `$TSIFT_LEASE_FILE`, then `$XDG_STATE_HOME/tsift/gpu-lease.json`,
  then `~/.tsift/gpu-lease.json`, then `./.tsift/gpu-lease.json` (override on
  every CLI command via `--lease-file`).
- Schema: `{ version, leases: { profile_id: [GpuLeaseRecord, ...] } }`.
  `GpuLeaseRecord` records `holder_pid`, `holder_command`,
  `acquired_at_unix_seconds`, `lease_mode`, `vram_baseline_mib`,
  `idle_ttl_seconds`, and free-form `notes`.
- Concurrency rules match `LeaseMode`: `Exclusive` (one holder per profile),
  `Shared` (multiple holders per profile), `CpuOrHash` (bypass — no registry
  entry required).
- A holder is stale when its pid is no longer alive (`kill -0`) or when
  `idle_ttl_seconds > 0` and the lease age exceeds the TTL. Stale holders are
  pruned on every acquire/release/show and are reclaimable by a new acquirer.
- Writes are atomic (temp file + `fsync` + rename); concurrent acquires rely
  on cooperative file rotation. There is intentionally no advisory lock — the
  registry is best-effort and a stale entry just gets reclaimed on the next
  acquire.

Pure logic lives in `apply_acquire` / `apply_release` / `prune_stale_leases`,
each taking an `is_alive: impl Fn(u32) -> bool` closure so tests can inject a
deterministic liveness check. File-backed wrappers `acquire_lease`,
`release_lease`, and `show_registry` use the real `is_pid_alive` (which shells
out to `kill -0`).

CLI surface: `tsift local-model lease acquire|release|show`. `acquire --strict`
exits non-zero on conflict so caller scripts can fail closed.

## Per-call profile preference (#gctrl2)

`ProfilePreference` lets a single call pin or downgrade the local model
without mutating global state. It is the per-call dial an agent-doc cycle uses
to request a small/hash profile during low-stakes phases of a long run.

- `ProfilePreference::Auto` — rank by free VRAM (existing behavior).
- `ProfilePreference::Pinned(id)` — pin to a specific profile id. The resolver
  still checks VRAM fit; if the pinned profile does not fit or does not
  support the requested role, the resolver falls back to the hash profile and
  reports `PinnedUnselectable`.
- `ProfilePreference::ForceHash` — force the deterministic CPU/hash fallback
  even when a GPU profile would fit. Shortcut on the CLI: `--profile hash`.

`resolve_profile_preference(preference, role, probe) -> ProfileResolution`
returns the chosen profile, whether it is selectable, a `ProfileResolutionSource`
(`auto_ranked` / `pinned` / `pinned_unselectable` / `forced_hash`), and a
human-readable reason. The hash profile is always a guaranteed-selectable
fallback so a call never has to abort because of a bad pin.

CLI surface:
- `tsift local-model resolve [--profile <id|hash>] [--role extract|embed|rerank] [--no-probe] [--json]`
  — the decision seam. Callers run this once to learn which profile a
  subsequent command would use.
- `tsift semantic --profile <id|hash>` and `tsift summarize --extract --profile <id|hash>`
  — informational plumbing. The resolved preference is recorded as a warning
  note on the response envelope (`profile preference <kind> -> <id> (<reason>)`)
  so a future provider seam can consume it. Until that seam lands, calls
  still use the existing cached/hash code paths.

## Profile swap lifecycle (#gctrl3)

`tsift local-model swap --from <id> --to <id>` is the one-command mid-run
downgrade path. It combines a source unload cleanup proof with a target
profile resolution against the post-unload probe, so a caller can decide in
one step whether it is safe to load the next profile (typically
`qwen3-32b-q4` → `qwen3-embedding-0.6b` or the hash fallback) without
orchestrating `unload` + `status` separately.

`SwapStatus` captures the combined outcome:

- `Swapped` — source unload cleanup proven AND target profile fits the post-unload probe.
- `SwappedToHash` — target is the CPU/hash profile; permitted once unload is proven.
- `UnloadProvenTargetUnselectable` — unload ok but target does not fit; caller should pick a smaller profile or hash.
- `UnloadNotProven` — source unload cleanup NOT proven; caller MUST NOT load the target. The report's `notes` include `DO NOT load target — source unload did not prove VRAM cleanup`.
- `NoOpSameProfile` — source and target are the same id.

Lease coordination stays the caller's job — they hold the holder-pid context
and can chain `tsift local-model lease release --profile <from>` → `swap` →
`tsift local-model lease acquire --profile <to>` in a script.

CLI surface: `tsift local-model swap --from <id> --to <id> [--provider-endpoint URL] [--provider-pid PID] [--idle-ttl-seconds S] [--no-probe] [--pre-used-mib M] [--post-used-mib M] [--tolerance-mib M] [--strict] [--json]`. `--strict` exits non-zero on `UnloadNotProven` or `UnloadProvenTargetUnselectable` so caller scripts fail closed.