-
Stefy Lanza (nextime / spora ) authored
Escalation alone only ratchets upward, so an opening bid that is too generous was permanent: 0.60 of a 24 GB card for an engine that needs 0.25 is 8 GB no other model could have. A successful boot states how much of the budget was left for the cache, and `footprint = budget - cache` is the only direct measurement of an engine's real cost there is — so it is now read back and used. - _pump_logs captures "Available KV cache memory: N GiB" as it STREAMS. By the time the server answers its health check that line has scrolled out of the 80-line tail behind vLLM's route listing, so it cannot be recovered afterwards. - On a boot with clear slack (> 1 GiB beyond the ~2 GiB target), a smaller share is computed from the measured footprint and recorded for the next start. - The learned store is now a record per key, {"ok": booted, "failed": too small}, and the downward half may never propose a share at or below "failed" + a step. That floor is what stops it oscillating between a budget that works and one that does not. - Over-provisioning now costs one boot instead of the life of the process, and the flat float format the previous version wrote still loads. 6 more tests (shrinking, the tight-fit no-op, the anti-oscillation floor, a missing cache figure, the old format, and the footprint round-trip): 16 in the file, 255 across the OCR/gate/cluster/remote-execution/placement files together. Co-Authored-By:Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
bb2389ff