-
Stefy Lanza (nextime / spora ) authored
Loading an HF model (e.g. Qwen3.5-9B, 4-bit) failed with 'Permission denied: ./offload'. Cause: the model's per-model offload_dir was the relative './offload' (a stale auto-saved default), which _cfg_or_global lets win over the global config; './offload' resolves to the CWD = the READ-ONLY /opt/coderai/app tree in the image. The config WAS respected — a relative offload path is just meaningless where the CWD isn't writable. * hf_loading.resolve_offload_dir(): an absolute offload_dir is respected as-is; a relative/empty one INHERITS the configured GLOBAL offload directory (global_args.offload_dir) when absolute, then CODERAI_OFFLOAD_DIR, then the user cache — never the CWD. Applied in the manager (both load sites, always passed), hf_loading, and defensively in the cuda backend. * main.py + entrypoint: a container-writable CODERAI_OFFLOAD_DIR (=/cache/offload, created by the entrypoint) is used when the GLOBAL config is still the bare './offload' default; explicit config wins. * run_oci.sh: forward HF_TOKEN / HUGGING_FACE_HUB_TOKEN from the host env so the engines authenticate to the HF Hub (the 'unauthenticated requests' warning) for higher rate limits + gated models. HF_HOME/cache dir was already honoured (main.py from config.models.hf_cache_dir). Bump version to 0.1.8. Co-Authored-By:
Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
fe7da690