• Stefy Lanza (nextime / spora )'s avatar
    hf: resolve relative offload_dir to the configured dir; forward HF_TOKEN · fe7da690
    Stefy Lanza (nextime / spora ) authored
    Loading an HF model (e.g. Qwen3.5-9B, 4-bit) failed with 'Permission denied:
    ./offload'. Cause: the model's per-model offload_dir was the relative './offload'
    (a stale auto-saved default), which _cfg_or_global lets win over the global
    config; './offload' resolves to the CWD = the READ-ONLY /opt/coderai/app tree in
    the image. The config WAS respected — a relative offload path is just meaningless
    where the CWD isn't writable.
    
    * hf_loading.resolve_offload_dir(): an absolute offload_dir is respected as-is; a
      relative/empty one INHERITS the configured GLOBAL offload directory
      (global_args.offload_dir) when absolute, then CODERAI_OFFLOAD_DIR, then the user
      cache — never the CWD. Applied in the manager (both load sites, always passed),
      hf_loading, and defensively in the cuda backend.
    * main.py + entrypoint: a container-writable CODERAI_OFFLOAD_DIR (=/cache/offload,
      created by the entrypoint) is used when the GLOBAL config is still the bare
      './offload' default; explicit config wins.
    * run_oci.sh: forward HF_TOKEN / HUGGING_FACE_HUB_TOKEN from the host env so the
      engines authenticate to the HF Hub (the 'unauthenticated requests' warning) for
      higher rate limits + gated models. HF_HOME/cache dir was already honoured
      (main.py from config.models.hf_cache_dir).
    
    Bump version to 0.1.8.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01RdMufYvtTbtGDWsiZVoXce
    fe7da690
hf_loading.py 17.7 KB