• Stefy Lanza (nextime / spora )'s avatar
    download HuggingFace models where the config says, in every process (v0.4.88) · c4539260
    Stefy Lanza (nextime / spora ) authored
    A deployment has to point HF_HOME somewhere before any config is read, so a
    container bakes it into the image. models.hf_cache_dir was then applied in
    exactly one place -- inside the server's main(), by mutating os.environ, which
    reaches no other process. So the front used the configured cache and everything
    else used the image's: supervisord's other children, and the isolated-venv
    workers, which were handed HF_HUB_CACHE and HUGGINGFACE_HUB_CACHE but not
    HF_HOME or TRANSFORMERS_CACHE, enough for a worker to re-fetch its model into
    the directory the image named.
    
    Measured on one install: 22 GB in the image's directory, 13 GB of it a
    byte-for-byte duplicate of the configured cache, three copies of one model, and
    the whole of it on the smaller of the two disks -- which was 99% full.
    
    The configured value is now applied from codai/__init__.py, before anything can
    import huggingface_hub, transformers or torch and read these variables once at
    import time. Every coderai process imports the package, so every one of them
    agrees without the launcher being involved. All four variables, since leaving
    any behind splits the cache again. main() keeps an authoritative pass against
    the config it actually loaded (--config may name another directory), the venv
    workers get the same four through child_env(), and the OCR fleet supervisor --
    stdlib-only on purpose -- reads the same key from the config.json it already
    opens, instead of building the path from CODERAI_CACHE_DIR.
    
    Nothing is invented: with no hf_cache_dir configured the environment is left
    exactly as found, and CODERAI_NO_HF_CACHE_ENV opts a process out.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
    c4539260