• Stefy Lanza (nextime / spora )'s avatar
    serve the LTX-2 family, fetch a Civitai checkpoint, and stop walking 900 GB of... · 9ae240b0
    Stefy Lanza (nextime / spora ) authored
    serve the LTX-2 family, fetch a Civitai checkpoint, and stop walking 900 GB of cache on the event loop
    
    Model installation work from the client's C list, plus one latency bug found
    on the way.
    
    LTX-2 is wired. This diffusers has LTX2Pipeline, LTX2ImageToVideoPipeline, a
    video AND an audio VAE — but video.py only mapped the LTX 0.9 family, so every
    LTX-2 entry in models.json sat parked: the name matched and the class did not.
    Split on the version in the name, since both families are just called "LTX"
    and a fine-tune keeps its base version (10Eros is LTX2.3, PinkCherry is
    LTX23).
    
    Stable Audio had its own entry in the type map pointing at AudioLDM2Pipeline —
    a different architecture with a different projection model and sample rate, so
    a configured stable-audio-open entry could only ever fail at load. It has its
    own loader now, keeps the stereo field (folding it to mono throws away the
    point of the model) and clips before the int16 cast, because a float above 1.0
    wraps to a loud negative sample and is heard as a click.
    
    ACE-Step gets requirements-acestep.txt and an explicit refusal naming what it
    needs, rather than being loaded as a MusicGen and failing with a shape error
    three minutes later. It ships its own package pinning a stack this venv cannot
    hold, so it needs an isolated-venv worker like CrisperWhisper and Canary; that
    worker does not exist yet and the refusal says so.
    
    A GGUF component that fails to load now fails the load. It was skipped with a
    log line — and a skipped component is simply absent from the dict, so the base
    pipeline loaded its own full-precision copy: the operator asked for a 14 GB
    quantized transformer and got a 38 GB bf16 one, while the VRAM estimate still
    said 14 GB and the planner therefore freed about a third of what the load
    needed. A silent downgrade to something that does not fit is worse than a
    refusal.
    
    `civitai:<modelVersionId>` references, because a large part of the community's
    fine-tunes are published only there, as one ComfyUI-style checkpoint — exactly
    the shape build_single_file_pipeline already handles. The only thing missing
    was a way to fetch one. The VERSION id, not the model id: a Civitai "model" is
    a family, and an entry naming the family would silently change underneath the
    operator on a new release. Content type and size are both checked, because an
    unauthenticated download answers with a web page at HTTP 200 and a naive fetch
    saves it under a .safetensors name — which surfaces hours later as an
    unreadable tensor header.
    
    LatentSync and MuseTalk are first-class presenter engines, ahead of wav2lip
    for a clip: wav2lip works at 96x96 and upscales, which is where the soft mouth
    on a 1080p reel comes from. Both are video-driven, so sadtalker still leads
    for a still portrait — it is the one that invents head motion. Recognised from
    a shim on PATH or a managed install whose weights are already present, never
    "could be downloaded": a compose job is the wrong place to fetch 5 GB.
    
    And the loop stall. huggingface_hub.scan_cache_dir is not a listing: it stats
    every blob of every revision of every repo, measured at 27.3 SECONDS on this
    machine's 899 GB cache. It was called from a per-model VRAM estimate, a GGUF
    path lookup, and app.internal_engine_state — which is POLLED — all
    synchronously on the event loop. The watchdog's verdict was 6631 stalls over
    0.5s with a worst of 13.0s, during every one of which no request of any kind
    was served. Fixed with both halves, because either alone is insufficient: the
    scan is memoised for 30s (short, so a model downloaded while the server runs
    still becomes visible) and the engine-state poll runs it through
    asyncio.to_thread, since memoising alone still puts the first 27-second call
    on the loop.
    
    Also: /v1/models reads `modes` from a video entry's long-standing
    `video_subtypes` as well as the newer spellings.
    
    Suite: 15 failed, 5194 passed — the same 15 as the baseline.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
    9ae240b0
test_hf_cache_scan_memoised.py 4.01 KB