• Stefy Lanza (nextime / spora )'s avatar
    normalise a language before transcribing, stop inflating weights that are... · 3f9b2680
    Stefy Lanza (nextime / spora ) authored
    normalise a language before transcribing, stop inflating weights that are already quantized, and never answer 200 with nothing (v0.5.9)
    
    Five bugs and five features from a client report. The bugs share a shape: the
    server held a belief about a model that nothing checked against the model.
    
    A1. A TTS voice is chosen for an accent, so the language a compose render
        carries is "en-us" / "pt-br". Passed to transformers' Whisper -- which
        accepts only the English NAME -- it raised "Unsupported language: en-us"
        and 500'd every captioned scene of every render. Reproduced verbatim
        against the running server before the fix. New codai/api/lang_codes.py
        normalises once, per backend: the NAME for the transformers pipeline, the
        ISO code for whisper.cpp, Canary and faster-whisper, and None (auto-detect)
        for a tag that maps to nothing -- refusing to transcribe because the caller
        named a dialect is never the better outcome. Language VALIDATION now
        compares by meaning, so a model declaring "english" no longer refuses
        "en-gb", and compose's model picker stops disqualifying the only model that
        can time words.
    
    A3. FLUX.2 (dev/klein/pro) is a unified generate-and-edit model whose name
        carries neither "edit" nor "kontext", so it fell through to the generation
        branch and advertised plain img2img -- telling clients to show a `strength`
        slider for a pipeline that has no such argument. Confirmed live:
        ['image_generation','image_to_image','inpainting']. diffusers 0.37 agrees
        it is an instruction edit (Flux2Pipeline takes `image`, not `strength`),
        which is what the runtime already decided on; only the advertised list
        disagreed.
    
    A4. The estimator normalises a disk size to the load precision. A GGUF was
        special-cased to 1.0 because its bytes ARE its footprint; an HF checkpoint
        published pre-quantized is the same case by another route and was not.
        Z-Image-Turbo-unsloth-bnb-4bit: 16.3 GB of 4-bit weights, 1.70x factor,
        estimated 30.03 GB against a 24 GB card. Nothing can free 30 of 24, so
        every still ran partly on the CPU -- ~72 s for one image from a few-step
        turbo model.
    
    A5. Qwen3.5-9B answered content "" with HTTP 200 for four weeks. All three
        rungs of the generation ladder raised ("Pointer argument (at 0) cannot be
        accessed from Triton (cpu tensor?)") and the last one assigned
        generated_text = "" and returned normally. Now raises. The Triton error is
        a quantized model whose weights were CPU-offloaded: such a load succeeds
        and then fails every request, so it is refused at load time and a quantized
        model tries full-GPU whatever the estimate says. That also breaks a loop --
        measured_vram_gb = GPU + offloaded + reserve means one offloaded load writes
        a number bigger than the card, and the next load reads it and offloads again.
    
    B1. A scene can carry its clip's own sound: visual.keep_audio, scene.audio_mix
        (voice|clip|both) and clip_volume. "clip" makes the scene as long as the
        clip and skips the TTS call; "both" ducks the clip under the voice. Folded
        into the narration track, so the music bed ducks under it like a voice.
    
    B2. Two characters talking: scene.speakers, each line in its own voice, each
        character lip-synced in their own column (or the frame cutting to whoever
        speaks). One sync run per CHARACTER, not per line -- each gets a track
        holding only their own lines, so the mouth moves on their words and stays
        closed while the other talks. Captions never mix two speakers and carry
        the attribution into the .srt/.vtt.
    
    B3. characters: ["alice","bob"] on images and video. The gap was never the
        field: IP-Adapter conditions on the MEAN of its references, so a second
        character's pictures produced one person who looked like neither. Reference
        sets are kept apart and handed to a model that takes several (FLUX.2,
        Qwen-Image-Edit); a model that cannot do it says so instead of blending.
    
    B4. POST /v1/moderation -- nsfw, faces, age, face_match over images or sampled
        video frames. It fails CLOSED: a missing model is a 503 naming it, not a
        result with the check quietly absent, because a caller cannot tell a
        missing check from a passed one. minor_detected is nullable (explicit
        content with no face cannot be cleared on age) and keys on the LOW end of
        an age interval, not a point estimate from a model with a multi-year error.
    
    C5. Per-model defaults (steps, guidance, sampler) that fill in only what the
        caller omitted, published in /v1/models alongside modes and licence -- so a
        client stops keeping a table keyed on the model NAME and an 8-step model
        stops being run for 30. A sampler switcher that refuses to cross the
        flow-matching boundary (a FLUX-family pipeline handed a DDIM produces noise,
        not a worse picture). Stable Audio gets its own pipeline: it was mapped to
        AudioLDM2, so a configured entry could only fail at load.
    
    Two of my own regressions, caught by the suite's guard tests and fixed here:
    /v1/moderation was unmapped in the remote gateway, and the moderation
    availability probe LOADED each model -- which turned the /v1/models listing
    into a 350 MB download on the event loop.
    
    Suite: 15 failed, 5146 passed -- the same 15 as the baseline.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
    3f9b2680
test_stt_language_normalization.py 5.84 KB