• Stefy Lanza (nextime / spora )'s avatar
    release the injected components BEFORE the teardown, not after · ecf9bbec
    Stefy Lanza (nextime / spora ) authored
    The release added in 9715d200 ran after _free_pipeline_vram, which is too
    late: that function ends in the teardown-leak breaker, which walks live CUDA
    tensors and nulls whatever still references them. By the time the release
    ran the components had already been stripped, so it found nothing on the GPU
    and printed nothing, while the breaker reported the full
    
        16.9 GB of this pipeline's tensors survived teardown;
        nulled 7581 external reference(s)
    
    exactly as before the fix. Verified live 2026-10-29: the new line never
    appeared and the figure was unchanged.
    
    The ordering is not cosmetic, because what the breaker nulls is the modules'
    own _parameters. It frees the VRAM and leaves the injected components gutted
    -- and those components are reused by the very next attempt. accelerate's
    offload hook then does
    
        current_device = next(module.parameters()).device
    
    on a module with no parameters left, which raises StopIteration, which in a
    worker thread becomes "StopIteration interacts badly with generators" and
    fails the generation. Measured: the run whose resident attempt failed died
    that way at encode_prompt, while the run that skipped the resident attempt
    (and so never ran the breaker) generated normally on the same pipeline.
    
    So the leak, the gutted components and that crash are one bug seen at three
    removes. Releasing first keeps the components intact, leaves the breaker with
    nothing to strip, and the retry gets working weights.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC
    ecf9bbec
test_injected_components_released.py 5.04 KB