Skip to content
Commit ecf9bbec authored by Stefy Lanza (nextime / spora )'s avatar Stefy Lanza (nextime / spora )
Browse files

release the injected components BEFORE the teardown, not after



The release added in 9715d200 ran after _free_pipeline_vram, which is too
late: that function ends in the teardown-leak breaker, which walks live CUDA
tensors and nulls whatever still references them. By the time the release
ran the components had already been stripped, so it found nothing on the GPU
and printed nothing, while the breaker reported the full

    16.9 GB of this pipeline's tensors survived teardown;
    nulled 7581 external reference(s)

exactly as before the fix. Verified live 2026-10-29: the new line never
appeared and the figure was unchanged.

The ordering is not cosmetic, because what the breaker nulls is the modules'
own _parameters. It frees the VRAM and leaves the injected components gutted
-- and those components are reused by the very next attempt. accelerate's
offload hook then does

    current_device = next(module.parameters()).device

on a module with no parameters left, which raises StopIteration, which in a
worker thread becomes "StopIteration interacts badly with generators" and
fails the generation. Measured: the run whose resident attempt failed died
that way at encode_prompt, while the run that skipped the resident attempt
(and so never ran the breaker) generated normally on the same pipeline.

So the leak, the gutted components and that crash are one bug seen at three
removes. Releasing first keeps the components intact, leaves the breaker with
nothing to strip, and the retry gets working weights.

Co-Authored-By: default avatarClaude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC
parent 9715d200
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment