• Stefy Lanza (nextime / spora )'s avatar
    longcat: place the lazily realised text encoder (v0.2.81 fixup) · bfcd3f22
    Stefy Lanza (nextime / spora ) authored
    A deferred encoder is not there when _place_pipeline runs, so pipe.to("cuda")
    has nothing to move. Under offload that was already right — the encoder belongs
    on the host and the wrapper brings it over for the encode — but with
    offload_strategy "" the encoder was built on the CPU and left there, while
    _get_t5_prompt_embeds sends it inputs on the card. The first cache miss would
    have died on a device mismatch.
    
    Placement cannot be re-derived on a miss: by then the offload argument is long
    gone. So _place_pipeline records where a realised encoder belongs, in each of
    its branches, and realise() replays it.
    
    A quantised encoder is exempt — device_map put it on the card as it loaded and
    bitsandbytes weights are not meant to be moved afterwards. That check now lives
    in _is_quantised_module, which _encoder_is_quantised delegates to, because a
    lazily realised encoder has to be asked the same question BEFORE it is attached
    to a pipeline and the two answers must not drift.
    
    test_quantisation_is_detected_from_the_model_not_the_config followed the names
    into the helper and now asserts the delegation too, so neither function can
    start consulting the config.
    
    Found by re-reading the committed diff, not by a test: nothing here runs the
    real pipeline, which is exactly the gap that let it through.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    bfcd3f22
Name
Last commit
Last update
..
test_admin_auth.py Loading commit data...
test_admin_remote_ui.py Loading commit data...
test_audio_ml_backends.py Loading commit data...
test_audio_ml_endpoints.py Loading commit data...
test_broker_config.py Loading commit data...
test_broker_dispatch.py Loading commit data...
test_broker_protocol.py Loading commit data...
test_broker_streaming.py Loading commit data...
test_burst_placement.py Loading commit data...
test_character_studio_fetch.py Loading commit data...
test_compose.py Loading commit data...
test_front_ocr_swap_gate.py Loading commit data...
test_image_has_c_compiler.py Loading commit data...
test_longcat_avatar_layout.py Loading commit data...
test_longcat_bsa.py Loading commit data...
test_longcat_cancellable.py Loading commit data...
test_longcat_checkpoint_resolve.py Loading commit data...
test_longcat_component_quant.py Loading commit data...
test_longcat_config_ui.py Loading commit data...
test_longcat_external_training.py Loading commit data...
test_longcat_int8_load.py Loading commit data...
test_longcat_lora.py Loading commit data...
test_longcat_orchestration.py Loading commit data...
test_longcat_packaging.py Loading commit data...
test_longcat_placement.py Loading commit data...
test_longcat_prompt_cache.py Loading commit data...
test_longcat_quantize.py Loading commit data...
test_longcat_scale.py Loading commit data...
test_longcat_tasks.py Loading commit data...
test_longcat_variants.py Loading commit data...
test_longcat_vendored_source.py Loading commit data...
test_longcat_venv_env.py Loading commit data...
test_longcat_vram_estimate.py Loading commit data...
test_longcat_worker.py Loading commit data...
test_manual_multimodal_test_client.py Loading commit data...
test_model_scan_cache.py Loading commit data...
test_ocr_olmocr.py Loading commit data...
test_ocr_vram_share_and_fallback.py Loading commit data...
test_remote_execution.py Loading commit data...
test_runpod_ocr_remotization.py Loading commit data...
test_studio_composed_surfaces.py Loading commit data...
test_township_api_key.py Loading commit data...
test_township_longcat.py Loading commit data...
test_township_page_js.py Loading commit data...
test_township_templates.py Loading commit data...
test_township_video_dataset.py Loading commit data...
test_upgrade_launcher.py Loading commit data...
test_video_as_b64_scope.py Loading commit data...
test_vllm_share_self_tuning.py Loading commit data...
test_vram_releasers.py Loading commit data...
test_weight_stores.py Loading commit data...
test_whisper_server_local_models.py Loading commit data...