• Stefy Lanza (nextime / spora )'s avatar
    pods: a compiler in the shared capability base, for Triton (v0.2.80) · ec6f5c30
    Stefy Lanza (nextime / spora ) authored
    Sweeping the pod images for the gap the LongCat one had: Triton compiles its
    CUDA driver module at RUNTIME, and only vllm and engines-kt carried a compiler.
    Thirteen other torch-bearing profiles did not.
    
    The risk is not theoretical and not confined to block-sparse attention.
    core.txt — which every capability image installs — carries bitsandbytes, and
    bitsandbytes reaches Triton's autotuner on import; that is how the NF4
    verification failed first time round, with "libcuda.so cannot found!" and then a
    failed gcc invocation. On a pod the first quantised forward would have died with
    "Failed to find C compiler", exactly as Surya did locally before v0.2.62.
    
    So gcc and libc6-dev go into Dockerfile.capability-base, where every profile
    image inherits them, rather than into thirteen profiles one at a time.
    
    capability-base-light is deliberately left without one: it ships no torch, so it
    has no Triton to build for. That is precisely why the LongCat image installs its
    own — it adds a 3.10 venv with torch and triton to a parent that has neither —
    and there is now a test saying so, so a later cleanup does not "simplify" it
    away.
    
    Also: the quantised text-encoder load now reads torch.cuda.current_device()
    instead of hardcoding 0. The worker masks the card with CUDA_VISIBLE_DEVICES so
    0 was right here, but assuming it would not survive an install that does its
    device selection differently.
    Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_0185zEBiy3KWB37xNGSrxL7w
    ec6f5c30
Name
Last commit
Last update
..
test_admin_auth.py Loading commit data...
test_admin_remote_ui.py Loading commit data...
test_audio_ml_backends.py Loading commit data...
test_audio_ml_endpoints.py Loading commit data...
test_broker_config.py Loading commit data...
test_broker_dispatch.py Loading commit data...
test_broker_protocol.py Loading commit data...
test_broker_streaming.py Loading commit data...
test_burst_placement.py Loading commit data...
test_character_studio_fetch.py Loading commit data...
test_compose.py Loading commit data...
test_front_ocr_swap_gate.py Loading commit data...
test_image_has_c_compiler.py Loading commit data...
test_longcat_avatar_layout.py Loading commit data...
test_longcat_bsa.py Loading commit data...
test_longcat_cancellable.py Loading commit data...
test_longcat_checkpoint_resolve.py Loading commit data...
test_longcat_component_quant.py Loading commit data...
test_longcat_config_ui.py Loading commit data...
test_longcat_external_training.py Loading commit data...
test_longcat_int8_load.py Loading commit data...
test_longcat_lora.py Loading commit data...
test_longcat_orchestration.py Loading commit data...
test_longcat_packaging.py Loading commit data...
test_longcat_placement.py Loading commit data...
test_longcat_quantize.py Loading commit data...
test_longcat_scale.py Loading commit data...
test_longcat_tasks.py Loading commit data...
test_longcat_variants.py Loading commit data...
test_longcat_vendored_source.py Loading commit data...
test_longcat_venv_env.py Loading commit data...
test_longcat_vram_estimate.py Loading commit data...
test_longcat_worker.py Loading commit data...
test_manual_multimodal_test_client.py Loading commit data...
test_model_scan_cache.py Loading commit data...
test_ocr_olmocr.py Loading commit data...
test_ocr_vram_share_and_fallback.py Loading commit data...
test_remote_execution.py Loading commit data...
test_runpod_ocr_remotization.py Loading commit data...
test_studio_composed_surfaces.py Loading commit data...
test_township_api_key.py Loading commit data...
test_township_longcat.py Loading commit data...
test_township_page_js.py Loading commit data...
test_township_templates.py Loading commit data...
test_township_video_dataset.py Loading commit data...
test_upgrade_launcher.py Loading commit data...
test_video_as_b64_scope.py Loading commit data...
test_vllm_share_self_tuning.py Loading commit data...
test_vram_releasers.py Loading commit data...
test_weight_stores.py Loading commit data...
test_whisper_server_local_models.py Loading commit data...