Skip to content
  • Stefy Lanza (nextime / spora )'s avatar
    15d208a7
    longcat: LoRA / QLoRA training, driven through SimpleTuner (v0.2.37) · 15d208a7
    Stefy Lanza (nextime / spora ) authored
    Plan item H.2. Also fixes a real bug shipped in 0.2.35.
    
    THE BUG: load_quantized_dit was imported from longcat_video.modules.longcat_video_dit,
    where it does not exist — it is in longcat_video.modules.quantization (beside
    QuantizedLinear and quantize_model), and it returns the AVATAR transformer, consistent
    with INT8 being an avatar-1.5-only build. The wrong import would have raised at the exact
    moment someone asked for INT8.
    
    TRAINING. coderai does not implement this loop, and the reasons are the design:
    
    1. LongCat's venv is standalone (Python 3.10 / torch 2.6 / transformers 4.41), so a
       trainer there cannot `from codai.api import loras` the way tools/lora_train_worker.py
       does — that overlay venv inherits the parent's site-packages, this one does not.
    2. The upstream repo has NO way to create trainable LoRA layers. Its DiT exposes
       load_lora() / enable_loras() / disable_all_loras() and nothing that initialises one,
       and th...
    15d208a7
    longcat: LoRA / QLoRA training, driven through SimpleTuner (v0.2.37)
    Stefy Lanza (nextime / spora ) authored
    Plan item H.2. Also fixes a real bug shipped in 0.2.35.
    
    THE BUG: load_quantized_dit was imported from longcat_video.modules.longcat_video_dit,
    where it does not exist — it is in longcat_video.modules.quantization (beside
    QuantizedLinear and quantize_model), and it returns the AVATAR transformer, consistent
    with INT8 being an avatar-1.5-only build. The wrong import would have raised at the exact
    moment someone asked for INT8.
    
    TRAINING. coderai does not implement this loop, and the reasons are the design:
    
    1. LongCat's venv is standalone (Python 3.10 / torch 2.6 / transformers 4.41), so a
       trainer there cannot `from codai.api import loras` the way tools/lora_train_worker.py
       does — that overlay venv inherits the parent's site-packages, this one does not.
    2. The upstream repo has NO way to create trainable LoRA layers. Its DiT exposes
       load_lora() / enable_loras() / disable_all_loras() and nothing that initialises one,
       and th...
Loading