video: param-weighted VRAM estimate, smarter auto offload, runtime accel LoRA

VRAM estimation (manager.py):
- Weight the effective quant multiplier by REAL per-component parameter
  shares (new _component_param_shares scans safetensors by component folder)
  instead of a blind 70/30 split. Wan2.2 is 99.6% quantizable (two 14B
  experts + text encoder 4-bit, only the 0.13B VAE dense), so the old 0.475x
  multiplier inflated ~25.8 GB -> 42.7 GB and forced needless offload. Now
  ~0.28x -> ~25.8 GB. VAE forced dense (conv-only, bnb can't quantize).

Auto offload decision (video.py):
- 'auto': when peak footprint exceeds free VRAM, go straight to `model` CPU
  offload (active component on GPU, near full-GPU speed) — no full-GPU gamble,
  no slow balanced+disk path.
- 'auto-borderline' (new mode): same, except a marginal overshoot (<=3 GB)
  tries full-GPU first to keep both experts resident and use free VRAM,
  falling back to model offload on OOM.

Acceleration LoRA (acceleration.py + video.py):
- Keep the distill/Lightning LoRA as an ACTIVE RUNTIME ADAPTER instead of
  fusing. Fusing into CPU-offloaded bitsandbytes 4-bit weights triggers a
  dequant->merge->requant per Linear on the CPU — minutes/hours per expert,
  appearing to hang (high CPU, empty VRAM). Runtime adapters apply at forward
  time on the GPU at negligible cost and natively cover transformer_2.
- _sync_video_loras preserves the accel adapters across per-request LoRA swaps
  and re-includes them in every set_adapters; _unload_video_loras deletes only
  per-request adapters, keeping accel.

UI (models.html):
- Add "Auto borderline-aware" offload strategy option + updated hint.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

codai/admin/templates/models.html

@@ -671,15 +671,17 @@ window.__DEFAULT_WHISPER_SERVER_PATH__ = {{ default_whisper_server_path|tojson }
         <div class="form-row" style="margin:0">
           <label class="form-label">Strategy</label>
           <select id="cfg-offload-strategy" class="form-input">
-            <option value="auto">Auto (pick from free VRAM)</option>
+            <option value="auto">Auto (over-VRAM → straight to model offload)</option>
+            <option value="auto-borderline">Auto borderline-aware (try full-GPU if marginally over)</option>
             <option value="none">None (GPU only)</option>
+            <option value="model">CPU offload (model — active component on GPU)</option>
+            <option value="group">Group offload + stream (block-level, prefetched)</option>
             <option value="balanced">Balanced (fill GPU, spill to CPU → disk)</option>
-            <option value="model">CPU offload (model — module-by-module)</option>
             <option value="sequential">CPU offload (sequential — most aggressive)</option>
             <option value="cpu">CPU RAM (legacy)</option>
             <option value="disk">Disk</option>
           </select>
-          <span class="form-hint">Auto picks full-GPU when the weights fit, else Balanced. Pick <b>Balanced</b> + lower the GPU % below for a model that's just over VRAM; <b>model</b>/<b>sequential</b> keep less on GPU (slower, but the safest fit).</span>
+          <span class="form-hint"><b>Auto</b> goes straight to <b>model</b> offload when the model's peak exceeds free VRAM (no full-GPU gamble). <b>Auto borderline-aware</b> is the same, except when the model only marginally overshoots (≤3 GB) it tries full-GPU first to keep both experts resident and use the free VRAM, falling back to model offload on OOM. <b>model</b> keeps only the active component on GPU (best for two-expert Wan2.2 — near full-GPU speed). <b>group</b> streams blocks with prefetch (lowest VRAM, fast; incompatible with bitsandbytes 4-bit → falls back to sequential). <b>balanced</b> fills GPU to the % below then spills.</span>
         </div>
         <div class="form-row" style="margin:0">
           <label class="form-label">Offload directory</label>

codai/models/acceleration.py

@@ -325,32 +325,31 @@ def apply_accel_to_pipeline(pipe, accel: Optional[dict]) -> None:
             raise RuntimeError("no distill adapter registered on the pipeline")
         try:
             pipe.set_adapters(loaded_adapters, [weight] * len(loaded_adapters))
-        except Exception:
-            pass
-        # Bake them in, then drop the adapter handles so per-request LoRAs are clean.
-        # CRITICAL: diffusers' Wan fuse_lora defaults to components=["transformer"],
-        # so without naming transformer_2 the low-noise expert's distill adapter is
-        # never fused — and the subsequent unload strips it off, leaving that expert
-        # undistilled. At 4 steps that collapses the clip to a solid colour. Fuse
-        # BOTH experts explicitly.
-        _fuse_components = ["transformer"]
-        if has_t2:
-            _fuse_components.append("transformer_2")
-        try:
-            pipe.fuse_lora(components=_fuse_components, lora_scale=weight)
-        except TypeError:
-            # Older diffusers without the `components` kwarg — best effort.
-            pipe.fuse_lora(lora_scale=weight)
-        try:
-            pipe.unload_lora_weights()
-        except Exception:
-            pass
+        except Exception as e:
+            log.warning("[accel] could not activate distill adapters %s: %s",
+                        loaded_adapters, e)
+        # Keep the distill LoRA(s) as ACTIVE RUNTIME ADAPTERS rather than fusing.
+        #
+        # Fusing merges the adapter into the base weights. For a bitsandbytes 4-bit
+        # model under CPU/disk offload the weights live on the CPU, so fuse_lora has
+        # to dequantize → merge → requantize every Linear on the CPU — minutes-to-
+        # hours per 14B expert, which looks exactly like a hang (high CPU, empty
+        # VRAM, no progress). Runtime adapters instead apply at forward time on
+        # whichever device the module is currently on (the GPU during the pass), at
+        # negligible cost, and `set_adapters` natively covers transformer_2 — so the
+        # low-noise expert is distilled too (the old fuse path needed an explicit
+        # components=[...,"transformer_2"] or it collapsed to a solid colour).
+        #
+        # `_sync_video_loras` preserves these accel adapters across per-request LoRA
+        # swaps and re-includes them in every set_adapters call.
+        pipe._coderai_accel_adapters = list(loaded_adapters)
+        pipe._coderai_accel_weight = weight
         try:
-            pipe._coderai_accel_fused = True
+            pipe._coderai_accel_fused = True  # accel is effective (distilled preset applies)
         except Exception:
             pass
-        log.info("[accel] fused distillation LoRA(s) %s (weight=%s) into %s%s",
-                 loaded_adapters, weight, type(pipe).__name__,
+        log.info("[accel] activated distillation LoRA(s) %s (weight=%s, runtime adapters) "
+                 "on %s%s", loaded_adapters, weight, type(pipe).__name__,
                  " (both experts)" if len(loaded_adapters) > 1 else "")
     except Exception as e:
         log.warning("[accel] failed to fuse acceleration LoRA (high=%s low=%s): %s "

codai/models/manager.py

@@ -1892,26 +1892,62 @@ class MultiModelManager:
         return 1.0
 
     def _effective_quant_multiplier(self, cfg: dict,
-                                    load_in_4bit: bool, load_in_8bit: bool) -> float:
+                                    load_in_4bit: bool, load_in_8bit: bool,
+                                    model_key: str = None,
+                                    resolved_name: str = None) -> float:
         """Fraction of full-precision weight size that stays resident.
 
-        Per-component quantization wins when present: the big weight-bearing
-        components (transformer(s), unet, text encoders) dominate, so we assume
-        the quantized components carry ~80 % of the weight and the rest (VAE,
-        etc.) stays dense. Falls back to the global 4/8-bit flags.
+        When per-component quantization is configured AND the model's real
+        per-component parameter shares can be scanned from disk, weight each
+        component's quant divisor by its ACTUAL share of the parameters — so a
+        pipeline that is e.g. 99 % 4-bit (Wan2.2: two 14B experts + text encoder
+        quantized, only a 0.13B VAE dense) is estimated at ~0.28× rather than the
+        blunt 0.475× a fixed 70/30 split would give (which inflated 25.8 GB →
+        42.7 GB and forced needless offload). Falls back to a fixed-share
+        heuristic, then to the global 4/8-bit flags.
         """
         comp_q = cfg.get("component_quantization") or {}
+
+        def _comp_divisor(name: str) -> float:
+            # VAEs are conv-only → bitsandbytes/quanto can't quantize them; they
+            # stay dense regardless of what the config asks for.
+            low = str(name).lower()
+            if low == 'vae' or low.endswith('_vae') or low.startswith('vae'):
+                return 1.0
+            if name in comp_q:
+                return self._quant_divisor(comp_q[name])
+            if load_in_4bit:
+                return 4.0
+            if load_in_8bit:
+                return 2.0
+            return 1.0
+
+        # --- Preferred: weight by real per-component parameter shares ---
+        shares = {}
+        if comp_q and model_key:
+            try:
+                shares = self._component_param_shares(model_key, resolved_name, cfg)
+            except Exception:
+                shares = {}
+        if shares:
+            total = sum(shares.values())
+            if total > 0:
+                mult = 0.0
+                for comp, numel in shares.items():
+                    div = _comp_divisor(comp)
+                    eff = (1.0 / div) if div > 1.0 else 1.0
+                    mult += (numel / total) * eff
+                # bitsandbytes keeps fp32 absmax + scale tensors and fp16 compute
+                # buffers alongside the packed 4-bit weights (~+12 %).
+                return mult * 1.12
+
         divisors = [self._quant_divisor(v) for v in comp_q.values()]
         divisors = [d for d in divisors if d > 1.0]
         if divisors:
             avg_div = sum(divisors) / len(divisors)
-            # Lean slightly HIGH: assume ~70 % of the footprint is shrunk by
-            # quantization and ~30 % (VAE, embeddings, fp16 compute buffers,
-            # bitsandbytes scale/absmax tensors) stays dense. Better to slightly
-            # over-estimate and evict enough than to under-estimate and OOM —
-            # but not so high it exceeds the card and forces needless offload.
-            # (e.g. all-4bit → ~0.475× raw, ×1.15 overhead ≈ 0.55× → ~+10% over
-            # the measured resident size.)
+            # No per-component scan available: assume ~70 % of the footprint is
+            # shrunk by quantization and ~30 % (VAE, embeddings, fp16 compute
+            # buffers, bitsandbytes scale/absmax tensors) stays dense.
             quantized_share = 0.7
             return quantized_share / avg_div + (1.0 - quantized_share)
         if load_in_4bit:
@@ -2042,6 +2078,80 @@ class MultiModelManager:
         self._storage_dtype_cache[ck] = result
         return result
 
+    # component (top-level subfolder) -> total params, per model. Cached.
+    _component_shares_cache: Dict[str, Dict[str, int]] = {}
+
+    def _component_param_shares(self, model_key: str, resolved_name: str,
+                                cfg: dict) -> Dict[str, int]:
+        """Map each pipeline component → its parameter count (numel).
+
+        Groups every ``.safetensors`` file by its TOP-LEVEL subfolder name
+        (``transformer``, ``transformer_2``, ``text_encoder``, ``vae`` …) — i.e.
+        the diffusers component layout — and sums the parameters per component.
+        Used to weight the effective quantization multiplier by each component's
+        real share of the model, so a pipeline that is 99 % quantized isn't
+        treated as if 30 % stays dense. Returns {} when it can't be determined.
+        """
+        ck = resolved_name or model_key
+        cached = self._component_shares_cache.get(ck)
+        if cached is not None:
+            return cached
+
+        import os
+        shares: Dict[str, int] = {}
+        candidates = []
+        for v in (resolved_name, cfg.get('path'), cfg.get('model_path'), cfg.get('model')):
+            if v and isinstance(v, str):
+                candidates.append(v)
+
+        def _record(root_dir: str, file_path: str):
+            rel = os.path.relpath(file_path, root_dir)
+            comp = rel.split(os.sep)[0]
+            # Files at the pipeline root (no subfolder) → bucket as 'root'.
+            if comp.endswith('.safetensors'):
+                comp = 'root'
+            numel, _ = self._safetensors_numel_bytes(os.path.realpath(file_path))
+            if numel > 0:
+                shares[comp] = shares.get(comp, 0) + numel
+
+        try:
+            scanned = False
+            for c in candidates:
+                if os.path.isdir(c):
+                    for r, _, files in os.walk(c):
+                        for fn in files:
+                            if fn.endswith('.safetensors'):
+                                _record(c, os.path.join(r, fn))
+                    scanned = True
+                    break
+            if not scanned:
+                from huggingface_hub import scan_cache_dir
+                from codai.models.cache import get_all_cache_dirs, is_huggingface_model_id
+                hf_dir = get_all_cache_dirs().get("huggingface")
+                for c in candidates:
+                    repo_id = c.split(":", 1)[1] if ":" in c else c
+                    if not (hf_dir and is_huggingface_model_id(repo_id)):
+                        continue
+                    info = scan_cache_dir(hf_dir)
+                    for repo in info.repos:
+                        if repo.repo_id != repo_id:
+                            continue
+                        revs = sorted(repo.revisions,
+                                      key=lambda rv: rv.last_modified, reverse=True)
+                        if revs:
+                            snap = str(revs[0].snapshot_path)
+                            for fobj in revs[0].files:
+                                fp = str(fobj.file_path)
+                                if fp.endswith('.safetensors'):
+                                    _record(snap, fp)
+                    if shares:
+                        break
+        except Exception:
+            shares = {}
+
+        self._component_shares_cache[ck] = shares
+        return shares
+
     @staticmethod
     def _load_bytes_per_elem(cfg: dict) -> float:
         """Bytes/elem for the configured LOAD precision (default fp16 = 2)."""
@@ -2189,7 +2299,7 @@ class MultiModelManager:
         # diffusion pipeline whose transformer/text_encoder are 4-bit isn't
         # wildly over-estimated at full precision.
         quant_mult = self._effective_quant_multiplier(
-            cfg, load_in_4bit, load_in_8bit)
+            cfg, load_in_4bit, load_in_8bit, model_key, resolved_name)
 
         # Precision normalization: used_vram_gb / disk-scan baselines are STORAGE
         # sizes at the on-disk dtype (e.g. Wan2.2 ships fp32 → 4 bytes/elem), but