the vision tower stays at full precision, and two embedders that needed a shim each

Four text-and-image embedders were added last commit; making them actually
answer took three fixes measured against the live models.

Quantization now skips the vision tower and the projector by default
(hf_loading.quant_skip_modules). For a vision-language EMBEDDER those are the
part being judged: the tower turns pixels into the vector that gets compared,
the projector is one small matrix whose error lands on every image, and
together they are a fraction of a 7B's weights -- so nf4 there saves almost
nothing and spends it in the worst place. An explicit empty quant_skip_modules
on the model entry still means "quantize all of it".

jina-clip-v2 and MM-Embed each needed one thing transformers 5 took away:

* `dtype=`, not `torch_dtype=`. The old name is still accepted for most
  models, but a model whose remote code forwards **kwargs into the config dies
  with "TypeError: attribute name must be string, not 'torch.dtype'", which
  reads like a corrupt checkpoint.
* `AutoModel.register(..., exist_ok=True)`. MM-Embed's modeling_nvmmembed
  registers LlavaNextConfig, which transformers already owns, and 5.x refuses a
  duplicate instead of ignoring it.
* `clip_loss`, deleted from modeling_clip while `contrastive_loss`, which it is
  two lines of, is still there. Restored from it rather than vendored.

Embeddings are also checked for NaN/Inf before they are returned
(_require_finite): a vector of NaNs scores every comparison equally and
silently, and that is what jina-clip-v2's image tower returns on this stack --
so it now answers with an error instead of nonsense.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014T3rZozi1Ehk96NG1WesgC

codai/models/hf_loading.py

@@ -87,6 +87,46 @@ def resolve_dtype(cfg: Optional[Dict[str, Any]], default: str = 'bf16'):
     }.get(precision, torch.bfloat16 if default == 'bf16' else torch.float32)
 
 
+#: Submodules left at full precision when a model is quantized: the vision tower
+#: and the projector that maps its output into the language model's space.
+#:
+#: A vision-language EMBEDDER lives or dies on those. The language side is doing
+#: comparatively coarse work -- pooling a hidden state -- while the tower turns
+#: pixels into the vector being compared, and the projector is a single small
+#: matrix whose error lands on every image. Quantizing them saves little (the
+#: tower is a fraction of a 7B's weights) and spends it in the one place the
+#: model is judged. nf4 on the language layers is the cheap part.
+#:
+#: Names, not classes, because transformers matches these against module names:
+#: `visual` is Qwen2-VL's tower, `vision_tower` + `multi_modal_projector` are
+#: LLaVA's, `vision_model`/`visual_projection` are CLIP-style, and `merger` is
+#: Qwen2-VL's patch merger. A name that does not exist in a given model is
+#: simply never matched, so one list is safe across families -- and harmless on
+#: a text-only model, which has none of them.
+_DEFAULT_QUANT_SKIP = (
+    'visual', 'vision_tower', 'vision_model', 'vision_encoder',
+    'multi_modal_projector', 'visual_projection', 'merger',
+    'mm_projector', 'image_newline',
+)
+
+
+def quant_skip_modules(cfg: Optional[Dict[str, Any]]):
+    """Which submodules to leave unquantized, or None to quantize everything.
+
+    `quant_skip_modules` on the model entry overrides the default. An explicit
+    empty list means "quantize all of it" -- an operator who asks for that gets
+    it, and the difference between "not configured" and "configured empty" is
+    the only way to say so.
+    """
+    c = _norm(cfg)
+    if 'quant_skip_modules' in c:
+        v = c.get('quant_skip_modules')
+        if not v:
+            return None
+        return list(v) if isinstance(v, (list, tuple)) else [str(v)]
+    return list(_DEFAULT_QUANT_SKIP)
+
+
 def build_quantization_config(cfg: Optional[Dict[str, Any]]):
     """Build a transformers BitsAndBytesConfig from the model config, or None."""
     c = _norm(cfg)
@@ -97,14 +137,17 @@ def build_quantization_config(cfg: Optional[Dict[str, Any]]):
     try:
         import torch
         from transformers import BitsAndBytesConfig
+        _skip = quant_skip_modules(cfg)
+        _extra = {'llm_int8_skip_modules': _skip} if _skip else {}
         if load_in_4bit:
             return BitsAndBytesConfig(
                 load_in_4bit=True,
                 bnb_4bit_quant_type='nf4',
                 bnb_4bit_compute_dtype=resolve_dtype(cfg),
                 bnb_4bit_use_double_quant=True,
+                **_extra,
             )
-        return BitsAndBytesConfig(load_in_8bit=True)
+        return BitsAndBytesConfig(load_in_8bit=True, **_extra)
     except Exception as e:
         print(f"  Quantization requested but unavailable: {e}")
         return None

packaging/linux/Dockerfile.update

@@ -54,6 +54,23 @@ RUN /opt/coderai/python/bin/python3 -m pip install --no-input --no-cache-dir "nv
 # without it (discovery just stays off), hence `|| true`.
 RUN /opt/coderai/python/bin/python3 -m pip install --no-input --no-cache-dir "zeroconf>=0.130" || true
 
+# --no-deps on every one of these, deliberately.
+#
+# `pip install --target <site-packages>` cannot see what is already installed,
+# so it resolves each package's dependency tree from scratch and reinstalls it.
+# Measured on the 2026-10-31 build, which pulled in fresh copies of
+# typing-extensions, setuptools, triton, sympy, safetensors, tokenizers 0.23.3,
+# huggingface-hub 2.4.0, fsspec, filelock, click, python-dateutil and even
+# cuda-toolkit, then reported:
+#
+#   tokenizers 0.22.2 requires huggingface-hub<2.0, but you have 2.4.0
+#   datasets 4.8.5 requires fsspec<=2026.2.0, but you have 2026.9.0
+#
+# i.e. three small additions silently re-pointed the core of the ML stack. That
+# is how an image gets a working geoclip and a broken transformers. Every
+# dependency these three actually need -- torch, torchvision, huggingface-hub,
+# safetensors, numpy, pandas, Pillow -- is already in the bundle, and the two
+# that are not (geopy, geographiclib) are named explicitly below.
 # Vosk, the CPU STT backend. It is declared in requirements.txt, which the FULL
 # build picks up through the pre-built venv bundle -- but this overlay never
 # touches the venv, so an image updated incrementally shipped without it and
@@ -68,7 +85,7 @@ RUN /opt/coderai/python/bin/python3 -m pip install --no-input --no-cache-dir "ze
 # the venv is not writable by the container user, and the engine -- which does
 # not load the user site -- went on reporting the module as missing while pip
 # reported success.
-RUN /opt/coderai/python/bin/python3 -m pip install --no-input --no-cache-dir \
+RUN /opt/coderai/python/bin/python3 -m pip install --no-input --no-cache-dir --no-deps \
         --target /opt/coderai/python/lib/python3.13/site-packages \
         "vosk>=0.3.45" || true
 
@@ -86,7 +103,7 @@ RUN /opt/coderai/python/bin/python3 -m pip install --no-input --no-cache-dir \
 # Checked against the built image rather than against requirements.txt --
 # a declared dependency the overlay never installs is invisible from the
 # source tree, which is what hid this one for as long as it hid vosk.
-RUN /opt/coderai/python/bin/python3 -m pip install --no-input --no-cache-dir \
+RUN /opt/coderai/python/bin/python3 -m pip install --no-input --no-cache-dir --no-deps \
         --target /opt/coderai/python/lib/python3.13/site-packages \
         "geoclip>=1.2.0" "geopy>=2.4" "geographiclib>=2.0" || true
 
@@ -95,7 +112,7 @@ RUN /opt/coderai/python/bin/python3 -m pip install --no-input --no-cache-dir \
 # incrementally-updated image (confirmed by listing the built image's packages,
 # not the source tree). It is also a hard requirement of jina-clip-v2, whose
 # EVA vision tower IS a timm model, so that embedder cannot load without it.
-RUN /opt/coderai/python/bin/python3 -m pip install --no-input --no-cache-dir \
+RUN /opt/coderai/python/bin/python3 -m pip install --no-input --no-cache-dir --no-deps \
         --target /opt/coderai/python/lib/python3.13/site-packages \
         "timm>=1.0.9" || true