vulkan: vision requests must bypass the fast chat-render path

The plain-Jinja fast path in generate_chat_stream renders the template
itself and generates via create_completion — which never invokes the
multimodal (mmproj) chat handler, so image_url parts were silently
dropped and vision models answered as if no image was sent. Streaming
vision requests now fall through to create_chat_completion.
Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014S8VtAvG499SsCbeESRK7V
parent 4d2aeb69
...@@ -16,7 +16,7 @@ ...@@ -16,7 +16,7 @@
# Canonical product version for CoderAI — single source of truth. Both the API # Canonical product version for CoderAI — single source of truth. Both the API
# metadata and the admin web UI read from here. # metadata and the admin web UI read from here.
__version__ = "0.1.31" __version__ = "0.1.32"
# Configure the CUDA caching allocator BEFORE torch is imported anywhere. # Configure the CUDA caching allocator BEFORE torch is imported anywhere.
# expandable_segments lets the allocator return freed pages to the driver even # expandable_segments lets the allocator return freed pages to the driver even
......
...@@ -2224,7 +2224,17 @@ class VulkanBackend(ModelBackend): ...@@ -2224,7 +2224,17 @@ class VulkanBackend(ModelBackend):
# via create_completion — avoids llama-cpp-python's slow per-call sandboxed # via create_completion — avoids llama-cpp-python's slow per-call sandboxed
# chat-template render. Falls through to create_chat_completion if the model # chat-template render. Falls through to create_chat_completion if the model
# has no usable chat_template or rendering fails. # has no usable chat_template or rendering fails.
_prompt, _fmt_stops = self._render_chat_prompt(messages) # NEVER take the fast path for vision requests: only create_chat_completion
# invokes the multimodal chat handler (mmproj) that actually embeds the
# images — a plain Jinja render would silently drop them and the model
# answers as if no image was sent.
_has_images = self.supports_vision and any(
isinstance(m.get('content'), list) and any(
isinstance(p, dict) and p.get('type') in ('image_url', 'input_image')
for p in m['content'])
for m in messages)
_prompt, _fmt_stops = ((None, None) if _has_images
else self._render_chat_prompt(messages))
if _prompt is not None: if _prompt is not None:
# Tokenize with add_bos=False (the chat template already emitted the BOS # Tokenize with add_bos=False (the chat template already emitted the BOS
# text — same as llama-cpp's handler, which tokenizes with # text — same as llama-cpp's handler, which tokenizes with
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment