embeddings image: accelerate is not optional (v0.3.82)

The pod booted healthy, answered /v1/models, advertised bge-m3 -- and
failed every embedding request with

  Model 'BAAI/bge-m3' failed to load: Using a `device_map`, `tp_plan`,
  `torch.device` context manager or setting `torch.set_default_device(device)`
  requires `accelerate`

sentence-transformers loads under a device_map context and transformers
refuses that without accelerate. It is in requirements.txt and in the
text, images, video and stt-crisper profiles; it was never in this one.

What makes it worth a comment in the file rather than just a line: the
failure does not look like a missing dependency from outside. The image
is healthy, the catalogue is right, and the error names the MODEL. On a
real A40 that cost 303 seconds of boot before anyone could find out, and
the next error after it ("'BAAI/bge-m3' is a text model") sends the
reader somewhere else entirely.
Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXRMe9ZPUQCCCtBai9uqt3
parent 252cc0bb
......@@ -16,7 +16,7 @@
# Canonical product version for CoderAI — single source of truth. Both the API
# metadata and the admin web UI read from here.
__version__ = "0.3.81"
__version__ = "0.3.82"
__tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
# Configure the CUDA caching allocator BEFORE torch is imported anywhere.
......
# /v1/embeddings and /v1/rerank
transformers>=4.35.0
sentence-transformers>=2.7.0
# NOT optional, and its absence does not look like a missing dependency.
# sentence-transformers loads under a `device_map`/`torch.device` context, and
# transformers refuses that without accelerate -- so the pod boots healthy,
# answers /v1/models, and then fails EVERY embedding request with
# Model 'BAAI/bge-m3' failed to load: Using a `device_map` ... requires `accelerate`
# which reads as a model problem rather than an image problem. Measured on a
# real A40: 303s of boot paid before anyone could find out.
accelerate>=0.26.0
geoclip>=1.2.0
geopy>=2.4
geographiclib>=2.0
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment