• Stefy Lanza (nextime / spora )'s avatar
    front: route to a pinned/owning engine that's busy, not a duplicate elsewhere · caa051b4
    Stefy Lanza (nextime / spora ) authored
    An engine mid-generation is GIL-blocked and fails the health poll, so it
    reads as unhealthy. pick_engine required e.healthy at every step, so a
    second request for a model pinned to that engine fell through to the
    least-loaded engine — which loaded a DUPLICATE copy (and ignored the
    model's configured n_ctx, e.g. 2048 vs 32000 → "exceeds context window").
    
    Honour the pin (and the assigned owner) when the engine is alive but
    transiently busy: route there so the request queues on its gen-lock and
    the owner handles serialization/eviction. Only fall back to another engine
    when the owner's process is actually dead. Adds Engine.is_alive() (process
    liveness) and registry.engine_owning() (health-agnostic owner lookup).
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    caa051b4
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...
video_editor.config.json Loading commit data...