tests: cover the remote-execution paths

18 tests over local stub servers — no GPU, no weights, no network:
worker service_url health enforcement, the mux service + RemoteMuxEngine
(streaming, stats, mid-stream cancel, error propagation, dead service),
remote GGUF text models (routing, /v1 normalisation, auth, chat, SSE,
no-local-weights), RunPod engine selection (HF->vLLM, GGUF->llama.cpp,
pins, the missing-hf_gguf error), and the gateway (transparency when
nothing is remote, capability and per-model routing, verbatim multipart
forwarding, chat exclusion, clean 502).

Also stop the mux service dumping a broken-pipe traceback when a client
cancels a stream — hanging up IS the cancel signal there.
Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
parent 3a4e2605
This diff is collapsed.
...@@ -135,6 +135,17 @@ class Handler(BaseHTTPRequestHandler): ...@@ -135,6 +135,17 @@ class Handler(BaseHTTPRequestHandler):
pass pass
class MuxServer(ThreadingHTTPServer):
"""A client that hangs up mid-generation is the cancel signal, not an error —
don't dump a broken-pipe traceback every time someone stops a stream."""
def handle_error(self, request, client_address):
exc = sys.exc_info()[1]
if isinstance(exc, (BrokenPipeError, ConnectionResetError)):
return
super().handle_error(request, client_address)
def main(): def main():
global ENGINE, KIND global ENGINE, KIND
ap = argparse.ArgumentParser(description=__doc__) ap = argparse.ArgumentParser(description=__doc__)
...@@ -159,7 +170,7 @@ def main(): ...@@ -159,7 +170,7 @@ def main():
flush=True) flush=True)
ENGINE = _build_engine(args.kind, cfg, args.model_dir, args.ctx) ENGINE = _build_engine(args.kind, cfg, args.model_dir, args.ctx)
print(f"[mux-service] {args.kind} ready, serving on {args.host}:{args.port}", flush=True) print(f"[mux-service] {args.kind} ready, serving on {args.host}:{args.port}", flush=True)
ThreadingHTTPServer((args.host, args.port), Handler).serve_forever() MuxServer((args.host, args.port), Handler).serve_forever()
if __name__ == "__main__": if __name__ == "__main__":
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment