-
Stefy Lanza (nextime / spora ) authored
Images, video, embeddings, rerank, OCR, TTS, STT, voice cloning, audio generation, stems, 3D and pipelines all load their model in-process, so none of them had a service to point elsewhere. Writing thirteen services would be the long way round; another coderai already exposes all of them, so it can be the service. RemoteGatewayMiddleware sits in front of the routes and, when a request is addressed to something configured as remote, replays it verbatim to that endpoint and streams the answer back — JSON and multipart alike, since the body is forwarded as raw bytes. Routing is per model (`service_url` on the models.json entry) first, then per capability (`remotes.endpoints` in config.json, or CODERAI_REMOTE_<CAP>_URL). Placement matters and is deliberate: it is registered first, so it ends up innermost — auth, rate limiting and request logging all run before it, and a request still has to be authorised here before being forwarded. Chat and completions are excluded: those go through RemoteOpenAIBackend, which keeps usage accounting, tool parsing and the manager's bookkeeping. When nothing is configured remote it returns on its first branch and never buffers a body (cached 5s so it doesn't walk the model list per request). Bodies over `max_body_mb` are served locally rather than held in RAM. A non-forwarded request replays its buffered body downstream untouched — tested, including multipart uploads that still parse locally. docs/remote-execution.md documents all four remotization mechanisms together. Co-Authored-By:
Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0182DhR1eFNPQyGmDrpHbedY
3a4e2605