-
Stefy Lanza (nextime / spora ) authored
Reranker (e.g. BAAI/bge-reranker-v2-m3): - new codai/api/rerank.py: /v1/rerank scores query
↔ document pairs with a native AutoModelForSequenceClassification cross-encoder (no new dep), cached + VRAM/thermal-managed via request_model like embeddings. Returns {index, relevance_score(sigmoid), document?} sorted desc, top_n honored. - router: /v1/rerank added to _INFERENCE_PATHS so the front routes it by model capability (transformers → nvidia engine), like /v1/embeddings. - app: include rerank_router. - capabilities: add `reranking` + `ocr` fields; detect 'rerank'/'cross-encoder' names as reranking. /v1/models discoverability: - list_models() now appends enabled OCR engines (paddle/doctr/surya) as type='ocr' entries (a node subsystem invoked at /v1/ocr?engine=<id>), so clients can discover them. - a reranker registered in models.json (embedding_models, capabilities ['reranking']) surfaces automatically via list_models, like bge-m3. Co-Authored-By:Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mw2KQiswmD69T45fTfjKwW
e44d41ac