• Stefy Lanza (nextime / spora )'s avatar
    rerank: add /v1/rerank cross-encoder endpoint + surface OCR in /v1/models · e44d41ac
    Stefy Lanza (nextime / spora ) authored
    Reranker (e.g. BAAI/bge-reranker-v2-m3):
    - new codai/api/rerank.py: /v1/rerank scores querydocument pairs with a native
      AutoModelForSequenceClassification cross-encoder (no new dep), cached +
      VRAM/thermal-managed via request_model like embeddings. Returns
      {index, relevance_score(sigmoid), document?} sorted desc, top_n honored.
    - router: /v1/rerank added to _INFERENCE_PATHS so the front routes it by model
      capability (transformers → nvidia engine), like /v1/embeddings.
    - app: include rerank_router.
    - capabilities: add `reranking` + `ocr` fields; detect 'rerank'/'cross-encoder'
      names as reranking.
    
    /v1/models discoverability:
    - list_models() now appends enabled OCR engines (paddle/doctr/surya) as
      type='ocr' entries (a node subsystem invoked at /v1/ocr?engine=<id>), so
      clients can discover them.
    - a reranker registered in models.json (embedding_models, capabilities
      ['reranking']) surfaces automatically via list_models, like bge-m3.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01Mw2KQiswmD69T45fTfjKwW
    e44d41ac
manager.py 241 KB