• Stefy Lanza (nextime / spora )'s avatar
    rerank: add /v1/rerank cross-encoder endpoint + surface OCR in /v1/models · e44d41ac
    Stefy Lanza (nextime / spora ) authored
    Reranker (e.g. BAAI/bge-reranker-v2-m3):
    - new codai/api/rerank.py: /v1/rerank scores querydocument pairs with a native
      AutoModelForSequenceClassification cross-encoder (no new dep), cached +
      VRAM/thermal-managed via request_model like embeddings. Returns
      {index, relevance_score(sigmoid), document?} sorted desc, top_n honored.
    - router: /v1/rerank added to _INFERENCE_PATHS so the front routes it by model
      capability (transformers → nvidia engine), like /v1/embeddings.
    - app: include rerank_router.
    - capabilities: add `reranking` + `ocr` fields; detect 'rerank'/'cross-encoder'
      names as reranking.
    
    /v1/models discoverability:
    - list_models() now appends enabled OCR engines (paddle/doctr/surya) as
      type='ocr' entries (a node subsystem invoked at /v1/ocr?engine=<id>), so
      clients can discover them.
    - a reranker registered in models.json (embedding_models, capabilities
      ['reranking']) surfaces automatically via list_models, like bge-m3.
    Co-Authored-By: 's avatarClaude Opus 4.8 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01Mw2KQiswmD69T45fTfjKwW
    e44d41ac
Name
Last commit
Last update
..
cache Loading commit data...
__init__.py Loading commit data...
acceleration.py Loading commit data...
capabilities.py Loading commit data...
gpu_lock.py Loading commit data...
gpu_query.py Loading commit data...
grammar.py Loading commit data...
hf_loading.py Loading commit data...
manager.py Loading commit data...
parser.py Loading commit data...
peft_compat.py Loading commit data...
pipeline_cache.py Loading commit data...
quant.py Loading commit data...
ram_monitor.py Loading commit data...
templates.py Loading commit data...
thermal.py Loading commit data...
tmp_janitor.py Loading commit data...
tool_call_grammar.gbnf Loading commit data...
turboquant.py Loading commit data...
utils.py Loading commit data...