• Stefy Lanza (nextime / spora )'s avatar
    serve one engine under several ids, and stop advertising what is not returned (v0.4.98) · 1ef6b8f6
    Stefy Lanza (nextime / spora ) authored
    A model id is a configuration, not a set of weights. Image models have worked
    that way for a long time -- one checkpoint, several published ids, one copy in
    VRAM -- and five things that should have followed from it did not.
    
    PER-MODEL LoRA STACKS. A diffusion pipeline could carry an acceleration LoRA
    and per-request LoRAs, with nothing in between: no way to say "this id is that
    base plus this style LoRA at 0.8". `loras` on a model entry says it, in the
    shape text models already use. `lora_path` is honoured too -- it had been in
    the config and the admin page for years while nothing on the diffusers image
    path read it.
    
    And `set_adapters` replaces the ACTIVE SET, so the image path's request-LoRA
    call was switching the pipeline's own adapters off: a 4-step distilled preset
    running un-distilled, and an id published as a fine-tune serving its base
    checkpoint. 200 OK, a plausible picture, nothing in the logs. The video path
    had always re-included them.
    
    A per-model LoRA that will not load now FAILS THE LOAD. The acceleration path
    logs and continues because the cost there is speed; here it is identity.
    
    SINGLE-FILE CHECKPOINTS. `from_single_file` was wired up only for GGUF
    components, so the one .safetensors Civitai ships for every community
    fine-tune could not be opened at all. A model path may now name a file, or
    `repo:file.safetensors` -- community repos hold every version side by side, so
    the repo id alone does not say which to serve.
    
    A SAMPLER IS A CLASS PLUS ITS OPTIONS. "DPM++ 2M SDE Karras" is
    DPMSolverMultistepScheduler and two settings; naming the class alone gave
    plain DPM++ 2M, a different sampler, silently. `scheduler_config` carries
    them.
    
    RASTERISATION, PER REQUEST. A scan is normally handed to the model as the
    scanner wrote it -- 0.1 ms and 147 KB against a render plus ~1.8 MB of PNG --
    but only between the target DPI and `page_image_max_dpi_factor` times it, and
    only install-wide. `native_dpi` removes the ceiling for one request; the floor
    stays, because a page passed through BELOW the target is a coarser page, which
    is what cost surya-ocr-2 its Italian accents once already. An uploaded image
    now carries its own bytes too: it was decoded and re-encoded to arrive at the
    picture the caller already had.
    
    SEVERAL IDS FOR ONE OCR ENGINE. `ocr.engine_configs` gives OCR what image
    models had: named configurations of one engine, each its own id, its own
    catalogue row, its own pool. An override naming a setting that does not exist
    is refused rather than quietly serving the base settings under another name.
    
    Which forced the last one into the open. `/v1/ocr/engines` advertised
    `surya: {regions: true, tables: true}` while the surya worker built every page
    with `"regions": [], "tables": []` hardcoded. surya ships a layout predictor
    and a table recogniser; they were never run. A client that chose surya BECAUSE
    the catalogue said it returned regions got empty arrays and no error.
    
    Both halves are now true by construction: the row is read from
    `surya_layout` / `surya_table_rec`, and those load the predictors. Regions come
    back with surya's own reading order (LayoutBox.position, explicit as `order`,
    -1 where an engine states none) and tables as cells with row/column ids.
    
    Loading is configured; spending is per request. `layout=true` chooses whether
    to pay for the pass on an engine that HOLDS the predictor -- it cannot make one
    acquire a model mid-page, which is how a footprint grows past what the VRAM
    accounting reserved. Asked for where none is loaded, it is refused with the
    configuration to use instead; answering 200 with empty regions would be the
    same lie wearing a request field.
    
    Pods differ from their siblings too: a pool whose entry names an
    `ocr_engine_config` boots pods with that configuration's settings, and a named
    config overriding `<engine>_model_id` reaches its own pods rather than sharing
    its sibling's.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
    1ef6b8f6