• Stefy Lanza (nextime / spora )'s avatar
    a waiting page can start a pod again, with OCR workers on (v0.4.85) · 36cc0fac
    Stefy Lanza (nextime / spora ) authored
    With `ocr.workers` set, the workers dispatch every page and never rent -- the
    controller owns the fleet -- but the controller only ever decided to grow
    inside its own `acquire()`, which it never calls while the workers serve the
    pages. The one signal the workers did send was keyed by POD, which is exactly
    what a cold fleet has none of: a worker holding pages with zero pods reported
    `{}`, and the controller read an idle install.
    
    So a cold OCR fleet never booted for any load, and a warm one never grew past
    `min_pods` -- "scale to 128 pods" was unreachable whenever the workers were on.
    Observed live: `workers: 2`, `min_pods: 0`, `pod_pipeline: require`, a page
    waiting 5.8 minutes for a pod that nothing would rent; the same request rented
    one within 20 seconds with `workers: 0`.
    
    Demand now travels on its own, keyed by pool rather than by pod, and the scaler
    pass acts on it -- single-threaded with the shedding it must not race, and
    still through `_grow_reason`, so max_pods and the parallel-boot caps are
    unchanged.
    
    Second, independent fix in the same area: under `ocr.pod_pipeline: require` a
    fallback chain now keeps only engines that can run on a pod. `auto` orders the
    chain fewest-ways-to-fail first, which put paddle and docTR -- neither of which
    has a pod-side pipeline -- at the front of it on an install whose whole rule is
    that this box never reads. The documented way out was `fallback_engines: off`,
    and that ALSO switched off the degraded-page recovery, which reads `auto` as
    "the fallback order": an install protecting its orchestrator thereby lost the
    mechanism written for the run that came out ~97% truncated with no error
    anywhere. Found live, in exactly that state.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
    36cc0fac
runpod_fleet_share.py 10.6 KB