0.3.66 serialised every pdfium call behind a lock, because pdfium permits one
caller at a time and calling it concurrently corrupts its state into "PDFium:
Data format error" on valid documents. Correct, and it made rasterisation
single-threaded for the whole install -- a ceiling of about one core.

pdfium IS safe one-document-per-process, so `ocr.raster_workers` (0 = auto at
half the cores capped at 4, 1 = in-process under the lock) hands documents to
worker processes. Measured on 12 concurrent 47-page PDFs at 150 DPI: 21.9s ->
10.3s, 2.13x. Not 4x -- the parent is single-threaded receiving the pixels and
rebuilding the images, and ~4.5 GB crosses the pipes. The ceiling is the parent.

Three things this had to get right, each of which cost a bug first:

- A worker is borrowed per CHUNK, not per document. The caller OCRs each window
  between pulls, which takes far longer than rasterising it, so holding a worker
  across that would idle it for most of its tenure and cap the install at
  `raster_workers` concurrent documents -- worse than the lock it replaces.

- A pool that cannot start must cost nothing. The first version waited 300s for
  a worker that could never arrive (missing script), stalling every document for
  five minutes: the optimisation becoming an outage, which is the exact failure
  its test was written for. Creation failure is now remembered rather than
  retried per document, and a busy pool falls back in-process after 30s.

- Workers are executed, never imported. `import codai.ocr` costs 377 MB and
  1.5s via codai.models.manager -> litellm; paying that per worker, in the
  subsystem whose other half exists to bound memory, would be self-defeating.
  The worker imports pypdfium2 and PIL only, ~25 MB. This is why it is not a
  ProcessPoolExecutor: spawn pickles a function by qualified name and re-imports
  the package to find it.

Also fixes a cost 0.3.65 introduced and nothing but a stopwatch would surface:
pricing a document called `pdf[i].get_size()` per page, which LOADS each page
object -- 593 ms for a 47-page document, paid UNDER THE LOCK, so every document
in the install queued behind every other being priced.
`pdf.get_page_size(i)` reads the same numbers from the page tree by index.
page_estimate: ~600 ms -> 9.6 ms.

Every failure path degrades to a correct in-process render: no pool, a dead
worker mid-document, a missing script, an unusable module, an abandoned
document. Correctness never depends on the optimisation working.

87 tests across the budget and pool suites. Full suite at the 25 pre-existing
failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXRMe9ZPUQCCCtBai9uqt3