0.3.65 moved PDF rendering into asyncio.to_thread so a long document would stop
stalling the event loop. That was correct in intent and broke OCR within the
hour: until then `load_pages` ran ON the loop, which serialised every
rasterisation BY ACCIDENT. Removing the accident let documents into pdfium
concurrently, and pdfium permits exactly one caller at a time. Concurrent use
corrupts its state and reports

    failed to rasterise PDF (pypdfium2): PDFium: Data format error

on documents that are perfectly valid -- so the failure accuses the input, and
the operator goes looking at their files. Three valid Diogene PDFs that poppler
parsed cleanly were rejected this way.

Measured: 1 document OK, 8 concurrent = 8 failures, 24 = 24 failures. After the
fix, 48 concurrent 47-page PDFs all return 47 pages.

Every pdfium call -- document open, page size probe, page render -- now runs
under one lock, held per CALL rather than per document so documents still
interleave page by page instead of queueing behind each other's full length.
Rendering still happens off the event loop, so this is strictly better than
0.3.64, which serialised AND blocked the loop.

Cost, stated plainly: rasterisation is single-threaded, a ceiling of about one
core. That is what 0.3.64 effectively was, so it is not a regression against it.
Rasterising in a process pool (pdfium is safe one-document-per-process) is the
scalability fix and is deliberately not bolted onto this hotfix.

Two process lessons, in the tests so they outlive the memory of this:
- this path is only ever exercised concurrently, so testing it sequentially
  tests the one case that cannot fail. I shipped it tested sequentially.
- the error STRING predated the change, and I used that to argue the behaviour
  did too. "This message is not new" is not evidence of "this is not new".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXRMe9ZPUQCCCtBai9uqt3