a coarser scan is not a saving, it is a different page (v0.4.12)

**The accent regression, reported from production hours after 0.4.7 shipped.**
Italian accents stopped surviving surya again. The cause was this afternoon's
scan passthrough, not the old detokenisation fault: its guard used an ABSOLUTE
150-DPI floor instead of the DPI the page would have been rendered at. surya's
own default is 250 (`surya_dpi`), and Aruba leaves it unset, so every scanned
page holding a 150-200 DPI JPEG went to the model at native size -- 25-40 % less
linear resolution than the render it replaced -- and surya-ocr-2 is measured to
read badly below 250.

The reasoning error is the part worth keeping: passing a scan through was
always meant to be "the same pixels, without the work". Fewer pixels is not a
saving. The floor is now the TARGET DPI, and `page_image_min_dpi` (default 0)
can only RAISE it. Verified against a 250-DPI engine: a 150 or 200 DPI scan is
rendered up as before; 250 and 300 DPI scans still pass through, which is where
the shortcut was always legitimate.

It also honours the REQUEST. `ocr_document` resolves
`use_dpi = int(dpi) if dpi else engine_dpi(engine)` and hands that one value to
the pod forward, the local render and (on the pod) this guard -- so "is this
scan detailed enough" is asked about what the caller asked for, never about a
default they overrode. Pinned, because a future change reading the engine
default would silently hand a 200-DPI scan to a request that wanted 300.

**And the rasterisation hop now encodes, instead of shipping raw pixels.**
Measured on A4 at 250 DPI (6.04 Mpix, surya's own default):

    raw RGB over the pipe        18.1 MB   + 109.7 ms for the parent to encode
                                           it for the engine anyway
    PNG level 1 in the worker     0.72 MB  104.7 ms, in the WORKER
    the parent's lazy open                   0.12 ms

The encode is not added, it MOVES: off the single parent process onto the
raster pool, which is parallel. The pipe carries 25x less, the second encode
disappears, and the parent holds 0.72 MB per page in flight instead of 18.1 MB
-- at 48 pages that is 35 MB instead of 870 MB, which is the quantity this
whole subsystem keeps getting into trouble over. PNG, not JPEG, unless the
operator asked: a page about to be read must not pass through a lossy codec by
default. Raw RGB stays the fallback, so the protocol still reads on an older
parent.

Seven existing tests had encoded the wrong floor and were corrected; one of
them hardcoded pixel counts that assumed A4 while the fixture is 609 pt, so it
measured 244 DPI while claiming 250.

11 new tests. 3368 passed, 13 pre-existing failures (broker, studio, manual
client) untouched.
Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KnfkxBpcZ1v6ypRFFyhrVM
parent 21035951
......@@ -16,7 +16,7 @@
# Canonical product version for CoderAI — single source of truth. Both the API
# metadata and the admin web UI read from here.
__version__ = "0.4.11"
__version__ = "0.4.12"
__tagline__ = "Complete orchestration, distribution and escalation of remotizable advanced inference."
# Configure the CUDA caching allocator BEFORE torch is imported anywhere.
......
......@@ -1183,13 +1183,14 @@
<span class="form-hint">Only used when the format above is JPEG.</span>
</div>
<div class="form-group" style="flex:1;min-width:150px">
<label class="form-label">Pass a scan through above (DPI)</label>
<input type="number" id="s-ocr-page-image-min-dpi" class="form-input" min="0" placeholder="150">
<span class="form-hint">A scan coarser than this is rendered at the configured DPI
instead, so the model still gets the pixel density the rest of the pipeline was
tuned for. A scan is also rendered when it is more than
<span id="s-ocr-page-image-factor-label">3</span>&times; the target, where
rendering down is cheaper than shipping it whole.</span>
<label class="form-label">Extra floor for passing a scan through (DPI)</label>
<input type="number" id="s-ocr-page-image-min-dpi" class="form-input" min="0" placeholder="0">
<span class="form-hint"><b>0 = the engine&rsquo;s own DPI</b>, which is the only
setting under which a passed-through scan cannot change what the model sees: a
coarser scan is rendered up instead, exactly as before. Raise this only to demand
<i>more</i> than the target. A scan is also rendered when it exceeds
<span id="s-ocr-page-image-factor-label">3</span>&times; the target, where rendering
down is cheaper than shipping it whole.</span>
</div>
<div class="form-group" style="flex:1;min-width:130px">
<label class="form-label">&hellip;and below this &times; DPI</label>
......@@ -2326,7 +2327,7 @@ async function loadSettings(){
document.getElementById('s-ocr-pod-http-async').value = (ocr.pod_http_async ?? true) ? '1' : '0';
document.getElementById('s-ocr-page-image-format').value = ocr.page_image_format ?? 'auto';
document.getElementById('s-ocr-page-image-quality').value = ocr.page_image_jpeg_quality ?? 95;
document.getElementById('s-ocr-page-image-min-dpi').value = ocr.page_image_min_dpi ?? 150;
document.getElementById('s-ocr-page-image-min-dpi').value = ocr.page_image_min_dpi ?? 0;
document.getElementById('s-ocr-page-image-max-factor').value = ocr.page_image_max_dpi_factor ?? 3;
document.getElementById('s-ocr-page-image-factor-label').textContent =
String(ocr.page_image_max_dpi_factor ?? 3);
......
......@@ -1099,14 +1099,18 @@ class OcrConfig:
pod_http_async: bool = True
page_image_format: str = "auto"
page_image_jpeg_quality: int = 95
# The bounds within which a scan may be passed through. Below the floor the
# embedded image is genuinely coarser than the configured DPI and rendering
# it up at least gives the model the pixel density the rest of the pipeline
# was tuned for; far above the target, rendering DOWN is cheaper than
# shipping the whole thing for the engine to scale. Expressed in real DPI
# rather than as a ratio because "is 150 DPI enough to read" is a question
# about documents, which an operator can answer.
page_image_min_dpi: float = 150.0
# An ADDITIONAL floor on passing a scan through, in real DPI. 0 means the
# floor is the engine's own target DPI, which is the only setting under
# which a passed-through scan cannot change what the model sees.
#
# It shipped as an absolute 150 and that was wrong: a 150-DPI scan then
# reached a 250-DPI engine at native size -- 40 % less linear resolution
# than the render it replaced -- and surya-ocr-2 lost Italian accents on
# those pages. Reported from production the same afternoon. The point of
# the passthrough was "the same pixels without the work"; fewer pixels is
# not a saving, it is a different page. Raise this only to demand MORE than
# the target.
page_image_min_dpi: float = 0.0
page_image_max_dpi_factor: float = 3.0
# How many pages of one document are rasterised at a time. The document is read
# in windows, so residency is O(window) instead of O(document): a 400-page
......
......@@ -144,7 +144,7 @@ def data_uri(mime: str, raw: bytes) -> str:
_OBJ_IMAGE = 3
def extract_embedded(page, dpi: int, min_dpi: float = 150.0,
def extract_embedded(page, dpi: int, min_dpi: float = 0.0,
max_dpi_factor: float = 3.0):
"""``(mime, raw, (w, h))`` if this page IS one embedded JPEG, else ``None``.
......@@ -169,11 +169,20 @@ def extract_embedded(page, dpi: int, min_dpi: float = 150.0,
error.
* **It covers the page.** 98 % in both axes, anchored at the origin, or the
model is being shown a crop.
* **Between ``min_dpi`` and ``max_dpi_factor`` x the target.** Below the
floor the scan is genuinely coarser than the configured DPI and rendering
it up at least hands the model the pixel density the rest of the pipeline
was tuned for; far above it, rendering DOWN to the target is cheaper than
shipping the whole thing and letting the engine scale it.
* **At least the TARGET DPI, and at most ``max_dpi_factor`` x it.** This
guard was wrong when it shipped and the error is worth recording: it used
an absolute floor (150 DPI) instead of the DPI the page would have been
rendered at. A 150-DPI scan then went to a 250-DPI engine at its native
size -- a 40 % linear resolution drop in what the model sees -- and
surya-ocr-2 lost Italian accents on exactly those pages, which is a
regression reported from production within hours of the change.
The justification for passing a scan through was never "fewer pixels", it
was "the same pixels, without the work". A coarser image is not a saving,
it is a different page. So the floor is the target itself, and
``min_dpi`` only RAISES it; above ``max_dpi_factor`` x the target,
rendering down is cheaper than shipping the whole thing for the engine to
scale.
"""
try:
if int(page.get_rotation() or 0) % 360 != 0:
......@@ -200,11 +209,18 @@ def extract_embedded(page, dpi: int, min_dpi: float = 150.0,
return None
if abs(left) / pw > 0.02 or abs(bottom) / ph > 0.02:
return None
# 2 % of slack on the floor, because pdfium derives the effective DPI
# from the image's own bounding box rather than the page: a plain
# 150-DPI scan of an A4 page reports 150.0 across and 149.957 down, and
# an exact floor would reject it over four hundredths of a dot.
lo = float(min_dpi) * 0.98
# The floor is what we WOULD have rendered; min_dpi only raises it.
# 0 (the default) therefore means "as detailed as the target", which is
# the only setting under which passing a scan through cannot change what
# the model sees.
floor = float(dpi)
if min_dpi and float(min_dpi) > floor:
floor = float(min_dpi)
# 2 % of slack, because pdfium derives the effective DPI from the
# image's own bounding box rather than the page: a plain 150-DPI scan of
# an A4 page reports 150.0 across and 149.957 down, and an exact floor
# would reject it over four hundredths of a dot.
lo = floor * 0.98
hi = float(dpi) * float(max_dpi_factor)
for eff in (float(md.horizontal_dpi), float(md.vertical_dpi)):
if not (lo <= eff <= hi):
......@@ -233,7 +249,7 @@ class Policy:
__slots__ = ("fmt", "jpeg_quality", "min_dpi", "max_dpi_factor")
def __init__(self, fmt: str = "auto", jpeg_quality: int = 95,
min_dpi: float = 150.0, max_dpi_factor: float = 3.0):
min_dpi: float = 0.0, max_dpi_factor: float = 3.0):
fmt = (fmt or "auto").strip().lower()
self.fmt = fmt if fmt in FORMATS else "auto"
try:
......@@ -243,7 +259,7 @@ class Policy:
try:
self.min_dpi = max(0.0, float(min_dpi))
except Exception:
self.min_dpi = 150.0
self.min_dpi = 0.0
try:
self.max_dpi_factor = max(1.0, float(max_dpi_factor))
except Exception:
......@@ -262,7 +278,7 @@ class Policy:
g = lambda k, d: getattr(cfg, k, d) if cfg is not None else d # noqa: E731
return cls(g("page_image_format", "auto"),
g("page_image_jpeg_quality", 95),
g("page_image_min_dpi", 150.0),
g("page_image_min_dpi", 0.0),
g("page_image_max_dpi_factor", 3.0))
def as_dict(self) -> dict:
......@@ -274,7 +290,7 @@ class Policy:
def from_dict(cls, d):
d = d or {}
return cls(d.get("fmt", "auto"), d.get("jpeg_quality", 95),
d.get("min_dpi", 150.0), d.get("max_dpi_factor", 3.0))
d.get("min_dpi", 0.0), d.get("max_dpi_factor", 3.0))
def encode(self, img):
"""``(mime, bytes)`` under this policy."""
......
......@@ -37,11 +37,28 @@ guards live in page_image.extract_embedded; `open` carries the policy.
-> {"cmd": "shutdown"}
<- {"ok": true} then exit
Raw RGB rather than PNG for a RENDERED page: the parent rebuilds it with
Image.frombytes, and a page about to be read by an OCR engine must not pass
through a lossy codec, while PNG would cost more CPU than the pipe it saves. A
page that was never rendered is a different matter -- its bytes are already a
JPEG, and re-rendering them to ship them raw would be work in both directions.
A rendered page is encoded HERE, losslessly, rather than shipped as raw RGB.
Measured on an A4 page at 250 DPI (6.04 Mpix), which is surya's own default:
raw RGB over the pipe 18.1 MB and the parent then spent
109.7 ms encoding it anyway
PNG level 1 encoded here 0.72 MB 104.7 ms, in THIS process
the parent's lazy Image.open 0.12 ms
So the encode does not get added, it MOVES: off the single parent process and
onto this pool, which is parallel, while the pipe carries 25x less and the
parent holds 0.72 MB per page in flight instead of 18.1 MB. That last number is
the one that matters -- a GPU-less orchestrator rasterising at 48 pages in
flight is the difference between 35 MB and 870 MB of page bitmaps.
The parent opens it lazily and passes the bytes onward, so the page is decoded
only if something actually wants pixels (ink measurement, a detector) and the
engine hop reuses these exact bytes instead of encoding a second time.
PNG, not JPEG, unless the operator asked for JPEG: a page about to be read by
an OCR engine must not pass through a lossy codec by default. Raw RGB remains
the fallback when no policy was sent, which keeps the protocol readable by an
older parent.
Errors answer {"ok": false, "error": str, "status": int} and leave the process
usable -- one undecodable document must not cost the pool a worker.
......@@ -187,9 +204,22 @@ def main() -> int:
continue
bitmap = page.render(scale=doc.scale)
img = bitmap.to_pil().convert("RGB")
raw = img.tobytes()
metas.append({"w": img.width, "h": img.height,
"nbytes": len(raw)})
# Encode here, under the policy, so the parent reuses these
# bytes rather than encoding its own copy of the same page.
enc = None
if doc.policy is not None:
try:
enc = doc.policy.encode(img)
except Exception:
enc = None # a render is worth more than a codec
if enc is not None:
mime, raw = enc
metas.append({"w": img.width, "h": img.height,
"nbytes": len(raw), "mime": mime})
else:
raw = img.tobytes()
metas.append({"w": img.width, "h": img.height,
"nbytes": len(raw)})
blobs.append(raw)
# Drop each page before rendering the next: this process is
# as bound by residency as its parent.
......
......@@ -150,7 +150,7 @@ def test_policy_from_a_config_that_predates_these_knobs():
class _Old:
dpi = 200
p = pi.Policy.from_cfg(_Old())
assert (p.fmt, p.jpeg_quality, p.min_dpi, p.max_dpi_factor) == ("auto", 95, 150.0, 3.0)
assert (p.fmt, p.jpeg_quality, p.min_dpi, p.max_dpi_factor) == ("auto", 95, 0.0, 3.0)
def test_policy_from_the_real_config():
......@@ -173,7 +173,7 @@ def test_policy_clamps_nonsense():
p = pi.Policy("auto", 9999, -5, 0.1)
assert p.jpeg_quality == 100 and p.min_dpi == 0.0 and p.max_dpi_factor == 1.0
p = pi.Policy("auto", "x", "y", "z")
assert (p.jpeg_quality, p.min_dpi, p.max_dpi_factor) == (95, 150.0, 3.0)
assert (p.jpeg_quality, p.min_dpi, p.max_dpi_factor) == (95, 0.0, 3.0)
# --------------------------------------------------------------------------- #
......@@ -183,7 +183,7 @@ def test_policy_clamps_nonsense():
def test_a_full_page_scan_is_passed_through():
pdfium = pytest.importorskip("pypdfium2")
pdf = pdfium.PdfDocument(_scan_pdf())
got = pi.extract_embedded(pdf[0], 200)
got = pi.extract_embedded(pdf[0], 150) # the fixture's own density
assert got is not None
mime, raw, size = got
assert mime == pi.MIME_JPEG
......@@ -192,6 +192,46 @@ def test_a_full_page_scan_is_passed_through():
assert Image.open(io.BytesIO(raw)).size == (1268, 1752)
def test_a_scan_coarser_than_the_TARGET_is_rendered_not_passed_through():
"""THE REGRESSION. This shipped with an absolute 150-DPI floor, so a
150-DPI scan reached a 250-DPI engine at native size -- 40 % less linear
resolution than the render it replaced -- and surya-ocr-2 lost Italian
accents on exactly those pages. Reported from production the same
afternoon.
Passing a scan through was always meant to be "the same pixels without the
work". Fewer pixels is not a saving, it is a different page.
"""
pdfium = pytest.importorskip("pypdfium2")
pdf = pdfium.PdfDocument(_scan_pdf()) # a 150-DPI scan
assert pi.extract_embedded(pdf[0], 250) is None, "a 150-DPI scan must NOT " \
"be sent to a 250-DPI engine instead of being rendered up"
assert pi.extract_embedded(pdf[0], 200) is None
# ...and it IS passed through to an engine that asked for no more.
assert pi.extract_embedded(pdf[0], 150) is not None
def test_min_dpi_only_raises_the_floor_never_lowers_it():
pdfium = pytest.importorskip("pypdfium2")
pdf = pdfium.PdfDocument(_scan_pdf())
# Asking for less than the target cannot re-open the hole.
assert pi.extract_embedded(pdf[0], 250, min_dpi=100.0) is None
# Demanding MORE than the target works.
assert pi.extract_embedded(pdf[0], 150, min_dpi=200.0) is None
assert pi.extract_embedded(pdf[0], 150, min_dpi=100.0) is not None
def test_surya_at_its_own_default_dpi_never_sees_a_coarser_scan():
"""surya_dpi defaults to 250 and surya is measured to read badly below it,
so the engine's own target is what the guard must compare against."""
from codai.config import OcrConfig
assert OcrConfig().surya_dpi == 250
assert OcrConfig().page_image_min_dpi == 0.0 # 0 = follow the target
pdfium = pytest.importorskip("pypdfium2")
pdf = pdfium.PdfDocument(_scan_pdf())
assert pi.Policy.from_cfg(OcrConfig()).extract(pdf[0], 250) is None
def test_a_scan_coarser_than_the_floor_is_rendered_instead():
"""75 DPI of information upscaled to 200 is not the same as 200 DPI of it."""
pdfium = pytest.importorskip("pypdfium2")
......@@ -218,7 +258,7 @@ def test_the_floor_tolerates_pdfiums_own_arithmetic():
pdf = pdfium.PdfDocument(_scan_pdf())
md = next(iter(pdf[0].get_objects())).get_metadata()
assert md.vertical_dpi < 150.0
assert pi.extract_embedded(pdf[0], 200, min_dpi=150.0) is not None
assert pi.extract_embedded(pdf[0], 150) is not None
def test_a_page_with_anything_else_on_it_is_rendered():
......@@ -276,8 +316,8 @@ def test_extraction_never_raises():
def test_the_png_policy_never_extracts():
pdfium = pytest.importorskip("pypdfium2")
pdf = pdfium.PdfDocument(_scan_pdf())
assert pi.Policy("png").extract(pdf[0], 200) is None
assert pi.Policy("auto").extract(pdf[0], 200) is not None
assert pi.Policy("png").extract(pdf[0], 150) is None
assert pi.Policy("auto").extract(pdf[0], 150) is not None
# --------------------------------------------------------------------------- #
......@@ -286,7 +326,7 @@ def test_the_png_policy_never_extracts():
def test_render_pages_passes_a_scan_through_in_process():
from codai.ocr.base import render_pages
pages = list(render_pages(_scan_pdf(), filename="s.pdf", dpi=200,
pages = list(render_pages(_scan_pdf(), filename="s.pdf", dpi=150,
raster_workers=1))
assert len(pages) == 1
got = pi.carried(pages[0])
......@@ -308,7 +348,7 @@ def test_the_rasteriser_pool_carries_the_bytes_across_the_pipe():
from codai.ocr import rasterize
rasterize.reset_pool()
try:
gen = rasterize.iter_pdf_pages(_scan_pdf(), 200, chunk=4, raster_workers=2)
gen = rasterize.iter_pdf_pages(_scan_pdf(), 150, chunk=4, raster_workers=2)
if gen is None:
pytest.skip("no rasteriser pool in this environment")
pages = list(gen)
......@@ -581,3 +621,157 @@ def test_the_worker_and_the_module_agree_on_the_attribute():
assert w._carried(img) == (pi.MIME_JPEG, raw)
w._carry(img, pi.MIME_JPEG, raw)
assert pi.carried(img) == (pi.MIME_JPEG, raw)
def test_the_floor_follows_THE_REQUEST_not_the_engine_default():
"""A client that asks for a DPI gets that DPI honoured, here too.
`ocr_document` resolves `use_dpi = int(dpi) if dpi else engine_dpi(engine)`
and passes it to the pod forward, to the local render AND (on the pod) to
this guard -- so "is this scan detailed enough" is asked about what THIS
request wanted, never about a default the caller overrode. A guard reading
the engine default instead would silently pass a 200-DPI scan to a request
that asked for 300.
"""
pdfium = pytest.importorskip("pypdfium2")
# A 250-DPI scan, as three different requests see it. Pixels derived from
# the fixture's own page size rather than hardcoded: 609 pt is not A4, and
# a number that assumes it measures 244 DPI while claiming 250.
def page250():
px_w = round(609.0 / 72 * 250)
px_h = round(842.0 / 72 * 250)
return pdfium.PdfDocument(_scan_pdf(px_w=px_w, px_h=px_h))[0]
assert pi.extract_embedded(page250(), 250) is not None # asked for 250: as good
assert pi.extract_embedded(page250(), 200) is not None # asked for less: better
assert pi.extract_embedded(page250(), 300) is None # asked for MORE: render it
def test_the_request_dpi_reaches_the_renderer_and_the_guard():
"""The plumbing the test above relies on, asserted rather than assumed."""
import inspect
from codai.ocr.manager import OcrManager
src = inspect.getsource(OcrManager.ocr_document)
assert "use_dpi = int(dpi) if dpi else self.engine_dpi(engine)" in src
# ...and that one value is what every read path is given.
assert src.count("use_dpi") >= 4
# --------------------------------------------------------------------------- #
# the rasterisation hop: encoded in the worker, not in the parent
# --------------------------------------------------------------------------- #
def _text_pdf(pages=1):
"""A rendered (not scanned) page, so the passthrough cannot apply."""
pdfium = pytest.importorskip("pypdfium2")
pdf = pdfium.PdfDocument.new()
for _ in range(pages):
page = pdf.new_page(595.28, 841.89)
path = pdfium.raw.FPDFPageObj_CreateNewRect(40, 40, 400, 600)
pdfium.raw.FPDFPage_InsertObject(page, path)
pdfium.raw.FPDFPage_GenerateContent(page)
out = io.BytesIO()
pdf.save(out)
return out.getvalue()
def test_a_rendered_page_arrives_from_the_pool_already_encoded():
"""The page a POOL rendered carries its own bytes, so the engine hop does
not encode a second copy of the same pixels."""
from codai.ocr import rasterize
rasterize.reset_pool()
try:
gen = rasterize.iter_pdf_pages(_text_pdf(), 250, chunk=4, raster_workers=2)
if gen is None:
pytest.skip("no rasteriser pool in this environment")
pages = list(gen)
finally:
rasterize.reset_pool()
assert len(pages) == 1
got = pi.carried(pages[0])
assert got is not None, "a rendered page must arrive encoded"
assert got[0] == pi.MIME_PNG # lossless by default
assert got[1][:8] == b"\x89PNG\r\n\x1a\n"
def test_the_pool_hop_is_lossless():
"""PNG, so the pixels the model sees are the pixels pdfium drew. A lossy
codec on this hop would change what is read for a saving on a local pipe."""
from codai.ocr import rasterize
from codai.ocr.base import render_pages
data = _text_pdf()
rasterize.reset_pool()
try:
gen = rasterize.iter_pdf_pages(data, 250, chunk=4, raster_workers=2)
if gen is None:
pytest.skip("no rasteriser pool in this environment")
pooled = list(gen)[0]
finally:
rasterize.reset_pool()
inproc = list(render_pages(data, filename="x.pdf", dpi=250, raster_workers=1))[0]
pooled.load()
assert pooled.convert("RGB").tobytes() == inproc.convert("RGB").tobytes()
def test_the_engine_hop_sends_the_pool_bytes_unchanged(monkeypatch):
"""What the move is FOR: one encode per page, not two."""
from codai.ocr import rasterize, subprocess_engine
rasterize.reset_pool()
try:
gen = rasterize.iter_pdf_pages(_text_pdf(), 250, chunk=4, raster_workers=2)
if gen is None:
pytest.skip("no rasteriser pool in this environment")
page = list(gen)[0]
finally:
rasterize.reset_pool()
sent = {}
class _Engine(subprocess_engine.SubprocessOcrEngine):
def __init__(self):
self.cfg = type("C", (), {})()
self._mux = False
self.name = "fake"
def _rpc(self, msg):
sent.update(msg)
return {"ok": True, "page": {}}
def _page_from_dict(self, d):
return d
class _Lock:
def __enter__(self): return self
def __exit__(self, *a): return False
_lock = _Lock()
_Engine().recognize_image(page)
assert sent["mime"] == pi.MIME_PNG
assert base64.b64decode(sent["image"]) == pi.carried(page)[1]
def test_the_operators_jpeg_choice_reaches_the_pool_hop():
from codai.ocr import rasterize
rasterize.reset_pool()
try:
gen = rasterize.iter_pdf_pages(_text_pdf(), 250, chunk=4, raster_workers=2,
policy=pi.Policy("jpeg", 90))
if gen is None:
pytest.skip("no rasteriser pool in this environment")
page = list(gen)[0]
finally:
rasterize.reset_pool()
got = pi.carried(page)
assert got is not None and got[0] == pi.MIME_JPEG
def test_a_worker_sent_no_policy_still_ships_a_page():
"""Raw RGB stays the fallback, so an older parent can still read this."""
import inspect
from codai.ocr.workers import raster_worker
src = inspect.getsource(raster_worker)
assert "img.tobytes()" in src
assert "if doc.policy is not None" in src
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment