• Stefy Lanza (nextime / spora )'s avatar
    size the summarizer by the window that refused us (v0.4.75) · cb3166dd
    Stefy Lanza (nextime / spora ) authored
    Both of these surfaced the moment a config said 131072 while the pod was still
    serving the 32768 it had been launched with -- a disagreement that is normal for
    a few minutes after an edit, and which compaction has to survive.
    
    **The summarizer was sized from the configured window.** It chunks the history
    it is about to summarize to fit the summarizing model's context, and that number
    came straight from the backend's configured value: 131072 against a pod serving
    32768, so every summarization request overflowed too. The slicer then caught the
    failure, left the oversized message exactly as it was, and compaction reported
    that it could not help. It now prefers a window an engine has already refused
    against, like the plan itself does.
    
    **The head's size cap was waived for its first message.** The intent was to keep
    at least one anchoring turn, but it meant a conversation opening with a pasted
    45,000-token document kept that document verbatim as the head, dropped the rest
    of the history, and overflowed anyway -- the one shape where compaction matters
    most. An opening turn too big to keep belongs to the slicer, which summarizes
    its middle.
    
    Also says on the model page what the RunPod "Max context" field actually is:
    the window the pod is LAUNCHED with, which the general context field above it
    does not reach, which takes effect on the next pod rather than the running one,
    and which the first model to create a shared pool sets for every model on it.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01S8LAP3ueNzQJtwzhgVbq66
    cb3166dd
Name
Last commit
Last update
codai Loading commit data...
docs Loading commit data...
packaging Loading commit data...
samples Loading commit data...
tests Loading commit data...
third_party Loading commit data...
tools Loading commit data...
.dockerignore Loading commit data...
.gitignore Loading commit data...
AI.PROMPT Loading commit data...
CODERAI_API_DOCUMENTATION.md Loading commit data...
CoderAI.gif Loading commit data...
DISTRIBUTION.md Loading commit data...
LICENSE.md Loading commit data...
MULTIMODAL_CAPABILITIES.md Loading commit data...
MULTIMODAL_UI_EXAMPLES.md Loading commit data...
README.md Loading commit data...
REQUEST-geoclip-embeddings.md Loading commit data...
REQUEST-vpr-embeddings.md Loading commit data...
amdgpu-coredump-05000.bin Loading commit data...
build-oci.sh Loading commit data...
build.ps1 Loading commit data...
build.sh Loading commit data...
coderai Loading commit data...
coderai-broker-implementation-reference.md Loading commit data...
coderai-integration.md Loading commit data...
commands Loading commit data...
osxbuild.sh Loading commit data...
package-oci.sh Loading commit data...
package-tarball.sh Loading commit data...
requirements-crisperwhisper.txt Loading commit data...
requirements-h3.txt Loading commit data...
requirements-longcat-train.txt Loading commit data...
requirements-longcat.txt Loading commit data...
requirements-melotts.txt Loading commit data...
requirements-nemo.txt Loading commit data...
requirements-nvidia.txt Loading commit data...
requirements-ocr-paddle.txt Loading commit data...
requirements-ocr.txt Loading commit data...
requirements-pyannote.txt Loading commit data...
requirements-surya.txt Loading commit data...
requirements-vllm.txt Loading commit data...
requirements-vulkan.txt Loading commit data...
requirements.txt Loading commit data...
run-oci.sh Loading commit data...
smoke-test-oci.sh Loading commit data...
todo.md Loading commit data...