- 06 Oct, 2026 23 commits
-
-
Stefy Lanza (nextime / spora ) authoredThree left over from earlier fixes in this work: - the frame tests asserted 93 was a valid TRAINING length. SimpleTuner's rule is 8n+1, not generation's 4n+1, and it rounds down silently — which is why frame_problems() rejects 93 and says a good clip can still be a bad sample. Reversed: the valid cases are 8n+1 now, and 93 has its own case asserting the message explains itself. - the trainer stopped dumping "Last output" when it started picking the error lines out of a bounded tail; the test still looked for the old wording. - the vendoring test counted the __pycache__ the container writes into the bind-mounted copy as "vendored non-source files". Generated, not vendored. Co-Authored-By:
Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0185zEBiy3KWB37xNGSrxL7w
-
Stefy Lanza (nextime / spora ) authored
Building the INT8 DiT needed ~42 GB because it materialised the bf16 model first; loading it needed ~55 GB for the same reason, plus every shard accumulated into one dict and then copied in. Both took the host down. Both now stream. The quantiser reads tensors one at a time through safe_open and flushes 4 GB shards, enumerating the Linear layers on a meta-device skeleton instead of a real model. _load_quantized_base_dit builds its skeleton on meta too, then assigns each shard into it as it is read. Measured on the 13.6B DiT: 51 GB bf16 -> 13.6 GB INT8, ~4 GB peak to write, and the load is 20 s at 0.6 GB RSS (safetensors mmap) with all 437 quantised Linears present and nothing stranded on meta. A checkpoint that leaves tensors on the meta device is now an error naming them, rather than a model that fails later somewhere unrelated. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Run against the real weights for the first time, it exhausted all 54 GB of RAM and all 4 GB of swap. MemAvailable reached 0 and I killed it to keep the server alive — the production container was two minutes from being the thing the OOM killer chose. It had loaded all six fp32 shards (13m42s) and reached "quantising every Linear to INT8". The peak is not the bf16 model alone: the INT8 copies are built while the originals are still referenced, and save_quantized_state_dict then materialises a full state dict on top. Roughly 2x bf16, not 1x. Three changes: - It now REFUSES when the host cannot take it, with the numbers: "~42 GB needed, 34 GB available". Estimated from the shards on disk (fp32, so bf16 is half) times 1.5, plus 4 GB for the interpreter and torch. min_free_gb overrides it for someone who knows better. A job that can kill the server should not be reachable from a button without this. - The load streams through a meta-device skeleton (low_cpu_mem_usage=True) instead of building a second full copy while loading. - gc.collect() between the phases, and the model is dropped before the save returns — the replaced bf16 weights are ~13 GB of garbage at exactly the moment the save wants to allocate. What this means on THIS host, honestly: 42 GB needed against 54 GB total with the server using ~20 GB. It will now refuse rather than try. Quantising here needs the server stopped, or a lower-peak implementation that never materialises the whole model (quantise shard by shard, write as it goes) — which is the real fix and is not in this commit. 22 tests. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Asked to stop an offloaded render, I killed the service process by hand — and had to, because nothing else could. Every diffusers video path calls task_registry.raise_if_cancelled() between denoise steps and shows up on the Tasks page; LongCat was never registered at all. So the LONGEST-running video job in the system was the only one with no stop button: a 93-frame bf16 render offloading across PCIe can go for a very long time. Its poll loop claimed otherwise. The docstring said it would "carry cancellation and the thermal gate across the process boundary" and the body did neither — it only mirrored progress. A comment describing intent rather than behaviour. It is now registered like the others, reports step progress to the task, and polls for cancellation every 2s. Because the work happens in ANOTHER PROCESS, cancelling cannot raise inside it: the service is stopped instead, which is exactly what eviction already does to it, so that path is well exercised. The request then ends 499 rather than returning a partial video as success, and the task is closed on every exit — cancelled, error or done. I also owe a correction: I told the user there was no cancel for /v1/video/generations. There is, for every other model — I searched for a /v1/video/* endpoint and missed the task registry. The gap was LongCat specifically, which is code I wrote. 7 tests. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
v0.2.62 added gcc to Dockerfile.oci. This install builds with --venv, and build_oci_image.sh uses Dockerfile.oci-venv for that — so the image came out without a compiler and surya would have gone on falling back exactly as before. Caught by checking inside the built image rather than trusting the build's exit code. The test had the same blind spot: it asserted against one Dockerfile. Both can ship, so it now checks both, parametrised by file. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Why a remote host was sending PaddleOCR requests: surya never ran. Every attempt ended as [ocr] engine 'surya' could not serve (vLLM exited (code 1) before becoming ready) … falling back to paddle -> doctr and the root cause, four frames down in the vLLM subprocess output: RuntimeError: Failed to find C compiler. Please specify via CC environment variable or set triton.knobs.build.impl. Triton JIT-compiles its CUDA driver module at RUNTIME, on first engine start. The image has no cc, gcc or clang — confirmed by looking inside it; the single dpkg match is gcc-*-base, runtime libs, not a compiler. So gcc and libc6-dev are now installed in the shipped stage, with the reason recorded beside them: a compiler in a runtime image looks like an obvious thing to delete. This affects EVERY vLLM engine, not just OCR. What made it invisible: the OCR fallback chain worked. All 203 /v1/ocr requests in the window returned 200 — on paddle, 218 times, after surya failed. Nothing in the HTTP log said anything was wrong, and the only outward sign was a remote host preferring an engine it should not have needed. The RunPod vLLM capability profile has carried gcc from the start. The main image never did. The test checks both so they cannot drift apart again. Needs a rebuild to take effect — a compiler cannot be bind-mounted in like the Python fixes. 4 tests. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Three clip generations failed while OCR and other work was running, with: RuntimeError: the venv at /AI/guffcache/longcat_venv was built but still does not report Python 3.10 with torch 2.6 / transformers 4.41 The venv reports exactly that. It holds torch 2.6.0+cu124, transformers 4.41.0, numpy 1.26.4, on Python 3.10.18 — read from its own dist-info, and the probe passes when run by hand. The check imports torch from a 6.2 GB venv. Timed on an IDLE box: 106 seconds, against a 120 second limit. Under the load that necessarily accompanies a generation request it expired, TimeoutExpired was caught by a bare `except Exception: return False`, and the caller turned "I could not finish asking" into "the answer is no" — then rebuilt, re-probed, and failed again. Both probes are now tri-state: True verified, False a real mismatch, None could not be determined. An unverified venv is USED; only a verified-bad one is rebuilt or rejected. The limit is 900s, and a genuine failure now prints what the probe actually said instead of asserting versions nobody checked. Nothing was wrong with the OCR/paddle side, and there were no VRAM, eviction, swap or queue failures in the window — the 1626 apparent "503"s are those digits occurring inside OCR'd Italian legal documents in the request debug dumps, and the 1792 "ERROR"s are mostly the same plus vLLM warnings. The only real exceptions in two hours were this one and the LongCat training chain. 32 tests. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Two rules, and I had conflated them. Generation obeys the VAE's 4n+1 — 93, 173 and 333 are whole LongCat segments, which is what the dataset builder and the township templates use. TRAINING is stricter: SimpleTuner's longcat_video family rounds DOWN to 8n+1 (frames % 8 == 1) and says nothing about it. So a job asking for 93 trained at 89, the config recorded 93, and frame_problems() validated the generation rule and passed it. Three places agreeing on a number that was not the one being used. training_frames() now applies the same rounding SimpleTuner does — verified against its adjust_video_frames() for every count from 0 to 400 — and the job is adjusted BEFORE validation, so a legal clip length is never rejected for being legal in the other sense. The adjustment is emitted to the progress stream and the log: num_frames 93 -> 89 (8n+1) frame_problems() now states the training rule and explains why the number looked valid, rather than asserting a constraint the trainer does not use. Found by reading SimpleTuner's model.py while answering a question about whether clips are needed at all — not from a failure. It would have shipped as a quiet discrepancy between the configured and actual frame count. 23 tests. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The CONFIG_PATH fix worked — SimpleTuner read the config, mapped it to CLI args and got as far as argparse, which then said: argparse.ArgumentError: the following arguments are required: --optimizer It has no default for it. The config now sets adamw_bf16, overridable per job. That choice matters here rather than being arbitrary: Adam normally carries two fp32 moments per parameter, which for a 13.6B base is more optimiser state than the card has. base_model_precision quantises the weights; adamw_bf16 keeps the state in bf16. Together they are what make this trainable on one consumer GPU. Also fixed how a SimpleTuner failure is reported. It raised with the last six lines of output, which for a traceback is frame noise — the line naming the cause was above the cut, which is why the first look at this error showed argparse internals and not the message. It now prefers lines that look like a cause (Error/Exception/required:/not found) and falls back to the raw tail. Verified against the installed package: every key the config emits exists in SimpleTuner's field registry, and adamw_bf16 is in optimizer_choices. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The venv built, SimpleTuner 4.9.3 installed, and the trainer RAN — the first time anything past the setup has executed. It then failed on how it was invoked: ValueError: JSON configuration file not found. Paths tried: config/config.json SimpleTuner takes no --config. Its loader picks a configuration BACKEND and asks that backend where to look; the JSON one reads CONFIG_PATH (a file, or a directory holding config.json) and otherwise defaults to ./config/config.json relative to the working directory. The --config we passed was simply ignored, so it fell through to the default and found nothing. Now the backend is named (SIMPLETUNER_CONFIG_BACKEND=json) and the path handed over in CONFIG_PATH, read from the installed package's own json_file.py rather than guessed. The same config is also written to <workdir>/config/config.json, so a loader change that ignored CONFIG_PATH would still find it. Worth recording what this run proved, since all of it was unexercised an hour ago: the vendored source resolves, the training venv rebuilds itself with a 3.12+ interpreter, SimpleTuner installs from PyPI into it, and the job reaches SimpleTuner's own argument parsing. The dataset built from the township clips has not been read yet — that is the next thing in the way. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The 3.10 attempt did not just fail — it left /AI/guffcache/longcat_venv-train behind, pyvenv.cfg saying version = 3.10.18. ensure_train_built skips creation when the directory exists, so the next run would have pip-installed into that same 3.10 and failed identically, forever. Fixing the interpreter choice in v0.2.56 was not enough on its own for anyone who had already tried once. An existing training venv is now checked against what SimpleTuner needs (>=3.12,<3.15) and rebuilt when it does not qualify, saying so in the log. The version is READ from pyvenv.cfg rather than obtained by running the interpreter: the Python that built it may be gone, and a failed exec is indistinguishable from a venv that is merely incomplete. Found by looking at the directory before telling the operator to press the button again, which is the only reason it is in this commit rather than in their next log. 24 tests. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The new diagnostics earned their keep immediately. Last run said only "returned non-zero exit status 1"; this one said: ERROR: Could not find a version that satisfies the requirement simpletuner (from versions: none) ERROR: No matching distribution found for simpletuner which reads like the package does not exist. It does. The line above it is the real one: "Ignored the following versions that require a different python version: … Requires-Python >=3.12,<3.14". SimpleTuner 4.9.3 needs >=3.12,<3.15, pip was being run under 3.10, so every release was filtered out and the remainder was none. My error. The 3.10 pin belongs to the INFERENCE venv — torch 2.6+cu124, transformers 4.41, numpy 1.26 — and I applied it to the training venv, which wants the opposite. They are separate venvs precisely because their requirements conflict; sharing an interpreter finder was the bug. ensure_train_built now picks an interpreter SimpleTuner can install into: a configured longcat.train_python, else this process's own (3.13, which satisfies >=3.12,<3.15), else python3.13/3.12 on PATH. So the training venv — unlike the inference one — needs nothing bundled and nothing provisioned. The manual instructions in the non-auto branch said python3.10 too; they now say 3.12+. Also renamed _py310_env to _clean_py_env, since it now guards a 3.13 child as well as the 3.10 one. The venv itself WAS created this time, so the earlier "No module named 'encodings'" did not recur — whether the stripped environment fixed it or it was transient, I still cannot say. 17 tests, including that ensure_train_built never reaches for the 3.10 finder. Co-Authored-By: -
Stefy Lanza (nextime / spora ) authored
training failed: Command '[…py310…, '-m', 'venv', '/AI/guffcache/longcat_venv-train']' returned non-zero exit status 1. An exit status is not a diagnosis. The actual reason was in the log only because the child's stderr happened to land there: Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding ModuleNotFoundError: No module named 'encodings' Every step of the venv build went through subprocess.run(check=True), which reports a number and discards what the process said. They now go through a runner that raises with the child's output, truncated at both ends if it is long. The next failure here names itself instead of needing a log archaeologist. The environment is also stripped for anything that spawns the 3.10 interpreter — PYTHONHOME, PYTHONPATH, VIRTUAL_ENV, PYTHONEXECUTABLE and friends — because that is exactly the class of variable that produces the error above, and the failure is fatal and silent in the parent. HF cache paths, CUDA_VISIBLE_DEVICES and everything else still reach the child. This covers the venv build, the pip installs, the service launch and the train/quantise workers; the service used dict(os.environ) directly before. Worth being straight about: I could NOT reproduce the failure. The same command from a throwaway container on the same image, same user, same mount, exits 0 and writes a complete venv, and the engine process has no PYTHONHOME, PYTHONPATH or VIRTUAL_ENV in its environment. So this is not a confirmed fix for that specific failure — it is the removal of the reason I cannot tell you what went wrong, plus hardening against the one cause the error text points at. The retry will either work or finally say why. 12 tests. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Caught by running the validators against the real checkpoint the moment it finished downloading: variant_problems('int8') -> ["INT8 weights exist only for avatar-1.5; this checkpoint reads as the base model"] That guard predates the quantise action and contradicts it: the model page can now BUILD base_model_int8/ for any checkpoint, so the button would have produced the weights and this would have refused to load them. The feature shipped one commit ago with a gate that made it useless. What decides it now is whether the weights are present, not which family wrote them — upstream still ships INT8 only for avatar-1.5, and that is still accepted without a local build, but a base checkpoint with base_model_int8/ is served too. Without them the refusal names the remedy ("use Build INT8 weights on the model page first") instead of stating a family rule the operator cannot act on. The checkpoint directory is passed in for that, since the gate could not otherwise see what had been built. Checked against the real 78 GB download: checkpoint_problems is clean, the vendored source passes source_problems, bf16 is ungated, int8 is refused before a build and accepted after, and upstream's avatar-1.5 path is unchanged. 16 tests. Co-Authored-By: -
Stefy Lanza (nextime / spora ) authored
A 13.6B bf16 DiT is ~27 GB: it does not fit a 24 GB card and streams across PCIe every step. The INT8 copy fits. So the model page gets a "Build INT8 weights" button beside the existing "INT8 DiT" switch — the switch said which weights to USE, and nothing could produce them. Deliberately an action, never a default. It is lossy and slow, the bf16 weights are never touched, and un-ticking the switch goes straight back to them. A test asserts nothing turns use_int8 on by itself. It runs in LongCat's isolated py3.10 venv (upstream's helpers, torch 2.6), so it is a worker script like training, not in-process — tools/longcat_quantize.py, driven by longcat_worker.quantize_int8() on the same job/JSON-lines protocol, so the existing progress plumbing needed no special case. Quantisation happens on the CPU on purpose: the point is to produce a model that fits the card, so getting there must not need the card. One thing upstream's code could not do for us. load_quantized_dit() hardcodes the AVATAR transformer class — correct for its avatar-only INT8 build, wrong for a quantised BASE checkpoint. The avatar families keep using it; the base family gets _load_quantized_base_dit(), the same procedure (build from the saved config, swap every non-skipped Linear for QuantizedLinear, load the sharded state dict) against LongCatVideoTransformer3DModel. Upstream's QuantizedLinear and skip patterns are reused rather than restated, so the layout cannot drift from what wrote it. The existing /admin/api/model-quantize routes a longcat-backed model here instead of GPTQ/AWQ, which are for transformers this is not; quantize-status reports the job the same way, so the page polls one endpoint. A model whose weights are not downloaded yet is refused rather than silently started. Not yet run against real weights — the 83 GB base checkpoint is still downloading. 43 tests. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
`longcat_video` — the pipeline, DiT, VAE, scheduler and INT8 quantisation helpers — lives inside the upstream repo and is not on PyPI, so resolve_source_dir looked for a checkout at ~/.coderai/LongCat-Video or a path baked at build time. Neither exists in a published image, which means anyone pulling it had to clone from GitHub before LongCat would run at all: a third-party network dependency in the middle of someone else's install, on whatever `main` happened to be that day. The 26 source files are now vendored under third_party/, MIT, pinned to commit 6b3f4b85. 496 KB, no weights, no wheels, nothing compiled. The resolver prefers them over every fallback, and an explicit longcat_source still wins for anyone running their own checkout. They ship automatically: the image does COPY . /opt/coderai/app, and third_party is not in .dockerignore — asserted by a test, since that is the sort of thing a later ignore rule breaks silently. The ISOLATED VENV is unaffected and still builds on first use from requirements-longcat.txt. Only the source is vendored; torch 2.6+cu124 and the rest are installed, not carried. While confirming the fp8 question: upstream's quantization.py implements INT8 only — QuantizedLinear holds weight_int8 + weight_scale, and there is no fp8 code anywhere in the package. The fp8 checkpoints on HF carry no loader either, and szwagros/LongCat-Video-Avatar-1.5-fp8's own card is a verbatim copy of upstream's and documents `--use_int8`, never fp8. So a VRAM-resident DiT goes through INT8, not fp8. 88 tests. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
✗ training failed: _lora_dir() missing 1 required positional argument: 'name' Progress, in that everything before it worked: the Train button built the dataset, the request carried dataset_config, the server accepted it and dispatched to the LongCat path. It then died on the first line of _train_externally that does real work — proof that function had never once executed. Two bugs in one line. _lora_dir(name) already resolves the per-LoRA directory AND routes a non-main DDP rank to scratch; the call treated it as the parent and joined the name on again, so even with the argument supplied it would have written <loras>/<name>/<name> and bypassed the DDP handling. The test calls the real _train_externally with only the worker stubbed — no GPU, no checkpoint, no venv — and checks it reaches the worker, that output_dir equals _lora_dir(name) with the name not doubled, that the job carries steps/rank/seed/ num_frames/dataset_config, that both refusal paths still refuse, and that the worker's **kwargs progress callback matches the receiver's signature (a mismatch there would only show up mid-training). Verified to FAIL on the real defect: restoring the old line turns 4 of the 6 red with exactly the TypeError above. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The log shows the 401 is gone — the latest attempt was accepted (200 OK, job-551586975a13) and reached the trainer. It then failed on the SAME error as before the key was fixed: RuntimeError: longcat LoRA training needs `dataset_config` Because there are two training paths and only one of them was wired. Step 4b (the batch stage) builds a SimpleTuner dataset from the rendered clips. The per-profile Train button on the Characters page — which is what was being pressed — assembled its kwargs independently and never passed dataset_config, so the server refused the job exactly as it is supposed to. Both paths now build the same dataset through the same function. When it cannot (no clips rendered yet, clips too short, too few), the button says so and names the remedy instead of letting the server answer with a stack trace. Surveyed against the real township output before claiming it works: 151 clips and outcomes carry prompts, and at the default 93-frame / 8-clip threshold all 6 fighters and all 4 environments qualify — khumalo has 19 usable clips, dlamini 11. Still true, and unchanged by this: no LongCat training has ever RUN. This gets the job past the server's gate; what SimpleTuner then does with it is unproven. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The 401 on video LoRA training was not a missing key. It was a DELETED one. Loading a template persisted by writing the template over township_config.json: _j.dump(tcfg, f, indent=2, sort_keys=True) A template is a PARTIAL — the LongCat starter carries 10 options, the config carries 60. So loading it turned the operator's config into those 10 keys and destroyed everything else, api_key and base_url included. Every subsequent call to CoderAI then failed with a bare 401 that said nothing about why. Found by comparing the live file against its own backups: township_config.json was 238 bytes with exactly the LongCat starter's 10 keys, timestamped during this session; every backup beside it was ~1700 bytes with 60 keys and a 75-character api_key. The token itself was never gone — it is still 'fighters' in auth.json. It now reads the existing config, updates it with the template, and writes the result. A template's values win; nothing it is silent about is touched. With no readable config yet, the template is the config, as before. This is mine, from 310ffc79 ("loading a template persists, not just applies") in v0.2.39. The persistence was right; replacing rather than merging was not. The operator's config has been restored in place from the newest full backup with the LongCat values kept on top: 60 keys, api_key present again. Two tests: that a merge keeps what the template omits, and that the handler no longer dumps the template over the file. Co-Authored-By: -
Stefy Lanza (nextime / spora ) authored
Training a video LoRA from the Characters page returned: POST /v1/loras/train -> 401: Invalid API key. Provide a valid Bearer token. CoderAI gates every /v1 endpoint through the same dependency and makes NO exemption for loopback, so the bundled tools — which call 127.0.0.1 from inside the same container — need a Bearer token exactly like a remote client. The container launches this tool without --api-key and its saved config had none, so there was no token to send. Nothing was wrong with the training request itself. Fourteen call sites each built their client from default_args.api_key directly. They now go through one resolver with a stated order: the running options (set by --api-key, a saved config, or the Connection card) → the config file on disk → CODERAI_API_KEY in the environment, which is how a launcher can supply it without a config edit. A blank value is treated as absent rather than sent as "Bearer ", which would have failed in a more confusing way. A 401 now says what to do — where to put a key in THIS tool — instead of echoing a server message that tells an operator nothing about this UI. The immediate fix needs no rebuild: paste the key into the Connection card on the Run page and press Save config. That writes township_config.json and updates the running options, so the next action authenticates. 11 tests, including that no call site reads default_args.api_key any more. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Uploading a photo to a fighter was never missing — the Characters page has had a file picker and a working /profile/upload-image endpoint all along. It LOOKED missing because the page's whole <script> block has been dead since v0.2.41 (fixed in 11b0f446), so the button did nothing at all. Anyone on 0.2.41-0.2.46 sees a picker that ignores them. What was genuinely absent is the other half: pasting a link. That now feeds the SAME endpoint, so a fetched picture goes through the same magic-byte check (the extension is never trusted), the same ref_NN numbering, and the same patch_profile sync to CoderAI. The caller gets one answer covering both sources. I wrote a second endpoint for this first and deleted it: /profile/upload-image already did everything except fetch, and a parallel route would have been a second place to keep the validation, the numbering and the sync correct. The fetch is guarded the same way as the character studio's, for the same reason — the request is made by the tool, which can reach coderai on 127.0.0.1 and the whole LAN: - every hop must resolve to a public address, checked by RESOLVING the host rather than matching the string; - redirects followed by hand, five hops, each re-checked (a public URL that 302s to 127.0.0.1 is the usual way past a one-time check); - http/https only; 64 MB ceiling enforced while streaming; - the content must decode as PNG/JPEG/WebP/GIF by magic bytes. --allow-private-fetch opts out, for pulling from your own NAS. Verified against a real profile: an uploaded JPEG became ref_01.jpg, a public link became ref_02.png, and a link to 127.0.0.1:8776 was refused with the reason. "synced": false is reported honestly when CoderAI is unreachable rather than claiming the profile was updated. 131 tests. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Paste a URL to a photo and the studio downloads it into the same uploads/ directory a picked file goes to. Nothing downstream changes: face extraction, the profile, the keyframes and LoRA training cannot tell the two apart, which is the point — "extract the character from it" is the pipeline that already exists. The part worth reviewing is that the STUDIO makes the request, not the browser. On this box that process can reach coderai on 127.0.0.1:8776, every admin API it exposes, and the whole LAN. So: - every hop must resolve to a public address. Private, loopback, link-local, reserved, multicast and unspecified are all refused, by resolving the host and checking the ADDRESS rather than pattern-matching the string; - redirects are followed by hand, five hops maximum, re-checking each one. A public URL that 302s to 127.0.0.1 is the standard way past a check done once; - only http/https. ftp://, file:// and the rest are refused by scheme; - 256 MB ceiling, enforced against Content-Length AND while streaming, since a server can lie about the first; - the content must be an image or a video, by URL suffix or, for the many image URLs with no suffix at all, by content type. --allow-private-fetch turns the address rule off for someone pulling from their own NAS. It is a flag rather than a default so it is the operator's decision. Verified against a running studio: loopback, 192.168/16, 169.254.169.254 and ftp:// are all refused with the reason; a real public image over the internet and a local one with the flag both land in uploads/ with the extension resolved from the content type when the URL had none. The tests also parse the page's script with node — the township page shipped a dead <script> for four versions because nothing ever parsed what it served, and this page now has a reason not to repeat that. 125 tests. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The template picker rendered as a box you could not choose from. The cause is much worse than the symptom: the main <script> block has not parsed since v0.2.41, so NOTHING in it ran — status polling, the template buttons, the per-stage buttons, Save config, Stop. The page still rendered perfectly, which is why screenshots looked right and nothing in the logs complained. One character did it. The page is emitted from a Python f-string, so side.textContent = running ? ('Running\n'+(label||'')) : ... puts a REAL newline inside the JS string literal. The literal never closes, the block dies with a SyntaxError, and every function it defines is gone. It needed \\n, which is what it has now. Found by probing the server: the browser never requested /templates, nor /status, nor anything else the init block does — and a marker written as the first statement of that block never ran either. That ruled out the picker and pointed at the parse. Two things in the picker itself, both real: - tplRefresh() was only ever called AFTER a save or a delete, from the first templates commit (c1a65e1d). A freshly loaded page never listed what was on disk, so the picker was empty even with the script working — and my v0.2.40 claim that "the picker is never an empty box" was simply wrong. - The select had no options at all before its first fill, which is why it read as an input field rather than a dropdown. It now ships a placeholder and is width- constrained. Verified against the real templates directory: all four script blocks parse, and the picker lists "big frames", "LongCat - few long scenes" and "Wan - many short scenes" with their dates and option counts. The test starts the real server, fetches the real page, and parses every script block with node. It is verified to FAIL on the defect: reintroducing the bare \n makes it fail with the SyntaxError. A dependency-free scanner was written first and deleted — it reported the known-broken block as clean, and a check that passes on a file proven broken is worse than no check. 115 tests. Co-Authored-By:
-
- 05 Oct, 2026 17 commits
-
-
Stefy Lanza (nextime / spora ) authored
The page was slow to open on a large configuration. Measuring first, because the obvious suspect was wrong: the disk scans are not the problem — scan_cache_dir over 59 repos is 0.42s and the GGUF walk 0.53s, and _scan_caches() is 0.08s once warm. The server-side stale-while-revalidate cache was already doing its job. What it was, in order of cost: THE CACHE DIED WITH THE PROCESS. It starts empty in every new one, and _cached() recomputes synchronously when it has nothing — so the first read after a restart pays the whole scan while someone sits looking at a spinner. Measured on the real configuration: 3.69s cold, 0.001s once the previous result is read back from disk. It is now persisted (atomically, under <config>/cache/) and seeded on first use. The stored SIGNATURE is deliberately not restored: it is a list of mtimes, and trusting it across a restart would serve a possibly hours-old list without revalidating. Leaving it unset means the stored result is returned instantly AND a refresh starts behind it. A missing, truncated or future-version file just falls back to scanning. THE PAGE WAITED ON ITSELF. First paint ran refreshLoadedStatus, then refreshQuantJobs, then the list — three round trips in a chain — while _loadEngineInfo did two more in another chain and then rendered the list a SECOND time. Inside the list load, /admin/api/cached-models and /admin/api/models also went out one after the other. Now: the two status reads go out together, the two list reads go out together, and concurrent loadCachedModels() calls collapse into one in-flight load, so first paint does one fetch and one render instead of three. A call made after it settles still refetches, which every post-action refresh relies on. IT PAINTED NOTHING UNTIL ALL OF THAT FINISHED. The last response is kept in sessionStorage and painted before any network call, then revalidated in place. It is a listing of what is on disk, so seconds of staleness cost nothing — and a quota error or private mode just means the page behaves as it does today. One thing the tests had to be corrected on: the first version asserted the background rescan had not run, which failed because _cached() starts the refresh thread and then reads state["data"] — with a trivial scan function the thread wins the race. That race is harmless (fresher data), and it was the assertion that was wrong: the property worth holding is that the CALLER is not blocked, so the test now uses a slow scan and asserts the read returns in under 0.2s with the stored value, and that the revalidation really did start. 112 tests. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
"Free disk" on the colibri GLM-5.2 model returned 200 OK three times and freed 4 KB. The 400 GB was in /AI/offloads/colibri/models--mastouri--…, and the HF cache held that model as a SYMLINK into it — so _do_delete_model deleted the link's entry, found nothing else it knew about, and truthfully reported success. Two separate failures, fixed separately. IT DID NOT SAY HOW MUCH. Freeing 4 KB and freeing 400 GB were the same {"success": true}. Every branch now measures what it removed and returns freed_bytes/freed_human; the UI prints it, and says so plainly when the answer is zero ("Nothing was on disk for this model — freed 0 B. Its files may live somewhere coderai has not been told about."). This is the part that would have made the problem visible on the first click instead of the third. IT ONLY LOOKED IN TWO PLACES. codai/models/weight_stores.py adds the rest: - The configured offload directory AND its siblings, one level. Engines are pointed at sibling roots by convention — coderai offloads to …/offloads/nvidia and colibri's weights landed in …/offloads/colibri. The scan widens where we LOOK; it never widens what we delete, which stays an exact models--<repo> name match. Nothing is ever removed by prefix or pattern. - register_weight_root() for an engine that does not follow that convention. Deliberately not called from colibri_worker/ds4_worker: at import time config_manager is not loaded yet, so the offload directory is '' and the registration would be dead code. The sibling scan runs at DELETE time, off config, and covers both. Measuring the size honestly took two passes: - symlinks are not followed or counted (an HF snapshot is a tree of links into blobs/, so following them doubles the number); - hardlinks are counted once by (device, inode), since a cache that links rather than copies would otherwise report the weights once per link. And purging a store leaves the cache entry that pointed INTO it dangling, which still looks like a present model to anything that only stats the name — so dangling links to a purged store are pruned too. Verified against the real model before touching it: a dry run found exactly one path, 399.8 GB, and nothing else among the nine roots it searched. The delete then freed it — /dev/nvme0n1p4 went from ~96% to 1% used, 444 GB free. (Worth recording: I first read df on /AI and reported the space had NOT come back, before noticing /AI/offloads is its own mount. Same wrong-filesystem mistake as earlier in this session. The deletion had worked the whole time.) 100 tests. Co-Authored-By: -
Stefy Lanza (nextime / spora ) authored
Step 4b failed in production: RuntimeError: longcat LoRA training needs `dataset_config` — a path to a SimpleTuner data-backend config describing your captioned video clips The refusal is right: LongCat trains through SimpleTuner, which wants captioned VIDEO clips, and synthesising a dataset from a fighter's reference stills would train something other than a video LoRA. But the UI offered no way to supply one, so the button could not succeed — and the tool was already sitting on exactly the corpus SimpleTuner asks for. Every clip it renders was rendered FROM a prompt, and that prompt is kept beside it in videos/prompts.json. So it now assembles the dataset rather than demanding it. Per fighter and per location, against a LongCat video model: <out>/video_datasets/character_<name>__<slug>/ clips/0000.mp4 -> ../../../videos/match_a_vs_b_clip03.mp4 clips/0000.txt the prompt that produced those pixels cache/vae cache/text data_backend.json sent as dataset_config manifest.json which render each clip came from Decisions worth stating: - SYMLINKS, not copies. A fighter appears in dozens of clips; a dataset per fighter would otherwise duplicate the whole render per fighter. - Captions come from the PLAN, never from filenames, so a caption is always the text that actually produced the pixels. - _2x/_2xfps derivatives are never picked up. They are the upscaler's and the frame interpolator's output; training on them teaches their artefacts. - An outcome clip is training data for BOTH fighters — they are both on screen. - Clips shorter than the training frame count are filtered out HERE, because SimpleTuner's loader drops them silently. A dataset where every clip is too short refuses with the numbers: "all 12 clips are shorter than 93 frames (longest 70)". Likewise "only 3 usable clips, need 8" and "no rendered clips yet (run the Videos step first)" — a skip with a reason, not a stack trace. - Rebuilt from scratch each time, so a symlink to a deleted render cannot fail deep inside SimpleTuner's loader instead of here. Two knobs on the Run page, in the config and on the CLI: Training clip frames (default 93 — one whole LongCat segment, and 4n+1 as the VAE requires) and Minimum usable clips (default 8; upstream suggests 50-100, this is a floor so a part-rendered show does not quietly train on three). The backend schema is upstream's documented one, including exactly one text_embeds backend carrying default: true. The client sends dataset_config only when it has one; the server has always accepted the field and the in-process Wan path ignores it. Checked against the real township render: 151 sources (102 clips + 49 outcomes), 19 usable for the busiest fighter at 93 frames. 82 tests. Co-Authored-By: -
Stefy Lanza (nextime / spora ) authored
Found in a production log, during a FULL VRAM CLEANUP ahead of a LoRA training job: Warning in external VRAM releaser: release_f5_engine() takes 0 positional arguments but 1 was given The manager calls every external releaser as `fn(needed_gb)` and sums what they return. release_f5_engine() declared no parameter, so every call raised TypeError straight into the manager's warning path: the cached F5-TTS engine held its VRAM through every eviction, including the exclusive ones that exist precisely because training needs the whole card. It failed quietly — the warning is one line in a cleanup that otherwise reports success. It now takes needed_gb and returns the VRAM delta it observed, like the other four releasers. The test checks the shape statically, across every register_external_vram_releaser call in the tree, so the next releaser cannot be registered with a signature the manager cannot call. Written against the real contract: one positional argument, and a value returned. (It first asserted a default was required too, which flagged _cosite_vram_releaser — but the manager always passes the argument, so that was the test inventing a rule.) 11 tests. Added with -f: .gitignore ignores test_*.py by default. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
The 0.2.40 "restyle" rewrote the stylesheet into tokens and kept every class and every piece of markup, so it rendered as the same page in a slightly different grey. It also could not have shown: the markup carried 270 inline style attributes, 32 of which re-shrank a button below the type scale it had just been given, and 47 of which hardcoded a colour. This changes the structure. THE SHELL. Three regions that do not move, instead of one 1180px column: - A left sidebar with the six sections, each one an icon plus a label, and the active one carried by aria-current. Under ~900px it becomes a strip of icons above the content rather than eating a third of a phone. - A sticky top bar holding the page title and, on the Run page, the controls that start, stop and persist a run. Starting a render used to mean scrolling past 66 settings to reach the button. The status pill sits with them. - A log dock fixed to the bottom. Watching a render used to mean scrolling to the end of the page and staying there. It remembers whether it was open, and opens itself the moment a run starts, which is the one time it is worth reading. Templates and the per-stage buttons move from under the settings to a card above them, in the order they are used: pick a template, see it in the form, run a stage. The Refresh links on Characters/Environments/Matches move to the top bar, and the per-page <h1> goes with them, since the bar already names the page. THE FORM. Above 1220px the cards flow in two columns, which roughly halves the height of a 66-setting form; the four wide cards span both. Each card is a container query root, so a row inside a half-width card collapses to the card's width rather than the window's. A label that wraps to two lines no longer pushes its field out of line with the one beside it. A BUG THIS SURFACED. The wardrobe inputs declared no type=, and [type=text] does not match a missing attribute, so every outfit field had been rendering as a white browser-default box on a dark page. The selector now covers them. The inline styles are mostly gone: 46 button size attributes folded into .btn-sm /.btn-xs, and 144 hardcoded hex values replaced by tokens, including four hues (violet, sage, tan, deep green) that the markup had been spelling out by hand and that are now declared once. THE STARTERS, answering what they left out. Both only carried video settings, so switching to LongCat left the outcome stings at Wan's 96-150 frames — neither 4n+1 nor segment-aligned, so the server had to round them. Both now also set the outcome range (LongCat: 93, one whole segment, to 173), frame interpolation (off for LongCat, which does not need it at 15fps), the upscale factor, the keyframe size and the match count (3 for LongCat, where a scene is ~4x the frames). Both cover the same keys, so switching between them is deterministic. What they still do not set, deliberately: the character and environment stages. Those are image-side work — the video model never sees the pool — so a template that pinned the pool size or the reference-image budget would reset an operator's setup for no reason. All six pages rendered and screenshotted at 1600px and 820px. 53 tests. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
THE UI. One style block, ~40 hardcoded hex values, 0.72-0.85rem type throughout, caps section labels, no focus ring, and an infinite striped animation with no reduced-motion guard. Rewritten as a token system, keeping every existing class so no page markup had to change: - Palette from the subject rather than a template: cool graphite (not a tinted black), title-belt gold for the single primary action, and red/blue corner as a SEMANTIC pair — every bout is fighter 1 vs fighter 2, so those two carry meaning here. - Numbers set in a monospace with tabular figures: frame counts, odds and durations are compared down a column. - 16px fields everywhere, not just on mobile — .85rem was cramped on a desktop too, and 16px is what stops iOS zooming on focus. - Numeric inputs capped at 11rem. A frame count is three digits; a box stretched across half the viewport makes the form longer to scan and harder to aim at. - Section headings instead of tracked-out caps labels, which were harder to read and made every group shout. - Sticky nav, with the active page carried by aria-current — it drives the underline AND tells a screen reader where you are, where the old inline colour did neither. - :focus-visible everywhere, and prefers-reduced-motion collapses the animated bar. Reviewed against renders of all six pages; the density pass alone fits roughly one extra section per screen on a form with 66 settings. TWO STARTER TEMPLATES, seeded on first use and never overwritten, so the picker is never an empty box. They describe the same show rendered by models with opposite strengths: Wan 8fps, 50-70 frame clips -> a 45s short is 6-8 scenes LongCat 15fps, 173-333 frames -> the same short is 2-4 scenes LongCat's numbers are its own: 15fps is the rate it was trained at (8 was a Wan-era compromise), 173 = 93+80 and 333 = 93+80x3 are whole segment counts, and all of them are 4n+1 — the VAE's rule, which the server also checks before a long run. Since the server owns the segment loop for LongCat, those long scenes are one request each rather than chained sub-renders with a re-encoded join and accumulating drift. 49 tests. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Loading applied the template to the live session but never wrote the active config file, which made "switch live" only half true: - the launcher starts the tool with --config <out_dir>/township_config.json, so the template silently reverted on the next restart; - _ref_gen_res_steps deliberately RE-READS that file from disk for keyframe_size and keyframe_steps (so Run-page edits apply without a restart), which meant those two options came from the stale file even in the same session — the loaded template disagreeing with itself. The load now writes the active config too, reports the path back as `persisted`, and says so in the log if the write fails rather than leaving a switch that looks applied and reverts later. Verified live: save a template with keyframe_size 1024x576, load it, and the active config file on disk carries it. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
A saved config was one JSON file; --config/--save let you keep several by juggling paths. A template is the same content under a NAME, in a known place, with list, load and delete — so "the LongCat long-shot setup" or "the quick 480p draft" is something you pick rather than a path you remember. Two properties beyond convenience: - A template is a COMPLETE configuration — every CONFIG_FIELDS key, 66 of them — not a patch. Loading one therefore cannot leave an option from the previous run in place. - The name is a SECURITY boundary, because names arrive from the CLI and from a browser. _safe_template_name strips separators and traversal and refuses a name with nothing usable; a test asserts that whatever is asked for, the resolved path stays inside the templates directory. "../../etc/passwd" becomes "etc-passwd". They live at <out_dir>/templates/<name>.json, because that is the directory the launcher already maps and persists — a template written to a container-local path would vanish on the next run. CLI: -T/--template NAME (layered over --config; explicit arguments still win, so a template can be a base you tweak per run), --save-template, --list-templates (with date, video model and option count), --delete-template. --config/--save are untouched. Web: GET /templates, POST /templates/load, POST /templates/delete, and a picker on the Run page with Load / Save as… / Delete. Saving is handled INSIDE /save-config rather than in its own route, so the large form-to-config mapping is not duplicated and cannot drift — a test pins that. Loading applies to default_args exactly as /save-config does, since the Run page renders from it and a load that only wrote a file would appear to do nothing until restart; it asks for confirmation first, because replacing every option is not an undoable click. A corrupt template is listed with its error rather than hiding the rest of the picker. 39 tests. Verified live against the web UI: save, list, load, delete, a missing name, and a traversal attempt contained. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Plan item I, the last of the LongCat work. The tool chains clips because the server generates one clip per request and VACE frame-tail conditioning is the only continuation a Wan model has. That machinery exists to fight drift — the dual-expert and VACE-extension notes are both about chained parts going wrong — and every join is a re-encode. LongCat was pretrained on continuation, so: - the client-side split is LIFTED for it (_chunk_max becomes unbounded): a clip's whole frame budget goes in ONE request and the server runs the segment loop. Chaining it from here would reintroduce exactly the drift the chaining was invented to avoid. Wan/VACE keeps the cap it always had; - between DELIBERATE shots the tool still chains, because a cut is a cut — but LongCat is tail-capable too, so it takes the same cond_frames handover. What differs is underneath, not the call; - the tail length follows the model: 13 of 93 frames, which is what LongCat's continuation pretraining used, against VACE's 5. Handing it 5 would condition it on less than it expects. The constant is asserted against tools/longcat_common.py so the two cannot drift apart; - the log says which continuation produced a shot, because diagnosing a seam starts there. Server side, so a client asks for the frames it wants rather than reimplementing the loop: num_frames is the PER-CALL count, and a request for more than one call's worth now derives total_frames and lets the segment loop cover it. An explicit num_segments or duration_seconds still wins. Passing 400 straight through would have asked the pipeline for a 400-frame single call. The uploader and the odds generation are untouched — they are downstream of the frames. 17 tests. No match has been rendered with it: that needs the weights. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Plan item H.2. Also fixes a real bug shipped in 0.2.35. THE BUG: load_quantized_dit was imported from longcat_video.modules.longcat_video_dit, where it does not exist — it is in longcat_video.modules.quantization (beside QuantizedLinear and quantize_model), and it returns the AVATAR transformer, consistent with INT8 being an avatar-1.5-only build. The wrong import would have raised at the exact moment someone asked for INT8. TRAINING. coderai does not implement this loop, and the reasons are the design: 1. LongCat's venv is standalone (Python 3.10 / torch 2.6 / transformers 4.41), so a trainer there cannot `from codai.api import loras` the way tools/lora_train_worker.py does — that overlay venv inherits the parent's site-packages, this one does not. 2. The upstream repo has NO way to create trainable LoRA layers. Its DiT exposes load_lora() / enable_loras() / disable_all_loras() and nothing that initialises one, and the adapters it loads (cfg_step_lora, refinement_lora, dmd_lora) use its own key layout. Reimplementing that layout unverified would produce adapters the pipeline cannot load: support in appearance only. SimpleTuner implements LongCat-Video properly — model_family "longcat_video", LoRA and quantised LoRA via int8-quanto / int4-quanto / fp8-torchao with no extra installs — so coderai generates its config and drives it: - lora_archs gains the arch with an `external` marker; load_classes refuses with that reason instead of an AttributeError that reads like a diffusers version problem. `target: video` still resolves Wan weights to Wan, so existing configs are untouched. - tools/longcat_train.py writes the config, runs SimpleTuner, parses its steps into the same JSON-lines protocol the overlay trainer uses, and reports the adapter it actually wrote rather than assuming one appeared. - Two VAE constraints are checked BEFORE a long run starts: (num_frames - 1) divisible by 4 — the same 4n+1 rule coderai already applies to Wan — and both sides divisible by 16. - A venv of its own, because SimpleTuner pins its own torch and would fight the inference venv's. train_auto_build is off by default, and the refusal carries the manual command. - QLoRA defaults ON (int8-quanto): a 13.6B transformer plus optimiser state does not fit a consumer card at bf16, so the quantised base is the normal path. `dataset_config` is REQUIRED and refused when absent or missing: SimpleTuner wants captioned video clips, and synthesising a dataset from the `images` field other targets use would train something other than a video LoRA, silently. 34 tests. No training run has been executed — the driver and its constraints are tested, the run against a real dataset is not. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Step 7, the last of the server-side work. Two things that are more than flags. CONTEXT PARALLELISM is N processes under torchrun, not N threads, and every rank must enter the same pipeline call or the collectives deadlock. So: - NCCL and init_context_parallel() run BEFORE the model loads — that call is what makes the DiT split its spatial dims, so load_pipeline cannot come first; - rank 0 owns the HTTP socket and BROADCASTS each job before entering the pipeline; the other ranks sit in cp_worker_loop waiting for one and discard their copy of the result. A rank-0-only server that simply called generate() would hang on the first collective; - the group is told to shut down when the server stops, or the non-zero ranks outlive it; - cp_size 1 (the default) initialises no distributed group at all; - asking for more ranks than there are visible GPUs is refused up front, because NCCL's own failure for that names nothing useful. THE POD IMAGE could not use the generic builder: Dockerfile.capability is python:3.12 on the cu128 torch index, and LongCat needs 3.10 on cu124 — two interpreters in one image, the light core running coderai and a 3.10 venv running the model. So video-longcat carries its own Dockerfile (the escape hatch the engines profile uses), builds the venv from the SAME requirements-longcat.txt a local install uses, asserts 3.10 / torch 2.6 / transformers 4.41 at build time rather than discovering a different resolution on a rented pod, clones the pipeline source (it is not on PyPI) and checks it imports, and bakes no weights. model_capability() now routes a `backend: longcat` entry to that image. Left alone it would land on the plain 'video' image, which cannot run LongCat at all — a pod that boots, passes its health check, accepts the request and then has nothing to generate with, exactly the trap the OCR profile documents for Surya. An explicit `capability` override still wins. 24 new tests, 212 across the seven LongCat suites. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Step 6. Read from upstream's avatar demo rather than assumed, because almost nothing carries over from the base model: - avatar is a DIFFERENT pipeline class, LongCatVideoAvatarPipeline, with its own methods generate_at2v / generate_ai2v / generate_avc; - it takes TWO guidance scales — text and audio — where the base pipeline takes one; - its distilled pass is dmd_lora at 8 steps with lora_network_dim=128/alpha=64 and dit.enable_loras(["dmd"]), NOT cfg_step_lora at 16; - use_int8 is a separate load_quantized_dit(subfolder="base_model_int8"), not a dtype; - both INT8 and the DMD LoRA exist only for avatar-1.5, which also REQUIRES distilled sampling; - the audio is librosa.load(sr=16000) then pipe.get_audio_embedding(...), and the encoder differs per family (chinese-wav2vec2-base for Avatar, whisper-large-v3 for Avatar-1.5). Unsupported combinations are refused BEFORE loading, because the alternative is quietly serving something that is not what was asked for: INT8 or DMD on anything but avatar-1.5, avatar-1.5 without distilled sampling, a base-model mode on avatar weights or the reverse, and GGUF, which needs ComfyUI's gguf loader and says so. FP8 loads the DiT as float8_e4m3fn. Community layouts differ between publishers (Kijai's scaled format is not a plain e4m3 dump), so a repo that needs its own loader fails with that reason rather than producing noise. Also: librosa is a real new dependency and went into requirements-longcat.txt, not only into the code that imports it. And the avatar audio comes from the request's `audio_file` field — `add_audio` is a BOOLEAN for the post-processing path, and using it would have passed True into the base64 encoder. 35 new tests, 188 across the six LongCat suites. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Step 5. The requirement is "full configuration in the interface", and the trap is silent: api_model_configure REBUILDS the entry from scratch and copies only an explicit whitelist, so a key with a UI control but no whitelist entry is written once and deleted on the next save from the models page. It looks saved until you reload. That is precisely why H3's documented h3_venv / in_process / dtype keys never survive, and it is not inherited here. - `longcat` is now an option in the hard-coded cfg-backend select — without that the backend cannot be pinned from the UI at all, which is the other half of why H3's `backend: "h3"` switch is unreachable. - A cfg-longcat field group, shown for that backend: variant (bf16/fp8/int8/gguf), default quality, offload, segments, conditioning frames, context-parallel GPUs, repo checkout, venv, and the avatar fields (reference frame index, mask range, INT8, distilled sampling, KV-cache offload). Populated on open and read back on save. - All thirteen keys added to the whitelist, with a comment saying what happens without it. - The settings now DO something: the dispatch reads num_segments, num_cond_frames, offload_kv_cache and quality off the entry as defaults, with the request still winning. A knob that is saved but never read is worse than no knob. - docs/longcat-video.md: why the venv exists, the two prerequisites, the download repos and why file_pattern matters (~83 GB), the checkpoint layout, the stages and presets, the segment arithmetic, the VRAM/eviction/segment-boundary behaviour, and an explicit list of what is NOT done — including that no generation has been verified against real weights. 62 tests, parameterised over every field so a future control added without a whitelist entry fails rather than silently losing data. Not in this step: there is no known-model catalogue in the codebase to pre-register the LongCat repos into — adding a model is operator-driven by model_id — so the repos and their file patterns are documented instead of hardcoded. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Step 4. It also CORRECTS step 2, whose generate() was written against an assumed interface. The real one, read from upstream's demo scripts, differs in four ways that all mattered: - there is no single callable: generate_t2v(height, width, …), generate_i2v(image, resolution, …), generate_vc(video, num_cond_frames, use_kv_cache, offload_kv_cache, …) and generate_refine(stage1_video, spatial_refine_only, …) are separate methods, and t2v takes height/width where i2v and vc take a resolution NAME; - each returns [0]-indexed frames normalised to [0,1], not an object with .frames — upstream converts with (tensor * 255).clamp(0,255).to(uint8); - the distilled stage is a use_distill=True FLAG on the same method, not a LoRA to fuse by hand, so _apply_stage_lora is gone; - the 50-step passes take a negative_prompt and the distilled one does not (its guidance is 1.0, so it would do nothing). On top of that: - The segment loop is upstream's: a call emits num_frames but its first num_cond_frames re-render the previous tail, so only the remainder is new and the fresh frames become the next call's conditioning. torch_gc() between segments, as upstream does. - Modes map onto the tasks: t2v, i2v/ti2v -> i2v, extend/vc/v2v -> vc. Continuation reuses the EXISTING cond_frames field, so a client that already chains clips through the VACE path needs no new vocabulary — and gets native pretrained continuation instead, which is what stops the colour drift chained conditioning suffers. - Long video: num_segments or duration_seconds, and the SERVER owns the loop. The response carries the tail a caller needs to continue by hand if it prefers to. - The segment boundary is now a real handover point. POST /yield asks a generation to stop at the next boundary; the releaser calls it before waiting, so an eviction arriving mid-request costs one segment instead of the whole generation, and the request still returns — shorter, with a warning that reaches the caller — rather than failing. Unsupported modes (the audio-driven avatar tasks) are still refused with a 400 rather than downgraded to t2v, which would read as a quality bug rather than a gap. 23 new tests pinning the upstream call contract so it cannot drift back, plus the segment arithmetic and the yield behaviour. Still no real generation: that needs the venv, the repo checkout and the ~83 GB checkpoint. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Step 3: make LongCat a citizen of the memory and swap machinery rather than a model that merely runs. Three gaps had to close. 1. THE ESTIMATE. _get_model_used_vram_gb had no branch for a `longcat:` key, so it fell through to scanning the checkpoint — ~83 GB across variants, which would have asked the manager to evict the whole box on every request. Added beside the ds4/colibri/vllm early returns: measured if known, else the starting estimate. 2. THE MEASUREMENT. No official VRAM figure exists for LongCat and the real peak moves with the variant, the stage and the offload mode, so the constant is a starting point and the service reports what it actually reached. It has to come from inside the worker: the pipeline loads lazily on the first request, so a host-side free-VRAM delta would also be counting whatever else was loading on the card. torch's max_memory_reserved is this process's own arenas — what the card genuinely cannot give to anything else. The HIGH-WATER mark is kept, because a draft run touches far less than a 720p refinement and reserving the smaller figure would under-evict later, and it is keyed per config_id/variant so an fp8 sibling cannot inherit a bf16 measurement. 3. GIVING THE CARD BACK. register_external_vram_releaser with the documented fn(needed_gb) -> float contract (not voice_clone's zero-argument one, which raises TypeError inside eviction and is therefore never reclaimed). The drain matters more here than anywhere: acquire_stt_backend POPS the model pool, so _is_key_busy is permanently False for this model and Pass 1 of release_idle_vram would tear the subprocess down mid-generation and lose a multi-minute request. The releaser waits for in-flight work, bounded by longcat.evict_drain_timeout_s (300 s). Also: video can now take a slot in the front per-model queue. Every acquire site was guarded by `_task_kind(path) == "text"`, so max_instances on a video model did nothing at the front and nothing bounded concurrent generations but the engine and the swap gate. _queues_at_front() enables it for video when — and only when — the operator has set max_instances: doing it unconditionally would change behaviour for every existing video model and start returning 503 at queue_max_size, which is not something to impose on deployments that never asked. Applied at all four sites (direct, keepalive, both brokered), and the `is_text` local it replaced was left dead, so it is gone. 21 tests, including the one that matters most: an eviction arriving mid-generation waits and the request still completes. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Step 2: text-to-video through LongCat-Video, single GPU, bf16. The engine exists now; i2v, continuation/long-video and the avatar variants are the next increment and are REFUSED with a 400 rather than silently downgraded. Step 1 missed a prerequisite, fixed here: LongCatVideoPipeline lives in the repo's own `longcat_video` package, not on PyPI (`from longcat_video.pipeline_longcat_video import LongCatVideoPipeline`), and it vendors its own Wan VAE and scheduler. So the repo SOURCE is bundled like Wav2Lip/SadTalker — code only, no weights — and the service prepends it to sys.path. - tools/longcat_common.py: the contracts both sides share, stdlib-only so it runs on 3.10 and 3.13 alike and is loaded BY PATH from each. Checkpoint layout (tokenizer/text_encoder/vae/scheduler/dit + lora/*.safetensors), the stage table, the presets, and the segment arithmetic — 93 frames per call of which 13 re-render the previous tail, so a segment ADDS 80, which is what makes upstream's "11 segments for a 1-minute video" come out at 59.5 s. Getting that wrong silently truncates long output. - tools/longcat_service.py: runs IN the 3.10 venv, serves /health /progress /generate /unload. Fuses the right LoRA per stage (stages 2 and 3 ARE their LoRAs — the distilled pass is 16 steps because of cfg_step_lora), and refuses 'refinement' with no earlier frames instead of generating something else. - codai/api/longcat_worker.py: venv/source resolution, process lifecycle, and VRAM registration through acquire_stt_backend so eviction reclaims it by stopping the subprocess — the only way to be sure a 13.6B DiT's pages are gone. ensure_built REFUSES by default with the exact manual command, because auto_build pulls torch 2.6 plus its CUDA runtime; the venv probe asserts 3.10/2.6/4.41 rather than a bare `import torch`, since the image rewrites pyvenv.cfg and a mis-pointed venv would import the main venv's torch 2.11. service_key includes config_id/variant: sibling configs on one path are DIFFERENT residencies and must not share a service or a measured footprint. - Detection honours the backend pin, then alias/served-name/path. The dispatch branch in video.py unwraps `_raw_cfg` first, because build_runtime_kwargs emits a FIXED key list for video models with no backend, alias or path in it — which is exactly why H3's documented `backend: "h3"` switch is dead. Not inherited. - The worker path reports progress and is cancellable: the service exposes /progress and the engine polls it onto the existing bar. H3's worker path has neither. - VideoGenerationRequest gains `quality` (draft|fast|best) and `stage` (base|distill|refinement), the preset and resumable-per-stage halves of the staging requirement. 31 tests. NOT covered, and so not claimed: a real generation — that needs the venv, the repo checkout and the ~83 GB checkpoint. Co-Authored-By:
-
Stefy Lanza (nextime / spora ) authored
Step 1 of the LongCat-Video integration: provision the runtime. No engine code yet. LongCat is the strictest isolation in the tree — it pins Python 3.10, torch 2.6+cu124, transformers 4.41 and numpy 1.26 against the main venv's 3.13 / 2.11+cu130 / 5.x / 2.4, four simultaneous conflicts, plus opencv-python against the bundled opencv-contrib. Installing it into the main venv would walk transformers and numpy backwards and break the stack, the failure already recorded for parler-tts. Crucially, coderai cannot build this venv itself: every venv bootstrap in codai/ uses `sys.executable -m venv`, which is 3.13. The only 3.10 in the tree is the standalone interpreter hand-built for the lip-sync tools and bundled as py310/, so LongCat rides that same interpreter in a venv of its own (its torch differs, so they cannot share one). - requirements-longcat.txt: upstream's pins off the cu124 index. flash-attn is deliberately NOT pinned — it is a source build needing ninja and a CUDA toolchain, and upstream accepts xformers, which ships a wheel; longcat.attention = "flash" is the opt-in. streamlit (upstream's demo UI) is left out. - build_oci_image.sh: declares LONGCAT_VENV and bundles it. The py3.10 interpreter is now discovered from EITHER py3.10 venv's pyvenv.cfg — it was read only from the lip-sync one, so a host with LongCat and no lip-sync venv would have bundled a venv with no interpreter to run it. - Dockerfile.oci-venv: copies longcat_venv and the pyvenv.cfg/bin-symlink rewrite now loops over both venvs. A bundled venv still carries the build host's `home =`, so without that its bin/python points at an interpreter the image does not have. - smoke_test_services.sh: asserts 3.10 + torch 2.6 + transformers 4.41, not just that torch imports — a venv re-pointed at the WRONG interpreter passes a bare import while being useless. - LongcatConfig (enabled, service_url, venv, python, auto_build, host, port, gpu, attention, ready_timeout, extra_args, extra_env), attached to Config, serialised by save_config, reconstructed in load() and exposed/parsed by the admin settings API. auto_build ships OFF: it pulls torch 2.6 plus its CUDA runtime. 16 tests. The config round-trip one caught a real hole: load() reconstructs each section explicitly, so without its line the operator could set LongCat up in the interface and lose it on the next restart. Co-Authored-By:
-