Studio Flow
Production orchestrator wired to ComfyUI, from brief to export
- double claims and 0 budget overruns, up to 4 concurrent workers
- 0
- jobs in parallel with 4 workers and 2 GPUs (0.93 with 1 worker)
- 3.77
- model switch instead of 11, thanks to model affinity
- 1
- passing tests (61 before), with no GPU or network
- 119
- The problem
- Chaining long renders on a shared machine without losing anything or rendering twice, and delivering only what a human approved.
- For whom
- An image production that drives ComfyUI and wants human review and proof before any delivery.
- The outcome
- A queue driving a real ComfyUI (2 shots rendered, hashes verified), resuming a render after a kill -9 without resubmitting it, and scheduling jobs by resources: 3.77 jobs in parallel with 4 workers and 2 GPUs, 0 double claims, 119 tests.
Context
Producing a batch of images means chaining long renders on a shared machine: knowing what actually ran, recovering cleanly after a failure, never rendering the same thing twice, and delivering only what a human approved, with proof.
What I built
- Real ComfyUI engine: graph submission, tracking, download, using the node pack’s templates (P07) with typed, bounded injection of shot parameters; image, video, animated and audio outputs supported.
- Resource-aware scheduler: GPU, memory, priorities, anti-starvation and model affinity. 0 double claims and 0 budget overruns up to 4 workers; claim in 0.8 to 6.2 ms, 26 MB worker.
- Crash recovery: the render ID is written to the database before submission; on restart, the job reattaches to the running render instead of resubmitting.
- Classified errors: transient ones retried with exponential backoff, permanent ones failed at once; file sha256 checked against the ComfyUI manifest.
- Dashboard with accounts (PBKDF2), roles, CSRF and tokens; JSON API and HMAC-signed webhook (300 s window, replay refused); versioned export of approved media only.
- n8n workflow run in a real n8n 2.44: HMAC-signed request received, job created, replay without duplicate (201 then 200).
Architecture 7 blocks
- 01 · InputInputsCLI, JSON plan, HMAC-signed webhook
- 02 · DataSQLitebriefs, shots, jobs, runs, media, events
- 03 · ToolSchedulerGPU, memory, priorities, anti-starvation, model affinity
- 04 · ToolWorkerAtomic claim (0.8–6.2 ms), retries, resume, heartbeat
- 05 · ModelComfyUIP07 templates, injected and bounded parameters
- 06 · HumanReviewApprove or reject each item
- 07 · OutputExportApproved only, manifest, CSV, sheet
Pipeline 5 steps
- queuedUnique client token, no duplicates.
- runningGraph submitted, ID stored.
- succeededMedia downloaded, hash verified.
- reviewApproved or rejected by a human.
- exportManifest, CSV, contact sheet.
Decisions the choice, the reason, the alternative ruled out
- D1Write the ComfyUI render ID to the database before submitting it.
After a crash, the job reattaches to the running render: shown by a kill -9 mid-render, a single ComfyUI prompt in total.
Ruled outRestarting orphaned jobs automatically, at the risk of rendering twice.
- D2Classify errors: transient ones retried, permanent ones failed at once.
A passing out-of-memory deserves a retry; a rejected graph does not. Bounded exponential backoff with jitter.
- D3Check the sha256 of the downloaded bytes against the manifest written by ComfyUI.
What is exported is proven identical to what was rendered; rechecked at export.
- D4Stay on the standard library, with a signed webhook that is closed without a secret.
Runs anywhere; HMAC signature, 300 s window and replay refusal covered by tests.
Ruled outFastAPI and an open webhook.
- D5Claim jobs according to declared resources (GPU, memory) with priorities, anti-starvation and model affinity.
0 double claims and 0 budget overruns up to 4 concurrent workers; 3.77 jobs in parallel with 4 workers and 2 GPUs; 1 model switch instead of 11.
Ruled outA single worker, one job at a time.
Results
Real runs on 20 Oct 2026 (ComfyUI 0.41, M1 Pro 16 GB)
| Scenario | Result |
|---|---|
| Run: 2 shots submitted to ComfyUI | 2 succeeded on first attempt, 2 matching hashes |
| End-to-end time per shot | 19.2 and 20.2 s; submission 2 ms, download 4–8 ms |
| Worker killed (kill -9) mid-render | Resume: reattached to the same render, a single ComfyUI prompt |
| Human review | Media “to review”, never auto-approved |
Resource-aware scheduler
| Measure | Result |
|---|---|
| Double claims, budget overruns (1 to 4 workers) | 0 / 0 |
| Jobs in parallel, 2 GPUs | 0.93 with 1 worker, 3.77 with 4 |
| Model switches | 1 instead of 11 (model affinity) |
| Claim time | 0.8 to 6.2 ms |
| Worker memory | 26 MB |
| Automated tests | 119 (61 before) |
Deliverables 5
- Dependency-free Python package: CLI, worker, simulation and ComfyUI engines, dashboard.
- Real ComfyUI run and crash-recovery proof, with logs.
- 119 tests (61 before), including a fake ComfyUI server (errors, OOM, tampered hash, lost prompt).
- Dashboard with accounts (PBKDF2), roles, CSRF and tokens; ComfyUI video, animated and audio outputs supported.
- n8n workflow validated by n8n’s own code, signed and idempotent webhook (201, then 200 on replay).
Environment
Hardware and measurement setup
- Real renders measured on a MacBook Pro M1 Pro 16 GB with ComfyUI 0.41; the scheduler places workers according to the declared GPUs and memory.
- Python 3.11+ and ffmpeg, no other dependency; dashboard served locally.
Reproduce README commands
Private code: read access under NDA, or a 30-minute live demo. Request access →
./demo.sh
python3 -m studio_flow templates
python3 -m studio_flow plan-import exemples/plan-vecteur-comfyui.json --submit
python3 -m studio_flow worker
python3 -m studio_flow --config demo/studio-flow.toml serve
python3 -m pytest- Python 3.11+ and ffmpeg; ComfyUI on 127.0.0.1:8188 for the real engine.
- The webhook stays closed until STUDIO_FLOW_WEBHOOK_SECRET is set.
Gallery 2
What this project means for you
The same know-how, on your problem.
Orchestrating your production
Shot-by-shot tracking, clean recovery and mandatory review: nothing reaches the client without approval.
Your ComfyUI, driven
Your workflows become parameterised templates, called by n8n, a script or your internal tool, without duplicates.
Proven robustness
Double delivery, crash, engine error, missing output, cancellation: every case is tested before go-live.