GPU Notebook
Sizing guide: what really fits in 16 GB
- measurements read from 7 projects
- 18
- video distillation speed-up (4 steps vs 16)
- ×7.8
- slowdown once swap saturates
- ×8.9
- peak of the 9B LoRA: the dedicated-GPU threshold, measured
- 12.3 GB
- The problem
- Choosing a local generation setup is often done by feel; it needs measured values, conditions included.
- For whom
- The portfolio’s local projects, and any team that has to size a generation machine.
- The outcome
- A verdict per task from 18 real measurements: 7B image in 141 s, distilled video in 40 s, 3B LLM at 28–53 tok/s; 1.2B LoRA in 18.6 min, 9B LoRA on a dedicated GPU (12.3 GB peak).
Context
The question from a client who wants “local AI”: what fits on a 16 GB machine, how long does it take, and where should you rent a GPU? The portfolio projects produced real measurements; this project reads them, compares them and draws a verdict per task.
What I built
- Standard-library ComfyUI and LLM benches: wall time, memory, image hashing, cold and warm kept apart.
- Automatic synthesis: one script reads 16 measurement files (CSV, JSON, logs) and writes the table and the charts in French and English, with the source of every number.
- Sizing guide: verdict per task, the setting that works, the point to move to a dedicated GPU, five rules drawn from the measurements.
- Benchmark-trap detection: ComfyUI execution cache (1 s, bit-identical image), GPU throughput dropping on the 3rd call.
Architecture 6 blocks
- 01 · InputConditionsMachine, versions, graph and settings recorded
- 02 · ToolComfyUI benchtxt2img, wall time, memory, image hashing
- 03 · ToolLLM benchllama-server, Metal vs CPU, prompt cache off
- 04 · DataMeasurementsCSV and JSON, one value per run, cold and warm
- 05 · ToolSynthesis16 files read across 7 projects, source of every number kept
- 06 · OutputSizing guideVerdict per task, FR/EN charts, dedicated-GPU thresholds
Pipeline 4 steps
- MeasureEach project writes its timings and memory peaks.
- ReadThe script finds the 18 measurements in their files.
- CompareTime and memory charts, FR and EN.
- AdviseVerdict per task and switch-over points.
Decisions the choice, the reason, the alternative ruled out
- D1Publish each run separately, with no percentiles on four values.
A median would hide the GPU throughput collapse from the 3rd warm call (52.9 → 30.0 → 28.2 tok/s).
Ruled outAverages or percentiles.
- D2Detect ComfyUI’s execution cache and exclude it from averages.
An already-computed graph returns in 1 s with a bit-identical image: a classic benchmarking trap.
Ruled outAveraging everything unchecked.
- D3llama.cpp (llama-server) with Qwen2.5-3B Q4_K_M, temperature 0, prompt cache off.
The same server switches between GPU and CPU: both series compare under identical conditions.
Ruled outOllama, ruled out.
- D4Read the numbers from the measurement files instead of copying them.
A sizing guide is only worth something if every number can be traced to its source; the synthesis regenerates after every new series.
Ruled outA hand-typed table.
- D5Keep process memory and Metal buffers apart.
Every memory figure is labelled for what it measures: no value is presented as GPU memory without a dedicated measurement.
Results
Verdict per task — MacBook Pro M1 Pro 16 GB
| Task | Local? | Measured setting |
|---|---|---|
| 768² image, 7B model | yes, batched | GGUF Q4 + 4 steps: 141 s (n = 55) |
| Photo retouch → 3600×2400 master | yes | full ComfyUI workflow: 171 s |
| 16-frame video clip | yes, distilled | 4 steps: 40 s (n = 8), 9.0 GB peak |
| 3B LLM generation | yes | llama.cpp GPU: 28–53 tok/s |
| LoRA on a 1.2B LLM | yes | 18.6 min, 2.2 GB peak |
| LoRA on a 9B LLM | no | Metal peak 12.3 GB → GPU ≥ 24 GB |
Every value is read from the file that produced it (source column of the synthesis table).
Qwen2.5-3B Q4_K_M, 128 generated tokens
| Run | Phase | Metal (tok/s) | CPU (tok/s) |
|---|---|---|---|
| 1 | cold | 52.5 | 10.0 |
| 2 | warm | 52.9 | 12.0 |
| 3 | warm | 30.0 | 12.8 |
| 4 | warm | 28.2 | 21.1 |
GPU throughput drops from the 3rd warm call: an average would have hidden it.
Deliverables 4
- ComfyUI bench and LLM bench in Python, standard library.
- Four measured series: SD 1.5 validation and warm, LLM on Metal and on CPU.
- Full conditions for each series: versions, graph, settings.
- Sizing guide, sourced synthesis table and two charts in French and English, regenerated with one command.
Environment
Hardware and measurement setup
- Every measurement on a MacBook Pro M1 Pro 16 GB, ComfyUI 0.41.0 and PyTorch on MPS, in real-workstation conditions: min–max spreads are shown.
Reproduce README commands
Private code: read access under NDA, or a 30-minute live demo. Request access →
cd portfolio/projets/p06-gpu-notebook
python3 guide/synthese.py # tableau + graphiques FR/EN, < 1 s
# nouvelle série image (ComfyUI lancé sur le port 8188) :
cd banc
python3 banc-comfyui.py --seeds 404,505,606 --steps 20 \
--prompt "wet dark basalt rock close-up, hard raking light, fine mist, mineral texture, photorealistic" \
--out ../mesures/AAAA-MM-JJ_nom-de-serie- MacBook Pro M1 Pro 16 GB, ComfyUI 0.41.0, PyTorch on MPS.
- Timings measured on a real workstation: min–max spreads are shown.
Gallery 2
What this project means for you
The same know-how, on your problem.
Sizing your machine
Before you buy or rent, I measure your real tasks on the hardware in question, every run shown with its conditions.
Choosing local or cloud
Throughput and timings measured in your conditions, to decide what runs on-site and what goes elsewhere.
Reproducible benches
Scripts that record versions and settings: the measurement can be redone identically after every update.