StudioLatentMatthieu Czarkowski · creative studio & engineering (home)
FREN · Passer en françaisStart a project

RELIEF Studio

Local studio: material LoRA, ControlNet, quantified evaluation

Role
Design, training, evaluation, development
Tools
  • PyTorch
  • Diffusers
  • PEFT
  • ControlNet Canny
  • CLIP
  • FastAPI
  • SQLite
Status
Functional
The studio: visual direction, LoRA adapter, and every image with its parameters.
The studio: visual direction, LoRA adapter, and every image with its parameters.
CLIP-image style gain vs base, 95% CI [0.105; 0.147]
+0,126
pairs where the adapter is closer to the style (scale 1)
24/24
text-adherence points, tunable through the adapter scale
−3.5
images, 4 materials: paper, metal, plaster, wood
144
The problem
A brand wants a recognisable material applied to its own shapes without losing their outlines, and to know what the style costs.
For whom
Studios and brands rolling out a visual direction; shown on four materials: paper, metal, plaster, wood.
The outcome
A four-material adapter trained in 52 min: style gain +0.126 [0.105; 0.147], 24 pairs out of 24, text adherence dosed by the scale (−3.5 points at scale 1).

Context

A brand wants a recognisable material applied to its own shapes without losing their outlines. v0.1 showed it on a single material (paper relief); v0.2 moves to four materials and an evaluation that also states what the style costs.

What I built

  • Base weights rebuilt by locally converting a checkpoint, with a parity check: held-out MSE matches the history within 4 × 10⁻⁵ relative.
  • relief-materials-v1 corpus: 144 images, 108 to learn and 36 to test, split by composition.
  • relief-materials-v1 adapter: 1,500 steps in 52 min (about 1.66 s per step), min-SNR, EMA, checkpoints and exact resume.
  • CLIP-image, CLIP-text, diversity and SSIM evaluation on 8 prompts (4 in domain, 4 out of domain) × 3 seeds, with confidence intervals; stacking several adapters in the UI.

Architecture 7 blocks

  1. 01 · InputDataset144 images, 4 materials, split by composition
  2. 02 · ToolVerified baseLocal conversion, parity within 4 × 10⁻⁵
  3. 03 · ModelAttention LoRA1,500 steps, min-SNR, EMA, frozen SD 1.5 base
  4. 04 · DataLibraryVersioned adapters, kohya export, stacking
  5. 05 · ToolControlNet CannyReference edges guide generation
  6. 06 · ToolEvaluationCLIP style, text, subject, diversity, SSIM, 95% CI
  7. 07 · OutputStudioWeb app, PNG and manifest per image

Pipeline 5 steps

  1. ParityBase reconverted, verified.
  2. Dataset144 images, 4 materials.
  3. Training1,500 steps, 52 min.
  4. EvaluationStyle, text, subject, diversity.
  5. StudioStacked adapters, outlines held.

Decisions the choice, the reason, the alternative ruled out

  1. D1
    Rebuild the base weights by local conversion, then prove parity.

    A rebuilt base must be proven: MSE on the 9 held-out cases matches the history within 4 × 10⁻⁵ relative, adapter included.

    Ruled outRe-downloading and assuming equivalence.

  2. D2
    Also measure what the style costs: text adherence, subject, diversity.

    The dosing is chosen on measurements: at scale 1, text adherence moves by −3.5 points; scale 0.7 preserves more of it.

  3. D3
    Test out of domain, on unseen subjects.

    Lighthouse, fox, portrait, ship: the style transfers (+0.151), it is not a copy of the corpus.

  4. D4
    A persistent SQLite queue and one GPU job at a time.

    Avoids saturating 16 GB of unified memory; the web server stays available during compute.

Results

relief-materials-v1 vs base SD 1.5 (24 pairs, 384², 24 steps)

ConditionStyle gain [95% CI]Pairs wonText-adherence change
Scale 1+0.126 [0.105; 0.147]100 %−3.5
Scale 0.7+0.100 [0.082; 0.118]96 %−2.1
Step 750, scale 1+0.111 [0.091; 0.132]100 %−2.9
Scale 1, out of domain+0.151 [0.133; 0.169]100 %−3.5

Style gain: CLIP-image similarity to held-out corpus images. The balance between style and prompt adherence is set through the adapter scale.

Deliverables 4

  • Responsive web app (FastAPI): generation, training, evaluation, stacked adapters.
  • relief-materials-v1 adapter and its step-750 checkpoint.
  • Evaluation report (table, board, HTML page) and parity report.
  • 144-image corpus with manifest and hashes; 67 tests.

Environment

Hardware and measurement setup

  • Trained and evaluated on an M1 Pro 16 GB (MPS): 1,500 steps in 52 min; a SQLite queue keeps one GPU job at a time.
  • API served on 127.0.0.1, for local use.

Reproduce README commands

Private code: read access under NDA, or a 30-minute live demo. Request access →

.venv/bin/relief doctor
.venv/bin/relief dataset build --name relief-materials-v1 --styles paper,metal,plaster,wood
.venv/bin/relief train --config configs/relief-materials-v1.toml
.venv/bin/relief evaluate --adapter relief-materials-v1 --adapter relief-materials-v1@step-000750 \
  --preset materials --references relief-materials-v1 --scales 0.7,1.0
.venv/bin/relief serve --port 7864
  • Tested on an M1 Pro 16 GB.
  • The API listens on 127.0.0.1, for local use; authentication is added before any network exposure.

Gallery 5

In-domain evaluation (paper and metal). Columns: base SD 1.5, adapter at 0.7 and 1, then checkpoint 750 at 0.7 and 1.
Out of domain: lighthouse and fox, subjects never seen in training. Same columns.
Before / after: ControlNet-guided restyling, edges kept.
Training an adapter.
Mobile view.

What this project means for you

The same know-how, on your problem.

01

Your material, your shapes

I teach a model a material and apply it to your shapes while keeping their outlines: posters, packaging, variants.

02

Measured dosing

Style gain and its cost, measured at every scale: you choose the trade-off knowingly.

03

A studio on your premises

A simple interface on your machines: your references and results stay on your disk.

Visuals for my brand →The same tool for my team