ATELIER Ops

From brief to production file, sourced, verified and approved

Role: Design, training, development, evaluation
Tools: Python
MLX
Qwen3-4B
LoRA
E5 / ONNX
Cross-encodeur (NumPy)
SQLite FTS5
FastAPI
Pydantic
MCP
Status: Functional

A file read like a document: steps from brief to review, cited sources alongside.

“supported” precision from automatic verification (0.41 with the citation alone): 0.90
of false or partial sentences flagged: 96%
unsupported sentences wrongly passed: 0 / 31
automated tests (14 before), browser acceptance 12 / 12: 30

The problem: Going from an idea to a shootable piece always takes the same steps: research, scope, write, check, get sign-off.
For whom: A creator or a small marketing team; used on LaRefVlog and shown with a sample dataset (the “Nacre” startup).
The outcome: A local, multi-user assistant that delivers a sourced production file with named approval, in 34 to 83 s on an M1 Pro; every claim is checked against its source (“supported” precision 0.90 vs 0.41 with the citation alone, 0 / 31 unsupported sentences wrongly passed).

Context

Going from an idea to a shootable piece always takes the same steps: find the information, pick an angle, write hooks, plan shots, check the facts, get sign-off.

First use: LaRefVlog. The solution is also shown with a sample dataset for a small team: the “Nacre” startup.

What I built

An MLX LoRA on quantised Qwen3-4B that extracts brief fields; it is only routed in for structured briefs, otherwise the base model answers.
Hybrid RAG: E5 embeddings on ONNX, FTS5/BM25, RRF fusion, workspace filtering before ranking.
Pydantic output contracts, checks on citations, duration, speaking rate and unsourced figures.
Automatic verification of every claim against the cited passage: a multilingual cross-encoder reimplemented in NumPy on CPU, without PyTorch, 1.3 s per sentence. An unsupported claim blocks approval until a reviewer fixes or confirms it.
Multi-user: accounts (scrypt passwords), four roles per workspace, sessions with CSRF, API tokens for CLI and MCP; isolation between workspaces tested, forged identifiers included.
Revisions, named approval, invalidation when a source changes; machine and human time stamped, CSV export and a measurement protocol; FastAPI, persistent queue, GPU lock.

Architecture 7 blocks

01 · InputBriefGoal, audience, channel, duration; otherwise, asks for details
02 · ModelExtractionQwen3-4B + LoRA for structured briefs, base model otherwise
03 · DataHybrid retrievalE5 (ONNX) and FTS5/BM25, RRF fusion, filtered by workspace
04 · ModelDraftingBase model: hooks, timed scenes, voice-over, sources
05 · ToolChecksPydantic, every sentence against its source (CPU cross-encoder), duration, speaking rate, figures
06 · HumanApprovalPer-workspace roles, named approval, voided if a source changes
07 · OutputExportMarkdown file; exporting is not publishing

Pipeline 6 steps

BriefGoal, audience, channel, duration.
ExtractionLoRA for structured briefs, base model otherwise.
RetrievalBrand book, product sheet, workspace notes.
DraftingHooks, timed scenes, voice-over, sources.
ChecksEvery sentence against its source, duration, figures, claims.
ApprovalRevision, named approval, export.

Decisions the choice, the reason, the alternative ruled out

D1
Two roles for the LLM: a LoRA to extract the brief, the base model to draft.
The stable task is learned in the weights; brands’ changing facts stay in the document base.
Ruled outOne fine-tuned model for everything, facts included.
D2
Route to the LoRA only when the brief has at least three explicit labels.
Measured on unseen probes: on free-form phrasing the base model does better (4/6 correct fields against 3/6); the router sends each brief to the better of the two.
Ruled outForcing the adapter on every brief.
D3
float32 vectors in SQLite, exact search fused with FTS5/BM25 via RRF.
Fits a modest local corpus; a large volume would justify a dedicated vector index and its own measurements.
Ruled outA dedicated vector database from day one.
D4
No simulated file in place of an invalid answer: validated JSON, a single repair attempt.
A visible error beats a presentable fake file.
Ruled outFilling the gaps with a template.
D5
No silent restart: on reboot, running tasks are marked “interrupted”.
An interrupted action is restarted explicitly; the waiting queue is kept.
Ruled outAutomatic resume.
D6
Check every sentence against the cited passage with a cross-encoder reimplemented in NumPy, on CPU.
An existing citation does not prove the sentence is true: verification lifts “supported” precision from 0.41 to 0.90 and flags 96% of false or partial sentences, at 1.3 s per sentence and without PyTorch.
Ruled outChecking only that the citation exists.

Results

Claim ↔︎ source verification (test split: 80 pairs, passages never seen in calibration)

Measure	Citation only	Automatic verification
“Supported” precision	0.41	0.90
False or partial sentences flagged	—	96%
Unsupported sentences wrongly passed	—	0 / 31
Cost on CPU (M1 Pro)	—	1.3 s per sentence

Base model vs LoRA v3 (first pass)

Protocol	Base	LoRA v3
Regression: valid JSON	7/8	7/8
Regression: correct fields	6/8	7/8
Unseen probes: valid JSON	5/6	5/6
Unseen probes: correct fields	4/6	3/6

The LoRA improves on structured briefs (correct fields: 6/8 → 7/8); a router enables it for those briefs and hands free-form phrasing to the base model.

Deliverables 6

Local multi-user web app (FastAPI, responsive UI): scrypt accounts, four roles per workspace, CSRF, tokens; authenticated 4-tool MCP server.
Automatic claim ↔︎ source verification, per-sentence status in the UI and export; machine and human time stamped, CSV export and a measurement protocol.
LoRA v3 adapter (2.6 MB, 160 steps) and its model card, base and LoRA compared on every protocol.
Two exported files: LaRefVlog and the “Nacre” sample dataset.
30 automated tests (forged-identifier isolation included) and browser acceptance 12 / 12; training and evaluation reports as JSON.
Interactive project presentation, viewable offline.

Environment

Hardware and measurement setup

Tested on macOS Apple Silicon (MacBook Pro M1 Pro, 16 GB) with MLX: 34 to 83 s per file.
Served on 127.0.0.1 by default; network listening requires TLS, allowed hosts and at least one account. No external API once the weights are downloaded; verification on CPU, 489 MB of memory at most.

Reproduce README commands

Private code: read access under NDA, or a 30-minute live demo. Request access →

python3 -m venv .venv
.venv/bin/pip install -r requirements-lock.txt
.venv/bin/pip install --no-deps -e '.[dev]'
.venv/bin/atelier download
.venv/bin/atelier seed
.venv/bin/atelier serve
.venv/bin/python -m pytest tests -q

Tested on macOS Apple Silicon (M1 Pro, 16 GB) with MLX.
The first weight download is several GB; after that, no external API.

Gallery 6

Production files: status, checks, briefs to complete.

Evaluations: base and LoRA compared on every protocol.

What this project means for you

The same know-how, on your problem.

A private preparation assistant

Connected to your documents and your review process, on your machine: we start with one format and a controlled set of sources.

A measured pilot

Five briefs prepared by hand, then with the tool, under the same review standard: time to an approvable file and factual errors compared, with no invented gains.

Adapting a model to your task

A narrow LoRA on a stable task, routed only where it does better: I measure base against adapter before going live.

Scope a project →The company offer