Fine-Tuning Lab
Take an engineering architecture task from governed examples to a tested Qwen artifact.
One evidence trail
Scenario → response → expert judgment → training record → checkpoint → release gate.
The existing review workbench contains 24 actual baseline outputs from 12 original scenarios. They are calibration material, not a completed expert-reviewed training set.
Review actual Qwen outputsRun the workflow locally
Download the dataset validator and SFT / DPO / GRPO scaffold. The browser planner below creates a specification; it does not launch GPU training.
Download training starterGPU execution and architecture-quality gains have not been validated. Full training needs compatible hardware and model downloads.
The end-to-end workflow
01 · Define task
Engineering architecture responses must state constraints, alternatives, failure modes, and verification steps.
Artifact: Task contract + acceptance rubric
02 · Collect & govern
Create original scenarios from approved requirements. For newsletters or email: establish rights, remove private content, extract general concepts, write new scenarios, and keep source provenance privately. Reading access is not permission to redistribute or train on copyrighted text.
Artifact: Approved scenario manifest; no raw mailbox export
03 · Format examples
SFT: messages with reviewed assistant targets. Preferences: identical prompt with chosen/rejected responses, rubric and reviewer provenance. RL: prompt or environment plus a tested reward contract.
Artifact: Versioned SFT / preference / rollout records
04 · Create splits
Keep related source documents and scenario families together. Separate calibration from training, validation, and frozen test. Toy Dataset Hub splits teach mechanics only.
Artifact: Group-aware split manifest + leakage audit
05 · Select base model
Start with the existing Qwen3-4B-Instruct-2507 baseline. Freeze model and tokenizer revision before comparing adapters.
Artifact: 24 actual outputs in the review workbench
06 · Choose method
Prompt and retrieval baseline → SFT if behavior needs teaching → DPO for adjudicated preferences → optional online RL when rewards and environments are verifiable.
Artifact: Objective tied to evidence
07 · Configure training
Select full tuning, LoRA, or QLoRA. Bound sequence length and memory; record rank, target modules, optimizer, seed, batch, accumulation, and checkpoints.
Artifact: Reproducible experiment specification
08 · Train & track
Run the downloadable Python starter in an isolated training environment. Start with a tiny mechanics run, then a GPU memory pilot. Log losses, evaluation, adapter hashes, and failures.
Artifact: Checkpoint + run manifest; no hosted GPU job here
09 · Evaluate
Use held-out architecture families, general capability regressions, safety cases, latency and serving memory. Blind human comparison to the frozen baseline.
Artifact: Signed release decision; uncertain cases withheld
10 · Package & deploy
Bundle base revision, adapter, tokenizer, prompt, policy, runtime dependencies and model card. Verify loading or merging, stage, canary, then promote.
Artifact: Versioned serving bundle + rollback artifact
11 · Monitor & improve
Track errors, drift, latency, feedback provenance and critical failures. Curate new failures into the next training cycle only after review.
Artifact: Fresh evaluation set and controlled next iteration
Configure an experiment
Effective batch: 16 examples on one device.
Illustration for 960 training records: approximately 180 optimizer steps. This is not a measured GPU memory or cost estimate.
Train adapters while freezing the base; preserve the exact base revision.
Experiment specification
{
"status": "specification_only_not_executed",
"task": "engineering_architecture",
"model_id": "Qwen/Qwen3-4B-Instruct-2507",
"revision": "cdbee75f17c01a7cc42f958dc650907174af0554",
"objective": "SFT",
"weight_update": "LoRA",
"lora_rank": 16,
"learning_rate": 0.0001,
"epochs": 3,
"microbatch": 2,
"gradient_accumulation": 8,
"devices": 1,
"seed": 42,
"max_sequence_length": 2048,
"required_data": "reviewed_messages",
"required_gates": [
"group_disjoint_splits",
"rights_and_privacy_review",
"heldout_quality",
"safety_veto",
"general_regression",
"serving_budget"
],
"recipe_library": "TRL 0.23.1 starter; verify compatibility before full GPU execution"
}
How a source becomes a training example
General concept: a data agent needs a canonical metric definition before a query.
Original scenario: “A dashboard uses paid order revenue while a warehouse column includes refunds. Design the agent’s clarification and validation path.”
Generate: produce two candidate responses from the frozen Qwen baseline. Keep exact prompt, decoding settings, model revision, output, and finish reason.
Review: experts score correctness, trade-offs, grounding, operability, and boundaries; record ties and critical failures.
Export: approved answer → SFT; clear safe preference → DPO/reward-model pair; testable tool task + environment + reward → RL rollout. An article alone is not an RL dataset.