Model Lab Make the hidden mechanics visible: compression, attention, sampling, and retrieval.
Autoencoder Transformer Embedding & RAG
A deterministic linear-bottleneck simulation on correlated 2D points. The “training” control interpolates toward a fixed projection; it is not gradient-based training of a neural autoencoder. An original PyTorch training loop is available in Python Playground.
Autoencoder experiment Simulate reconstruction Generate new sample
Normal reconstruction MSE 0.0018
Noise 0.2 · 40 simulated epochs · 1 latent dimension(s)
Simulated normal reconstruction loss 0.35 0.00 Start End Anomaly contrast Injected sample [0.8, −0.8] has reconstruction error 0.6400 .
Illustrative threshold 0.15: Flagged .
Identity capacity can reconstruct the anomaly well and erase the detection signal.
Threshold is illustrative, not calibrated. Noise can create false positives.
Sample Normal 1
Clean [-0.95, -0.95]
Noisy [-1, -1.032]
Latent [-1.016]
Reconstruction [-0.98, -0.98]
MSE 0.0016
Sample Normal 5
Clean [-0.55, -0.55]
Noisy [-0.475, -0.451]
Latent [-0.463]
Reconstruction [-0.447, -0.447]
MSE 0.0004
Sample Normal 10
Clean [-0.05, -0.05]
Noisy [0.035, 0.028]
Latent [0.031]
Reconstruction [0.03, 0.03]
MSE 0.0000
Sample Normal 15
Clean [0.45, 0.45]
Noisy [0.443, 0.362]
Latent [0.403]
Reconstruction [0.388, 0.388]
MSE 0.0019
Sample Normal 20
Clean [0.95, 0.95]
Noisy [0.992, 0.994]
Latent [0.993]
Reconstruction [0.958, 0.958]
MSE 0.0013
Sample Injected anomaly
Clean [0.8, -0.8]
Noisy [0.8, -0.8]
Latent [0]
Reconstruction [0, 0]
MSE 0.6400 Autoencoder sample reconstruction trace Sample Clean Noisy Latent Reconstruction MSE Normal 1 [-0.95, -0.95] [-1, -1.032] [-1.016] [-0.98, -0.98] 0.0016 Normal 5 [-0.55, -0.55] [-0.475, -0.451] [-0.463] [-0.447, -0.447] 0.0004 Normal 10 [-0.05, -0.05] [0.035, 0.028] [0.031] [0.03, 0.03] 0.0000 Normal 15 [0.45, 0.45] [0.443, 0.362] [0.403] [0.388, 0.388] 0.0019 Normal 20 [0.95, 0.95] [0.992, 0.994] [0.993] [0.958, 0.958] 0.0013 Injected anomaly [0.8, -0.8] [0.8, -0.8] [0] [0, 0] 0.6400
Read the lesson Bring this lab to your team Original teaching examples · No production training claims