Autoencoders · Suman Bhadra Notes

Learn by rebuilding yourself

An autoencoder is a neural network trained to copy its input to its output. That sounds pointless — until you force the data through a narrow bottleneck in the middle. To reconstruct the input from so few numbers, the network has to discover what actually matters and throw away the rest. No labels needed: the input is the target.

Encoder x → z

Compresses the input into a small latent code z — the bottleneck.

Decoder z → x̂

Rebuilds the input from that code as faithfully as it can.

Loss ‖x − x̂‖²

Reconstruction error — how far the rebuilt output is from the original. Train to shrink it.

Squeeze, then rebuild

Follow one input through the hourglass: down to the latent code, back out to a reconstruction. Early in training the rebuild is rough; as the weights learn, the output snaps back toward the original.

What they're good for

Dimensionality reduction nonlinear PCA

The latent code is a compressed view of the data — like PCA, but able to bend around curved structure.

Anomaly detection high error = odd

Train on normal data only. Anything it reconstructs badly is unlike what it saw — a likely anomaly.

Denoising noisy in, clean out

Feed a corrupted input, ask for the clean original — the bottleneck can't store the noise.

From autoencoders to generative models

A variational autoencoder (VAE) makes the latent space smooth and continuous, so you can sample new points and decode them into brand-new data. That generative leap connects autoencoders to diffusion models and the rest of generative AI.