You Don’t Need Many Labels to Learn

Generative models can discover meaningful structures in data without any labels, which drastically reduces the amount of supervision needed to build a powerful classifier. A Gaussian Mixture Variational Autoencoder (GMVAE), for instance, can be trained in an unsupervised way to organize data like handwritten letters into distinct clusters. The challenge then becomes assigning class labels to these clusters using only a tiny fraction of labeled data. A "soft decoding" approach proves superior by using the full probability distribution from the model, comparing a new image's cluster probabilities against the known probability profiles of each class. This probabilistic method effectively handles uncertainty and cases where clusters contain multiple classes, outperforming simpler hard-assignment strategies.