Semi-Supervised Learning
Semi-supervised learning is a hybrid approach to machine learning that sits between supervised (all labeled data) and unsupervised (no labeled data) learning. It is specifically designed for situations where you have a massive amount of data, but only a tiny fraction of it is labeled.
☆ How it works?
The general process often follows these steps:
- Initial Training
The model is trained on the small set of labeled data, just like in supervised learning.
- Pseudo-Labeling
The model is then used to predict labels for the unlabeled data. Because the model isn't perfect yet, these are called "pseudo-labels."
- Filtering
The predictions with the highest confidence are added to the training set as if they were real labels.
-
Retraining
The model is retrained on the now-expanded dataset (original labeled data + high-confidence pseudo-labeled data). This cycle repeats to refine accuracy.
☆ Key Concepts and Assumptions
For semi-supervised learning to work, algorithms rely on three core assumptions:
- Continuity Assumption
Points that are close to each other in the data space are likely to share the same label.
- Cluster Assumption
Data naturally forms groups (clusters). If two points are in the same cluster, they likely belong to the same category.
- Manifold Assumption
High-dimensional data (like images) actually lies on a lower-dimensional "surface" or manifold. The model tries to learn this surface to better separate classes.
☆ Common Techniques
| Technique | How it works |
|---|---|
| Self-Training | The model labels its own unlabeled data and uses the best guesses for its next round of training. |
| Co-Training | Two different models look at the same data from different "views" (e.g., text vs. images) and teach each other. |
| Graph-Based SSL | Data points are treated as "nodes" in a web. Labels "spread" like a virus from labeled nodes to their closest unlabeled neighbors. |
☆ Real-World Use Cases
- Medical Imaging
- Speech Recognition
- Web Classification