Self-Supervised Learning

Published:

Self-supervised learning teaches a model using data that has no human labels, but it still gives the model a clear task to learn from. The trick is that the task is created from the data itself. For example, the model might hide part of a sentence and try to predict the missing words, or hide part of an image and try to reconstruct what was removed. By repeatedly solving these “fill in the blank” style tasks, the model learns useful patterns about language or images.

The key difference from semi-supervised learning is that self-supervised learning doesn’t mix labeled and unlabeled data during training. It learns entirely from unlabeled data by generating its own targets. After this pretraining step, the model is often fine-tuned with a smaller labeled dataset for a specific job, like classification or detection.

Follow us on Facebook and LinkedIn to keep abreast of our latest news and articles