Day 17

Day 17 of 1% Better.

Today, I learned about self-supervision is and why it changed AI.

Here's the simplest way to think about it:

1. Traditional AI needs labeled data.

↳ People manually label examples.
↳ For example:
🐱 → Cat
🐶 → Dog
↳ This process is time-consuming.

2. Self-supervision is different.

↳ The model learns from the data itself.
↳ It creates its own learning signals instead of relying on human-created labels.
↳ This removes the need for large amounts of manually labeled data.

3. How does an LLM learn?

↳ It reads billions of text sequences.
↳ It learns by predicting the next token.
↳ Every sentence becomes a learning opportunity.

4. Why is it important?

↳ Self-supervision made it possible to train on massive amounts of internet text.
↳ More data helped models become more capable.
↳ This is one of the key reasons today's Large Language Models exist.