Towards AI

FEBRUARY 11, 2025

2020) Scaling Laws for Neural Language Models [link] First formal study documenting empirical scaling laws Published by OpenAI The Data Quality Conundrum Not all data is created equal. Some argue that while scaling has driven progress so far, we may eventually exhaust high-quality training data, leading to diminishing returns.

Data Quality 53

Data Quality Data Engineering Data Engineering Data Engineering 53

Snorkel AI

JANUARY 16, 2024

As humans, we learn a lot of general stuff through self-supervised learning by just experiencing the world. We have papers from 2020 where we showed that these models hallucinate less than regular parametric models. You need data that’s labeled and curated for your use case. AR: It’s not an either/or.

AI 52

AI AI Supervised Learning Algorithm 52

Snorkel AI

JANUARY 16, 2024

As humans, we learn a lot of general stuff through self-supervised learning by just experiencing the world. We have papers from 2020 where we showed that these models hallucinate less than regular parametric models. You need data that’s labeled and curated for your use case. AR: It’s not an either/or.

Supervised Learning 52

Supervised Learning AI AI Algorithm 52