Hacker News

JULY 1, 2025

I'm JD, a Software Engineer with experience touching many parts of the stack (frontend, backend, databases, data & ETL pipelines, you name it). With over 3 years of working with ETL pipelines and REST API integrations and development, I understand how to develop and maintain robust and scalable data systems.

Python 54

Python AWS SQL ML 54

Applied Data Science

MAY 6, 2021

Building end-to-end data science solutions means developing data collection, feature engineering, model building and model serving processes. Airflow is a Python based open source orchestration tool developed in-house by Airbnb in 2014 to help their internal workflows. It’s a lot of stuff to stay on top of, right?

Data Scientist 130

Data Scientist Python Data Science Database 130

Data Science Blog

JUNE 27, 2023

Kappa – Architecture Jay Kreps introduced the Kappa architecture in 2014 as an alternative to the Lambda architecture. It offers the advantage of having a single ETL platform to develop and maintain. It is well-suited for developing data systems that emphasize online learning and do not require a separate batch layer.

Big Data 130

Big Data Big Data Apache Kafka Database 130

phData

NOVEMBER 9, 2023

Effectively this is a way to store the source of truth and build (or rebuild) your downstream data products (including data warehouses) from it. What is the Difference Between a Data Lake and a Data Warehouse? Historically, there were big differences.

Data Warehouse 98

Data Warehouse Data Lakes Clustering Cloud Data 98

DagsHub

APRIL 7, 2024

Image generated with Midjourney In today’s fast-paced world of data science, building impactful machine learning models relies on much more than selecting the best algorithm for the job. Data scientists and machine learning engineers need to collaborate to make sure that together with the model, they develop robust data pipelines.

Machine Learning 59

Machine Learning Machine Learning ML ML 59

The MLOps Blog

OCTOBER 20, 2023

Jupyter notebooks have been one of the most controversial tools in the data science community. Nevertheless, many data scientists will agree that they can be really valuable – if used well. When data science was sexy , notebooks weren’t a thing yet. In 2014, Project Jupyter evolved from IPython.

SQL 52

SQL Data Scientist Database Python 52