Data Science Blog

JUNE 27, 2023

The batch views within the Lambda architecture allow for the application of more complex or resource-intensive rules, resulting in superior data quality and reduced bias over time. On the other hand, the real-time views provide immediate access to the most current data.

Big Data 130

Big Data Big Data Apache Kafka Database 130

Towards AI

AUGUST 16, 2023

MLOps is the intersection of Machine Learning, DevOps, and Data Engineering. Data quality: ensuring the data received in production is processed in the same way as the training data. MIT Press, ISBN: 978–0262028189, 2014. [2] References [1] E. Alpaydin, Introduction to Machine Learning, 3rd ed.,

Machine Learning 98

Machine Learning Machine Learning AI AI 98

phData

FEBRUARY 2, 2024

Founded in 2014 by three leading cloud engineers, phData focuses on solving real-world data engineering, operations, and advanced analytics problems with the best cloud platforms and products. This search for efficiency led us to create the Data Source tool, which is part of the phData Toolkit.

SQL 52

SQL Database Data Quality Data Engineering 52

Alation

SEPTEMBER 28, 2022

DataOps is a set of technologies, processes, and best practices that combine a process-focused perspective on data and the automation methods of the Agile software development methodology to improve speed and quality and foster a collaborative culture of rapid, continuous improvement in the data analytics field.

DataOps 52

DataOps Data Pipeline Data Quality Analytics 52

DagsHub

OCTOBER 23, 2024

General Purpose Tools These tools help manage the unstructured data pipeline to varying degrees, with some encompassing data collection, storage, processing, analysis, and visualization. DagsHub's Data Engine DagsHub's Data Engine is a centralized platform for teams to manage and use their datasets effectively.

Machine Learning 52

Machine Learning Machine Learning Data Lakes AI 52

Hacker News

JULY 1, 2025

Prior to that, I spent a couple years at First Orion - a smaller data company - helping found & build out a data engineering team as one of the first engineers. We were focused on building data pipelines and models to protect our users from malicious phonecalls. Background in backend software engineering.

Python 53

Python AWS SQL ML 53