Data Science Blog

JUNE 27, 2023

Kappa – Architecture Jay Kreps introduced the Kappa architecture in 2014 as an alternative to the Lambda architecture. It offers the advantage of having a single ETL platform to develop and maintain. As a result, the development and maintenance efforts for both layers should not be underestimated.

Big Data 130

Big Data Big Data Apache Kafka Database 130

Applied Data Science

MAY 6, 2021

To keep myself sane, I use Airflow to automate tasks with simple, reusable pieces of code for frequently repeated elements of projects, for example: Web scraping ETL Database management Feature building and data validation And much more! What’s Airflow, and why’s it so good? What makes it my go to?

Data Scientist 130

Data Scientist Python Data Science Database 130

phData

NOVEMBER 9, 2023

If you go back to 2014, data warehouse platforms were built using legacy architectures that had drawbacks when it came to cost, scale, and flexibility. Data Processing: Snowflake can process large datasets and perform data transformations, making it suitable for ETL (Extract, Transform, Load) processes.

Data Warehouse 98

Data Warehouse Data Lakes Clustering Cloud Data 98

phData

JANUARY 19, 2024

Now, we’ll make a GET request to the following endpoint, which is set up to look for analytics books released between 2014 and 2024. The custom connector works very similarly to the API extract feature in Matillion ETL. Check out the API documentation for our sample. With that, you can cover most of the necessary connections.

Data Pipeline 52

Data Pipeline Data Warehouse ETL Azure 52

DagsHub

APRIL 7, 2024

The project was created in 2014 by Airbnb and has been developed by the Apache Software Foundation since 2016. Flexibility: Its use cases are wider than just machine learning; for example, we can use it to set up ETL pipelines. Hopefully, you can use it as a cheatsheet that will help you make a decision for your next project!

Machine Learning 59

Machine Learning Machine Learning ML ML 59

The MLOps Blog

OCTOBER 20, 2023

In 2014, Project Jupyter evolved from IPython. These last thoughts about traceability, reproducibility, and lineage will be the starting point for the next article in my series on Software Patterns in Data Science and ML Engineering , which will focus on how to uplevel your ETL skills.

SQL 52

SQL Database Data Scientist Python 52

DagsHub

OCTOBER 23, 2024

is similar to the traditional Extract, Transform, Load (ETL) process. BLEU on the WMT 2014 English- to-German translation task, improving over the existing best results, including ensembles, by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8

Machine Learning 52

Machine Learning Machine Learning Data Lakes AI 52