IBM Journey to AI blog

JANUARY 5, 2023

The first generation of data architectures represented by enterprise data warehouse and business intelligence platforms were characterized by thousands of ETL jobs, tables, and reports that only a small group of specialized data engineers understood, resulting in an under-realized positive impact on the business.

Data Quality 52

Data Quality Data Lakes Data Warehouse Business Intelligence 52

phData

SEPTEMBER 28, 2023

Dataflows represent a cloud-based technology designed for data preparation and transformation purposes. Dataflows have different connectors to retrieve data, including databases, Excel files, APIs, and other similar sources, along with data manipulations that are performed using Online Power Query Editor.

Power BI 78

Power BI Data Preparation Machine Learning Machine Learning 78

ODSC - Open Data Science

FEBRUARY 24, 2023

There are many well-known libraries and platforms for data analysis such as Pandas and Tableau, in addition to analytical databases like ClickHouse, MariaDB, Apache Druid, Apache Pinot, Google BigQuery, Amazon RedShift, etc. With Great Expectations , data teams can express what they “expect” from their data using simple assertions.

Exploratory Data Analysis 126

Exploratory Data Analysis Data Visualization Data Analysis Data Analysis 126

The MLOps Blog

MARCH 15, 2023

Data Processing : You need to save the processed data through computations such as aggregation, filtering and sorting. Data Storage : To store this processed data to retrieve it over time – be it a data warehouse or a data lake. Relational database connectors are available.

Data Pipeline 59

Data Pipeline ETL SQL Data Quality 59

Alation

JANUARY 13, 2022

Thoughtworks says data mesh is key to moving beyond a monolithic data lake. Spoiler alert: data fabric and data mesh are independent design concepts that are, in fact, quite complementary. Thoughtworks says data mesh is key to moving beyond a monolithic data lake 2. Gartner on Data Fabric.

Data Lakes 59

Data Lakes Data Governance Data Quality Data Warehouse 59

The MLOps Blog

JUNE 27, 2023

It integrates with Git and provides a Git-like interface for data versioning, allowing you to track changes, manage branches, and collaborate with data teams effectively. Dolt Dolt is an open-source relational database system built on Git. Metaplane supports collaboration, anomaly detection, and data quality rule management.

Machine Learning 134

Machine Learning Machine Learning ML ML 134

The MLOps Blog

MAY 17, 2023

ETL data pipeline architecture | Source: Author Data Discovery: Data can be sourced from various types of systems, such as databases, file systems, APIs, or streaming sources. We also need data profiling i.e. data discovery, to understand if the data is appropriate for ETL.

ETL 59

ETL Data Pipeline ML ML 59