databricks

JUNE 4, 2025

Events Data + AI Summit Data + AI World Tour Data Intelligence Days Event Calendar Blog and Podcasts Databricks Blog Explore news, product announcements, and more Databricks Mosaic Research Blog Discover the latest in our Gen AI research Data Brew Podcast Let’s talk data!

AWS 130

AWS AI AI Data Science 130

IBM Journey to AI blog

FEBRUARY 2, 2023

In today’s world, data warehouses are a critical component of any organization’s technology ecosystem. The rise of cloud has allowed data warehouses to provide new capabilities such as cost-effective data storage at petabyte scale, highly scalable compute and storage, pay-as-you-go pricing and fully managed service delivery.

Data Warehouse 58

Data Warehouse Cloud Data Analytics Analytics 58

Smart Data Collective

SEPTEMBER 8, 2021

The ETL process is defined as the movement of data from its source to destination storage (typically a Data Warehouse) for future use in reports and analyzes. The data is initially extracted from a vast array of sources before transforming and converting it to a specific format based on business requirements. Conclusion.

ETL 126

ETL Hadoop Data Warehouse Data Pipeline 126

Data Science Connect

JANUARY 27, 2023

It is the process of collecting, storing, managing, and analyzing large amounts of data, and data engineers are responsible for designing and implementing the systems and infrastructure that make this possible. Learn about data modeling: Data modeling is the process of creating a conceptual representation of data.

Data Engineering 130

Data Engineering Data Engineering Data Engineer Data Engineering 130

Pickl AI

APRIL 6, 2023

Accordingly, one of the most demanding roles is that of Azure Data Engineer Jobs that you might be interested in. The following blog will help you know about the Azure Data Engineering Job Description, salary, and certification course. How to Become an Azure Data Engineer?

Azure 52

Azure Data Engineering Data Engineering Data Engineer 52

Pickl AI

SEPTEMBER 5, 2024

Summary: This blog provides a comprehensive roadmap for aspiring Azure Data Scientists, outlining the essential skills, certifications, and steps to build a successful career in Data Science using Microsoft Azure. What is Azure?

Azure 52

Azure Data Scientist Data Science Machine Learning 52

phData

SEPTEMBER 5, 2023

In this step-by-step guide, we will walk you through setting up a data ingestion pipeline using Azure Data Factory (ADF), Google BigQuery, and the Snowflake Data Cloud. By the end of this tutorial, you’ll have a seamless pipeline that fetches and syncs your GA4 raw events data to Snowflake efficiently.

Azure 52

Azure Analytics Analytics Data Pipeline 52

phData

NOVEMBER 8, 2024

Open Table Format (OTF) architecture now provides a solution for efficient data storage, management, and processing while ensuring compatibility across different platforms. In this blog, we will discuss: What is the Open Table format (OTF)? Delta Lake became popular for making data lakes more reliable and easy to manage.

Data Lakes 52

Data Lakes Data Warehouse Database Azure 52

Dataversity

MAY 28, 2024

Most enterprises today store and process vast amounts of data from various sources within a centralized repository known as a data warehouse or data lake, where they can analyze it with advanced analytics tools to generate critical business insights.

Analytics 52

Analytics Analytics Data Warehouse Data Lakes 52

phData

OCTOBER 2, 2023

In this blog post, we’ll explore what a feature store is in the first place, explore a few of the key advantages (and disadvantages), and touch on when is the right time for your organization to build or adopt a feature store. It also supports a vast selection of data sources, including the Snowflake Data Cloud , Redshift, and HDFS.

Data Warehouse 98

Data Warehouse ML ML Machine Learning 98

phData

JUNE 5, 2025

Through its versatile platform, organizations can build efficient data warehouses and harness the power of data monetization via Secure Data Shares, accessible through the Snowflake Marketplace. As of the time this blog was written, Snowflake does not cater to these needs.

Database 52

Database Azure AWS SQL 52

phData

MAY 25, 2023

In a perfect world, Microsoft would have clients push even more storage and compute to its Azure Synapse platform. One of the easiest ways for Snowflake to achieve this is to have analytics solutions query their data warehouse in real-time (also known as DirectQuery).

Power BI 84

Power BI Analytics Analytics Azure 84

phData

FEBRUARY 14, 2023

Data integration is essentially the Extract and Load portion of the Extract, Load, and Transform (ELT) process. Data ingestion involves connecting your data sources, including databases, flat files, streaming data, etc, to your data warehouse. Snowflake provides native ways for data ingestion.

Data Warehouse 59

Data Warehouse Azure AWS Database 59

Pickl AI

MARCH 19, 2025

Data engineering is all about collecting, organising, and moving data so businesses can make better decisions. Handling massive amounts of data would be a nightmare without the right tools. In this blog, well explore the best data engineering tools that make data work easier, faster, and more reliable.

Data Engineering 52

Data Engineering Data Engineering Data Engineer Data Engineering 52

phData

APRIL 29, 2024

In our previous blog, Top 5 Fivetran Connectors for Financial Services , we explored Fivetran’s capabilities that address the data integration needs of the finance industry. Now, let’s cover the healthcare industry, which also has a surging demand for data and analytics, along with the underlying processes to make it happen.

SQL 52

SQL Data Warehouse Azure Cloud Data 52

phData

SEPTEMBER 2, 2024

In this blog, we will cover the best practices for developing jobs in Matillion, an ETL/ELT tool built specifically for cloud database platforms. The blog will be divided into three broad sections: Design, SDLC, and Security, each with its best practices. What Are Matillion Jobs and Why Do They Matter? Below are the best practices.

ETL 52

ETL Data Warehouse SQL Database 52

IBM Journey to AI blog

SEPTEMBER 11, 2023

Db2 can run on Red Hat OpenShift and Kubernetes environments, ROSA & EKS on AWS, and ARO & AKS on Azure deployments. Customers can also choose to run IBM Db2 database and IBM Db2 Warehouse as a fully managed service. Db2 database SaaS is a fully managed service for a high – performance, transactional workload.

Database 101

Database SQL Data Warehouse Machine Learning 101

IBM Journey to AI blog

JANUARY 18, 2023

They defined it as : “ A data lakehouse is a new, open data management architecture that combines the flexibility, cost-efficiency, and scale of data lakes with the data management and ACID transactions of data warehouses, enabling business intelligence (BI) and machine learning (ML) on all data. ”.

Data Lakes 52

Data Lakes Data Warehouse Azure Apache Hadoop 52

phData

SEPTEMBER 19, 2023

By employing robust data modeling techniques, businesses can unlock the true value of their data lake and transform it into a strategic asset. With many data modeling methodologies and processes available, choosing the right approach can be daunting. Want to learn more about data governance?

Data Lakes 52

Data Lakes Data Modeling Data Models Data Warehouse 52

DrivenData Labs

MAY 21, 2024

Data storage ¶ V1 was designed to encourage data scientists to (1) separate their data from their codebase and (2) store their data on the cloud. We have now added support for Azure and GCS as well. Teams that primarily access hosted data or assets (e.g., These options include DVC, Pachyderm and Quilt.

Data Science 233

Data Science Python Data Scientist Data Warehouse 233

phData

SEPTEMBER 15, 2023

In order to unlock the potential of these tools, your CRM data must remain synced between Salesforce and Snowflake. Salesforce Sync Out offers an excellent and cost-efficient solution for seamlessly ingesting Salesforce data into Snowflake. Warehouse for loading the data (start with XSMALL or SMALL warehouses).

Data Warehouse 52

Data Warehouse Tableau Data Silos Analytics 52

phData

NOVEMBER 28, 2023

In this blog, we will show you how easy it is to get your Data Productivity Cloud environment up and running and how you can start your studies on the platform. What is Matillion Data Productivity Cloud? Creating Your Account First things first, let’s create your Matillion account in order to deploy your Data Productivity Cloud.

Data Warehouse 52

Data Warehouse Data Pipeline ETL Azure 52

phData

NOVEMBER 12, 2024

Snowflake is one of the most powerful cloud-based data warehouses on the market, offering a scalable solution built for analytics. However, when storing sensitive data in Snowflake, it’s crucial to implement every security measure possible to protect it from unauthorized access and potential breaches.

Azure 52

Azure Data Warehouse AWS Data Analyst 52

IBM Journey to AI blog

JANUARY 10, 2023

This allows data that exists in cloud object storage to be easily combined with existing data warehouse data without data movement. The advantage to NPS clients is that they can store infrequently used data in a cost-effective manner without having to move that data into a physical data warehouse table.

Data Warehouse 52

Data Warehouse Data Analysis Data Analysis SQL 52

phData

MARCH 7, 2023

From keeping an active backup to consolidating or broadcasting data between platforms, GoldenGate is a very versatile tool that can handle many different use cases. Prerequisites In this blog, we focus on ingesting data into the Snowflake Data Cloud with GoldenGate and so we will pick up the replication process within GoldenGate.

Hadoop 59

Hadoop Database Data Warehouse AWS 59

phData

MAY 4, 2023

This blog was co-written by Arnab Mondal & David Beyer. In the modern era, data reigns supreme, often regarded as the new oil of this century. To maintain a competitive edge, organizations must not only amass vast quantities of data but also skillfully harness its potential. What is Teradata?

SQL 52

SQL Data Warehouse Azure Big Data 52

ODSC - Open Data Science

JANUARY 18, 2024

If you are a data scientist, you may be wondering if you can transition into data engineering. The good news is that there are many skills that data scientists already have that are transferable to data engineering. In this blog post, we will discuss how you can become a data engineer if you are a data scientist.

Data Engineering 52

Data Engineering Data Engineering Data Engineer Data Engineering 52

phData

AUGUST 9, 2024

Typically, this data is scattered across Excel files on business users’ desktops. Cloud Storage Upload Snowflake can easily upload files from cloud storage (AWS S3, Azure Storage, GCP Cloud Storage). The downside is that dbt/source control can be challenging for non-technical users.

ETL 52

ETL Data Warehouse Data Governance Tableau 52

phData

JULY 18, 2023

With the birth of cloud data warehouses, data applications, and generative AI , processing large volumes of data faster and cheaper is more approachable and desired than ever. First up, let’s dive into the foundation of every Modern Data Stack, a cloud-based data warehouse.

Data Warehouse 52

Data Warehouse Analytics Analytics Cloud Data 52

IBM Journey to AI blog

AUGUST 3, 2023

IBM Security® Guardium® Data Protection empowers security teams to safeguard sensitive data through discovery and classification, data activity monitoring, vulnerability assessments and advanced threat detection.

Azure 71

Azure Database Analytics Analytics 71

Pickl AI

JULY 25, 2024

From extracting information from databases and spreadsheets to ingesting streaming data from IoT devices and social media platforms, It’s the foundation upon which data-driven initiatives are built. Batch Processing In this method, data is collected over a period and then processed in groups or batches.

Apache Kafka 52

Apache Kafka Data Lakes Data Warehouse Data Quality 52

Pickl AI

JUNE 7, 2024

At the heart of this process lie ETL Tools—Extract, Transform, Load—a trio that extracts data, tweaks it, and loads it into a destination. Choosing the right ETL tool is crucial for smooth data management. This blog will delve into ETL Tools, exploring the top contenders and their roles in modern data integration.

ETL 40

ETL Data Quality Data Pipeline Data Warehouse 40

The MLOps Blog

MAY 17, 2023

Focus Area ETL helps to transform the raw data into a structured format that can be easily available for data scientists to create models and interpret for any data-driven decision. A data pipeline is created with the focus of transferring data from a variety of sources into a data warehouse.

ETL 59

ETL Data Pipeline ML ML 59

phData

SEPTEMBER 25, 2024

Under this category, tools with pre-built connectors for popular data sources and visual tools for data transformation are better choices. Fivetran is a data integration platform that helps businesses move data from various sources to various destinations, such as data warehouses, databases, or cloud storage.

ETL 52

ETL Database Data Warehouse Analytics 52

Pickl AI

APRIL 26, 2024

This comprehensive blog outlines vital aspects of Data Analyst interviews, offering insights into technical, behavioural, and industry-specific questions. It covers essential topics such as SQL queries, data visualization, statistical analysis, machine learning concepts, and data manipulation techniques.

Data Analyst 52

Data Analyst Data Analysis Data Analysis Machine Learning 52

phData

JUNE 3, 2024

In this blog, we will take you through Snowflake’s top features , what they can do for you, and how to get the most out of your Snowflake account. Elasticity and Scalability The best benefit of having your data on Snowflake’s cloud-based platform is its elasticity and scalability. What are Snowflake’s Top Features?

Machine Learning 52

Machine Learning Machine Learning Database Cloud Data 52

Pickl AI

JULY 25, 2023

Data Engineering: Laying the Foundation for Data Success While Data Science deals with data analysis and insights, Data Engineering focuses on the design, construction, and maintenance of robust data pipelines and infrastructure. Data Modeling: Entity-Relationship (ER) diagrams, data normalization, etc.

Data Engineering 52

Data Engineering Data Engineering Data Engineer Data Engineering 52