phData

NOVEMBER 9, 2023

Snowflake’s Data Cloud has emerged as a leader in cloud data warehousing. As a fundamental piece of the modern data stack , Snowflake is helping thousands of businesses store, transform, and derive insights from their data easier, faster, and more efficiently than ever before.

Data Warehouse 98

Data Warehouse Data Lakes Clustering Cloud Data 98

Smart Data Collective

APRIL 29, 2020

Data management approaches are varied and may be categorised in the following: Cloud data management. The storage and processing of data through a cloud-based system of applications. Master data management. Extraction, Transform, Load (ETL). Data transformation.

Data Warehouse 98

Data Warehouse Azure SQL ETL 98

Dataversity

MAY 17, 2021

There are advantages and disadvantages to both ETL and ELT. The post Understanding the ETL vs. ELT Alphabet Soup and When to Use Each appeared first on DATAVERSITY. To understand which method is a better fit, it’s important to understand what it means when one letter comes before the other.

ETL 52

ETL Data Lakes Data Warehouse Database 52

phData

AUGUST 9, 2023

With ELT, we first extract data from source systems, then load the raw data directly into the data warehouse before finally applying transformations natively within the data warehouse. This is unlike the more traditional ETL method, where data is transformed before loading into the data warehouse.

Data Warehouse 52

Data Warehouse ETL Cloud Data Big Data 52

phData

MARCH 8, 2023

Fivetran is an automated data integration platform that offers a convenient solution for businesses to consolidate and sync data from disparate data sources. With over 160 data connectors available, Fivetran makes it easy to move data out of, into, and across any cloud data platform in the market.

Data Warehouse 96

Data Warehouse Data Engineering Data Engineer Data Engineering 96

phData

JULY 17, 2023

Over the past few decades, the corporate data landscape has changed significantly. The shift from on-premise databases and spreadsheets to the modern era of cloud data warehouses and AI/ LLMs has transformed what businesses can do with data. This is where Fivetran and the Modern Data Stack come in.

Data Warehouse 52

Data Warehouse Data Pipeline Cloud Data ETL 52

phData

SEPTEMBER 2, 2024

In this blog, we will cover the best practices for developing jobs in Matillion, an ETL/ELT tool built specifically for cloud database platforms. It offers a cloud-agnostic data productivity hub called Matillion Data Productivity Cloud. What Are Matillion Jobs and Why Do They Matter?

ETL 52

ETL Data Warehouse SQL Database 52

Alation

JANUARY 17, 2023

It is known to have benefits in handling data due to its robustness, speed, and scalability. A typical modern data stack consists of the following: A data warehouse. Data ingestion/integration services. Reverse ETL tools. Data orchestration tools. How Did the Modern Data Stack Get Started?

Data Warehouse 52

Data Warehouse ETL Tableau Cloud Data 52

phData

OCTOBER 9, 2023

As organizations embrace the benefits of data vault, it becomes crucial to ensure optimal performance in the underlying data platform. One such platform that has revolutionized cloud data warehousing is the Snowflake Data Cloud. However, joining tables using a hash key can take longer than a sequential key.

ETL 69

ETL Clustering Data Warehouse SQL 69

Dataconomy

SEPTEMBER 4, 2023

Define data ownership, access controls, and data management processes to maintain the integrity and confidentiality of your data. Data integration: Integrate data from various sources into a centralized cloud data warehouse or data lake.

Analytics 203

Analytics Analytics Big Data Analytics Big Data Analytics 203

phData

JUNE 6, 2024

Data gets ingested, centralized, and deployed within your cloud data warehouse. This allows companies to use their pre-existing data tools and prevents the need for costly setups. Companies need to bring in data from a wide variety of sources to get a holistic view of the customer.

Data Warehouse 52

Data Warehouse Cloud Data ETL Data Models 52

How to Learn Machine Learning

APRIL 26, 2025

Data Cleaning and Preparation The tasks of cleaning and preparing the data take place before the analysis. This includes duplicate removal, missing value treatment, variable transformation, and normalization of data.

Data Science 52

Data Science Data Analyst Data Scientist Machine Learning 52

Smart Data Collective

OCTOBER 17, 2022

If you haven’t already, moving to the cloud can be a realistic alternative. Cloud data warehouses provide various advantages, including the ability to be more scalable and elastic than conventional warehouses. Can’t get to the data.

Big Data 119

Big Data Big Data Data Engineering Data Engineer 119

phData

APRIL 20, 2023

In today’s world, data-driven applications demand more flexibility, scalability, and auditability, which traditional data warehouses and modeling approaches lack. This is where the Snowflake Data Cloud and data vault modeling comes in handy. What is Data Vault Modeling?

Data Warehouse 72

Data Warehouse Data Governance Clustering Database 72

phData

APRIL 29, 2024

Understanding Fivetran Fivetran is a popular Software-as-a-Service platform that enables users to automate the movement of data and ETL processes across diverse sources to a target destination. For a longer overview, along with insights and best practices, please feel free to jump back to the previous blog.

SQL 52

SQL Data Warehouse Azure Cloud Data 52

Precisely

APRIL 3, 2023

How can an organization enable flexible digital modernization that brings together information from multiple data sources, while still maintaining trust in the integrity of that data? To speed analytics, data scientists implemented pre-processing functions to aggregate, sort, and manage the most important elements of the data.

ETL 52

ETL Analytics Analytics Database 52

phData

NOVEMBER 28, 2023

Matillion is also built for scalability and future data demands, with support for cloud data platforms such as Snowflake Data Cloud , Databricks, Amazon Redshift, Microsoft Azure Synapse, and Google BigQuery, making it future-ready, everyone-ready, and AI-ready. Why Does it Matter? No problem.

Data Warehouse 52

Data Warehouse Data Pipeline ETL Azure 52

phData

FEBRUARY 14, 2023

Data integration is essentially the Extract and Load portion of the Extract, Load, and Transform (ELT) process. Data ingestion involves connecting your data sources, including databases, flat files, streaming data, etc, to your data warehouse. Snowflake provides native ways for data ingestion.

Data Warehouse 59

Data Warehouse Azure AWS Database 59

phData

APRIL 28, 2025

A Matillion pipeline is a collection of jobs that extract, load, and transform (ETL/ELT) data from various sources into a target system, such as a cloud data warehouse like Snowflake.

AI 52

AI AI SQL ETL 52

phData

JULY 18, 2023

With the birth of cloud data warehouses, data applications, and generative AI , processing large volumes of data faster and cheaper is more approachable and desired than ever. This typically results in long-running ETL pipelines that cause decisions to be made on stale or old data.

Data Warehouse 52

Data Warehouse Analytics Analytics Cloud Data 52

phData

JANUARY 5, 2023

Python has proven proficient in setting up pipelines, maintaining data flows, and transforming data with its simple syntax and proficiency in automation. Having been built completely for and in the cloud, the Snowflake Data Cloud has become an industry leader in cloud data platforms.

Python 52

Python Data Engineering Data Engineering Data Engineer 52

phData

OCTOBER 17, 2023

The story is all too common – a business user requests some data, the data team creates/prioritizes a ticket, and said ticket is completed after some number of months (or weeks if you’re lucky) – just to have the data be wrong, and the whole process starts again. Those are scary for data teams to change.

ETL 52

ETL Data Pipeline Data Engineering Data Engineer 52

phData

JANUARY 4, 2023

Why You Should Consider Migrating from Netezza to Snowflake With Netezza, IBM was one of the first companies to provide a data warehousing solution to allow organizations to analyze and manage large amounts of data. However, as technology has evolved, the need for more advanced, agile data warehousing solutions has become apparent.

Data Warehouse 52

Data Warehouse SQL Database ETL 52

IBM Journey to AI blog

JULY 17, 2023

It is supported by querying, governance, and open data formats to access and share data across the hybrid cloud. Through workload optimization across multiple query engines and storage tiers, organizations can reduce data warehouse costs by up to 50 percent.

AI 74

AI AI Machine Learning Machine Learning 74

phData

JULY 10, 2024

Unlike traditional BI tools, its user-friendly interface ensures that users of all technical levels can seamlessly interact with data. The platform’s integration with cloud data warehouses like Snowflake AI Data Cloud , Google BigQuery, and Amazon Redshift makes it a vital tool for organizations harnessing big data.

SQL 52

SQL Data Warehouse Analytics Analytics 52

Dataversity

SEPTEMBER 6, 2021

The rush to become data-driven is more heated, important, and pronounced than it has ever been. Businesses understand that if they continue to lead by guesswork and gut feeling, they’ll fall behind organizations that have come to recognize and utilize the power and potential of data. Click to learn more about author Mike Potter.

Data Warehouse 98

Data Warehouse Data Lakes ETL Cloud Data 98

Alation

OCTOBER 27, 2022

Few actors in the modern data stack have inspired the enthusiasm and fervent support as dbt. This data transformation tool enables data analysts and engineers to transform, test and document data in the cloud data warehouse. But what does this mean from a practitioner perspective?

Data Analyst 52

Data Analyst Data Scientist Analytics Analytics 52

phData

JUNE 14, 2023

In recent years, data engineering teams working with the Snowflake Data Cloud platform have embraced the continuous integration/continuous delivery (CI/CD) software development process to develop data products and manage ETL/ELT workloads more efficiently.

Data Pipeline 52

Data Pipeline Database SQL Data Engineering 52

phData

FEBRUARY 5, 2024

There are many different third-party tools that work with Snowflake: Fivetran Fivetran is a tool dedicated to replicating applications, databases, events, and files into a high-performance data warehouse, such as Snowflake.

SQL 52

SQL Database Data Quality Data Warehouse 52

phData

JANUARY 19, 2024

Matillion is also built for scalability and future data demands, with support for cloud data platforms such as Snowflake Data Cloud , Databricks, Amazon Redshift, Microsoft Azure Synapse, and Google BigQuery, making it future-ready, everyone-ready, and AI-ready. Additional setup is typically optional.

Data Pipeline 52

Data Pipeline Data Warehouse ETL Azure 52

Mlearning.ai

JULY 10, 2023

In my 7 years of Data Science journey, I’ve been exposed to a number of different databases including but not limited to Oracle Database, MS SQL, MySQL, EDW, and Apache Hadoop. You can use stored procedures to handle complex ETL processes, make API calls, and perform data validation.

SQL 52

SQL Database Apache Hadoop Data Science 52

Dataversity

APRIL 1, 2021

In the data-driven world we live in today, the field of analytics has become increasingly important to remain competitive in business. In fact, a study by McKinsey Global Institute shows that data-driven organizations are 23 times more likely to outperform competitors in customer acquisition and nine times […].

Analytics 69

Analytics Analytics Data Silos Data Lakes 69

Alation

JANUARY 25, 2022

On the policy front, a feature like Policy Center empowers users to enforce and track policies at scale; this ensures that people use data compliantly, and organizations are prepared for compliance audits. See Gartner’s “ How DataOps Amplifies Data and Analytics Business Value ”).

DataOps 52

DataOps SQL ML ML 52

Dataversity

MARCH 17, 2021

I do not think it is an exaggeration to say data analytics has come into its own over the past decade or so. What started out as an attempt to extract business insights from transactional data in the ’90s and early 2000s has now transformed into an […]. The post Is Lakehouse Architecture a Grand Unification in Data Analytics?

Analytics 52

Analytics Analytics Data Lakes Data Warehouse 52

phData

OCTOBER 28, 2024

Modern low-code/no-code ETL tools allow data engineers and analysts to build pipelines seamlessly using a drag-and-drop and configure approach with minimal coding. One such option is the availability of Python Components in Matillion ETL, which allows us to run Python code inside the Matillion instance.

Python 52

Python ETL AWS Database 52

Alation

APRIL 14, 2023

Last week, the Alation team had the privilege of joining IT professionals, business leaders, and data analysts and scientists for the Modern Data Stack Conference in San Francisco. In “The modern data stack is dead, long live the modern data stack!” Cloud costs are growing prohibitive.

Data Pipeline 52

Data Pipeline Data Warehouse Cloud Data ETL 52

phData

SEPTEMBER 27, 2024

If the event log is your customer’s diary, think of persistent staging as their scrapbook – a place where raw customer data is collected, organized, and kept for future reference. In traditional ETL (Extract, Transform, Load) processes in CDPs, staging areas were often temporary holding pens for data.

Data Models 52

Data Models Data Modeling Apache Kafka Data Lakes 52