ODSC - Open Data Science

FEBRUARY 6, 2024

This open-source streaming platform enables the handling of high-throughput data feeds, ensuring that data pipelines are efficient, reliable, and capable of handling massive volumes of data in real-time. Its open-source nature means it’s continually evolving, thanks to contributions from its user community.

Data Engineering 52

Data Engineering Data Engineer Data Engineering Data Engineering 52

Mlearning.ai

FEBRUARY 16, 2023

The demand for information repositories enabling business intelligence and analytics is growing exponentially, giving birth to cloud solutions. The ultimate need for vast storage spaces manifests in data warehouses: specialized systems that aggregate data coming from numerous sources for centralized management and consistency.

Data Warehouse 52

Data Warehouse Business Intelligence Business Intelligence Database 52

Tableau

JUNE 8, 2021

Domain experts, for example, feel they are still overly reliant on core IT to access the data assets they need to make effective business decisions. In all of these conversations there is a sense of inertia: Data warehouses and data lakes feel cumbersome and data pipelines just aren't agile enough.

Tableau 52

Tableau Data Lakes Data Warehouse SQL 52

phData

APRIL 23, 2024

Designing New Data Pipelines Takes a Considerable Amount of Time and Knowledge Designing new ingestion pipelines is a complex undertaking that demands significant time and expertise. Engineering teams must maintain a complex web of ingestion pipelines capable of supporting many different sources, each with its own intricacies.

Data Governance 52

Data Governance Data Pipeline Data Warehouse Cloud Data 52

phData

JANUARY 19, 2024

Matillion’s Data Productivity Cloud is a versatile platform designed to increase the productivity of data teams. It provides a unified platform for creating and managing data pipelines that are effective for both coders and non-coders. Additional setup is typically optional.

Data Pipeline 52

Data Pipeline Data Warehouse ETL Azure 52

phData

FEBRUARY 14, 2023

Data integration is essentially the Extract and Load portion of the Extract, Load, and Transform (ELT) process. Data ingestion involves connecting your data sources, including databases, flat files, streaming data, etc, to your data warehouse. Snowflake provides native ways for data ingestion.

Data Warehouse 59

Data Warehouse Azure AWS Database 59

phData

JULY 19, 2023

In this blog post, we’ll dive into the amazing advantages of using Fivetran , a powerful data integration platform that will revolutionize the way you handle your data pipelines. They established an Information Architecture for Snowflake Data Cloud , enabling automated database and role creation.

Data Warehouse 52

Data Warehouse Database Data Pipeline SQL 52

phData

JULY 19, 2023

In this blog post, we’ll dive into the amazing advantages of using Fivetran , a powerful data integration platform that will revolutionize the way you handle your data pipelines. They established an Information Architecture for Snowflake Data Cloud , enabling automated database and role creation.

Data Warehouse 52

Data Warehouse Database Data Pipeline SQL 52

phData

JANUARY 24, 2024

Fivetran includes features like data movement, transformations, robust security, and compatibility with third-party tools like DBT, Airflow, Atlan, and more. Its seamless integration with popular cloud data warehouses like Snowflake can provide the scalability needed as your business grows.

Data Warehouse 52

Data Warehouse Data Pipeline Data Governance Data Quality 52

The MLOps Blog

MAY 17, 2023

We also discuss different types of ETL pipelines for ML use cases and provide real-world examples of their use to help data engineers choose the right one. What is an ETL data pipeline in ML? Xoriant It is common to use ETL data pipeline and data pipeline interchangeably.

ETL 59

ETL Data Pipeline ML ML 59

phData

MARCH 29, 2024

Founded in 2014 by three leading cloud engineers, phData focuses on solving real-world data engineering, operations, and advanced analytics problems with the best cloud platforms and products. Over the years, one of our primary focuses became Snowflake and migrating customers to this leading cloud data platform.

Data Engineering 52

Data Engineering Data Engineer Data Engineering Data Engineering 52

phData

JULY 17, 2023

Over the past few decades, the corporate data landscape has changed significantly. The shift from on-premise databases and spreadsheets to the modern era of cloud data warehouses and AI/ LLMs has transformed what businesses can do with data. This is where Fivetran and the Modern Data Stack come in.

Data Warehouse 52

Data Warehouse Data Pipeline Cloud Data ETL 52

Alation

JANUARY 17, 2023

The modern data stack is a combination of various software tools used to collect, process, and store data on a well-integrated cloud-based data platform. It is known to have benefits in handling data due to its robustness, speed, and scalability. Data ingestion/integration services. Data orchestration tools.

Data Warehouse 52

Data Warehouse ETL Cloud Data Tableau 52

phData

OCTOBER 12, 2023

When the data or pipeline configuration needs to be changed, tools like Fivetran and dbt reduce the time required to make the change, and increase the confidence your team can have around the change. These allow you to scale your pipelines quickly. Governance doesn’t have to be scary or preventative to your cloud data warehouse.

Data Warehouse 52

Data Warehouse Database Cloud Data Data Pipeline 52

phData

MARCH 8, 2023

Fivetran is an automated data integration platform that offers a convenient solution for businesses to consolidate and sync data from disparate data sources. With over 160 data connectors available, Fivetran makes it easy to move data out of, into, and across any cloud data platform in the market.

Data Warehouse 96

Data Warehouse Data Engineering Data Engineer Data Engineering 96

Dataversity

APRIL 1, 2021

In the data-driven world we live in today, the field of analytics has become increasingly important to remain competitive in business. In fact, a study by McKinsey Global Institute shows that data-driven organizations are 23 times more likely to outperform competitors in customer acquisition and nine times […].

Analytics 69

Analytics Analytics Data Silos Data Lakes 69

phData

FEBRUARY 5, 2024

There are many different third-party tools that work with Snowflake: Fivetran Fivetran is a tool dedicated to replicating applications, databases, events, and files into a high-performance data warehouse, such as Snowflake. How can organizations minimize downtime and ensure business continuity during the migration process to Snowflake?

SQL 52

SQL Database Data Quality Data Warehouse 52

Smart Data Collective

OCTOBER 17, 2022

If you haven’t already, moving to the cloud can be a realistic alternative. Cloud data warehouses provide various advantages, including the ability to be more scalable and elastic than conventional warehouses. Can’t get to the data. Data pipeline maintenance.

Big Data 105

Big Data Big Data Data Engineering Data Engineer 105

phData

JANUARY 5, 2023

Python has proven proficient in setting up pipelines, maintaining data flows, and transforming data with its simple syntax and proficiency in automation. Having been built completely for and in the cloud, the Snowflake Data Cloud has become an industry leader in cloud data platforms.

Python 52

Python Data Engineering Data Engineer Data Engineering 52

Flipboard

AUGUST 17, 2023

Amazon Redshift is the most popular cloud data warehouse that is used by tens of thousands of customers to analyze exabytes of data every day. AWS Glue is a serverless data integration service that makes it easy to discover, prepare, and combine data for analytics, ML, and application development.

ML 93

ML ML AWS Data Warehouse 93

Precisely

APRIL 3, 2023

It simply wasn’t practical to adopt an approach in which all of an organization’s data would be made available in one central location, for all-purpose business analytics. To speed analytics, data scientists implemented pre-processing functions to aggregate, sort, and manage the most important elements of the data.

ETL 52

ETL Database Analytics Analytics 52

Alation

APRIL 14, 2022

This two-part series will explore how data discovery, fragmented data governance , ongoing data drift, and the need for ML explainability can all be overcome with a data catalog for accurate data and metadata record keeping. The Cloud Data Migration Challenge. Data pipeline orchestration.

Data Governance 59

Data Governance ML ML Cloud Data 59

IBM Journey to AI blog

JULY 17, 2023

It is supported by querying, governance, and open data formats to access and share data across the hybrid cloud. Through workload optimization across multiple query engines and storage tiers, organizations can reduce data warehouse costs by up to 50 percent.

AI 67

AI AI Machine Learning Machine Learning 67

Tableau

DECEMBER 7, 2022

But good data—and actionable insights—are hard to get. Traditionally, organizations built complex data pipelines to replicate data. Those data architectures were brittle, complex, and time intensive to build and maintain, requiring data duplication and bloated data warehouse investments.

Tableau 53

Tableau Data Warehouse Data Pipeline Data Visualization 53

phData

MAY 25, 2023

One big issue that contributes to this resistance is that although Snowflake is a great cloud data warehousing platform, Microsoft has a data warehousing tool of its own called Synapse. Gateways are being used as another layer of security between Snowflake or cloud data source and Power BI users.

Power BI 84

Power BI Analytics Analytics Azure 84

Snorkel AI

MAY 26, 2023

What’s really important in the before part is having production-grade machine learning data pipelines that can feed your model training and inference processes. And that’s really key for taking data science experiments into production. And so that’s where we got started as a cloud data warehouse.

SQL 52

SQL ML ML Python 52

Snorkel AI

MAY 26, 2023

What’s really important in the before part is having production-grade machine learning data pipelines that can feed your model training and inference processes. And that’s really key for taking data science experiments into production. And so that’s where we got started as a cloud data warehouse.

SQL 52

SQL ML ML Python 52

phData

NOVEMBER 9, 2023

Snowflake’s Data Cloud has emerged as a leader in cloud data warehousing. As a fundamental piece of the modern data stack , Snowflake is helping thousands of businesses store, transform, and derive insights from their data easier, faster, and more efficiently than ever before.

Data Warehouse 98

Data Warehouse Data Lakes Clustering Cloud Data 98

phData

OCTOBER 17, 2023

Fivetran also takes care of all the manual elements of building and maintaining a data pipeline that is not business-related so that data teams don’t have to.

ETL 52

ETL Data Pipeline Data Engineering Data Engineer 52

phData

JULY 18, 2023

With the birth of cloud data warehouses, data applications, and generative AI , processing large volumes of data faster and cheaper is more approachable and desired than ever. First up, let’s dive into the foundation of every Modern Data Stack, a cloud-based data warehouse.

Data Warehouse 52

Data Warehouse SQL Cloud Data Analytics 52

Alation

APRIL 19, 2022

Data modernization is the process of transferring data to modern cloud-based databases from outdated or siloed legacy databases, including structured and unstructured data. In that sense, data modernization is synonymous with cloud migration. Data Pipeline Automation. Advanced Tooling.

Data Governance 52

Data Governance Cloud Data Database Data Lakes 52