Smart Data Collective

APRIL 29, 2020

The extraction of raw data, transforming to a suitable format for business needs, and loading into a data warehouse. Data transformation. This process helps to transform raw data into clean data that can be analysed and aggregated. Data analytics and visualisation. Microsoft Azure.

Data Warehouse 98

Data Warehouse Azure SQL ETL 98

Pickl AI

SEPTEMBER 5, 2024

Summary: This blog provides a comprehensive roadmap for aspiring Azure Data Scientists, outlining the essential skills, certifications, and steps to build a successful career in Data Science using Microsoft Azure. What is Azure?

Azure 52

Azure Data Scientist Data Science Machine Learning 52

phData

SEPTEMBER 5, 2023

Google Analytics 4 (GA4) is a powerful tool for collecting and analyzing website and app data that many businesses rely heavily on to make informed business decisions. However, there might be instances where you need to migrate the raw event data from GA4 to Snowflake for more in-depth analysis and business intelligence purposes.

Azure 52

Azure Analytics Analytics Data Pipeline 52

Dataconomy

SEPTEMBER 4, 2023

Diagnostic analytics: Diagnostic analytics goes a step further by analyzing historical data to determine why certain events occurred. By understanding the “why” behind past events, organizations can make informed decisions to prevent or replicate them. Ensure that data is clean, consistent, and up-to-date.

Analytics 203

Analytics Analytics Big Data Analytics Big Data Analytics 203

phData

JUNE 5, 2025

In the fast-moving world of data, the Snowflake AI Data Cloud has established itself as an essential part of the Modern Data Stack. Through its versatile platform, organizations can build efficient data warehouses and harness the power of data monetization via Secure Data Shares, accessible through the Snowflake Marketplace.

Database 52

Database Azure AWS SQL 52

Pickl AI

FEBRUARY 23, 2025

Statistics : A survey by Databricks revealed that 80% of Spark users reported improved performance in their data processing tasks compared to traditional systems. Google Cloud BigQuery Google Cloud BigQuery is a fully-managed enterprise data warehouse that enables super-fast SQL queries using the processing power of Googles infrastructure.

Big Data 52

Big Data Big Data Apache Hadoop Apache Kafka 52

Pickl AI

MARCH 19, 2025

Apache Kafka Apache Kafka is a distributed event streaming platform used for real-time data processing. It helps data engineers collect, store, and process streams of records in a fault-tolerant way, making it crucial for building reliable data pipelines.

Data Engineering 52

Data Engineering Data Engineer Data Engineering Data Engineering 52

phData

FEBRUARY 14, 2023

Data integration is essentially the Extract and Load portion of the Extract, Load, and Transform (ELT) process. Data ingestion involves connecting your data sources, including databases, flat files, streaming data, etc, to your data warehouse. Snowflake provides native ways for data ingestion.

Data Warehouse 59

Data Warehouse Azure AWS Database 59

phData

APRIL 29, 2024

Recognizing these specific needs, Fivetran has developed a range of connectors, including dedicated applications, databases, files, and events, which can accommodate the diverse formats used by healthcare systems. Addressing these needs may pose challenges that lead to the implementation of custom solutions rather than a uniform approach.

SQL 52

SQL Data Warehouse Azure Cloud Data 52

phData

MARCH 7, 2023

Note: Recommended to only edit the configuration marked as TODO gg.target=snowflake #The Snowflake Event Handler #TODO: Edit JDBC ConnectionUrl gg.eventhandler.snowflake.connectionURL=jdbc:snowflake://.snowflakecomputing.com/?warehouse= The S3 Event Handler #TODO: Edit the AWS region #gg.eventhandler.s3.region= snowflakecomputing.com/?warehouse=

Hadoop 59

Hadoop Database Data Warehouse AWS 59

Mlearning.ai

FEBRUARY 16, 2023

The ultimate need for vast storage spaces manifests in data warehouses: specialized systems that aggregate data coming from numerous sources for centralized management and consistency. In this article, you’ll discover what a Snowflake data warehouse is, its pros and cons, and how to employ it efficiently.

Data Warehouse 52

Data Warehouse Business Intelligence Business Intelligence Database 52

ODSC - Open Data Science

JANUARY 27, 2023

Training Your PyTorch Model Using Components and Pipelines in Azure ML In this post, we’ll explore how you can take your PyTorch model training to the next level, using Azure ML. This is where the BigQuery data warehouse comes into play.

Algorithm 52

Algorithm Machine Learning Machine Learning Azure 52

phData

JUNE 11, 2025

Whether you’re running small-scale analytics or managing enterprise-level data warehouses, these tips will help drive performance and meaningful business outcomes for your organization. Storage Costs Our first tip involves taking a closer look at managing how your data is stored, organized, and accessed.

Clustering 59

Clustering SQL Database Data Lakes 59

IBM Journey to AI blog

SEPTEMBER 11, 2023

Db2 can run on Red Hat OpenShift and Kubernetes environments, ROSA & EKS on AWS, and ARO & AKS on Azure deployments. Customers can also choose to run IBM Db2 database and IBM Db2 Warehouse as a fully managed service. Many consider a NoSQL database essential for high data ingestion rates.

Database 101

Database SQL Data Warehouse Machine Learning 101

Pickl AI

NOVEMBER 4, 2024

Role of Data Engineers in the Data Ecosystem Data Engineers play a crucial role in the data ecosystem by bridging the gap between raw data and actionable insights. They are responsible for building and maintaining data architectures, which include databases, data warehouses, and data lakes.

Data Engineering 52

Data Engineering Data Engineer Data Engineering Data Engineering 52

ODSC - Open Data Science

FEBRUARY 6, 2024

Google BigQuery When it comes to cloud data warehouses, Snowflake, Amazon Redshift, and Google BigQuery are often at the forefront of discussions. Each platform offers unique features and benefits, making it vital for data engineers to understand their differences. Interested in attending an ODSC event?

Data Engineering 52

Data Engineering Data Engineer Data Engineering Data Engineering 52

ODSC - Open Data Science

MAY 29, 2023

Research and new offerings in AI fuel the field of data science. At ODSC events, we love to highlight our partners’ latest developments, products, services, and research, so that you can see how organizations from around the world are changing the landscape of AI. The event is only a few weeks away, so don’t delay!

Machine Learning 52

Machine Learning Machine Learning Data Science AI 52

ODSC - Open Data Science

OCTOBER 7, 2024

Kaggle Grandmaster Jiwei Liu and NVIDIA experts will answer all your questions about the upcoming event. Syncari : Tailored for data teams, Syncari integrates and unifies data across systems, enhancing real-time AI-driven decision-making.

Data Scientist 52

Data Scientist Data Science Machine Learning Machine Learning 52

ODSC - Open Data Science

JANUARY 18, 2024

They’ll also work with software engineers to ensure that the data infrastructure is scalable and reliable. These professionals will work with their colleagues to ensure that data is accessible, with proper access. The reason this is an important skill is that ETL is a critical process for data warehousing and business intelligence.

Data Engineering 52

Data Engineering Data Engineer Data Engineering Data Engineering 52

IBM Journey to AI blog

AUGUST 3, 2023

IBM Security® Guardium® Data Protection empowers security teams to safeguard sensitive data through discovery and classification, data activity monitoring, vulnerability assessments and advanced threat detection.

Azure 71

Azure Database Analytics Analytics 71

phData

JUNE 3, 2024

Taking it one step further, if you don’t want your data traversing the public internet, you can implement one of the private connections available from the cloud provider your Snowflake account is created on, i.e., Azure Private Link, AWS Privatelink, or Google Cloud Service Private Connect.

Machine Learning 52

Machine Learning Machine Learning Database Cloud Data 52

databricks

JUNE 12, 2025

Events Data + AI Summit Data + AI World Tour Data Intelligence Days Event Calendar Blog and Podcasts Databricks Blog Explore news, product announcements, and more Databricks Mosaic Research Blog Discover the latest in our Gen AI research Data Brew Podcast Let’s talk data!

SQL 236

SQL Data Engineer Data Engineering Data Engineering 236

Alation

APRIL 14, 2022

This two-part series will explore how data discovery, fragmented data governance , ongoing data drift, and the need for ML explainability can all be overcome with a data catalog for accurate data and metadata record keeping. The Cloud Data Migration Challenge. Automatic sampling to test transformation.

Data Governance 59

Data Governance ML ML Cloud Data 59

ODSC - Open Data Science

JULY 11, 2023

Co-location data centers: These are data centers that are owned and operated by third-party providers and are used to house the IT equipment of multiple organizations. Edge data centers: These are data centers that are located closer to the edge of the network, where data is generated and consumed, rather than in central locations.

Data Lakes 40

Data Lakes AI AI Cloud Computing 40

phData

NOVEMBER 13, 2023

Delphi Prerequisites and Compatibility: It requires a dbt Cloud Team or Enterprise account and supports popular data warehouses like Snowflake, BigQuery, Databricks, and Redshift. These jobs can be triggered via schedule or events, ensuring your data assets are always up-to-date.

Database 52

Database Business Intelligence Business Intelligence Data Silos 52

Data Science Blog

FEBRUARY 7, 2023

Schon damals in Ansätzen, aber spätestens heute gilt es zu recht als Best Practise, die Datenanbindung an ein Data Warehouse zu machen und in diesem die Daten für die Reports aufzubereiten. Ein Data Warehouse ist eine oder eine Menge von Datenbanken. Was gerade zum Trend wird, ist der Aufbau eines Data Lakehouses.

Business Intelligence 286

Business Intelligence Business Intelligence Data Warehouse Data Lakes 286

Data Science Blog

SEPTEMBER 3, 2024

Dabei darf gerne in Erinnerung gerufen werden, dass Process Mining im Kern eine Graphenanalyse ist, die ein Event Log in Graphen umwandelt, Aktivitäten (Events) stellen dabei die Knoten und die Prozesszeiten die Kanten dar, zumindest ist das grundsätzlich so. Es handelt sich dabei also um eine Analysemethodik und nicht um ein Tool.

Data Science 130

Data Science Power BI Azure Data Warehouse 130

phData

OCTOBER 28, 2024

Understanding Matillion and Snowflake, the Python Component, and Why it is Used Matillion is a SaaS-based data integration platform that can be hosted in AWS, Azure, or GCP and supports multiple cloud data warehouses. If not, it will retry after a certain duration (E.g., 30 minutes).

Python 52

Python ETL AWS Database 52

phData

SEPTEMBER 27, 2024

Methods that allow our customer data models to be as dynamic and flexible as the customers they represent. In this guide, we will explore concepts like transitional modeling for customer profiles, the power of event logs for customer behavior, persistent staging for raw customer data, real-time customer data capture, and much more.

Data Modeling 52

Data Modeling Data Models Apache Kafka Data Lakes 52