AWS Machine Learning Blog

OCTOBER 28, 2024

The workflow includes the following steps: Within the SageMaker Canvas interface, the user composes a SQL query to run against the GCP BigQuery data warehouse. For information about using data connectors in queries, see Running federated queries. You can now use the connector in your Athena queries.

Machine Learning 108

Machine Learning Machine Learning ML ML 108

Data Science Blog

NOVEMBER 15, 2023

Simple Data Model for a Process Mining Event Log As part of data engineering, the data traces that indicate process activities are brought into a log-like schema. A simple event log is therefore a simple table with the minimum requirement of a process number (case ID), a time stamp and an activity description.

Data Models 245

Data Models Data Modeling Business Intelligence Business Intelligence 245

Data Science Dojo

JULY 3, 2024

Big Data Technologies : Handling and processing large datasets using tools like Hadoop, Spark, and cloud platforms such as AWS and Google Cloud. Data Processing and Analysis : Techniques for data cleaning, manipulation, and analysis using libraries such as Pandas and Numpy in Python.

Data Science 150

Data Science Machine Learning Machine Learning Data Visualization 150

phData

MARCH 8, 2023

Fivetran is an automated data integration platform that offers a convenient solution for businesses to consolidate and sync data from disparate data sources. With over 160 data connectors available, Fivetran makes it easy to move data out of, into, and across any cloud data platform in the market.

Data Warehouse 96

Data Warehouse Data Engineering Data Engineer Data Engineering 96

Snorkel AI

MAY 26, 2023

[link] Ahmad Khan, head of artificial intelligence and machine learning strategy at Snowflake gave a presentation entitled “Scalable SQL + Python ML Pipelines in the Cloud” about his company’s Snowpark service at Snorkel AI’s Future of Data-Centric AI virtual conference in August 2022. Welcome everybody.

SQL 52

SQL ML ML Python 52

Snorkel AI

MAY 26, 2023

[link] Ahmad Khan, head of artificial intelligence and machine learning strategy at Snowflake gave a presentation entitled “Scalable SQL + Python ML Pipelines in the Cloud” about his company’s Snowpark service at Snorkel AI’s Future of Data-Centric AI virtual conference in August 2022. Welcome everybody.

SQL 52

SQL ML ML Python 52

Analytics Vidhya

MARCH 10, 2023

Introduction Snowflake is a cloud-based data warehousing platform that enables enterprises to manage vast and complicated information by providing scalable storage and processing capabilities. It is intended to be a fully managed, multi-cloud solution that does not need clients to handle hardware or software.

Analytics 306

Analytics Analytics Data Warehouse Data Engineer 306

phData

NOVEMBER 8, 2024

Data Versioning and Time Travel Open Table Formats empower users with time travel capabilities, allowing them to access previous dataset versions. Versioning also ensures a safer experimentation environment, where data scientists can test new models or hypotheses on historical data snapshots without impacting live data.

Data Lakes 52

Data Lakes Data Warehouse Database Azure 52

IBM Journey to AI blog

MAY 15, 2024

There are several styles of data integration. Data engineers build data pipelines, which are called data integration tasks or jobs, as incremental steps to perform data operations and orchestrate these data pipelines in an overall workflow.

Data Pipeline 85

Data Pipeline ETL SQL Database 85

phData

JULY 17, 2023

Over the past few decades, the corporate data landscape has changed significantly. The shift from on-premise databases and spreadsheets to the modern era of cloud data warehouses and AI/ LLMs has transformed what businesses can do with data. What is the Modern Data Stack? Data modeling, data cleanup, etc.

Data Warehouse 52

Data Warehouse Data Pipeline Cloud Data ETL 52

Dataversity

SEPTEMBER 28, 2022

Data engineering is a fascinating and fulfilling career – you are at the helm of every business operation that requires data, and as long as users generate data, businesses will always need data engineers. The journey to becoming a successful data engineer […].

Data Engineering 52

Data Engineering Data Engineer Data Engineering Data Engineering 52

phData

AUGUST 1, 2024

A prime example of this is automating repetitive code performed in many models or implementing a new feature introduced in your cloud data warehouse. Scenarios Now, we need to build the SQL statements. In this case, we have to create it before loading the data. In our case, we need to set up the temporary table SQL first.

SQL 52

SQL Database Data Warehouse Cloud Data 52

phData

JUNE 14, 2023

In recent years, data engineering teams working with the Snowflake Data Cloud platform have embraced the continuous integration/continuous delivery (CI/CD) software development process to develop data products and manage ETL/ELT workloads more efficiently. What Are the Benefits of CI/CD Pipeline For Snowflake?

Data Pipeline 52

Data Pipeline Database SQL Data Engineering 52

phData

MARCH 29, 2023

However, many analysts and other data professionals run into two common problems: They are not given direct access to their database They lack the skills in SQL to write the queries themselves The traditional solution to these problems is to rely on IT and data engineering teams. Only use the data you need.

Database 52

Database SQL Analytics Analytics 52

phData

MARCH 27, 2024

The SnowPro Advanced Administrator Certification targets Snowflake Administrators, Snowflake Data Cloud Administrators, Database Administrators, Cloud Infrastructure Administrators, and Cloud Data Administrators. I found the Data Engineering Simplified’s playlists particularly beneficial during my studies.

Data Analyst 52

Data Analyst Data Engineering Data Engineer Data Engineering 52

phData

SEPTEMBER 2, 2024

Best practices are a pivotal part of any software development, and data engineering is no exception. This ensures the data pipelines we create are robust, durable, and secure, providing the desired data to the organization effectively and consistently. Below are the best practices.

ETL 52

ETL Data Warehouse SQL Database 52

Alation

APRIL 4, 2023

Profiling delivers a birds-eye view of the statistics of the data, such as minimum, maximum, median, and null values. This empowers users to judge data’s quality and fitness for purpose quickly. This expanded connector to Databricks Unity Catalog does just that, delivering to joint customers a comprehensive view of all cloud data.

DataOps 52

DataOps Data Engineering Data Engineer Data Engineering 52

phData

FEBRUARY 7, 2024

The Snowflake Data Cloud is a leading cloud data platform that provides various features and services for data storage, processing, and analysis. A new feature that Snowflake offers is called Snowpark, which provides an intuitive library for querying and processing data at scale in Snowflake.

Python 52

Python ML ML SQL 52

phData

JULY 24, 2024

Utilizing AI and machine learning (ML) models can sound like a daunting task, but it is achievable, especially with the ML engineering experts at phData by your side to guide you in your data journey. Many data engineering consulting companies can answer these questions, and you may have the in-house talent to do it yourself.

Machine Learning 52

Machine Learning Machine Learning Data Engineering Data Engineer 52

phData

AUGUST 6, 2024

Organizations must ensure their data pipelines are well designed and implemented to achieve this, especially as their engagement with cloud data platforms such as the Snowflake Data Cloud grows. For customers in Snowflake, Snowpark is a powerful tool for building these effective and scalable data pipelines.

Data Pipeline 52

Data Pipeline Python Data Engineering Data Engineer 52

phData

JUNE 26, 2024

Cleaning and preparing the data Raw data typically shouldn’t be used in machine learning models as it’ll throw off the prediction. Data engineers can prepare the data by removing duplicates, dealing with outliers, standardizing data types and precision between data sets, and joining data sets together.

Machine Learning 52

Machine Learning Machine Learning Data Engineering Data Engineer 52

phData

OCTOBER 12, 2022

It comes with a rather lightweight intellisense, and highlights for both SQL and Jinja use. The real power is the ability to run your models and view the outputs, or even have your SQL compiled to verify that your Jinja or SQL compiles into the correct model. Our team of data experts are happy to assist. Reach out today!

SQL 52

SQL Data Warehouse Data Visualization Cloud Data 52

phData

DECEMBER 4, 2023

However, if there’s one thing we’ve learned from years of successful cloud data implementations here at phData, it’s the importance of: Defining and implementing processes Building automation, and Performing configuration …even before you create the first user account. And once again, for loading data, do not use SQL Inserts.

Clustering 98

Clustering Database SQL Data Pipeline 98

AWS Machine Learning Blog

DECEMBER 1, 2023

Proper data preparation leads to better model performance and more accurate predictions. SageMaker Canvas allows interactive data exploration, transformation, and preparation without writing any SQL or Python code. The following diagram shows the SageMaker Canvas data flow after adding visual transformations.

Machine Learning 118

Machine Learning Machine Learning Data Preparation ML 118

phData

JANUARY 4, 2023

Data Sharing Enterprises can easily create data sharing relationships with direct, governed, and secure sharing in near-real time. With Snowflake, organizations can be data consumers, data providers, or both. Ready to Get Started in the Migration to Snowflake?

Data Warehouse 52

Data Warehouse SQL Database ETL 52

Mlearning.ai

FEBRUARY 16, 2023

Data warehousing is a vital constituent of any business intelligence operation. Companies can build Snowflake databases expeditiously and use them for ad-hoc analysis by making SQL queries. Machine Learning Integration Opportunities Organizations harness machine learning (ML) algorithms to make forecasts on the data.

Data Warehouse 52

Data Warehouse Business Intelligence Business Intelligence Database 52

phData

MAY 25, 2023

One big issue that contributes to this resistance is that although Snowflake is a great cloud data warehousing platform, Microsoft has a data warehousing tool of its own called Synapse. The June 2021 release of Power BI Desktop introduced Custom SQL queries to Snowflake in DirectQuery mode.

Power BI 84

Power BI Analytics Analytics Azure 84

phData

JULY 18, 2024

Many data engineering consulting companies could also answer these questions for you, or maybe you think you have the talent on your team to do it in-house. Expertise Here at phData, we strive to be experts in data engineering, analytics, and machine learning. Why phData? Why should you choose phData to help?

Machine Learning 52

Machine Learning Machine Learning Natural Language Processing Data Engineering 52

Alation

SEPTEMBER 7, 2021

Organizations need to ensure that data use adheres to policies (both organizational and regulatory). In an ideal world, you’d get compliance guidance before and as you use the data. Imagine writing a SQL query or using a BI dashboard with flags & warnings on compliance best practice within your natural workflow.

Data Governance 59

Data Governance Data Scientist Data Quality Data Profiling 59

Pickl AI

APRIL 9, 2025

The Top List of ETL Tools In the world of data management, Extract, Transform, Load (ETL) tools play a crucial role in enabling organizations to efficiently process and analyse large volumes of data. Apache NiFi Apache NiFi is an open-source data integration tool that provides an intuitive user interface for designing data flows.

ETL 52

ETL Data Warehouse AWS Business Intelligence 52

phData

MARCH 29, 2024

Founded in 2014 by three leading cloud engineers, phData focuses on solving real-world data engineering, operations, and advanced analytics problems with the best cloud platforms and products. Over the years, one of our primary focuses became Snowflake and migrating customers to this leading cloud data platform.

Data Engineering 52

Data Engineering Data Engineer Data Engineering Data Engineering 52

Alation

AUGUST 30, 2022

These range from data sources , including SaaS applications like Salesforce; ELT like Fivetran; cloud data warehouses like Snowflake; and data science and BI tools like Tableau. This expansive map of tools constitutes today’s modern data stack. But different users have different needs.

Data Governance 52

Data Governance Tableau Data Quality Data Analyst 52

Alation

JANUARY 17, 2023

These tools are used to manage big data, which is defined as data that is too large or complex to be processed by traditional means. How Did the Modern Data Stack Get Started? The rise of cloud computing and cloud data warehousing has catalyzed the growth of the modern data stack.

Data Warehouse 52

Data Warehouse ETL Tableau Cloud Data 52

IBM Journey to AI blog

JULY 17, 2023

Within watsonx.ai, users can take advantage of open-source frameworks like PyTorch, TensorFlow and scikit-learn alongside IBM’s entire machine learning and data science toolkit and its ecosystem tools for code-based and visual data science capabilities. Savings may vary depending on configurations, workloads and vendor.

AI 74

AI AI Machine Learning Machine Learning 74

phData

OCTOBER 17, 2023

In our previous blog , we discussed how Fivetran and dbt scale for any data volume and workload, both small and large. Now, you might be wondering what these tools can do for your data team and the efficiency of your organization as a whole. Can these tools help reduce the time our data engineers spend fixing things?

ETL 52

ETL Data Pipeline Data Engineering Data Engineer 52

phData

SEPTEMBER 4, 2024

ThoughtSpot is a cloud-based AI-powered analytics platform that uses natural language processing (NLP) or natural language query (NLQ) to quickly query results and generate visualizations without the user needing to know any SQL or table relations. Why Use ThoughtSpot?

Analytics 52

Analytics Analytics SQL ETL 52

phData

FEBRUARY 14, 2023

Matillion Matillion is a complete ETL tool that integrates with an extensive list of pre-built data source connectors, loads data into cloud data environments such as Snowflake, and then performs transformations to make data consumable by analytics tools such as Tableau and PowerBI.

Data Warehouse 59

Data Warehouse Azure AWS Database 59