AWS Machine Learning Blog

SEPTEMBER 18, 2023

With Ray and AIR, the same Python code can scale seamlessly from a laptop to a large cluster. Amazon SageMaker Pipelines allows orchestrating the end-to-end ML lifecycle from data preparation and training to model deployment as automated workflows. You can specify resource requirements in actors too.

Machine Learning 103

Machine Learning Machine Learning ML ML 103

phData

DECEMBER 4, 2023

Thirty seconds is a good default for human users; if you find that queries are regularly queueing, consider making your warehouse a multi-cluster that scales on-demand. Cluster Count If your warehouse has to serve many concurrent requests, you may need to increase the cluster count to meet demand.

Clustering 98

Clustering Database SQL Data Pipeline 98

Flipboard

AUGUST 17, 2023

AWS Glue is a serverless data integration service that makes it easy to discover, prepare, and combine data for analytics, ML, and application development. Here we use RedshiftDatasetDefinition to retrieve the dataset from the Redshift cluster.

ML 93

ML ML AWS Data Warehouse 93

DagsHub

APRIL 21, 2024

A traditional machine learning (ML) pipeline is a collection of various stages that include data collection, data preparation, model training and evaluation, hyperparameter tuning (if needed), model deployment and scaling, monitoring, security and compliance, and CI/CD.

Machine Learning 59

Machine Learning Machine Learning ML ML 59

phData

FEBRUARY 7, 2024

Server Side Execution Plan When you trigger a Snowpark operation, the optimized SQL code and instructions are sent to the Snowflake servers where your data resides. This eliminates unnecessary data movement, ensuring optimal performance. Snowflake spins up a virtual warehouse, which is a cluster of compute nodes, to execute the code.

Python 52

Python ML ML SQL 52

The MLOps Blog

JUNE 27, 2023

See also Thoughtworks’s guide to Evaluating MLOps Platforms End-to-end MLOps platforms End-to-end MLOps platforms provide a unified ecosystem that streamlines the entire ML workflow, from data preparation and model development to deployment and monitoring. Flyte Flyte is a platform for orchestrating ML pipelines at scale.

Machine Learning 134

Machine Learning Machine Learning ML ML 134

Snorkel AI

MAY 26, 2023

What’s really important in the before part is having production-grade machine learning data pipelines that can feed your model training and inference processes. And that’s really key for taking data science experiments into production. So it’s not as simple as taking everything and converting it to SQL.

SQL 52

SQL ML ML Python 52