AWS Machine Learning Blog

NOVEMBER 30, 2023

Amazon SageMaker is a fully managed service that enables developers and data scientists to quickly and easily build, train, and deploy machine learning (ML) models at scale. SageMaker makes it easy to deploy models into production directly through API calls to the service. SageMaker provides a variety of options to deploy models.

ML 97

ML ML AWS Python 97

AWS Machine Learning Blog

NOVEMBER 16, 2023

We propose using this capability with the Amazon SageMaker platform of services to improve regression model accuracy in an ML use case, and independently, for the automated tagging of visual images. SageMaker Data Wrangler is purpose-built to simplify the process of data preparation and feature engineering.

AWS 82

AWS ML ML Machine Learning 82

AWS Machine Learning Blog

NOVEMBER 30, 2023

We are excited to share that Amazon SageMaker HyperPod is now generally available to enable training foundation models with thousands of accelerators up to 40% faster by providing a highly resilient training environment while eliminating the undifferentiated heavy lifting involved in operating large-scale training clusters.

Clustering 106

Clustering AWS Machine Learning Machine Learning 106

AWS Machine Learning Blog

FEBRUARY 29, 2024

In this post, we present a framework for automating the creation of a directed acyclic graph (DAG) for Amazon SageMaker Pipelines based on simple configuration files. It then dynamically creates a SageMaker Pipelines DAG based on the steps declared in the configuration files and the interactions and dependencies among steps.

AWS 86

AWS ML ML Machine Learning 86

AWS Machine Learning Blog

DECEMBER 13, 2023

Amazon SageMaker MLOps is a suite of features that includes Amazon SageMaker Projects (CI/CD), Amazon SageMaker Pipelines and Amazon SageMaker Model Registry. SageMaker Pipelines allows for straightforward creation and management of ML workflows, while also offering storage and reuse capabilities for workflow steps.

AWS 94

AWS ML ML Data Preparation 94

AWS Machine Learning Blog

APRIL 11, 2023

Amazon SageMaker Studio can help you build, train, debug, deploy, and monitor your models and manage your machine learning (ML) workflows. Amazon SageMaker Pipelines enables you to build a secure, scalable, and flexible MLOps platform within Studio. In our example, we create a SageMaker pipeline running a single processing step.

AWS 71

AWS ML ML Data Scientist 71

AWS Machine Learning Blog

OCTOBER 2, 2023

Solution overview The sample use case used for this series is a visual quality inspection solution that can detect defects on metal tags, which could be deployed as part of a manufacturing process. SageMaker Ground Truth provides out-of-the-box templates for many different labeling task types, including drawing bounding boxes.

AWS 87

AWS Internet of Things ML ML 87

AWS Machine Learning Blog

NOVEMBER 29, 2023

Amazon SageMaker Studio provides a fully managed solution for data scientists to interactively build, train, and deploy machine learning (ML) models. Amazon SageMaker notebook jobs allow data scientists to run their notebooks on demand or on a schedule with a few clicks in SageMaker Studio.

ML 78

ML ML Data Scientist Python 78

AWS Machine Learning Blog

MAY 30, 2023

In this series of posts, we share lessons learned about optimizing costs in Amazon SageMaker. In Part 1 , we showed how to get started using AWS Cost Explorer to identify cost optimization opportunities in SageMaker. SageMaker manages creating the instance and related resources.

AWS 77

AWS ML ML EDA 77

AWS Machine Learning Blog

DECEMBER 12, 2023

This post shows you how you can create a web UI, which we call Chat Studio, to start a conversation and interact with foundation models available in Amazon SageMaker JumpStart such as Llama 2, Stable Diffusion, and other models available on Amazon SageMaker. We cover the following steps: Deploy SageMaker foundation models.

AWS 93

AWS ML ML AI 93

AWS Machine Learning Blog

OCTOBER 2, 2023

However, using purpose-built services like Amazon SageMaker and AWS IoT Greengrass allows you to significantly reduce this effort. If you’re just getting started with MLOps at the edge on AWS, refer to MLOps at the edge with Amazon SageMaker Edge Manager and AWS IoT Greengrass for an overview and reference architecture.

AWS 85

AWS ML ML Internet of Things 85

AWS Machine Learning Blog

MAY 16, 2024

In this post, we share how LotteON improved their recommendation service using Amazon SageMaker and machine learning operations (MLOps). The main AWS services used are SageMaker, Amazon EMR , AWS CodeBuild , Amazon Simple Storage Service (Amazon S3), Amazon EventBridge , AWS Lambda , and Amazon API Gateway.

AWS 88

AWS ML ML Deep Learning 88

AWS Machine Learning Blog

SEPTEMBER 26, 2023

This post shows you how to build and host an ML application with custom containers on Amazon SageMaker. Amazon SageMaker offers built-in algorithms and pre-built SageMaker docker images for model deployment. But, if these don’t fit your needs, you can bring your own containers (BYOC) for hosting on Amazon SageMaker.

ML 96

ML ML Python AWS 96

AWS Machine Learning Blog

MAY 30, 2023

In this post, we share lessons learned and walk you through the various ways to analyze your SageMaker usage and identify opportunities for cost optimization. It allows you to filter and group by values such as AWS service, usage type, cost allocation tags, EC2 instance type, and more.

AWS 63

AWS ML ML Machine Learning 63

AWS Machine Learning Blog

DECEMBER 14, 2023

Amazon SageMaker Studio offers a broad set of fully managed integrated development environments (IDEs) for machine learning (ML) development, including JupyterLab, Code Editor based on Code-OSS (Visual Studio Code Open Source), and RStudio. Both JupyterLab and Code Editor can be launched using a flexible workspace called Spaces.

ML 92

ML ML AI AI 92

AWS Machine Learning Blog

NOVEMBER 14, 2023

In this post, we walk through best practices for managing LoRA fine-tuned models on Amazon SageMaker to address this emerging question. Working with FMs on SageMaker Model Registry In this post, we walk through an end-to-end example of fine-tuning the Llama2 large language model (LLM) using the QLoRA method.

ML 84

ML ML AWS SQL 84

AWS Machine Learning Blog

AUGUST 21, 2023

Amazon SageMaker Data Wrangler reduces the time it takes to collect and prepare data for machine learning (ML) from weeks to minutes. SageMaker Data Wrangler supports fine-grained data access control with Lake Formation and Amazon Athena connections. The EMR cluster should be created with encryption in transit.

AWS 80

AWS Data Lakes Clustering Data Preparation 80

AWS Machine Learning Blog

MARCH 15, 2023

The term legacy code refers to code that was developed to be manually run on a local desktop, and is not built with cloud-ready SDKs such as the AWS SDK for Python (Boto3) or Amazon SageMaker Python SDK. The best practice for migration is to refactor these legacy codes using the Amazon SageMaker API or the SageMaker Python SDK.

AWS 117

AWS Machine Learning Machine Learning Data Scientist 117

AWS Machine Learning Blog

FEBRUARY 13, 2024

In this post, we discuss how BigBasket used Amazon SageMaker to train their computer vision model for Fast-Moving Consumer Goods (FMCG) product identification, which helped them reduce training time by approximately 50% and save costs by 20%. Use SageMaker and Amazon FSx for Lustre for efficient data augmentation.

AWS 94

AWS AI AI ML 94

AWS Machine Learning Blog

APRIL 4, 2024

In this post, we describe how Nielsen Sports modernized a system running thousands of different machine learning (ML) models in production by using Amazon SageMaker multi-model endpoints (MMEs) and reduced operational and financial cost by 75%. The following figure shows an example of our tagging system.

AWS 71

AWS Machine Learning Machine Learning ML 71

AWS Machine Learning Blog

APRIL 25, 2023

Amazon SageMaker Model Training helps data scientists run fully managed large-scale training jobs on AWS’s compute infrastructure. SageMaker Training also helps data scientists with advanced tools such as Amazon SageMaker Debugger and Profiler to debug and analyze their large-scale training jobs.

Machine Learning 76

Machine Learning Machine Learning ML ML 76

AWS Machine Learning Blog

APRIL 24, 2024

The solution uses Amazon SageMaker JumpStart as the core service for model deployment, fine-tuning, and reinforcement learning. In the pipeline, we used Amazon OpenSearch Service for the vector database, and deployed a fine-tuned Mistral-7B-Instruct model on Amazon SageMaker. Answer the score inside of tags.

AI 103

AI AI Data Science AWS 103

AWS Machine Learning Blog

AUGUST 4, 2023

This is the second post in a series discussing the integration of Salesforce Data Cloud and Amazon SageMaker. Use the newly launched capability of the Amazon SageMaker Data Wrangler connector for Salesforce Data Cloud to prepare the data in SageMaker without copying the data from Salesforce Data Cloud.

ML 75

ML ML AWS AI 75

AWS Machine Learning Blog

JULY 25, 2023

Amazon SageMaker JumpStart is an ML hub offering algorithms, models, and ML solutions. With SageMaker JumpStart, ML practitioners can choose from a growing list of best performing open source FMs. For the complete list, check out Getting started with Amazon SageMaker JumpStart.

AWS 76

AWS AI AI ML 76

AWS Machine Learning Blog

AUGUST 29, 2023

In this post, we describe how to create an MLOps workflow for batch inference that automates job scheduling, model monitoring, retraining, and registration, as well as error handling and notification by using Amazon SageMaker , Amazon EventBridge , AWS Lambda , Amazon Simple Notification Service (Amazon SNS), HashiCorp Terraform, and GitLab CI/CD.

AWS 70

AWS Data Quality Data Scientist Python 70

AWS Machine Learning Blog

MAY 30, 2023

In this series of posts, we share lessons learned about optimizing costs in Amazon SageMaker. In Part 1 , we showed how to get started using AWS Cost Explorer to identify cost optimization opportunities in SageMaker. You can also apply additional filters such as account number, EC2 instance type, cost allocation tag, Region, and more.

AWS 66

AWS ML ML Machine Learning 66

Flipboard

MARCH 22, 2023

In this post, we show how to configure a new OAuth-based authentication feature for using Snowflake in Amazon SageMaker Data Wrangler. Finally, we show how to export the data flow and train a model using SageMaker Autopilot. Access to Amazon SageMaker , an instance of Amazon SageMaker Studio , and a user for Studio.

AWS 100

AWS Data Preparation Azure ML 100

AWS Machine Learning Blog

JUNE 27, 2023

Amazon SageMaker JumpStart is a machine learning (ML) hub that can help you accelerate your ML journey. With SageMaker JumpStart, you can discover and deploy publicly available and proprietary foundation models to dedicated Amazon SageMaker instances for your generative AI applications.

AWS 64

AWS ML ML Machine Learning 64

AWS Machine Learning Blog

APRIL 7, 2023

To reduce the barrier to entry of ML at the edge, we wanted to demonstrate an example of deploying a pre-trained model from Amazon SageMaker to AWS Wavelength , all in less than 100 lines of code. In this post, we demonstrate how to deploy a SageMaker model to AWS Wavelength to reduce model inference latency for 5G network-based applications.

AWS 77

AWS Clustering ML ML 77

AWS Machine Learning Blog

JULY 3, 2023

In computer vision (CV), adding tags to identify objects of interest or bounding boxes to locate the objects is called labeling. Amazon SageMaker Ground Truth – Alternatively, you can manage your own data labeling workflows and workforce to label data. For a full list of features, refer to Amazon SageMaker Data Labeling Features.

Deep Learning 78

Deep Learning Deep Learning AWS Machine Learning 78

AWS Machine Learning Blog

MAY 25, 2023

To understand the overall structure of a RAG-based approach, refer to Question answering using Retrieval Augmented Generation with foundation models in Amazon SageMaker JumpStart. We then demonstrate how to use LangChain for tying everything together: Interfacing with LLMs hosted on Amazon SageMaker.

AWS 74

AWS Clustering Python ML 74

DagsHub

AUGUST 22, 2023

Amazon SageMaker is a fully managed machine learning service. Combining SageMaker with DagsHub provides a single source of truth to the project, managed in one place, including code, data, models, experiments, annotations, and now - computing resources and automation. What is Amazon SageMaker?

Machine Learning 52

Machine Learning Machine Learning AWS Algorithm 52

AWS Machine Learning Blog

MARCH 10, 2023

Amazon SageMaker Data Wrangler reduces the time it takes to aggregate and prepare data for machine learning (ML) from weeks to minutes in Amazon SageMaker Studio. Solution overview With SageMaker Studio setups, data professionals can quickly identify and connect to existing EMR clusters. Choose Open studio.

Clustering 73

Clustering AWS ML ML 73

AWS Machine Learning Blog

APRIL 22, 2024

Amazon SageMaker HyperPod is purpose-built to accelerate foundation model (FM) training, removing the undifferentiated heavy lifting involved in managing and optimizing a large training compute cluster. With SageMaker HyperPod, you can train FMs for weeks and months without disruption. Add any optional tags, then choose Next.

Clustering 81

Clustering AWS Machine Learning Machine Learning 81

Mlearning.ai

FEBRUARY 6, 2023

Use SageMaker Processing Jobs to easily run inference on your large Dataset with Hugging Face’s Transformer Models Photo by Alex Kulikov on Unsplash This blog will give you a complete walk through of running a distributed batch inference on large data in production. We’ll be using Amazon Sagemaker, a fully managed machine learning service.

AWS 52

AWS ML ML Python 52

Flipboard

MARCH 13, 2023

Therefore, VMware Carbon Black and AWS chose to build a custom MLOps pipeline using Amazon SageMaker for its ease of use, versatility, and fully managed infrastructure. In this post, VMware Carbon Black and AWS architects discuss how we built and managed custom ML workflows using Gitlab , Amazon MWAA, and SageMaker.

ML 102

ML ML AWS Data Scientist 102

AWS Machine Learning Blog

JANUARY 17, 2023

Amazon SageMaker JumpStart is the Machine Learning (ML) hub of SageMaker providing pre-trained, publicly available models for a wide range of problem types to help you get started with machine learning. Amazon SageMaker JumpStart provides one-click, end-to-end solutions for many common ML use cases. JumpStart solution templates.

AWS 68

AWS Machine Learning Machine Learning Natural Language Processing 68