AWS Machine Learning Blog

APRIL 11, 2023

Amazon SageMaker Studio can help you build, train, debug, deploy, and monitor your models and manage your machine learning (ML) workflows. Amazon SageMaker Pipelines enables you to build a secure, scalable, and flexible MLOps platform within Studio. In our example, we create a SageMaker pipeline running a single processing step.

AWS 72

AWS ML ML Data Scientist 72

AWS Machine Learning Blog

NOVEMBER 30, 2023

Amazon SageMaker is a fully managed service that enables developers and data scientists to quickly and easily build, train, and deploy machine learning (ML) models at scale. SageMaker makes it easy to deploy models into production directly through API calls to the service. SageMaker provides a variety of options to deploy models.

ML 98

ML ML AWS Python 98

AWS Machine Learning Blog

NOVEMBER 16, 2023

We propose using this capability with the Amazon SageMaker platform of services to improve regression model accuracy in an ML use case, and independently, for the automated tagging of visual images. SageMaker Data Wrangler is purpose-built to simplify the process of data preparation and feature engineering.

AWS 84

AWS ML ML Machine Learning 84

AWS Machine Learning Blog

NOVEMBER 30, 2023

We are excited to share that Amazon SageMaker HyperPod is now generally available to enable training foundation models with thousands of accelerators up to 40% faster by providing a highly resilient training environment while eliminating the undifferentiated heavy lifting involved in operating large-scale training clusters.

Clustering 107

Clustering AWS Machine Learning Machine Learning 107

AWS Machine Learning Blog

FEBRUARY 29, 2024

In this post, we present a framework for automating the creation of a directed acyclic graph (DAG) for Amazon SageMaker Pipelines based on simple configuration files. It then dynamically creates a SageMaker Pipelines DAG based on the steps declared in the configuration files and the interactions and dependencies among steps.

AWS 87

AWS ML ML Machine Learning 87

AWS Machine Learning Blog

DECEMBER 13, 2023

Amazon SageMaker MLOps is a suite of features that includes Amazon SageMaker Projects (CI/CD), Amazon SageMaker Pipelines and Amazon SageMaker Model Registry. SageMaker Pipelines allows for straightforward creation and management of ML workflows, while also offering storage and reuse capabilities for workflow steps.

AWS 96

AWS ML ML Data Preparation 96

AWS Machine Learning Blog

OCTOBER 2, 2023

Solution overview The sample use case used for this series is a visual quality inspection solution that can detect defects on metal tags, which could be deployed as part of a manufacturing process. SageMaker Ground Truth provides out-of-the-box templates for many different labeling task types, including drawing bounding boxes.

AWS 88

AWS Internet of Things ML ML 88

AWS Machine Learning Blog

NOVEMBER 29, 2023

Amazon SageMaker Studio provides a fully managed solution for data scientists to interactively build, train, and deploy machine learning (ML) models. Amazon SageMaker notebook jobs allow data scientists to run their notebooks on demand or on a schedule with a few clicks in SageMaker Studio.

ML 79

ML ML Data Scientist Python 79

AWS Machine Learning Blog

DECEMBER 12, 2023

This post shows you how you can create a web UI, which we call Chat Studio, to start a conversation and interact with foundation models available in Amazon SageMaker JumpStart such as Llama 2, Stable Diffusion, and other models available on Amazon SageMaker. We cover the following steps: Deploy SageMaker foundation models.

AWS 94

AWS ML ML AI 94

AWS Machine Learning Blog

MAY 30, 2023

In this series of posts, we share lessons learned about optimizing costs in Amazon SageMaker. In Part 1 , we showed how to get started using AWS Cost Explorer to identify cost optimization opportunities in SageMaker. SageMaker manages creating the instance and related resources.

AWS 78

AWS ML ML EDA 78

AWS Machine Learning Blog

OCTOBER 2, 2023

However, using purpose-built services like Amazon SageMaker and AWS IoT Greengrass allows you to significantly reduce this effort. If you’re just getting started with MLOps at the edge on AWS, refer to MLOps at the edge with Amazon SageMaker Edge Manager and AWS IoT Greengrass for an overview and reference architecture.

AWS 86

AWS ML ML Internet of Things 86

AWS Machine Learning Blog

MAY 16, 2024

In this post, we share how LotteON improved their recommendation service using Amazon SageMaker and machine learning operations (MLOps). The main AWS services used are SageMaker, Amazon EMR , AWS CodeBuild , Amazon Simple Storage Service (Amazon S3), Amazon EventBridge , AWS Lambda , and Amazon API Gateway.

AWS 89

AWS ML ML Deep Learning 89

AWS Machine Learning Blog

SEPTEMBER 26, 2023

This post shows you how to build and host an ML application with custom containers on Amazon SageMaker. Amazon SageMaker offers built-in algorithms and pre-built SageMaker docker images for model deployment. But, if these don’t fit your needs, you can bring your own containers (BYOC) for hosting on Amazon SageMaker.

ML 98

ML ML Python AWS 98

AWS Machine Learning Blog

DECEMBER 14, 2023

Amazon SageMaker Studio offers a broad set of fully managed integrated development environments (IDEs) for machine learning (ML) development, including JupyterLab, Code Editor based on Code-OSS (Visual Studio Code Open Source), and RStudio. Both JupyterLab and Code Editor can be launched using a flexible workspace called Spaces.

ML 94

ML ML AI AI 94

AWS Machine Learning Blog

MAY 30, 2023

In this post, we share lessons learned and walk you through the various ways to analyze your SageMaker usage and identify opportunities for cost optimization. It allows you to filter and group by values such as AWS service, usage type, cost allocation tags, EC2 instance type, and more.

AWS 64

AWS ML ML Machine Learning 64

AWS Machine Learning Blog

NOVEMBER 14, 2023

In this post, we walk through best practices for managing LoRA fine-tuned models on Amazon SageMaker to address this emerging question. Working with FMs on SageMaker Model Registry In this post, we walk through an end-to-end example of fine-tuning the Llama2 large language model (LLM) using the QLoRA method.

ML 85

ML ML AWS SQL 85

AWS Machine Learning Blog

AUGUST 21, 2023

Amazon SageMaker Data Wrangler reduces the time it takes to collect and prepare data for machine learning (ML) from weeks to minutes. SageMaker Data Wrangler supports fine-grained data access control with Lake Formation and Amazon Athena connections. The EMR cluster should be created with encryption in transit.

AWS 81

AWS Data Lakes Clustering Data Preparation 81

AWS Machine Learning Blog

MARCH 15, 2023

The term legacy code refers to code that was developed to be manually run on a local desktop, and is not built with cloud-ready SDKs such as the AWS SDK for Python (Boto3) or Amazon SageMaker Python SDK. The best practice for migration is to refactor these legacy codes using the Amazon SageMaker API or the SageMaker Python SDK.

AWS 118

AWS Machine Learning Machine Learning Data Scientist 118

AWS Machine Learning Blog

JULY 25, 2023

These FMs can perform a wide range of tasks that span multiple domains, like writing blog posts, generating images, solving math problems, engaging in dialog, and answering questions based on a document. Amazon SageMaker JumpStart is an ML hub offering algorithms, models, and ML solutions.

AWS 78

AWS AI AI ML 78

AWS Machine Learning Blog

MARCH 10, 2023

Amazon SageMaker Data Wrangler reduces the time it takes to aggregate and prepare data for machine learning (ML) from weeks to minutes in Amazon SageMaker Studio. Solution overview With SageMaker Studio setups, data professionals can quickly identify and connect to existing EMR clusters. LDAP No Auth Choose Next.

Clustering 75

Clustering AWS ML ML 75

AWS Machine Learning Blog

APRIL 4, 2024

In this post, we describe how Nielsen Sports modernized a system running thousands of different machine learning (ML) models in production by using Amazon SageMaker multi-model endpoints (MMEs) and reduced operational and financial cost by 75%. The following figure shows an example of our tagging system.

AWS 72

AWS Machine Learning Machine Learning ML 72

AWS Machine Learning Blog

APRIL 24, 2024

The solution uses Amazon SageMaker JumpStart as the core service for model deployment, fine-tuning, and reinforcement learning. In the pipeline, we used Amazon OpenSearch Service for the vector database, and deployed a fine-tuned Mistral-7B-Instruct model on Amazon SageMaker. Answer the score inside of tags.

AI 104

AI AI Data Science AWS 104

AWS Machine Learning Blog

APRIL 25, 2023

Amazon SageMaker Model Training helps data scientists run fully managed large-scale training jobs on AWS’s compute infrastructure. SageMaker Training also helps data scientists with advanced tools such as Amazon SageMaker Debugger and Profiler to debug and analyze their large-scale training jobs.

Machine Learning 77

Machine Learning Machine Learning ML ML 77

AWS Machine Learning Blog

AUGUST 4, 2023

This is the second post in a series discussing the integration of Salesforce Data Cloud and Amazon SageMaker. Use the newly launched capability of the Amazon SageMaker Data Wrangler connector for Salesforce Data Cloud to prepare the data in SageMaker without copying the data from Salesforce Data Cloud.

ML 77

ML ML AWS AI 77

AWS Machine Learning Blog

MAY 25, 2023

To understand the overall structure of a RAG-based approach, refer to Question answering using Retrieval Augmented Generation with foundation models in Amazon SageMaker JumpStart. We then demonstrate how to use LangChain for tying everything together: Interfacing with LLMs hosted on Amazon SageMaker.

AWS 75

AWS Clustering Python ML 75

AWS Machine Learning Blog

JUNE 27, 2023

Amazon SageMaker JumpStart is a machine learning (ML) hub that can help you accelerate your ML journey. With SageMaker JumpStart, you can discover and deploy publicly available and proprietary foundation models to dedicated Amazon SageMaker instances for your generative AI applications.

AWS 65

AWS ML ML Machine Learning 65

AWS Machine Learning Blog

AUGUST 29, 2023

In this post, we describe how to create an MLOps workflow for batch inference that automates job scheduling, model monitoring, retraining, and registration, as well as error handling and notification by using Amazon SageMaker , Amazon EventBridge , AWS Lambda , Amazon Simple Notification Service (Amazon SNS), HashiCorp Terraform, and GitLab CI/CD.

AWS 71

AWS Data Quality Data Scientist Python 71

AWS Machine Learning Blog

APRIL 7, 2023

To reduce the barrier to entry of ML at the edge, we wanted to demonstrate an example of deploying a pre-trained model from Amazon SageMaker to AWS Wavelength , all in less than 100 lines of code. In this post, we demonstrate how to deploy a SageMaker model to AWS Wavelength to reduce model inference latency for 5G network-based applications.

AWS 78

AWS Clustering ML ML 78

AWS Machine Learning Blog

MAY 30, 2023

In this series of posts, we share lessons learned about optimizing costs in Amazon SageMaker. In Part 1 , we showed how to get started using AWS Cost Explorer to identify cost optimization opportunities in SageMaker. You can also apply additional filters such as account number, EC2 instance type, cost allocation tag, Region, and more.

AWS 67

AWS ML ML Machine Learning 67

Mlearning.ai

FEBRUARY 6, 2023

Use SageMaker Processing Jobs to easily run inference on your large Dataset with Hugging Face’s Transformer Models Photo by Alex Kulikov on Unsplash This blog will give you a complete walk through of running a distributed batch inference on large data in production. Note: /opt/ml and all its subdirectories are reserved by SageMaker.

AWS 52

AWS ML ML Python 52

DagsHub

SEPTEMBER 7, 2023

By using AWS SageMaker for customizable pipeline solutions triggered by code base modifications and DagsHub for data and code management and experiment tracking - building and managing those pipelines becomes a manageable task. How to setup SageMaker for Continuous Training?

Machine Learning 52

Machine Learning Machine Learning ML ML 52

AWS Machine Learning Blog

APRIL 11, 2023

With the integrated development environment (IDE) of Amazon SageMaker Studio with Jupyter Lab 3 , we can build, run, and serve Streamlit web apps from within that same environment for development purposes. The code for this blog can be found in this GitHub repository. The code for this blog can be found in this GitHub repository.

ML 71

ML ML AWS Data Scientist 71

AWS Machine Learning Blog

JULY 3, 2023

In computer vision (CV), adding tags to identify objects of interest or bounding boxes to locate the objects is called labeling. Amazon SageMaker Ground Truth – Alternatively, you can manage your own data labeling workflows and workforce to label data. For a full list of features, refer to Amazon SageMaker Data Labeling Features.

Deep Learning 79

Deep Learning Deep Learning AWS Machine Learning 79

DagsHub

AUGUST 22, 2023

Amazon SageMaker is a fully managed machine learning service. Combining SageMaker with DagsHub provides a single source of truth to the project, managed in one place, including code, data, models, experiments, annotations, and now - computing resources and automation. What is Amazon SageMaker?

Machine Learning 52

Machine Learning Machine Learning AWS Algorithm 52

AWS Machine Learning Blog

APRIL 22, 2024

Amazon SageMaker HyperPod is purpose-built to accelerate foundation model (FM) training, removing the undifferentiated heavy lifting involved in managing and optimizing a large training compute cluster. With SageMaker HyperPod, you can train FMs for weeks and months without disruption. Add any optional tags, then choose Next.

Clustering 82

Clustering AWS Machine Learning Machine Learning 82

AWS Machine Learning Blog

MAY 13, 2024

In part 1 of this blog series, we discussed how a large language model (LLM) available on Amazon SageMaker JumpStart can be fine-tuned for the task of radiology report impression generation. For instance, “Generate radiology report impressions based on the following findings and output it within tags.”

AI 100

AI AI AWS ML 100

AWS Machine Learning Blog

MARCH 6, 2024

In this post, we demonstrate how to efficiently fine-tune a state-of-the-art protein language model (pLM) to predict protein subcellular localization using Amazon SageMaker. We start by downloading a public dataset using Amazon SageMaker Studio. The following diagram illustrates this workflow. apply(lambda x: len(x)).between(100,

AWS 88

AWS Machine Learning Machine Learning ML 88