Remove 2012 Remove Big Data Analytics Remove Python
article thumbnail

Four approaches to manage Python packages in Amazon SageMaker Studio notebooks

Flipboard

This post presents and compares options and recommended practices on how to manage Python packages and virtual environments in Amazon SageMaker Studio notebooks. You can manage app images via the SageMaker console, the AWS SDK for Python (Boto3), and the AWS Command Line Interface (AWS CLI). Define a Dockerfile.

Python 123
article thumbnail

Use LangChain with PySpark to process documents at massive scale with Amazon SageMaker Studio and Amazon EMR Serverless

AWS Machine Learning Blog

Apache Spark and its Python API, PySpark , empower users to process massive datasets effortlessly by using distributed computing across multiple nodes. In this post, we build a Docker image that includes the Python 3.11 You can modify the role to include any additional services that EMR Serverless needs to access at runtime.

AWS 120
article thumbnail

Machine learning with decentralized training data using federated learning on Amazon SageMaker

AWS Machine Learning Blog

It serializes these configuration dictionaries (or config dict for short) to their ProtoBuf representation, transports them to the client using gRPC, and then deserializes them back to Python dictionaries. Flower FL strategies Flower allows customization of the learning process through the strategy abstraction.