AWS Machine Learning Blog

FEBRUARY 1, 2023

Although larger models tend to be more powerful, training such models requires significant computational resources. Creation and attachment of the FSx for Lustre file system to the EKS cluster is mediated by the Amazon FSx for Lustre CSI driver.

AWS 90

AWS Clustering Deep Learning Deep Learning 90

AWS Machine Learning Blog

APRIL 19, 2024

Karpenter monitors for any pending pods that can’t run due to lack of sufficient resources in the cluster. If such pods are detected, Karpenter adds more nodes to the cluster to provide the necessary resources. discovery: do-eks-yaml-karpenter iam: withOIDC: true addons: - name: aws-ebs-csi-driver version: v1.26.0-eksbuild.1

Clustering 94

Clustering AI AI AWS 94

AWS Machine Learning Blog

MARCH 2, 2023

script will create the VPC, subnets, auto scaling groups, the EKS cluster, its nodes, and any other necessary resources. The scripts in aws-do-eks/Container-Root/eks/deployment/csi/ provide instructions to mount Amazon EFS on an EKS cluster. You can specify custom AMIs or specific zones for different instance types. eks-create.sh

Clustering 74

Clustering AWS Deep Learning Deep Learning 74