Sitemap
Heartbeat

Comet is a machine learning platform helping data scientists, ML engineers, and deep learning engineers build better models faster

Follow publication

Large Language Models: A Complete Guide

30 min readMay 29, 2023

--

Photo by Drew Beamer on Unsplash

Introduction

A small portion of the LLM ecosystem; image from scalevp.com

Part 1: Training LLMs

LLMs use a combination of machine learning and human input; image from OpenAI

Data preparation and preprocessing

Practitioners will rarely encounter data in the wild that doesn’t need substantial “cleaning”; image from Counting Stuff
Three examples of tokenization methods; image from FreeCodeCamp

The ML process is cyclical — find a workflow that matches. Collaborate across teams, reproduce experiments, and more with a strong MLOps strategy. Check out our expert solutions for overcoming common ML team problems.

Model selection and architecture

BERT model architecture; image from TDS

Hyperparameter tuning

Hyperparameters are like a series of dials that can be adjusted to alter model performance; Photo by Jonathan Farber on Unsplash
Grid search vs random search grid layouts; image from Bengio and Bergstra
Bayesian optimization; image from CERN

Fine-tuning

Fine-tuning an NLP model involves retraining an entire model on a new dataset at a very low learning rate; image from TDS

Data augmentation

Thesaurus-based substitution; image from Amit Chaudhary
Mask prediction substitutions; image from Amit Chaudhary
Examples of random insertion, random swapping, and random deletion; image from Amit Chaudhary

Transfer learning

Transfer learning uses knowledge acquired from previous training and applies it to a new task; image from data-science-blog.com
Transfer learning improves model performance in at least three different ways; image from MachineLearningMastery.com

Ensembling

Stacking is one method of ensemble learning; image from TDS
Bagging consists in fitting several base models on different bootstrap samples and build an ensemble model that “average” the results of these weak learners; image from TDS
Boosting consists in, iteratively, fitting a weak learner, aggregate it to the ensemble model and “update” the training dataset to better take into account the strengths and weakness of the current ensemble model when fitting the next base model; from TDS

Evaluation and testing

Common evaluation metrics for classifiers; image from Kaggle

Mitigating bias

Bias-variance tradeoff; image from cs.cornell.edu

Ethical considerations

Elements of data governance; image from imperva.com

Security and privacy

Part 2: Deploying LLMs

Infrastructure

Source: AWS re:Invent

Data management

Security

Monitoring and maintenance

User interface

Workflow

Part 3: Improving Large Language Models

Some tools and libraries available for improving LLMs

Conclusion

--

--

Heartbeat
Heartbeat

Published in Heartbeat

Comet is a machine learning platform helping data scientists, ML engineers, and deep learning engineers build better models faster

Eberechi Dan
Eberechi Dan

Written by Eberechi Dan

Data Engineer | Back-End Dev | Technical Writer @cometml, @inPlainEngHQ | Open Source Contributor @chaossproj | ML/NLP Research 🇳🇬

Responses (1)

Write a response