5 stages of Generative AI project lifecycle
In Generative AI projects, there are five distinct stages in the lifecycle, centred around a Large Language Model
1️⃣ Pre-training : This involves building an LLM from scratch. The likes of BERT, GPT4, Llama 2, have undergone pre-training on a large corpus of data. Billions of parameters are trained. Pre-training is an Unsupervised Learning task and the objective is text generation or next token prediction. Pre-training is a compute intensive phase and the training phase lasts for days and even months. The task is complex and everything from the training corpus to the transformer architecture is decided in the pre-training phase. The result of pre-training are Foundation Models
2️⃣ Prompt Engineering : Once the foundation model is ready, text can be generated by providing the model with a prompt. The model generates a completion on the prompt. This process is called inference. No training happens during prompt engineering. None of the model weights are touched. The only examples given are in-context. Prompt engineering is the simplest of phases in the LLM lifecycle. The objective of prompt engineering is to improve performance on the generated text.
3️⃣ Fine-tuning : Probably, the most important phase of an llm lifecycle is when it is trained to perform well on certain desired tasks. This is done by providing examples of prompts and completions to the foundation model. Fine-tuning is a Supervised Learning task. A complete fine-tuning requires as much memory as pre-training a foundation model. The weights of the foundation model are updated in fine-tuning. PEFT or Parameter Efficient Fine Tuning, reduces the memory requirement of fine-tuning while maintaining performance levels.
4️⃣ Reinforcement Learning from Human/AI feedback : RLHF or RLAIF proved to be the turning point in acceptance of LLMs. The primary objective of RLH/AIF is to align the llm to the human values of Helpfulness, Harmlessness and Honesty . This is done using rewards. The rewards are initially given by a human in RLHF and then a rewards model is generated. Applying the principles of constitutional AI, RLAIF is used to scale human feedback. The result is a model that is aligned to human values.
5️⃣ Compression, Optimisation and Deployment : The final stage is where the LLM is ready to be used in an application. In this stage, the model is optimised for faster inference and lesser memory. Sometimes, a smaller llm derived from the original llm is used in production.
What stages of the lifecycle does your application use? Do let me know in the comments below.
