Video: Lecture 15: Coding the self attention mechanism with key, query and value matrices

Curso: Building LLMs from scratch
Idioma:   Course LanguageDificuldade:  
Básico

Categorias: Design,
Descrição:
In this playlist, we will learn about the entire process of building a Large Language Model (LLM) from scratch. Nothing will be assumed. Everything will be spelled out.

Progresso:

Lecture 1: Building LLMs from scratch: Series introduction
Lecture 2: Large Language Models (LLM) Basics
Lecture 3: Pretraining LLMs vs Finetuning LLMs
Lecture 4: What are transformers?
Lecture 5: How does GPT-3 really work?
Lecture 6: Stages of building an LLM from Scratch
Lecture 7: Code an LLM Tokenizer from Scratch in Python
Lecture 8: The GPT Tokenizer: Byte Pair Encoding
Lecture 9: Creating Input-Target data pairs using Python DataLoader
Lecture 10: What are token embeddings?
Lecture 11: The importance of Positional Embeddings
Lecture 12: The entire Data Preprocessing Pipeline of Large Language Models (LLMs)
Lecture 13: Introduction to the Attention Mechanism in Large Language Models (LLMs)
Lecture 14: Simplified Attention Mechanism - Coded from scratch in Python | No trainable weights
Lecture 15: Coding the self attention mechanism with key, query and value matrices
Lecture 16: Causal Self Attention Mechanism | Coded from scratch in Python
Lecture 17: Multi Head Attention Part 1 - Basics and Python code
Lecture 18: Multi Head Attention Part 2 - Entire mathematics explained
Lecture 19: Birds Eye View of the LLM Architecture
Lecture 20: Layer Normalization in the LLM Architecture
GELU Activation Function in the LLM Architecture
Shortcut connections in the LLM Architecture
Coding the entire LLM Transformer Block
Coding the 124 million parameter GPT-2 model
Coding GPT-2 to predict the next token
Measuring the LLM loss function
Evaluating LLM performance on real dataset | Hands on project | Book data
Coding the entire LLM Pre-training Loop
Temperature Scaling in Large Language Models (LLMs)
Top-k sampling in Large Language Models
Saving and loading LLM model weights using PyTorch
Loading pre-trained weights from OpenAI GPT-2
Introduction to LLM Finetuning | Python Coding with hands-on-example
Dataloaders in LLM Classification Finetuning | Python Coding | Hands on LLM project
Coding the model architecture for LLM classification fine-tuning
Coding a fine-tuned LLM spam classification model | From Scratch
Introduction to LLM Instruction Fine-tuning | Loading Dataset | Alpaca Prompt format
Data Batching in LLM instruction fine-tuning | Hands on project | Live Python coding
Dataloaders in Instruction Fine-tuning
Instruction fine-tuning: Loading pre-trained LLM weights
LLM fine-tuning training loop | Coded from scratch
Evaluating fine-tuned LLM using Ollama
Build LLMs from scratch 20 minutes summary

Reportar Building LLMs from scratch

logo Courses Tube
O CoursesTube foi criado para guiar estudantes em sua jornada de aprendizado e aperfeiçoamento profissional em sua carreiras, organizando, categorizando, classificando e ordenando materiais gratuitos encontrados na internet. Não nos responsabilizamos pelos conteúdos dos cursos, falas, materiais ou produtos dos idealizadores. Todos os videos são embedados legalmente seguindo a política do Youtube.
Links
SuporteIndique um Curso
Parceiros
TopOne Automobilismo VirtualEnter The Shadows Radio
Redes sociais
facebook pageinstagram page
Me pague um café?
@2022 - CoursesTube | Políticas de privacidade | Termos de uso | pt-BR