LLM Fine-Tuning from Scratch

Master the complete fine-tuning pipeline from basics to covering tokenization, dataset curation, LoRA & QLoRA techniques, training on Hugging Face, evaluating model performance, and deploying your custom LLM via API.

Topics

1.1

Introduction to Large Language Models (LLMs)

1.2

Understanding Tokenization

1.3

Hugging Face Tokenizers Library

Study Resources

→The Illustrated Transformer →Hugging Face Tokenizers Documentation →Tokenization for LLMs: A Comprehensive Guide

How It Works

Learn

Study curated videos and articles

Practice

Solve problems, build things

Verify

Submit your work as proof

Identity

Get a verified skill on your profile

References

1.The Illustrated Transformer↗

2.Hugging Face Tokenizers Documentation↗

3.Tokenization for LLMs: A Comprehensive Guide↗

4.Hugging Face Datasets Documentation↗

5.Deploying LLMs: A Practical Guide↗

6.The Alpaca Dataset: A Strong, Replicable Instruction-Following Model↗

7.PEFT: Parameter-Efficient Fine-Tuning of Foundation Models↗

8.LoRA: Low-Rank Adaptation of Large Language Models↗

9.QLoRA: Efficient Finetuning of Quantized LLMs↗

10.Hugging Face Trainer Documentation↗

11.Hugging Face Evaluate Library↗

12.A Survey of Metrics for Evaluating LLMs↗

13.Hugging Face Inference Endpoints↗

14.FastAPI Documentation↗

LLM Fine-Tuning from Scratch

Module 1: Foundations of LLMs & Tokenization

Topics

Study Resources

Module 2: Dataset Curation & Preparation

Module 3: Introduction to Fine-Tuning & PEFT

Module 4: Advanced PEFT: QLoRA & Training with Hugging Face

Module 5: Model Evaluation & Benchmarking

Module 6: Deployment & API Integration

How It Works

References