Status: [LIVE]
Endpoints found:
https://abubakarsiddik31.github.io/llm-from-scratch/](https://abubakarsiddik31.github.io/llm
Source: https://github.com/abubakarsiddik31/llm-from-scratch
# Hands-On LLM Implementation
### Build everything from scratch. Ship to production.
**A practical, implementation-first guide to training and deploying Large Language Models**
[](https://www.python.org/)
[](https://pytorch.org/)
[](LICENSE)
π **Read it as a book:** [https://abubakarsiddik31.github.io/llm-from-scratch/](https://abubakarsiddik31.github.io/llm-from-scratch/) *(built with mdBook from [`book/`](book/))*
[Projects](#-roadmap) β’ [Getting Started](#-getting-started) β’ [Directory Structure](#-directory-structure)
---
## Overview
This roadmap is for people who **understand the theory** and want to **build**.
You'll implement every component of an LLM pipeline from scratch:
> **Tokenization β Pre-training β Fine-tuning β Optimization β Production Deployment**
**Philosophy:** learn by doing. Each project builds on the previous one, and the last one is a fully deployed LLM application.
---
## Prerequisites
| Requirement | Details |
|-------------|---------|
| **GPU** | CUDA-capable (8GB+ VRAM recommended) |
| **Python** | 3.10+ |
| **Env** | `uv`, `conda` or `mamba` |
| **Knowledge** | Neural networks & attention mechanism (theoryβwe'll implement the rest) |
---
## Roadmap
### 10 Phases β’ 32 Projects β’ From Zero to Production
### Phase 1 β Foundation: Build Your First LLM
| Project | Topic | Status |
|---------|-------|--------|
| **1** | Character-Level GPT | β
Complete |
| **2** | BPE Tokenizer | β
Complete |
| **3** | Contextual Embeddings (BERT-style) | β
Complete |
| **3B** | SimCSE (Sentence Embeddings) | β
Complete |
| **4** | Pre-train 125M Model | β
Complete |
**Focus:** Multi-head attention, Transformer blocks, Training loop, Text generation, Masked Language M