3,555 prompts from top AI collections
← Back to Market
[LIVE] [yashgarg4/nanogpt-tinystories] 1 endpoint(s)
Gemini API Proxy/Gemini
2,210 characters
Status: [LIVE] Endpoints found: https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yashgarg4/nanogpt Source: https://github.com/yashgarg4/nanogpt-tinystories # nanogpt-tinystories **A ~30M-parameter GPT-2-style transformer, built from scratch in pure PyTorch and trained on TinyStories until it writes coherent little children's stories.** ![Python](https://img.shields.io/badge/Python-3.13-3776AB?logo=python&logoColor=white) ![PyTorch](https://img.shields.io/badge/PyTorch-2.x-EE4C2C?logo=pytorch&logoColor=white) ![Model](https://img.shields.io/badge/params-30M-blue) ![Val loss](https://img.shields.io/badge/val%20loss-1.88-brightgreen) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/yashgarg4/nanogpt-tinystories/blob/main/notebooks/train_colab.ipynb) No `transformers` library, no `AutoModel`, no pretrained weights. Every core component — tokenization, embeddings, causal self-attention, the transformer block, the training loop, and sampling — is implemented **and explained** by hand. The goal is **understanding transformers from the inside**. The code is heavily commented to match. ## What it does Trained from **random initialization** on TinyStories, it continues a prompt like `"Once upon a time"` into a short, mostly-coherent story — a real result from this repo's model (val loss 1.88, temperature 0.8, top-k 200): > Once upon a time, there was a little girl named Lily. She loved to play with her > jewelry and pretend she was a princess. One day, she went to the park to play > with her jewelry. But when she saw her toy box, a big dog tried to take it from > her. Lily was sad and started to cry. Her mommy told her not to worry and that > the dog was not a real friend... Later that day, Lily's friend came over to play. > Lily was so happy to have her shiny jewelry box back. She promised to always be > careful and respectful with it. From that day on, Lily and her friends played > with her jewelry box, and the dog never grabbed it again. More in [`samples/`](samples/): the same model at different temperatures ([`gallery.md`](samples/gallery.md))
Download .txt