Toy LLM Demo
Return to klaas.org homeSee a tiny language model learn in your browser.
I took an evening to learn more about how an LLM works and, had an LLM produce a 'toy' LLM including three distinct components: 1) ingest training data and tokenize. 2) Change the weights of the tokens via training. 3) Use the trained model.
About
See a toy LLM in your browser. The stages are: 1) ingest data. 2) train weights on this data. 3) use the model via prompts.
1) Ingest Local Text
The first step is to take training data (only text file(s)) to read, tokenize (break into discrete pieces such as, each word), and assign weights for what token follows another token. If you use a single text file with 6 words, you can then see the model can only predict words that are present.
2) Train Weights
Epochs = how many full passes over your ingested tokens. Learning rate = how quickly weights adjust each pass.
3) Prompt the Engine
Output will appear here.
Model Snapshot
Top tokens by current weight.