Build an LLM from Scratch 2: Working with text data

Описание:

This is a supplementary video going over text data preparations steps (tokenization, byte pair encoding, data loaders, etc.) for LLM training. 00:00 2.2 Tokenizing text 14:02 2.3 Converting tokens into token IDs 23:56 2.4 Adding special context tokens 30:26 2.5 Byte pair encoding 44:00 2.6 Data sampling with a sliding window 1:07:10 2.7 Creating token embeddings 1:15:45 2.8 Encoding word positions