How LLMs work for AI Engineers
Large language models predict the next token using transformer networks trained on large text corpora.
Stage: Foundations · Part of the AI Engineer roadmap · Last reviewed 19 November 2026
What to learn in How LLMs work
- Tokens and context windows
- Transformers at a high level
- Temperature and sampling
How LLMs work interview questions
What is a context window?
The maximum number of tokens a model can consider at once, including both the prompt and the response.
Where How LLMs work fits in the AI Engineer roadmap
How LLMs work belongs to the Foundations stage. Code and core concepts.
Other units in this stage
Back to the full AI Engineer roadmap