Models for LLM and RAG Engineers
LLM models and prompting covers how to call hosted large language model APIs and shape their output. It includes messages, system prompts, temperature, token limits and context windows, structured outputs, and prompting techniques such as few-shot examples, which together decide how reliably a model follows instructions.
Stage: LLM Foundations · Part of the LLM and RAG Engineer roadmap · Last reviewed 19 November 2026
What to learn in Models
- Call a chat completion API and handle streaming, errors and rate limits
- Write system prompts and few-shot examples that steer output format
- Request structured JSON output and validate it against a schema
- Estimate token usage and keep prompts within the context window
- Tune temperature and top-p for deterministic or creative tasks
Models interview questions
What does the temperature setting control?
Temperature scales how random the next-token choice is. Low values make output more deterministic and focused, which suits extraction and code. Higher values sample more varied tokens, which suits brainstorming. It does not make the model more accurate or add knowledge.
Why does the context window matter in application design?
The context window limits how many tokens of prompt and output fit in one call. Long documents, history and retrieved passages compete for that space, and cost and latency grow with it, so applications must choose, summarise or truncate what they send.
Where Models fits in the LLM and RAG Engineer roadmap
Models belongs to the LLM Foundations stage (intermediate level), which takes about 33 hours. How models behave.
Other units in this stage
Back to the full LLM and RAG Engineer roadmap