Module 1: Generative AI Landscape
How Large Language Models Work (Intuitively)
You do not need to be an engineer to use generative AI well, but you do need a working mental model of what is happening when you type a request and text comes back. People who understand the basics write better prompts, trust the output appropriately and spot problems faster. This lesson gives you that model in plain language.
Prediction, one piece at a time
A large language model (LLM) is a computer program trained to predict what text should come next. Given the words "The capital city of Kenya is", a well-trained model predicts that "Nairobi" is a very likely continuation. When you ask a question, the model generates an answer by repeatedly predicting the next piece of text, adding it, and predicting again, until the response is complete.
Those pieces are called tokens. A token is often a whole short word, part of a longer word, or a punctuation mark. Models read and write in tokens, which is why limits and prices for these tools are usually expressed in tokens rather than words.
Because the model chooses each next token from a range of likely options, the same prompt can produce different answers on different attempts. Tools often allow some randomness so that responses feel natural rather than repetitive.
How a model learns
Building a modern assistant happens in broad stages:
- Pre-training. The model is trained on a very large collection of text, such as books, websites and code. By practising next-token prediction across this text, it learns grammar, facts, writing styles and patterns of reasoning. It does not store the text like a library; it adjusts billions of internal numbers, called parameters, that capture patterns.
- Fine-tuning. The model is then trained further on examples of helpful conversations, so it learns to follow instructions and answer questions rather than just continue text.
- Feedback training. Developers use human and automated feedback to steer the model towards responses that are helpful, honest and safe.
A model's knowledge comes from its training data, which has a cutoff date. It does not know about events after that date unless the tool gives it access to search or to documents you provide.
The context window: the model's working memory
Everything the model can "see" in a conversation (your instructions, any documents you paste, and its own previous replies) must fit within its context window, measured in tokens. Information inside the window can be used directly. Information outside it, such as a conversation from last week in a different chat, is not available unless the tool has a memory feature or you provide it again.
This explains several everyday experiences. A model can summarise a long report you paste in, because the report is in its context. It may lose track of details in an extremely long conversation. And it cannot know your company's pricing unless you tell it.
Worked example: why a model gets something wrong
Achieng, who runs a small tour company in Naivasha, asks an assistant: "What are the current park entry fees for Hell's Gate National Park?" The model replies confidently with specific figures. When she checks the Kenya Wildlife Service website, the figures are different.
What happened? The model predicted text that looked like a plausible answer, drawing on patterns from its training data. That data may have been outdated, or the model may have combined patterns incorrectly. It did not look anything up. If her tool had web search turned on, it might have found the current page, but she would still need to check the source.
A better approach would be to ask: "Draft a short paragraph for my website explaining that park fees are set by the Kenya Wildlife Service and change from time to time, and that visitors should check the official KWS website before travelling." Here the model does what it is good at (clear writing), and the facts come from the official source.
Mental model summary
| Think of it as | Not as |
|---|---|
| A very well-read writing partner that predicts plausible text | A database that looks up verified facts |
| Working only with what is in its training and its current context | Knowing everything about you and your business |
| Able to produce different answers each time | A calculator that always returns the same result |
Common mistakes to avoid
- Treating fluent, confident writing as proof of accuracy.
- Assuming the model knows recent events or your private information.
- Expecting identical answers every time you ask the same question.
- Pasting a long document and assuming every detail was used; ask the model to quote the parts it relied on.
Exercise
Open any generative AI assistant you have access to and ask it the same question three times in new chats, for example 'Suggest a name for a juice bar in Westlands'. Note how the answers differ. Then ask it a factual question about something recent in your industry, and check the answer against an official or reliable source. Write three sentences on what this tells you about how the model works.
Key points
- An LLM generates text by predicting the next token, repeatedly.
- Its knowledge comes from training data with a cutoff date, unless the tool adds search or your documents.
- The context window is the model's working memory for a conversation.
- Fluent output is not the same as verified fact; check important claims.