Applied AI Academy

A model reads fragments, remembers nothing, and runs nothing — the rest is plumbing you own

Tokens, context and memory, tool calling, connectors and the five orchestration patterns

The question: What actually happens between your sentence and a system that books the meeting — and which part of it is the model?

Every conversation about AI architecture uses four words — tokens, context, tools, orchestration — and most people using them could not draw what happens. That matters commercially, because almost everything you can control, price or secure lives in the plumbing rather than in the model. This lesson draws the pipe.

What the lesson covers

A model does not read letters. A **tokenizer** turns your text into numbered fragments first — roughly four characters each in English, often a whole common word, sometimes a single character — and the model only ever sees the numbers. Three consequences follow immediately. Price is per token, so wording has a cost. The context limit is counted in tokens, so verbosity is a capacity decision. And the model genuinely cannot see spelling, which is why an otherwise brilliant system miscounts the letters in a word: it was never shown them. This is also where multilingual unfairness begins, before the model is even invoked.

The second thing to unlearn is memory. A model has none. Every call is answered from scratch, using only what is in the **context window** at that moment, and the window is rebuilt on every turn. What feels like memory is four separate mechanisms doing the work: the conversation being replayed back in full each time, a summary substituted once it no longer fits (**compaction**, which is lossy and silent), files or retrieved documents pasted in, and an explicit memory store your application writes to and reads from. Knowing which one is operating tells you why a system forgot something — and more context is not more knowledge: as the window fills, irrelevant material competes for attention and quality falls. What you leave out is a design decision.

Now the correction that changes how you think about the whole category. **A model cannot run anything.** You give it a list of tools, each with a name, a description and an argument schema. When it decides one is needed it does not execute it — it returns a structured request: this tool, these arguments, and then it stops. Your code receives that request, decides whether to honour it, runs it, and appends the result to the context so the model can continue. Every permission question, every audit trail, every kill switch lives in that gap. The model proposes; your code disposes. It also explains a class of failure: a badly worded tool description is a badly chosen tool, because the description is the documentation the model reads at decision time.

**Plugins**, **connectors** and **MCP servers** are three names for packaging that same thing. A plugin is a vendor-specific bundle inside one product. A connector is usually a hosted integration that carries your credentials to a service. An MCP server is the open, standard version — one socket, so any host can use any tool. All three are the same shape underneath: a described capability plus an identity that decides what it may reach. When somebody says "we connected it to Salesforce", the question is never whether the connection works. It is which tools that connection exposes, and whose permissions they run under.

Finally, **orchestration**: the code that decides which model call happens next. Five patterns cover nearly everything in production. **Chaining** — each call takes the last one's output, with programmatic checks between the links. **Routing** — classify the input, then send it to a specialist path. **Parallelisation** — split into independent parts, or run the same task several times and aggregate. **Orchestrator-workers** — one call decomposes the task, delegates, and synthesises. **Evaluator-optimiser** — one call generates, another critiques, and it loops. The first four are workflows: you drew the path. Only the fifth begins to blur into an agent, which is a system that chooses its own path. Prefer the workflow every time you can draw the path, because it is cheaper, testable, and fails in ways you anticipated.

Key points

Framework — Chain · Route · Parallel · Orchestrate · Evaluate

Five orchestration patterns, in rising order of how much the model decides. Chaining: fixed steps with checks between. Routing: classify, then specialise. Parallelisation: split or vote. Orchestrator-workers: the model decomposes and delegates. Evaluator-optimiser: generate, critique, repeat. Draw the path first — if you can draw it, build a workflow. Reach for an agent only when the path genuinely cannot be known in advance, and then pay for it in evaluation.

The lab

See each layer once with your own eyes: count real tokens in your own languages, read a real tool call, and pick the right orchestration pattern for a real task.

Deliverable: One workflow drawn as a named orchestration pattern, with its checks marked, one properly written tool schema, and your token ratio for the languages you actually operate in.

Open this lesson, its lab and its quiz

Sources and further reading