AI Stuff Part 1

Strip away the jargon and a large language model does one boringly simple thing: it predicts the next word.

Everything else — RAG, agents, MCP, tool use — is machinery bolted around that one guess to feed it the right information at the right moment.

I keep getting asked variations of “so how does this actually work?” This is the answer I wish I could hand over every time — the whole modern AI stack in plain English, no maths required.

It builds up in layers: the brain at the center, then the system wrapped around it, then the specific acronyms people trip over. There are diagrams throughout, plus three demos you can actually poke at.

This is part one, covering the foundations — how models work and how the pieces fit together. Later parts will go deeper on the individual mechanisms.

1. The core: what an AI model actually is

At the heart of every modern AI is a large language model — an LLM. You give it some text, it guesses what comes next, one small chunk at a time. Then it feeds its own output back in and guesses again, over and over, until it has written a full answer.

That’s genuinely it. Writing code, explaining physics, holding a conversation — all of it comes out of that one repeated guess.

A few words that go with that picture.

Tokens

The model doesn’t read whole words. It chops text into pieces called tokens — roughly ¾ of a word each. “Cats” might be one token; “unbelievable” might be three. Every input and output is really a stream of tokens.

Type something and watch it get chopped up:

Notice what happens: common words survive intact, while unusual words, long words, and punctuation break apart. This is also why models are sometimes bad at counting letters in a word — they never saw the letters, only the chunks.

Parameters (or weights)

These are the billions of internal numbers the model learned during training. Think of them as billions of tiny knobs, tuned until the guessing got good. When someone says a model is “70B”, they mean 70 billion knobs.

Transformer

The specific neural-network design almost every modern LLM uses. Its superpower is attention: when processing a word, it looks back at all the other words and decides which ones matter most.

In “the cat sat on the ___”, attention lets it notice cat and sat to land on mat. That single idea is why AI got dramatically better after 2017.

Training vs. using it

Two totally separate phases, and conflating them causes most of the confusion:

Training builds the brain; inference uses it. This is also exactly why a model has a knowledge cutoff — it only knows what existed when it was trained. Nothing you say in a chat teaches it anything permanent.

2. Zooming out: the whole AI system

Here’s the part most people miss. The raw LLM is just the engine. A product like ChatGPT or Claude is a whole system built around that engine.

The model on its own cannot browse the web, remember past chats, or open your files. Those abilities all come from components bolted on around it.

The prompt and the context window

Everything the model “sees” at once is the context window — its short-term working memory. Into it goes:

• your question
• hidden instructions (the system prompt, which sets the AI’s rules and personality)
• any relevant files or past messages the application chose to include

The model has no memory beyond this window. Every technique below is really about getting the right stuff into that window at the right moment. If you remember one thing from this post, make it that.

Embeddings and vector databases

An embedding turns text into a list of numbers that captures its meaning. Texts about similar things end up with similar numbers, even when they share no words at all.

A vector database stores millions of these and instantly finds the ones closest in meaning to your question. This is how AI does “search by meaning” instead of “search by exact keyword” — and it’s the engine behind the next section.

3. RAG — giving the AI knowledge it wasn’t trained on

RAG stands for Retrieval-Augmented Generation.

The problem it solves: the model’s knowledge is frozen at training time, and it knows nothing about your private documents. RAG fixes this by fetching relevant information first, then handing it to the model alongside your question.

The trick is those embeddings. Your documents get chopped into chunks and stored in a vector database. When you ask something, the system finds the chunks closest in meaning and stuffs them into the context window — so the model answers from real retrieved facts instead of guessing from memory.

Try the retrieval step. Pick a question and watch which chunks get pulled — note that matches happen on meaning, not shared words:

Why RAG matters: it keeps answers current and grounded, cuts down on made-up hallucinations, and lets an AI work with private data it never saw in training — all without the huge cost of retraining the model.

4. Tools, agents, and MCP

The model can also do things, not just talk.

Tool use

Also called function calling. The application tells the model: “here are some tools you can call — a calculator, a web search, a way to send email.” When the model decides it needs one, it outputs a request like “search the web for X”. The application runs that action and feeds the result back into the context.

To you it looks seamless. Under the hood, it’s the model asking and the app doing. The model never touches the outside world itself.

Agents

An agent is just an AI doing that in a loop to accomplish a bigger goal: think → use a tool → look at the result → decide the next step → repeat, until the task is done.

A basic chatbot answers in one shot. An agent might search, read three pages, run some code, and check its own work before replying.

MCP

Now the problem MCP solves. Every tool and app — Google Drive, Slack, your database — has its own way of connecting. Without a standard, developers hand-build a custom connector for every single combination. Three apps and four tools means twelve integrations.

MCP — the Model Context Protocol — is a shared “plug shape” that fixes this. It’s USB-C for AI: build your app to speak MCP once, and it connects to any tool that also speaks MCP.

5. A few more terms you’ll bump into

Fine-tuning — taking an already-trained model and training it a bit more on specialized examples (legal contracts, your company’s writing style) so it gets better at that niche.

The distinction that matters: RAG hands the model facts at question time; fine-tuning bakes a skill or style into the model itself. RAG is cheaper and easier to update; fine-tuning goes deeper but costs more.

Hallucination — when a model states something false with total confidence. It happens because the model is fundamentally guessing plausible next words, and a plausible-sounding wrong answer can score just as well as a right one. RAG and tool use are the main ways to reduce it.

Multimodal — a model that handles more than text: images, audio, sometimes video. Same core idea, just trained to turn pictures and sound into tokens too.

Temperature — a dial for randomness. Low temperature makes the model pick the most likely next word every time (focused, repetitive); high temperature lets it take chances (creative, less predictable).

Drag the dial and watch the same probabilities get sharpened or flattened:

Prompt engineering — the craft of wording your request and instructions well. Since everything rides on what’s in the context window, how you phrase things genuinely changes output quality.

Putting it all together

Zoom all the way out and the whole picture is one simple loop wrapped in helpful machinery:

• A transformer-based LLM sits at the center doing next-token prediction.
• Around it, an application layer decides what goes into the context window.
• It pulls in facts through RAG, connects to the outside world through tools and MCP, and runs in a loop when acting as an agent.

Training builds the brain once. Everything else is about feeding that brain the right context at the right moment.

That framing is genuinely useful when debugging AI systems too. When an answer comes back wrong, the question is almost never “is the model broken?” — it’s “what was actually in the context window when it answered?” Nine times out of ten, that’s where the bug lives.

That’s the foundation. In a follow-up I want to go a level deeper on the pieces that deserve their own post — how attention actually works inside a transformer, how RAG chunking and re-ranking make or break retrieval quality, and what it takes to build an agent that doesn’t get stuck in its own loop.