What AI, language models, and the application actually do — and where human judgment still has to sit
Nov 5, 2026 - Samuel R. Lammie, with Claude (Anthropic)
Disclosure: one co-author of this post is a large language model built by a company that profits from AI adoption. Read "The black box" with that in mind.
Two words doing too much work
When I made a small investment in Teknowledge in the mid-1980s, at the height of the expert-systems boom, "artificial intelligence" meant hand-written if-then rules captured from human specialists. Today the same two words cover a satellite-imagery classifier, a chatbot, and an agent that can search, calculate, and act across several programs. The vocabulary has flattened distinctions that matter.
Three layers are worth keeping apart. The first (1) is the field: artificial intelligence, the whole discipline. The second (2) is the model: a large language model, one kind of AI among many. The third (3) is the application: the model wrapped with an interface and tools so people can actually use it. In the plate at the end, the field is the outer frame, the application sits inside it, and the model is the core. Most confusion about these systems, and most misplaced trust in them, comes from treating the three as one thing.
Layer one: the field
Artificial intelligence is the broad discipline of building computer systems that do things we associate with human intelligence: recognizing patterns, interpreting language, solving problems, planning, deciding. Some systems follow explicit rules. Others learn statistical patterns from examples.
The family is older and wider than the current conversation suggests. Rule-based and expert systems apply rules people wrote, the way a screening tool flags a permit application when specified conditions are met. Traditional machine learning learns from data to classify or predict, the way a model estimates wildfire susceptibility from terrain, vegetation, and weather. Computer vision reads images, picking roads, forest cover, or damaged buildings out of satellite scenes. Speech systems transcribe a meeting or read text aloud. Reinforcement learning develops strategy through trial, action, and reward. Robotics joins perception, planning, and physical motion. Generative AI, the newest branch in public view, produces text, images, audio, video, and code.
These categories overlap. Some describe how a system learns; others describe what it does. Neural networks sit underneath many of them, but not all: decision trees, rules, and plain statistics still carry a great deal of working AI.
Layer two: the model
A large language model is one kind of AI, trained on vast amounts of text to learn patterns in language and the relationships among the concepts language carries. "Large" refers mainly to the scale of its neural network: billions of learned numerical values, called parameters. Training adjusts those values, over and over, to improve the model's predictions of text. Out of that process come capabilities nobody wrote down as rules: answering questions, summarizing documents, drafting letters, writing code, working through an argument.
When you type a prompt, the model converts your words into numbers and passes them through many layers of computation. It then writes its answer one token at a time, a token being a word, part of a word, or a punctuation mark. Each new token is shaped by your prompt, by whatever else is in the conversation, and by everything the model has already written.
That process can produce real analysis. It can also produce a convincing mistake, delivered in the same even voice.
The black box
Researchers understand how these networks are built and how they are trained. What remains hard is explaining why a particular network produced a particular answer. The learned patterns are spread across enormous numbers of interacting parameters. No single rule can be pointed to, and no single source can be traced. Interpretability research is making real but partial progress at locating some of the machinery; it does not yet let anyone audit an ordinary answer line by line.
A model's account of its own reasoning is the least reliable class of evidence it offers. When you ask a model to explain how it reached an answer, the explanation is itself generated text, produced the same way as the answer. It may be plausible, but it is not a faithful record of the computation. This consequence deserves its own sentence. Fluency is not accuracy, and a confident explanation is not a transparent one.
Layer three: the application
A chat application such as ChatGPT or Claude is the interface, not the model itself. Depending on how it is configured, the application can connect the model to other machinery: web search to retrieve outside sources, document retrieval to find material in files you supply, a calculator or code environment to run the numbers, image and speech capabilities, and connections that let it take actions in other software. The model interprets and writes; the tools fetch, compute, and act.
Each connection extends reach. Each also adds a place to fail. The system can pick a weak source, misread a document, or run the right calculation on the wrong figure, and the finished answer will read just as smoothly either way.
Two more terms round out the vocabulary. A multimodal system handles more than one form of information, discussing a photograph, reading a chart, answering a spoken question. An agent pairs a model with tools and a process for pursuing a goal across several steps: search, read the files, run the analysis, assemble the report. How independently it works depends on its design, its permissions, and who is watching. That last variable is the one worth asking about.
Oversight language tends to blur a distinction here. A governor binds before the action: the agent cannot spend, send, or delete until a person releases it. A gate works after the fact: a log, a review, a correction once the result is already in the world. Much of what is offered as oversight of AI agents is a gate dressed as a governor. The policy says a human is "in the loop," but the agent runs at machine rate and the review runs at document rate, so the human arrives to inspect what has already happened. The test is Stanislav Petrov's. In September 1983 Petrov, a Soviet duty officer, watched an early-warning system report incoming American missiles, judged it a false alarm, and declined to pass it up the chain as an attack. He was right, and the system had left him room to refuse. The question is not whether a document names someone who may refuse, but whether that refusal holds load at the moment the agent acts.
Where the value is
What language models changed is the interface. Ordinary language now works as a way into information and software. People can explain, summarize, draft, compare, organize,and explore at a speed and scale that was not available five years ago.
So the reliability of the whole depends on the task, the evidence, the tools, and the checks someone actually runs. The model can draft the script that delineates a watershed in minutes. It cannot walk the creek, and it does not answer for what gets decided downstream. Use it to extend your reach. Check the claims that carry weight against evidence, verify the arithmetic, and keep the consequential decisions in human hands.
“The three layers, and where verification and validation have to sit.”
Samuel R. Lammie, with Claude (Anthropic). Developed from an explainer first drafted with ChatGPT (OpenAI), then restructured and rewritten. Header illustration generated with ChatGPT (OpenAI); closing plate by Claude. Disclosure: Anthropic builds large language models and benefits from their adoption; the caution about a model's account of itself applies to the co-author of this post.
Tapadh leat.
