Oversight That Holds Load

A design test for AI governance, written for the committees that will hold the oversight pen

Nov 11, 2026 · Samuel R. Lammie, with Claude (Anthropic) and ChatGPT (OpenAI)

Disclosure: the co-authors of this piece are large language models built by companies that profit from AI adoption and whose systems would fall under the oversight it describes. Weigh its contributions accordingly.

The 120th Congress convenes in January with new committee chairs and new oversight priorities. Whoever holds those gavels will hear a great deal about artificial intelligence: frameworks, pilots, task forces, guardrails. This piece offers them one test to apply to all of it. For any AI provision, ask where it binds. A governor slows the machine itself. A stamp is a named person's approval before the irreversible step. An after-action review learns from what already happened. Most of what is sold as AI oversight is an after-action review dressed as a stamp. The difference decides whether anyone can actually stop a system once it is running.

I write as a retired federal land manager, not as a partisan. My federal career ran from Bureau of Indian Affairs forestry on Apache reservations in Arizona, through the National Park Service, to more than two decades in the Forest Service, ending as the Northern Region's geospatial program manager. In 2025 I completed Carnegie Mellon's certificate in Managing Artificial Intelligence Systems, so I come to this from both sides: as someone who ran the systems agencies depend on, and as someone trained in how AI systems should be governed. The test below is the one I would want any committee to apply, whichever party chairs it.

Where things stand

Federal AI policy now runs on three instruments, and none of them is a statute.

The first governs how agencies use AI. OMB Memorandum M-25-21, issued April 3, 2025, requires each agency to name a Chief AI Officer and identify "high-impact" AI, meaning systems whose output is a principal basis for decisions that significantly affect rights or safety. For those systems it sets minimum risk practices, including "human oversight, intervention, and accountability" and a "timely human review and a chance to appeal." It also contains the strongest sentence in current federal AI policy: if a high-impact use case does not comply, "the agency must safely discontinue use of the AI functionality."

The second governs who may regulate AI at all. Executive Order 14365, signed December 11, 2025, created a Justice Department AI Litigation Task Force to challenge state AI laws. It directed Commerce to list "onerous" state laws within 90 days, tied some federal funding to state alignment, and asked for a legislative proposal to preempt conflicting state rules. It carves out child safety, data center infrastructure and state procurement. An executive order cannot preempt state law on its own; that takes Congress.

The third is still a draft. In June 2026, Representatives Lori Trahan (D) and Jay Obernolte (R) released a discussion draft of the Great American AI Act that would preempt state laws on AI model development for three years while leaving states free to regulate deployment and use. It drew fire from both directions: from Democrats, civil rights and labor groups for preempting too much, and from many Republicans for preempting too little. In July the two sponsors introduced one piece of it as a stand-alone bill requiring independent audits of large frontier developers.

That is the ground the new chairs inherit: a use-side memo with a real brake in its text, a preemption push with no statute behind it, and a bipartisan draft that splits development from use. All three can be read with the same test.

The test: governor, stamp, or after-action review

A governor on an old steam engine did not report that the engine had run too fast. It closed the throttle before the engine could. That is the first stage: friction built into the system itself, through permissions, rate limits, spending caps and required confirmations. A governor slows the machine. It does not decide.

The second stage is the stamp: a named, accountable person's final approval before the irreversible step. Engineers stamp drawings; surveyors stamp plats. A stamp says a qualified person looked, judged, and will answer for it. It is the only stage where refusal lives.

The third stage is the after-action review, the wildland fire community's term for learning from what happened: an audit, a report, an inspector general finding, a hearing. By then the agent is already out the door. In earlier pieces I called this a gate. After-action review is the more honest term, because a gate implies it stops something.

All three have their place. Governors buy time, and after-action reviews are how institutions learn. But only the stamp puts a person between the system and the harm, and AI systems increasingly act faster than any review can follow.

Applied to the current instruments, the test sorts quickly. M-25-21's discontinue clause is a governor at the program level: noncompliant systems must stop. Its compliance reporting is an after-action review. Its human-oversight requirement is a stamp only if the human approves before an AI-driven decision takes effect; afterward, it is a review. Executive Order 14365 is neither, because it governs who may regulate rather than how systems behave. The Trahan–Obernolte draft would remove state-level governors on model development for three years. The federal instrument offered so far is an independent audit, which is an after-action review. Audits are worth having. They are not a stamp.

The question for a committee is never "does this provision mention oversight?" It is "does this provision bind in the machine, at the stamp, or only in the review afterward, and on whom?"

The rate problem

Stability is a rate, not a state. Living systems run at nested speeds: a leaf turns over in a season, a hardwood stand in a century, a soil in millennia. Systems fail when one part runs at a speed the parts that govern it cannot match. A forest plan revised every fifteen years cannot govern a beetle outbreak that moves in two seasons.

AI oversight has the same flaw. An agent acts at machine rate, in seconds. The instruments that oversee it run at document rate: annual inventories, compliance reports, hearings scheduled months out. A human "in the loop" who reviews a weekly digest of decisions already executed is not in the loop. That human is conducting an after-action review, inspecting what has already happened.

The design response is not to make hearings faster. It is to put the governor where the rate is: in the system's permissions, its spending limits, its required confirmations before irreversible actions. Those are engineering choices, and they can be written into procurement contracts and agency policy. Committees can ask for them by name. And for every irreversible action, they can ask who holds the stamp.

Voluntary in name

The NIST AI Risk Management Framework is voluntary. So are most industry safety commitments. But "voluntary" describes the text, not the effect. When a framework is written into procurement requirements, cited by courts as the standard of care, or demanded by insurers, it becomes mandatory in effect. When a mandatory requirement carries no audit, no consequence and no published compliance record, it becomes voluntary in effect.

The test is the same in both directions: follow the instrument to the point where someone bears a cost for ignoring it. If no one does, it is guidance, whatever the cover page says.

This matters for M-25-21 directly. Its discontinue clause is mandatory in text. Whether it holds in practice depends on whether anyone outside the agency can see which high-impact systems were found noncompliant, and whether any were actually switched off. A committee can ask that question with a single letter.

The Petrov clause

In September 1983, Stanislav Petrov, a Soviet duty officer, watched an early-warning system report incoming American missiles. He judged it a false alarm and did not pass it up the chain as an attack. He was right. The system had a human in the loop, and at the moment of action the human held.

Petrov's case is usually told as a story about courage. For a committee, it is better read as a design specification. Three things had to be true for his refusal to count. He had to be positioned before the irreversible step, not after it. He had to have enough time and information to judge. And he had to be able to refuse without the refusal being overridden by procedure.

Every human-oversight provision in AI policy can be held to those three conditions. Is the reviewer placed before the action or after it? Is the reviewer staffed, trained and given the time to actually review, or is one person approving hundreds of outputs a day? Is a refusal protected, so that a federal employee who halts a system is backed rather than disciplined? A requirement that fails any of the three is not a stamp, however it is worded. It is an after-action review.

The second condition is already failing. In remote sensing, a classifier that maps forest cover from satellite imagery is only as good as its ground truth: field plots where a person walked in, measured the stand and recorded what was actually there. Without those plots the map still looks finished. It is simply unverified. Every Petrov clause assumes a reviewer with that kind of field knowledge and the time to use it.

In October 2026, USDA's IT program manager told a Washington panel, of keeping people overseeing AI work: "I think that's untenable in our industry or in government — we just don't have the resources anymore." That is the second condition conceded in public, before anyone has tested the first or the third.

I managed geospatial work for the Forest Service's Northern Region, 25 million acres across Montana, Idaho and the Dakotas. An oversight framework that requires human review without staffing the reviewers has a stamp on paper and no one to hold it. Committees overseeing AI deployment should also oversee the workforce that has to check it.

The question is not whether a document names someone who may refuse. It is whether that refusal holds load at the moment the system acts.

The first 90 days

None of this requires new legislation to begin. A committee with oversight jurisdiction can start with four requests.

Ask for the discontinue ledger. Request from each major agency its inventory of high-impact AI under M-25-21, which systems were found noncompliant, and which were actually discontinued. The memo's compliance deadlines have passed, so the answers should already exist. A clause that has never been exercised has not yet been shown to hold load.

Ask where the human sits. For each high-impact system, ask whether human review happens before the AI-driven decision takes effect or after. Ask how many decisions each reviewer handles per day. The answers sort stamps from after-action reviews faster than any policy document.

Ask what protects a refusal. Ask whether any agency has written guidance protecting an employee who halts an AI system in good faith. If none exists, that is the cheapest way to make a stamp real: one paragraph of policy.

Put the verification workforce in the same frame. When an agency reports AI adoption, ask in the same hearing what has happened to the field and technical staff who would catch its errors. Adoption and verification capacity should be reported together, because one is meaningless without the other.

Any preemption bill can then be read through the same test. If it removes state governors, what replaces them: a federal governor, a stamp, or only an after-action review?

The throttle

The ridges above Leading Creek are bare this week, and with the leaves down you can see how the hillside is actually built. Oversight is the same. Strip away the vocabulary of frameworks and principles, and the structure underneath is simple: at some point between a request and a result in the world, either someone can stop the machine or no one can. The new committees will decide where that point sits. They should decide it on purpose.

---

Sources

• OMB M-25-21, Accelerating Federal Use of AI through Innovation, Governance, and Public Trust (govinfo)
• Akin Gump summary of M-25-21
• White & Case on Executive Order 14365
• Global Policy Watch on the Great American AI Act draft
• Washington Examiner on the stand-alone audit bill
• Tech Times on USDA and human oversight of AI
• Forkast on the M-25-21 deadline

---

Layers of the Machine

What AI, language models, and the application actually do — and where human judgment still has to sit

Nov 5, 2026 - Samuel R. Lammie, with Claude (Anthropic)                                                                                          

                                                               "Three Layers of the Machine"  

Disclosure: one co-author of this post is a large language model built by a company that profits from AI adoption. Read "The black box" with that in mind.

Two words doing too much work

When I made a small investment in Teknowledge in the mid-1980s, at the height of the expert-systems boom, "artificial intelligence" meant hand-written if-then rules captured from human specialists. Today the same two words cover a satellite-imagery classifier, a chatbot, and an agent that can search, calculate, and act across several programs. The vocabulary has flattened distinctions that matter.

Three layers are worth keeping apart. The first (1) is the field: artificial intelligence, the whole discipline. The second (2) is the model: a large language model, one kind of AI among many. The third (3) is the application: the model wrapped with an interface and tools so people can actually use it. In the plate at the end, the field is the outer frame, the application sits inside it, and the model is the core. Most confusion about these systems, and most misplaced trust in them, comes from treating the three as one thing.

Layer one: the field

Artificial intelligence is the broad discipline of building computer systems that do things we associate with human intelligence: recognizing patterns, interpreting language, solving problems, planning, deciding. Some systems follow explicit rules. Others learn statistical patterns from examples.

The family is older and wider than the current conversation suggests. Rule-based and expert systems apply rules people wrote, the way a screening tool flags a permit application when specified conditions are met. Traditional machine learning learns from data to classify or predict, the way a model estimates wildfire susceptibility from terrain, vegetation, and weather. Computer vision reads images, picking roads, forest cover, or damaged buildings out of satellite scenes. Speech systems transcribe a meeting or read text aloud. Reinforcement learning develops strategy through trial, action, and reward. Robotics joins perception, planning, and physical motion. Generative AI, the newest branch in public view, produces text, images, audio, video, and code.

These categories overlap. Some describe how a system learns; others describe what it does. Neural networks sit underneath many of them, but not all: decision trees, rules, and plain statistics still carry a great deal of working AI. 

Layer two: the model

A large language model is one kind of AI, trained on vast amounts of text to learn patterns in language and the relationships among the concepts language carries. "Large" refers mainly to the scale of its neural network: billions of learned numerical values, called parameters. Training adjusts those values, over and over, to improve the model's predictions of text. Out of that process come capabilities nobody wrote down as rules: answering questions, summarizing documents, drafting letters, writing code, working through an argument.

When you type a prompt, the model converts your words into numbers and passes them through many layers of computation. It then writes its answer one token at a time, a token being a word, part of a word, or a punctuation mark. Each new token is shaped by your prompt, by whatever else is in the conversation, and by everything the model has already written.

That process can produce real analysis. It can also produce a convincing mistake, delivered in the same even voice.

The black box

Researchers understand how these networks are built and how they are trained. What remains hard is explaining why a particular network produced a particular answer. The learned patterns are spread across enormous numbers of interacting parameters. No single rule can be pointed to, and no single source can be traced. Interpretability research is making real but partial progress at locating some of the machinery; it does not yet let anyone audit an ordinary answer line by line.

A model's account of its own reasoning is the least reliable class of evidence it offers. When you ask a model to explain how it reached an answer, the explanation is itself generated text, produced the same way as the answer. It may be plausible, but it is not a faithful record of the computation. This consequence deserves its own sentence. Fluency is not accuracy, and a confident explanation is not a transparent one.

Layer three: the application

A chat application such as ChatGPT or Claude is the interface, not the model itself. Depending on how it is configured, the application can connect the model to other machinery: web search to retrieve outside sources, document retrieval to find material in files you supply, a calculator or code environment to run the numbers, image and speech capabilities, and connections that let it take actions in other software. The model interprets and writes; the tools fetch, compute, and act.

Each connection extends reach. Each also adds a place to fail. The system can pick a weak source, misread a document, or run the right calculation on the wrong figure, and the finished answer will read just as smoothly either way.

Two more terms round out the vocabulary. A multimodal system handles more than one form of information, discussing a photograph, reading a chart, answering a spoken question. An agent pairs a model with tools and a process for pursuing a goal across several steps: search, read the files, run the analysis, assemble the report. How independently it works depends on its design, its permissions, and who is watching. That last variable is the one worth asking about.

Oversight language tends to blur a distinction here. A governor binds before the action: the agent cannot spend, send, or delete until a person releases it. A gate works after the fact: a log, a review, a correction once the result is already in the world. Much of what is offered as oversight of AI agents is a gate dressed as a governor. The policy says a human is "in the loop," but the agent runs at machine rate and the review runs at document rate, so the human arrives to inspect what has already happened. The test is Stanislav Petrov's. In September 1983 Petrov, a Soviet duty officer, watched an early-warning system report incoming American missiles, judged it a false alarm, and declined to pass it up the chain as an attack. He was right, and the system had left him room to refuse. The question is not whether a document names someone who may refuse, but whether that refusal holds load at the moment the agent acts.

Where the value is

What language models changed is the interface. Ordinary language now works as a way into information and software. People can explain, summarize, draft, compare, organize,and explore at a speed and scale that was not available five years ago.

So the reliability of the whole depends on the task, the evidence, the tools, and the checks someone actually runs. The model can draft the script that delineates a watershed in minutes. It cannot walk the creek, and it does not answer for what gets decided downstream. Use it to extend your reach. Check the claims that carry weight against evidence, verify the arithmetic, and keep the consequential decisions in human hands.

 

“The three layers, and where verification and validation have to sit.”

Samuel R. Lammie, with Claude (Anthropic). Developed from an explainer first drafted with ChatGPT (OpenAI), then restructured and rewritten. Header illustration generated with ChatGPT (OpenAI); closing plate by Claude. Disclosure: Anthropic builds large language models and benefits from their adoption; the caution about a model's account of itself applies to the co-author of this post.

Tapadh leat.

 

The Missing Position Description

Part 1 of 3: A Presidential Standard

Oct 11, 2026 · Samuel R Lammie

The one federal job without a position description

The Constitution sets three qualifications for president: 35 years of age, natural-born citizenship, and 14 years of residence. That is the entire standard.

As a federal program manager, I hired against position descriptions and sat on rating panels. No applicant for a career federal position, from a GS-11 Geographer up, could clear the first screen without documented specialized experience, a published qualification standard, and a panel scoring the record against it. A senior executive must show five Executive Core Qualifications before the government will hand over a program. The office that commands all of them has no such standard.

The founders were right to leave the choice to voters. But voters need something to judge against, and no one publishes it. So here is one: a position description for the presidency, posted in the open before any candidate's name is attached. It endorses no one.

The filter, not the vote

I don't believe our elections are rigged. I do believe the field is filtered before any of us votes. Primaries reward money raised, attention captured, and intensity among the base. None of those is a qualification.

A candidate can clear all three with no record of running anything, keeping a secret, or telling the truth. A published standard puts a second filter beside the first, one built on the record instead of the reach.

One caution belongs up front. Résumés do not predict greatness. Herbert Hoover arrived with one of the strongest résumés of the century: mining engineer, organizer of Belgian relief, Commerce Secretary. The Depression overran him. Lincoln brought one House term and a law practice.

So this standard is a floor, not a forecast. Like any federal qualification standard, it screens out. It does not promise excellence.

The framers could afford a thin standard because they built strong checks. Congress was meant to correct a poorly chosen president. When that check weakens, as it largely has in recent years, the qualifications of the person in the chair carry more of the weight. Congress deserves its own standard, and I will come back to it.

 Position description: President of the United States

The government already wrote most of this standard, just not for the president. In October 2025, OPM revised the five Executive Core Qualifications every senior executive must meet. The first is now "Commitment to the Rule of Law and the Principles of the American Founding." A career executive must demonstrate it on paper. The person who appoints them need not.

PD element	Entry
Title	President of the United States
Series and grade	None established
Pay	$400,000 per year (3 U.S.C. § 102)
Accountable to	The Constitution, Congress, the courts, the electorate
Statutory qualifications	Age 35; natural-born citizen; 14 years a resident (Art. II, § 1); no more than two elected terms (22nd Amendment)
Major duties	Commander in chief; "take care that the laws be faithfully executed"; treaties and appointments with Senate consent; reprieves and pardons; report on the state of the union
Oath	"Preserve, protect and defend the Constitution of the United States"
Proposed qualification standard	The 17 criteria below, grouped under the five current ECQs plus four the presidency adds

 

Here is how the criteria map onto the standard OPM already uses for senior executives:

Executive Core Qualifications (ECQ)	Criteria that measure it
Commitment to the Rule of Law and the Principles of the American Founding	Constitutional fidelity; criminal and civil record; classified-information stewardship; public–private separation
Driving Efficiency	Resource stewardship
Merit and Competence	Executive experience; national security experience; military service; judgment under crisis
Leading People	Temperament; communication; coalition building
Achieving Results	Policy results
Added for the presidency	Honesty; personal ethics; adversary posture and alliances; formative grounding

 

The rating schedule

Seventeen criteria carry 100 points. Eight, worth 45 points, are scored from the public record against fixed tiers, so any two raters should land within a point. The other nine, worth 55, take judgment. For those I show the evidence on both sides and leave the score to you. Each criterion is scored 1 to 10.

Record-scored criteria (45 points)

Criterion	Points	How it is scored
Executive experience	7	10 multi-year presidency · 8–9 governor of a large state, 4+ years · 6–7 big-city mayor, cabinet officer, or governor under 4 years · 4–5 military command or private CEO without public office · 1–3 legislative only
National security experience	5	Tenure, not performance. 10 commander in chief, multiple years · 8–9 under 4 years, or a cabinet security post · 6–7 seats on both Armed Services and Intelligence · 4–5 one security committee, or a governor's Guard command · 1–3 none
Classified-information stewardship	5	10 no documented incidents · 6–8 an inadvertent-mishandling finding · 3–5 documented negligent retention · 1–2 charged with willful retention or obstruction, whatever the trial outcome. A clean record with no clearance-level exposure is marked untested.
Military service	4	10 career service with combat · 7–9 career service · 4–6 short-term or enlisted service · 2–3 Guard or Reserve, no deployment · 1 none. Military-school attendance is not service.
Resource stewardship	4	Money and organizations, public or private, net of starting advantages and failures; score the larger record. 9–10 large public budget or enterprise run well for years, with audited results · 7–8 large budget or enterprise with mixed results, or mid-size with a clean record · 5–6 durable small or mid-size organization, including on outside capital · 3–4 founder or executive of a small venture · 1–2 no record, or serial failure
Public–private separation	6	10 no business interests, or full divestment · 8–9 blind trust or small disclosed holdings · 6–7 holdings with a documented controversy · 4–5 retained ownership with documented conflicts in office · 1–3 family business transacting with foreign governments or parties before the government
Criminal and civil record	7	10 no charges or adverse findings · 7–8 civil liability for fraud or misconduct · 4–6 misdemeanor conviction · 1–3 felony conviction. Pending appeals are noted, not credited.
Honesty	7	10 × (1 − share of PolitiFact-checked statements rated Mostly False or worse). Report the number of checks; under 20 checks, treat the score as provisional.

 

Judgment criteria (55 points)

Criterion	Points	The question
Constitutional fidelity	9	Does the record show respect for elections, courts, and limits on power?
Policy results	8	What measurable outcomes trace to this person's decisions?
Judgment under crisis	7	How did they decide under time pressure and uncertainty?
Adversary posture and alliances	6	How do they deal with hostile regimes, and do allies trust them?
Temperament	6	Self-command and proportion toward opponents
Coalition building	5	Can they build majorities beyond their base?
Personal ethics	5	Do they keep their word, and how do they conduct themselves?
Formative grounding	5	Did their upbringing and early service prepare them to serve?
Communication	4	Can they explain, persuade, and steady the public?

 

The index is the weighted average of the criteria actually scored, times 10. It always reports coverage, the share of points scored, beside it. A 90 built on 20% coverage means very little.

How it earns trust

A standard is only as good as its resistance to the person applying it, me included. Five safeguards are built in.

1. Published before it is applied. No candidate is named in this post. You can judge the criteria before you know whom they favor.
2. Three tiers kept separate. The record scores the record-scored rows, readers score the judgment rows, and readers set the weights. My defaults are shown, never required.
3. Calibrated on history first. The next post scores past presidents. If the standard rates Hoover high and Lincoln low, I will say so and explain why the floor still matters.
4. Tested across the aisle. Someone who does not share my politics will score two candidates blind. Where we differ by more than two points, the rubric gets tightened.
5. Same rubric for both parties. The 2028 post scores the leading Republicans and Democrats side by side.

One disclosure. Two of these criteria, classified-information stewardship and public–private separation, took shape while I was scoring a sitting president. I kept them because any trustee of the office should be measured on them. Publishing them here first, with no names attached, lets you decide whether that holds.

What comes next

This is the first of three posts. Grading Past Presidents tests the standard history. The 2028 Bench applies it to the likely field in both parties.

If a criterion is wrong, missing, or weighted badly, tell me before Post 2. A position description is a draft until the panel has argued over it.

A free people may hire anyone they choose. They should at least post the job.

Developed with Claude as an analytical collaborator.

Sources

• U.S. Constitution, Article II: qualifications, oath, powers
• 3 U.S.C. § 102: presidential compensation
• OPM Executive Core Qualifications: updated October 1, 2025
• PolitiFact personality scorecards: source for the honesty criterion