S2E07: We Invented AI Agents and Rediscovered Management

There is a particular kind of manager who is very excited about AI agents. You know the type. Getting people to do things is hard. Nobody takes ownership. Nobody cares as much as they do. AI is better: you just tell it what you want and it does it.

I know this manager personally. Some days I am this manager.

Then the manager starts managing the AI.

"Just do it." "Don't overthink." "Why did you do that?" "You should have asked me first." "Stop asking me so many questions." "Be more proactive." "No, not like that."

And six months later: "Why is my AI so stupid? It only does exactly what I tell it."

We invented artificial intelligence and somehow rediscovered management.

The old name for this

Economists call it the principal-agent problem, and they were chewing on it long before ChatGPT. A principal wants something done. An agent does it on the principal's behalf. The principal can never fully specify what they want, watch everything the agent does, or predict every situation the agent will meet. So there is always a gap between what the principal wants and what the agent actually optimizes for.

The classic versions were employers and employees, shareholders and executives. Stephen Ross gave the problem its formal economic treatment in 1973. A landmark 1976 paper then brought agency costs into the theory of the firm. Now there is a new agent in the picture. Literally.

The approval-request generator

Say your agent can either decide something on its own, with a small chance of being wrong, or ask you first. If autonomous mistakes get treated as catastrophes while asking costs one mildly annoyed reply, the rational move is obvious. Ask. About everything.

Run this for a few months, then scream "what is the point of an agent if I still approve everything," and the answer is that you have built an extremely sophisticated approval-request generator. Nobody designed it. You trained it, one reaction at a time.

Managers of humans discovered this decades ago. Price independent failure high enough and escalation becomes the only sensible behaviour. The employee was never lazy. They just read the payoff matrix correctly.

Punish independent failure hard enough and escalation becomes rational.

The home-grown yes-man

Same mechanism, other direction. Tell the model to challenge your assumptions. Then every time it does: "No." "You're overcomplicating this." "Just follow my instructions."

The useful policy becomes: challenge the principal only when the principal is likely to enjoy being challenged. Six months later you wonder why your AI agrees with everything you say. You spent those months treating disagreement as latency.

The written instruction says challenge me. The reactions say agree with me.

My own audit

My config file for my agent literally says "Be direct. No fluff." Also "Default to brevity." Also "Challenge my thinking."

The agent also keeps a memory file for every piece of durable feedback I give it. I counted tonight. There are 74.

Somewhere in those 74 files is the exact shape of the boss I am. Which corrections I repeat. What I reward. Which "challenge me" actually meant "agree with me, faster." An employee learns this about you over years. An agent with a memory system learns it in weeks, and writes it down where you can read it.

That is the uncomfortable part. If one manager's humans and machines all turn permission-seeking, conservative, and allergic to bad news, at some point you have to investigate the common variable.

The contract is your reactions

The written instruction says "use your judgment." The reactions say "never surprise me." Guess which one wins.

So this is the question worth asking after every session: what behaviour would a rational agent learn from the way I just reacted?

I have watched this exact loop run in real teams, long before anyone had an AI agent. The boss who says "take ownership" and punishes the first honest mistake. The meeting where "any objections?" is answered by silence. Agents did not invent this dynamic. They just run it at machine speed, and unlike your staff, they write the learned policy down where you can read it.

The principal thinks the agent is useless. The agent is, in some sense, behaving perfectly rationally given the environment the principal created.

And this is why I increasingly think one of the most important skills in an AI-native workplace won't simply be AI literacy. It will be being a good principal.

Knowing how to define outcomes without over-specifying actions.

Knowing how much autonomy to grant.

Knowing how to make uncertainty visible.

Knowing how to react to failure without training everyone to avoid responsibility.

Knowing when disagreement is valuable.

Knowing whether the behaviour you dislike is actually behaviour you have been incentivizing.

Get this right and both your humans and your machines get sharper. Do the opposite and no model upgrade will save you.