It is designed by engineers who hold a premise true, unlike most people in the world: that more capability is better. More capability is different. But not better.

As AI models get more capable, and harnesses designed around them mature, this same capability mischaracterization compounds from task execution to judgement. In other words, we mistake competence at execution for competence at judgement. The very mistake that allows us to think that because AI can use the software of a digital marketer, it can be a digital marketer.

Doing the task is not the same as doing the job.

I lay the blame squarely on the direction software engineering as a domain has gone, because software engineers believe peculiar things about their particular hard thing: that other hard things are the same kind of hard.

Writing code was difficult because it asked the programmer to think with unusual precision. The systems we built had no implicit behavior. A button that changed color had to be programmed to. Even something as simple as the text you highlighted had to be told to turn blue. This was a departure from the real world where we internalize patterns. So computers didn’t just do what you told them to, they didn’t do the things you hadn’t told them to.

I chose the ubiquitous examples of buttons and highlighted text for a reason. To demonstrate that a large amount of code we write in the domain of software engineering is surprisingly unoriginal. Everyone knows what highlighting text looks like, and what buttons do when you click on them.

The field is full of reusable frameworks, coding libraries, and abstractions precisely because the way we solved problems was general. A system that allowed users to log into a platform safely turned into a framework, the framework implemented as a library, and the library used everywhere. The same is true for databases, for interfaces, and almost everything you can think of.

We defined correctness easily. Code that behaved the way we expected it to. As AI models were trained to solve these close-ended tasks, we chose to confirm a much larger idea: that all work, all skills, maybe all professions were the same kind of difficult thing, a collection of somewhat difficult, close-ended tasks.

So AI software, a harness, grows around capability. We give the models more close-ended tasks. We give it memory, tools, and computers. Even the ability to spawn more agents and delegate work. But sales is not programming with different buttons. Neither is marketing, or writing, or management.

This is why the anthropomorphism of AI bothers me, and it should worry you too.

The models solved the kind of hard problem we programmers understood best, and we took that as evidence that enough capability would eventually add up to a person.

Perhaps one day, such AI models that can truly exercise judgement will exist. That’ll be a world where we have a formal definition of consciousness and an understanding of how generalization emerges. In other words, that will be a post-AGI world. We aren’t there yet.

I hope it’s clear now that my concern with harnesses isn’t that they are incapable of doing the right work. It’s that they limit the ability of an organization to arrange itself in a way that it becomes greater than the sum of its parts. I discuss three clear problems below, and I hope that gives you direction for how to think about making better software with AI for teams.

Software experiences don’t just help us execute. The best enterprise software has always been an exercise in improving the judgement of its user. Not reducing it, as the proponents of AI today wish. Not maintaining it. Not increasing execution alone. Good enterprise software experiences increase a user's abilities precisely by making decisions about what the user should do or see or think about next.

Terence Tao in his recent article explained the problem with AI models brute forcing solutions to math problems, which demonstrates the same problem organizations are running into. In fact, this very problem existed pre-AI in most instances where digital transformations failed.

If AI brute forces, or abstracts the entire process of performing a task, the discoveries you would have made about the problem space, and yourself, are lost. Those realizations and learnings made you grow, and the organization or domain of work you belonged to was better off for it. In fact, the goals you set right now for AI to execute for you only exist because someone did the hard work without AI, and realized there was a meaningful task or destination that exists.

Execution changes your beliefs. The mental fatigue of doing a hard job changes your abilities. Reviewing AI outputs does neither.

With digital transformations in the past, and even today, people chose to not adopt software even if it was well made because the product experience started from a point much further in the journey than they were. For example, a digital marketing platform that overwhelms users with the options to schedule posts, or extract learnings from user comments, will go ignored if the user has no idea of the value of these tasks in the first place. The work will flow naturally only for those who have been down that road before and know the hard parts of the job, and the valuable parts of the job.

There’s a reason more content creators use Tiktok and Capcut over Adobe Premiere. Those features don’t make sense to them. They weren’t built for them. They can never enter a flow state and show their abilities on the latter.

The second problem with harnesses is memory.

Current AI memory systems are a hybrid of computer memory and our rough theory of human memory.

The trouble is that we don't actually understand how human memory works very well, and that gap is where problems show up. We don't know why a fact from three months ago suddenly comes to mind during a meeting. We don't know why an apparently trivial interaction changes how we interpret something much later. Worse still, compaction and summarization in harness memories depend on AI to write the summaries.

There are attempts in earnest to solve this problem. For example, RLM approaches save everything, and allow agents to search over the data themselves until they find the answer. My problem with this approach is that it surfaces an overwhelming amount of information for the user to sift through.

If you ask an AI to find every contractual reference to a termination clause, exhaustive retrieval may work perfectly well. If you ask it to prepare a pitch, write an article, understand a customer or advise a team, factual retrieval is only one small part of the job.

Search is useful when you know what to ask; judgement often depends on noticing the thing you did not know to ask about. If you outsource the very facts you work with, and make no effort to decide which facts are worth remembering, you will find yourself unable to do the work you were capable of doing in a pre-AI world.

The final problem is the easiest to demonstrate because it is what makes harnesses so impressive.

They do *a lot* of things.

You give one instruction and watch an agent search the web, open pages, write documents, update records, create tasks and call other agents.

A system can spend ten million tokens, perform two hundred actions and produce fifty artifacts without improving a single important decision.

Yet the user gets the unmistakable sensation that something happened. This kind of dopamine-spike chasing in a world where all of our brains are fried because of social media is incredibly dangerous. And it’s the reason there’s a glut of capability and not enough real value being delivered in organizations.

I really believe the benefits we’re chasing after with AI, having better companies, growing our business, enjoying our work, are all within reach. But they require a more thoughtful approach to building software experiences. Not brute forcing and adding capabilities with no regard for how people actually use them.

I don’t think the answer is less AI.

I think the answer is to stop measuring AI software by how much of the human being it can remove.

The mistake behind harnesses is the same mistake discussed throughout this essay. We confuse more capability with better software. We abstract the work, abstract the memory of the work, and then celebrate the amount of output produced at the end.

But organizations don’t become great because more things happen inside them. They become great because people notice the right things, learn faster, remember what matters, and make better decisions because of it.

That is what we should design AI software around.

Automate execution where judgement isn’t valuable. Engineer information where judgement is.

Sometimes that means an agent should complete the whole task. Sometimes it means the AI should do nothing more than bring the right piece of information to the right person at the right moment. The latter may look less impressive in a demo. It may create far more value.

This is why giving everyone a Copilot licence, or putting a chat box on top of every system, is not much of an AI strategy. You have given people capability. You have not decided what that capability is for.

We can build software that removes the boring parts of work without removing the parts that make people better at it. Software that helps people see more, remember more, understand more, and arrive at better decisions with less fatigue.