AI doesn’t make a team productive

After writing about cloud, microservices, and the illusion of scale, a pattern became hard to ignore: every time a new technology shows up, it gets treated as the answer. With AI, the story repeats. Only faster, and with far more money on the table.

And a question comes up that almost no one asks before adopting it: how will we know if AI is actually making us more productive?

The answer is less comfortable than it looks.

The feeling of speed is not speed

A controlled study by METR followed experienced developers solving tasks in their own repositories. Half of the tasks with AI, half without. The result was the opposite of what everyone expected: with AI, they were 19% slower. (full source here)

That is not the detail that matters. This is: the developers themselves believed they had been 20% faster.

It is not a small measurement error. It is a gap between what we feel and what actually happened.

That should unsettle anyone who plans to measure AI’s impact by asking people whether AI helped them. Perceived productivity is one of the least reliable signals there is. And it is exactly the one most companies are using.

Producing more is not delivering more

AI is excellent at generating. More code, more pull requests, more commits. And that is precisely where the trap is.

A team can double the amount of code and deliver nothing new to the client. It can open three times the pull requests and choke its own review process. It can write faster and generate more rework afterward.

Typing speed was never the bottleneck in engineering. The bottleneck is deciding what to build, understanding the domain, reviewing, integrating, and maintaining. AI mostly speeds up the part that was already fast.

It is the same mistake as the illusion of scale, wearing new clothes: confusing motion with progress.

AI is a mirror

The most recent industry research all points in the same direction. AI does not level teams. It widens the distance between them.

Mature teams, with a clear delivery flow, automated tests, consistent review, and defined ownership, adopt AI and get better. Immature teams adopt the same AI and get faster at producing mess. The instability that was already there starts to show up sooner, and bigger.

AI does not fix a team’s disorganization. Just as cloud did not. Just as microservices did not.

It only makes an immaturity that was already there more visible, and more expensive.

So what should you measure

If AI amplifies what a team already is, then the right question stops being “how much did AI speed us up.” It becomes: is the team delivering what was agreed, on time, with quality?

That did not change with AI. It never changed.

In a consultancy, productivity is not the amount of code. It is predictability: delivering the contracted scope within the promised time. It is quality: what you deliver does not come back as rework. It is consistency: being able to repeat that project after project, client after client.

AI enters as one variable inside that equation. Not as the whole equation. You do not measure AI. You measure delivery, and you watch what AI does to it.

The risk is not that AI will not work. It is that it works too fast on a team that still does not know where it is going.

Because when that happens, AI does not amplify what is right. It amplifies what is wrong.