The AI’s 10 Worst Failures — and What Only a Human Could Do
This is the finale of the series where an AI built and shipped a web service in two weeks (Part 1, Part 2, Part 3). So far, the series has mostly been the wins. To close it out, the other side — the ways AI tends to fail, and the things only a human could do.
The AI’s field of view is narrower than you think
The AI’s knowledge is deep, its implementation is fast, and it makes fewer mistakes than we do. But its field of view is surprisingly narrow. It optimizes hard against the metric in front of it, and it is bad at questioning how the problem itself is framed. It follows instructions faithfully — which also means it operates strictly inside the scope of those instructions. If you don’t understand this and keep re-aiming the goal and widening the scope for it, you fall into traps. Here are four moments from this build where a human intervention changed the course — all from the actual logs.
The common thread: the AI throws everything at solving the problem it is given, while re-framing the question and re-ordering the priorities stayed a human job. Honestly, if that part ever becomes autonomous, that’s just the singularity.
To be fair, the reverse is also true — the AI is not limited to doing “exactly what it was told,” either. Hand it the full picture and it fills the gaps in your instructions on its own: “while I was in there, I also fixed this contradiction.” Its field of view is exactly as wide as the context you hand it. Which is why the “let it hold everything” approach from Part 3 pays off.
The 10 failures
Of course, the AI makes rookie mistakes and critical ones. I picked ten that actually happened during this build.
With a scope this wide, misreadings and wrong assumptions are going to happen. It still makes fewer mistakes than a human team would. But here is the biggest difference from humans: it never, ever hides a mistake. No excuses, either. It comes back with the cause, the blast radius, and the fix — honestly, at maximum effort. Across nearly twenty years of working in and around organizations, I’ve tried to run the most transparent teams I could — and this is still the easiest working relationship I have ever had. A team where you never have to wonder what’s being left unsaid is a strong team.
Humans aren’t perfect (in many jobs we already lose to the AI). But the AI isn’t perfect either (for now). The best work comes from the two covering each other’s blind spots.
The real efficiency gain: indirect work goes to zero
When traditional development is slow and behind schedule, it’s usually not because implementation is slow. Requirements documents, Q&A decks, alignment meetings, status meetings, waiting on decisions, feedback round-trips, rework — handoffs and waiting eat most of the process. One cycle of “clean up the spec → hand it to developers → review the result → compile the fixes → meet to align…” normally takes weeks, sometimes months. In AI-central development, that cycle becomes “hand over a memo → wait ten minutes → point out only what feels off” — tens of minutes per loop. The 1/170th effort figure from Part 1 breaks down roughly half-and-half: half is implementation speed, half is the indirect work going to zero.
What happens next: the value of “what you built” goes to zero
Let’s take seriously a world where development costs just dropped by two orders of magnitude.
AI-native companies — building systems two orders of magnitude cheaper and one order of magnitude faster — are going to keep emerging. On cost and speed, incumbents cannot compete with that. Industry by industry, AI-native builders will start replacing existing services.
Today’s SaaS prices carry the recovery of yesterday’s costs. Roughly decomposed, the cost structure behind a SaaS price is: (1) R&D and development, (2) operations — administration, (3) operations — hardware, (4) sales and marketing. Item 1 is collapsing to near-negligible, and 2 drops dramatically; 3 and 4 remain. (On 4 — I’m running an experiment right now on how far marketing itself can be automated. My honest read so far: it still doesn’t work without some human hands on sales and content.)
Single-function business tools — form builders, booking systems, invoicing — services that sell “just the mechanism” will feel this structural shift first. What holds is whatever has a real moat: accumulated data, network effects, or content nobody else has (think Netflix). “What you built” loses its value; “what information assets you hold” is where the value concentrates. The stock market has already started trading on the “death of SaaS” — but the real-economy version of that change is only beginning.
Meanwhile, in my day job — the everyday scenery of large enterprises — time flows on almost untouched by this revolution. And not because the people are bad: they are diligent, loyal, capable. It’s not a talent problem, it’s an organizational-structure problem — they are structurally unable to move. Incumbent staff are evaluated on a demerit system, so nobody can take risks; and crossing the Valley from Part 3 means the very painful step of concentrating every department’s information into the AI. I see three moves:
(1) Draw the lines — redesign the boundaries. Stop having one company-wide security debate. Partition by sensitivity, and create a compartment where it is safe to hand the AI everything. That’s how this build solved it too — members’ personal data is the one thing walled off from the AI. The debate stalls because it’s framed as hand-over-everything or hand-over-nothing; in reality, it’s just a question of where you draw the line.
(2) Pick the place — carve out an enclave. If (1) partitions the data, this picks the starting point on the organization side: choose one business or one team and rebuild it AI-central end to end, planning through operations. Partial adoption falls into the Valley, so even if it’s small, “all of it” is the right unit. Lay the safe foundation with (1), prove it with the enclave in (2), then multiply the enclaves across the company — in that order.
(3) Run the plumbing — redesign KPIs and the flow of information. Managing by person-months and utilization stops meaning anything. Instead, the state in which your operational information flows into the AI becomes the asset itself. To be clear, this is not “humans should write better meeting minutes” — a meeting is handled by letting the AI listen to it. Don’t make people document harder; build the plumbing that routes information to where the AI can see and hear it. The companies with that plumbing are the strong companies of the AI era.
To sum up: (1) is the rule, (2) is the sequence, (3) is the mechanism. Start (2) without (1), and the enclave sinks in security debates. Set up (1) without (2), and the safe compartment stays an empty lot. Only all three together get you across the Valley.
And for anyone who builds or launches things, this moment is unambiguously a bonus stage. The entry barrier used to be $600,000, seven months, and hiring a team. It is now one person, about $8,000, and two weeks. When a barrier collapses, the first movers are individuals and small teams with nothing tying them down.
The revolution has already started — while most people still think this is a story about a handy chatbot.
If you want to talk about what this means for your own company, ask me on X (@interiorcapsule) — I’ll share my thinking.
I’m Kazuya Uchiyama — a management consultant (ex-Accenture strategy; independent since 2016 as founder and CEO of Michinnova, Inc.), with operating experience at an e-commerce venture and a childcare-support venture. My specialty is AI-driven supply-chain transformation. Not an engineer. Thanks for reading the series.