Your Company Is Forgetting How to Think — and It’s Not AI’s Fault

On the procurement desk sat three proposal summaries. With the headers stripped, no one could tell which one GPT wrote, which one Claude, which one Gemini. Not because they were identical, but because for the scenarios the company actually used them for (writing analysis, summarizing, drafting replies), they were already “good enough.” Good enough means it didn’t matter who you picked.

So people will tell you: models are turning into utilities, and the real competition is no longer at the model layer but at the data layer. Palantir’s CEO has been saying this loudly. He has his own business to look after, but the direction he’s pointing isn’t entirely wrong: companies feed data in, get convenience back, and trade away something else in the process.

Look at the data first. A company used AI to analyze internal meetings. The contract explicitly said “not for training.” A year later, employees discovered that when they tested the same questions in a competitor’s prompt box, the phrasing patterns coming back increasingly resembled their own team’s never-released analysis. No proof. They just stopped using it. That’s the data-sovereignty kind of risk: traceable, litigable, solvable with a product you can buy.

But there’s another kind of thing no product can guard against. It doesn’t trip any alarms.

Quarterly review meeting. The AI recommended cutting inventory for three product lines, citing slowing turnover in the historical data. After execution, costs dropped 17% the next quarter. Clean numbers. No one clicked the button on the third page of the dashboard labeled “Prediction Deviation Review.” The AI had gotten three things wrong last time, but the new results had already overwritten the old ones. That button sat there for six quarters. Zero clicks.

It wasn’t just operations. The marketing department received an AI-generated competitive intelligence digest every week. The brand manager used to comb through the raw data herself, cross-check two sources before landing on a judgment. Now she just clicked “Read.” Last week the AI mislabeled a small brand’s promotion as a “strategic pivot,” and she copied it verbatim into the memo for the director. After the meeting she checked the original screenshot and saw the mistake. She corrected the memo. She did not correct the process. Next week, she’d click “Read” again.

An organization learns to judge not by hiring smart people but by grinding through loop after loop of “wrong, why wrong, how to fix it next time.” That loop has friction built into it. AI smooths the friction away. A person taps “Confirm,” and on to the next thing. The reflection step is technically still possible, but in practice, efficiency pressure shoves it out. No one has a performance metric called “rate of organizational reflection on AI recommendations.”

The same thing was happening in product R&D. AI screened three hundred candidate compounds and flagged the five most promising. The synthesis team ran the list, four months ahead of schedule. The project review meeting lasted twenty minutes and only covered the two that worked. The three that failed never made it into the minutes. An old researcher stood in the hallway after the meeting. He remembered one of them, a marginal structure he would have given a retest well to back when they screened by hand ten years ago. He didn’t need to anymore. He stood there for a moment, then went to get water.

The losses don’t show up on financial statements. Year-end review. A mid-level manager deleted “team judgment capability building” from his objectives and replaced it with “AI-assisted decision coverage rate raised to 92%.” His finger hovered over the trackpad for two seconds. Save. In those two seconds, he knew the thing that was missing couldn’t be quantified, but it could be felt. He closed the review file, typed “judgment in-house training” into the browser, found it, scrolled through two pages of the course outline, and closed the window. Tomorrow there were three more AI-generated proposals to review.

Then look across the industry. Two competing firms sent a risk alert email to their clients on the same day. The phrasing, the sequence, the exact placement of the word “uncertainty” were nearly identical. A client called to ask: “Are you guys using the same vendor?” It wasn’t plagiarism; it was convergence. When an entire industry uses the same AI to digest information and generate judgments, decision frameworks quietly pull close. No one stole anything. The difference walked away on its own.

One company’s strategy unit tried to break the inertia. They set a rule: after every AI recommendation, the team had to submit a backup plan with at least two different directions, and at least one of them couldn’t rely on AI generation. The first time they enforced it, the conference room went silent for ten minutes. Someone opened a blank document. The cursor blinked eleven times. They typed three words, deleted them, typed a full sentence, deleted that too. They realized they were no longer very good at starting from nothing.

It’s easy to conclude here: stop using AI, then. That’s wrong. An old captain now uses GPS. He can still read an approaching storm from the smell of the sea wind and the rhythm of the swell. But he has stopped teaching the third mate these things. Last winter, on the final scheduled training day, he was supposed to teach the third mate how to read the angle between cirrostratus clouds and wind waves. That day the dispatch office called a last-minute reroute, the third mate was pulled to verify route data, and the training was canceled. The old captain never brought that lesson up again. It’s not that GPS broke. He just stopped teaching. The third mate now navigates entirely by screen and can keep the ship steady. But if one day the screens go dark, the swell’s rhythm will still be there, and the person standing above it won’t be someone who can read it anymore.

Organizations won’t stop using AI, and they won’t stop using efficiency. But maybe, before clicking “Confirm,” someone could ask one more question: “The last time it got something wrong—do we actually understand why?” Just that one sentence. It needs no system purchase, and it won’t appear in any slide deck. At a Monday standing meeting, someone actually tried it. After the AI delivered a new round of inventory recommendations, the operations lead didn’t click through. He said, “Wait two minutes. That batch of returns we had last time—the reason the model missed it, does anyone still remember?” The room went quiet for a few seconds. Then someone opened an old folder that hadn’t been clicked on in a very long time.

That skipped step, trying to find its way back.