Every AI hiring post I see is looking for ML engineers. Of course, they're important.

But the actual bottleneck at most companies isn't "can we build this." It's "should we build this, and if so, how."

That's not a technical question. It's a judgment question. And judgment is what's missing.

I've watched teams spend months on AI features that should have been a rules-based filter. I've seen products ship that work beautifully in demos and then create absolute mayhem in production because nobody thought through failure modes. I've seen a process that took four minutes manually turn into something that takes 45 minutes to review and fix because someone decided it needed to be "AI-powered."

In each case, the technology worked. The decision to use it didn't.

And that decision wasn't made by an engineer. It was made by a founder or exec or PM who didn't have a framework for evaluating when AI helps and when it just adds complexity. We have a hiring problem, but it's not the hiring problem we think it is.

The people who are actually valuable right now aren't (just) the ones who can fine-tune a model. They're the ones who can look at a workflow, figure out which 20% actually benefits from AI, scope it for "good enough" instead of "impressive demo," and kill the project entirely if the complexity isn't worth it. That's not an ML engineer. That's a product-minded, operationally-aware person who happens to understand what AI can and can't do. And this person is almost never in the room when AI decisions get made. They're not in the room because we haven't created the role, haven't defined what it looks like, haven't built a career path for it. We keep hiring builders when what we need are editors.

What does this person actually do? They're the one who asks "what happens when this is wrong" before anything gets built. They say "let's map every failure mode and assign a probability and a cost". They're the one who maps the human-AI handoff: not just what the model outputs, but who reviews it, how they review it, what they do when it's garbage, and whether that review process eats more time than it saves. This is where most AI projects die. Not in the model, but in the operations around the model. The model works fine. The process around the model is a disaster. And nobody figured this out in advance because nobody's job was to figure it out. They're the one who can look at an exec's AI roadmap and say "three of these five features will cost more to maintain than they'll ever save". They're the one who knows that the right answer is sometimes "don't," and has the organizational standing to make that stick.

Most companies don't have this person. They have engineers who want to build interesting things and leadership who wants "AI-powered" in the press release. The incentives are all pointed toward shipping, not toward scoping. Product managers get handed AI projects but aren't given the context to push back on whether the project should exist. They're measured on shipping, not on ROI, so they ship. Ops teams are told to "figure out how to use AI" without any framework for evaluating where it actually helps, so they pilot seventeen different tools and none of them stick because nobody did the upfront work of figuring out which problems were actually worth solving this way.

Everyone's building. Nobody's scoping. And scoping is the entire game right now.

Here's a couple of things I've personally adviced founders to do tha might be useful for you-

1. Stop hiring for AI skills and start hiring for AI judgment: That might be a senior PM who's spent time in operations and understands where processes actually break. It might be a generalist who knows how to scope a problem before solving it, who's allergic to building anything before the ROI math is done. It might be someone already on your ops team who's been skeptical of the AI push and can articulate why. That skepticism isn't resistance. Promote them. The point is to find someone who won't be seduced by what's possible and will stay focused on what's worth it. Technical sophistication is secondary. Judgment is primary.

2. Change how you greenlight AI projects. Right now, most companies greenlight based on "this would be cool" or "competitors are doing this" or "we need AI on the roadmap." That's how you end up with fourteen half-built AI features and no ROI. Before anything gets built, require a failure mode audit. What happens when this model is wrong? How often will it be wrong? Who catches the errors? What's the cost of catching them in time, in money, in customer trust? If the answer is "a human reviews everything anyway," stop and do real math. You're not automating a task. You're adding a step before the task. That might still be worth it! Sometimes the AI does 80% of the work and the human just validates, and that's a genuine win. But often you've just created a new job called "AI output reviewer" and you're paying a full salary for someone to fix what the AI gets wrong. Nobody checks this math in advance. Everyone's surprised when the efficiency gains don't materialize.

3. Run a "should this be AI" audit on your existing roadmap. Take everything you've tagged as an AI feature and pressure-test it. What's the accuracy threshold required for this to be useful? If you need 99% accuracy and the model gives you 85%, you don't have a solution. You have a demo. What's the maintenance burden? Models drift. APIs change. Vendors raise prices. That "set it and forget it" AI feature is going to need babysitting forever. Have you budgeted for that? What happens when the third-party model you're building on gets deprecated or acquired or enshittified? Do you have a Plan B, or are you building on sand? I guarantee at least a third of what's on your AI roadmap will fail this audit. Kill it now. It's cheaper than killing it after six months of engineering time and a press release you'll have to walk back.

4. Create a permission structure for killing AI projects mid-flight. This is cultural, not technical, and it's where most companies fail. Right now, most teams are rewarded for shipping and punished for stopping. Nobody wants to be the person who killed the AI project. So projects that should die keep lurching forward, consuming resources, until they get abandoned and nobody talks about them. Make it explicitly okay, celebrated, even, to pull the plug when the ROI isn't there. The person who kills a bad project early should be rewarded more than the person who ships a bad project on time. This sounds obvious but almost nobody does it. The incentives are too strong in the other direction. Shipping feels like progress. Killing feels like failure. But shipping something that doesn't work is the actual failure. It just takes a lot more time and money to become visible.

5. Separate your "AI for efficiency" work from your "AI for product" work. These are completely different games and they require different evaluation frameworks. AI for efficiency is about cost savings. It lives in ops, in internal tools, in back-office functions. The math is straightforward: does this save more money than it costs, including maintenance and error-handling? If yes, do it. If no, don't. AI for product is about differentiation. It lives in your customer-facing features, your core experience. The math is harder: does this make the product meaningfully better in a way customers will pay for or stay for? "Meaningfully" is doing a lot of work in that sentence. A feature that's slightly better with AI isn't worth the complexity. A feature that's transformatively better might be worth significant complexity. But most AI features land in the middle and that's the danger zone. That's where you burn resources for gains nobody notices. Most companies blur these two categories together and end up doing neither well. They ship customer-facing AI features with an efficiency mindset ("it's good enough") and evaluate internal AI tools with a product mindset ("it's not magical enough"). Completely backwards.

6. Accept that most of what you're being sold right now is garbage. The AI vendor ecosystem is a mess of overpromising and underdelivering. Every tool claims to "transform" something. Almost none of them do. Your job is not to find the best AI tool. Your job is to figure out which of your problems are actually AI-shaped, and then find tools that solve those specific problems without creating new ones. That's a much smaller list than the vendors want you to believe. The sales demos are spectacular. The production implementations are almost always disappointing. Not because the technology is bad, but because the problems being solved weren't the right problems in the first place. You automated something that wasn't the bottleneck. You added intelligence to a process that needed simplification.

The companies that are going to win with AI aren't the ones shipping the most AI features. They're the ones who figured out which features were actually worth shipping, built those well, and skipped everything else. That sounds simple but it requires a kind of discipline that most organizations don't have. It requires someone whose job is to say no. It requires evaluation frameworks that most companies haven't built. It requires incentive structures that reward judgment, not just output.

The skill that matters most right now isn't building AI. It's knowing when not to. And until we start hiring for that skill, training for that skill, and rewarding that skill, we're going to keep watching companies burn money and time on impressive demos that never quite work in production. The hiring posts will keep asking for ML engineers. The AI roadmaps will keep getting longer. The ROI will keep not materializing. And everyone will keep wondering why AI isn't delivering on its promise, when the answer is sitting right there:

We built the wrong things, because we hired the wrong people to decide what to build.