Why I'm Optimistic About AI Despite Not Trusting AI Companies

I started with a lazy version of a good feeling. Ten years ago the smartest people I knew were building apps to make you watch more ads. Now a lot of them are working on frontier AI. That felt like an upgrade in what humanity was pointing its brainpower at, and I wanted to sit with why.

The feeling doesn’t survive contact with the details as cleanly as I’d hoped. A huge amount of applied AI work today is still ads and engagement, just with better models underneath. Recommendation systems, feed ranking, ad targeting. The infrastructure got smarter and the goal stayed the same. And talent has always followed capital more than it follows impact. The 2010s app wave wasn’t people deciding photo-sharing was the highest use of their minds. It was where the money was. Same is true now.

So the clean story falls apart. What I actually think is more like this: the ceiling went up. The range of what AI could do, good and bad, is wider than anything the app era put on the table. That’s not the same as it being a guaranteed net good. It means the variance got bigger.

The analogies I reached for, and where they broke

I kept trying to find the right reference class for this moment.

First I thought about the space program. Sputnik to the moon in twelve years, a huge chunk of a generation’s engineers pointed at one hard problem. The velocity matches. But Apollo had one buyer, one goal, and a finish line. Land the man, bring him home, done. AI has dozens of companies with their own commercial roadmaps racing each other, and no moon-landing moment where anyone gets to say mission accomplished. It’s a permanent buildout, not a sprint.

Then I thought about the early app era, where a hundred players show up and a few incumbents survive. That one holds better, but mostly for the application layer. The AI wrapper startups will consolidate the way social apps did. The frontier model labs won’t, because their moat is compute and capital, not network effects. You can’t bootstrap a competitor to a frontier model out of a dorm room the way you could bootstrap Facebook.

The thing that stuck with me from the app comparison is uglier. The winners of the 2010s weren’t the ones building the most good. They were the ones who won the attention game. If AI consolidates the same way, whether it’s good for people depends entirely on what the dominant business model turns out to be.

The part I hadn’t thought through

If I had to bet on the next ten years, I’d bet on broad benefit. AI changing how people work, and doing the most good in sectors that historically couldn’t attract great talent because the pay and the prestige weren’t there. Rural medicine. Teacher-strapped schools. Elder care. A model doesn’t care what a job pays. That’s a real mechanism, not a hope.

But when I said out loud why I believed the benefit would actually land, my first answer was weak. I said people learned their lesson from social media and would be more careful this time. That’s not true, and I knew it the moment I typed it. Social media is still running the same engagement playbook fifteen years later. The lesson-learning happened in essays and academia, not in the business models that won.

So I had to find where my optimism actually rests, and it isn’t there.

Where it actually rests

It rests on backlash. Specifically, on backlash that arrives fast and stays constant, and on that pressure being what forces regulation rather than the other way around.

The lawmakers of the social media era weren’t online natives. The people who are 30 to 50 now grew up with this stuff, got burned by it, and are watching what it does to their own kids. That’s a different set of regulators, structurally, than the Congress that couldn’t work out how Facebook made money. And the young people getting squeezed out of entry-level jobs by AI right now are a live grievance, not a hypothetical one. Grievances like that become voting blocs.

Then there’s the part I find almost funny. Social media is the thing that will keep AI in check. The reason backlash can be fast and constant now is that ideas move through communities at a speed they never did before, and we have that speed because of the platforms everyone spent a decade criticising. The disease built the antibody.

Someone will point out that social media had constant loud criticism the whole time and the business model never changed. Fair. My answer is that the criticism of social media wasn’t fast enough early enough. It built up after the damage was already normal. The scrutiny machine that exists now didn’t exist when Facebook was becoming what it became. This time the “this could be dangerous” conversation started the same week the tools launched. Talk about the risk came two seconds after the announcement, not five years after the harm.

Awareness isn’t restraint, and I know that

Here’s the honest weak spot in my own position, the one I want on the record so I don’t quietly forget it.

Early awareness of danger is not the same as restraint. The Manhattan Project scientists knew exactly what they were building before they built it. Tobacco companies had their own cancer research decades before regulation. In every case the people closest to the danger saw it clearest and earliest, and it didn’t stop them. Being the one who names the risk can even make it easier to keep going, because it lets you look responsible while you proceed.

I still think the nuclear story lands closer to net good than not. No world war since 1945. No weapon used in anger for over half a century. But the reason wasn’t that the scientists felt the weight of it. It was mutually assured destruction and an entire external architecture of treaties and inspections, forced restraint rather than chosen restraint. And even that nearly failed on single human judgment calls more than once.

So when I say I think the labs will behave, I don’t mean because Dario or anyone else holds a moral line. I mean because the public, through every mechanism it now has to apply pressure, will make staying in line the cheaper option. The check is external, same as it was for nuclear. I’ve just swapped arms-control treaties for a faster, messier, decentralised version made of media and communities and people who won’t stop being loud.

That’s the whole bet. Not that AI is good. Not that the people building it are wise. That the pressure gets there in time, and that this time the pressure is fast enough to matter.

I don’t know if I’m right. Ask me in a few years whether the noise actually moved anything, or whether it turned into background hum the way it did last time.