The Danger Isn't AI Writing Code. It's AI Having Permissions

I've been using AI coding tools long enough to catch myself thinking:

"If AI can fix the code on its own now, do we even need developers much longer?"

Look at the recent security incidents, though, and I think a different question matters more. Not how well AI writes code. How far we let it reach.

AI making mistakes isn't the new problem

Developers have always made mistakes. We write bad code. We fat-finger config values. We ship things to production that break.

That's why software development grew layers of safety. Someone writes the code. Someone reviews it. Tests run. CI/CD passes. An approver signs off. Then, finally, it goes to production.

Why make it that tedious? Because the whole system was built on the assumption that people screw up.

So the fact that AI produces bad code doesn't shock me. Of course it does. It makes mistakes too.

The interesting part is what happens next.

AI is growing hands

Early generative AI wasn't especially dangerous. You asked, it answered. If the answer was wrong, you just didn't copy it.

The shape was AI → human → system. A person sat in the middle.

The agent era is changing that shape. AI spots a problem, reads the code, drafts a fix, opens a PR, runs the tests, and in some setups reaches all the way into the deployment pipeline.

The shape is turning into AI → system. AI didn't just get a brain. It got hands.

That's where the security story changes completely.

Why the recent Autofix case is interesting

Wiz's research team recently demonstrated a script injection vulnerability in a public Snowflake GitHub repo, found by an autonomous AI agent. Wiz attributed the flaw to a change tied to the GitHub Copilot Autofix flow; GitHub pushed back, saying a human wrote the contribution. Whoever's right about authorship, the case showed what happens when AI-generated fixes get wired into CI/CD and other downstream systems.

What caught my attention isn't "the AI wrote bad code."

It's the trust boundary between three things: what the AI can read, what changes it can produce, and the automation sitting behind it. That boundary is a new place to attack.

This is only going to get bigger, because companies are handing AI more and more work right now.

It's like giving a developer root

If you've run infrastructure, this one's easy. You don't hand root on a production box to a developer, no matter how good they are. Not because you distrust them. Because people make mistakes.

So you build RBAC. Separate ServiceAccounts. Restrict Secret access. Apply NetworkPolicy. Split up deploy permissions.

Kubernetes is the obvious example. One misbehaving Pod doesn't get to touch the whole cluster.

Why? To shrink the blast radius when something does go wrong.

Yet with AI, people somehow run the logic in reverse.

"It can't do its job unless it can also access this."

"Connect GitHub." "Connect Jira." "Let it read Slack." "Let it run CI/CD." "Hook it into cloud."

The more capable the AI gets, the more the permissions pile up. At some point you've built one very smart, very privileged account with reach into a dozen systems.

So AI security may not be about accuracy

Everybody fixates on model accuracy. A 95% model. A 99% model.

Production doesn't care. At 99.9% accuracy and millions of operations a year, failure is a certainty.

Enterprise systems have always been designed around that. Not "a system that doesn't fail" but "a system that survives failing."

AI needs the same treatment. Don't expect the AI not to make mistakes. Build it so that when it does, the damage is bounded.

I think that becomes a foundational principle of AI agent architecture.

Least privilege applies to AI agents too

Say the AI finds a code vulnerability on its own. You can hand out authority in stages.

Level 1

AI finds the vulnerability and tells a human.

Level 2

AI proposes the fix.

Level 3

AI opens the PR automatically.

Level 4

AI runs the tests too.

Level 5

AI deploys to production without approval.

Level 5 looks the coolest from a purely technical angle. Full automation. But in a corporate system, Level 5 isn't automatically the right design.

The better move is to design where the AI has to stop, based on how risky the work is. Payment system? Maybe it stops at opening a PR. Internal test environment? Let it deploy.

The AI's capability isn't the deciding factor. The system decides how much authority and responsibility the AI gets.

The developer's job shifts here too

Plenty of people say AI means developers won't be needed. Some coding work really will shrink.

But other parts get more important. What do we ask the AI to do? What data does it get to see? Which systems can it touch? How far does automation run? Where does a human have to approve? When it fails, how far does the damage spread?

None of that is a coding problem. It's an architecture problem.

The good developer in the AI era may not be the one who writes code fastest. It may be the one who designs systems where AI can work safely.

At the end of automation, always: permissions

We've been building automation for a long time. Shell scripts, then CI/CD, then Infrastructure as Code, then Kubernetes. Now AI agents.

Every time automation got more powerful, the same question came back: how much authority do we give this thing?

AI is no different. It's more urgent, actually, because AI makes judgment calls on its own.

That's why I don't think enterprise AI will be won on "which model are you using." The sharper question is: what permission structure is that model operating inside?

AI writing code doesn't scare me. AI being wrong sometimes is entirely predictable.

The dangerous part is handing an AI that can be wrong far too much authority, and then calling it automation.