What a program manager, one prompt, and a 45-minute deployment taught a room full of enterprise leaders about the real bottleneck in applied AI.
Recent surveys tell a story most executives already feel: 88 percent of organizations are using AI in some form. Half of employees use AI tools that their IT departments don’t know about. Sixty percent of people have built something — an app, a dashboard, an automation, without any IT oversight at all. A workforce of accidental software builders has appeared almost overnight.
And yet very little of what they build ever reaches anyone beyond the person who built it.
That gap between an AI-built prototype and a secure, governed production application was the subject of SEI’s tenth AI event, “Applied AI: From Efficiency to Opportunity,” hosted in Atlanta on August 18. Rather than talk about the gap in the abstract, we crossed it live, taking a real application from one consultant’s laptop to a deployed, access-controlled enterprise tool in about 45 minutes. Here is what happened and what it says about how organizations should think about AI right now.
One prompt, then a wall
Michelle Eckard is a principal at SEI who manages a large IT program for a client. She is, in her own words, not a software developer. Her problem was familiar to anyone who has run a program office: a financial workbook with more than ten tabs that could tell her she was over budget but never why. Answering that question meant hours of filtering, cross-referencing, and manual math every month.
So she typed one question into her client’s approved AI workspace: “Show me who is charging time to my project.”
It worked. So she asked for a filter. Then a search. Then drill-downs by team lead and work stream, burn-rate views, and status indicators that flag which resources are trending over budget and explain why. No requirements documents, no tickets, no backlog prioritization. Each feature suggested the next one, and within weeks, she had a full financial management application built through plain-English conversation. When the numbers didn’t match the source spreadsheet, she didn’t debug the code. She told the AI, “The numbers aren’t matching,” and it found the faulty logic, fixed it, and tested the correction.
Her teammates wanted it. Then her client’s leadership saw it, and said the words every accidental builder both hopes for and dreads: “This is fantastic, we want to productionize it for the entire PMO.”
“That’s the moment where I went, in my head, from wow, this is really cool, to oh no, what do I do now,” she told the room. “I completely hit a wall.”
The wall is real, and it exists for a good reason
Michelle’s wall is not a training or talent problem. It’s a structural one, and nearly every hand in the room went up when we asked who had hit it.
One attendee, a finance program manager at a major technology company, built a tool to orchestrate his team’s month-end close process, with tasks, dependencies, and error tracking, in a few days. It works, but it’s also stuck because making it production-ready requires stakeholder interviews and requirements he doesn’t have the bandwidth to gather. Another attendee, a new hire at a food company, turned a 50-page benefits PDF into a question answering agent for himself in an afternoon. When he tried to publish it for colleagues, it entered an IT queue. It’s been sitting in that ticket for eight weeks.
The uncomfortable truth is that the controls causing these delays are necessary. Security reviews, architecture standards, data classification, access permissions, and approved platforms. No responsible enterprise can drop them, and as our software engineering principal, Adam Monica, put it, every one of those steps exists because something went wrong without it. The problem is not the controls. The problem is that the knowledge behind them is scattered across teams, tickets, and tribal memory. Navigating it, Adam observed, feels like an old role-playing game: talk to everyone in town, find the wizard, retrieve the sword, and maybe eventually you get to slay the dragon. AI collapsed the first half of the software lifecycle from months to hours. The second half still takes months, and that’s where the value is dying.
What we demonstrated: governance that ships
The answer we showed in Atlanta is not to bypass the controls. It’s to make them machine-readable so an AI agent can build within them from the first line of code.
Three building blocks make that possible.
- First, policies: the security, architecture, and data rules an organization already has, codified into a format AI can consume, in plain text, not buried in slide decks and SharePoint folders.
- Second, tools: the deployment pipelines, cloud platforms, and scanning infrastructure that enterprises have spent decades maturing, exposing so that an agent can use them on a person’s behalf.
- Third, the agent itself: an AI workflow with access to both, guided by skills, which are simply written instructions that encode how the organization works. SEI’s internal implementation of this pattern is called Greenlight, but the technologies are interchangeable. The principles are what matter.
With those pieces in place, Adam took the zip file of Michelle’s app and gave an agent one instruction: leadership wants to make this available to the organization — help me do that. The agent identified what production actually required, flagged that the app had no access control, pushed back on an overly permissive sharing choice by citing a specific internal security policy, refactored the front end into the company’s approved stack, generated the deployment keys, pushed the code through the pipeline, and passed the security and compliance scans. Forty-five minutes after starting, Michelle’s app was live in the cloud behind corporate single sign-on, with role-based access control, looking exactly as it did on her laptop.
Michelle’s comment from the audience landed harder than any slide: “I wasn’t supposed to know how to do any of that.”
The most important moment was a failure
A pipeline that always passes is not governance, it’s theater. So we broke it on purpose.
Mid demo, an admin edited one of the organization’s data protection policies, adding a requirement that any application handling personal data must enforce a bounded retention period. The application code barely changed. The next run through the pipeline failed, correctly, citing the new policy by name.
That failure is the entire point. Policies change constantly, and in a landscape where AI has expanded the attack surface faster than most security teams can track, an organization’s rules need to be enforced the moment they are updated, not rediscovered in an audit a year later. The same code, checked against living policies, produced a different and correct answer. Notably, the check that matters runs in the pipeline, independent of the building agent, so even a developer who tried to push something through would be stopped at the gate.
What this means for the economics of AI
There is a cost conversation attached to all of this, and it came up pointedly in the Q&A. Agentic building is token-based and not cheap. Twenty program managers, each running 45 minutes of AI build time a day, add up quickly, and cost controls, budgets, and even cost policies enforced in the pipeline need to be part of the design.
But the sharper insight is about where AI spend goes to die. Organizations are spending heavily on tokens to generate prototypes that never leave someone’s laptop. That is ROI evaporating into the same gap Michelle hit. As John Longo, SEI’s intelligent automation practice lead, framed it: there is a very different conversation between “we built 15 prototypes” and “we spent 200 dollars per app, and we have 200 production apps saving an hour a day for 5,000 employees.”
Where to start
For leaders watching their organizations fill up with brilliant, stranded prototypes, the path we laid out in Atlanta comes down to a few moves.
- Codify your policies into an AI-readable format; they almost certainly already exist, scattered across documents, and consolidating them is the highest-leverage step.
- Expose your existing pipelines and platforms to agents rather than building parallel infrastructure. Establish who curates those policies and how they are kept current, because stale governance is no governance, and treat it as a business strategy rather than an IT-side task.
- Scope deployments deliberately, since an app serving 20 people needs a different review than one serving the enterprise, and your framework should know the difference.
- Expect your engineers to move up the stack, not out the door: the work shifts from writing every line to maintaining the guardrails, handling edge cases, and tackling problems too complex for the pipeline.
The technology to build is already in your employees’ hands, whether you sanctioned it or not. The organizations that win over the next few years will be the ones that give all that creative energy a safe road to production, rather than a wall.
SEI helps organizations move AI from prototype to production with the security, governance, and architecture enterprise software demands. This event was the tenth in our applied AI series.
