A year into modernizing a large system under the Health and Human Services, I've had a front-row seat to one of the most fascinating conversations in GovTech: how do we responsibly integrate AI into systems that serve millions of Americans?

I'll be honest—when I joined this effort, I expected the AI conversation to be all about cutting-edge models and rapid prototyping. But I've discovered something more nuanced—the real challenge isn't the technology itself, but how we build the foundation for AI to actually serve the public good.

The Reality Check

Working on enterprise system modernization means dealing with decades of legacy Java systems, strict FedRAMP compliance, and stakeholders who've seen plenty of "transformative" tech promises come and go. Every day, I'm deep in Spring Boot migrations, OAuth flows, and containerization strategies. But our architecture discussions increasingly circle back to a critical question: how do we build systems today that can responsibly leverage AI tomorrow?

I got a front-row seat to this challenge when an executive order required us to scan thousands of grant documents for specific language. Without a reliable way to extract and tag those phrases, analysts were manually hunting through hundreds of pages, risking missed insights and compliance errors. What followed wasn’t a triumphant AI implementation story—it was weeks of methodical technical investigation into OAuth scopes, file-ingestion patterns, API rate limits, and schema mappings to ensure that our chosen AI service could securely and accurately process each grant without exposing sensitive data.

I found myself provisioning S3 buckets, debugging OAuth scope permissions, testing file upload patterns, and documenting every finding. Should we push documents via API or let the AI platform pull from our bucket? How do we map scan results back to specific grant records? What happens when a webhook fails—do we have a polling fallback? These aren't the questions that make it into AI conference keynotes, but they're exactly what determines whether AI actually works in production.

There  is a gap between AI's promises and the unglamorous groundwork required. While tech conferences showcase demos that scan documents in seconds, I'm spending days just figuring out if our authentication tokens have the right scopes. We're not asking "what can AI do?"—we're asking "how do we make AI work within FISMA constraints, with audit trails, in a system that serves millions?"

Building Bridges, Not Walls

I’ve learned a vital lesson: technology succeeds when it enhances human collaboration rather than replacing it. Too often, solutions chase end-to-end automation—only to stumble when real users find the output inscrutable. The best tools empower experts to work faster and more accurately, adapting to human workflows instead of forcing humans to adapt.

The most promising AI applications aren’t the ones chasing full automation. For example, imagine an AI tool that scans lengthy grant guidelines to pull out key deadlines and required signatures, then highlights those snippets for the grant reviewer. Rather than sifting through 50 pages, the analyst sees exactly where to focus, confirming accuracy and context in seconds. This kind of augmentation can cut review time in half and let experts spend more effort on strategic decisions—showing how small, human-centered AI features can deliver outsized value.

The Security-First Reality

In my world of backend engineering, every integration starts with security. When AI features come up, the conversation immediately turns to fundamentals: How will we authenticate requests? Where will sensitive data reside? Who can access which resources? What audit trails do we need? And what formal agreements (ISA, MOU, etc.) must we put in place before data crosses organizational boundaries?”

Those questions might sound restrictive, but I’ve come to view them as empowering. By baking security into our architecture from the start—establishing clear authentication flows, enforcing data governance policies, and maintaining comprehensive logs—we create the trust and resilience that responsible AI depends on. It’s not glamorous work, but it’s absolutely essential.

Start Small, Think Big

Something  I've learned from engineering is that successful transformations start with small, tangible wins. In the context of AI for government, this might mean:

• Start with internal tools that help staff, not public-facing systems

• Focusing on augmentation rather than automation

• Choosing use cases with clear metrics and low risk

• Building feedback loops with actual users from day one

The technical spike I mentioned? It revealed something important: even a "simple" AI integration requires answering dozens of questions. Webhook or polling? Synchronous or asynchronous? How do we handle versioning when documents get updated? Where exactly in the workflow do we surface AI results to reviewers?

Each question leads to more investigation, more documentation, more alignment meetings. But that's not failure—that's what responsible innovation looks like. We're not racing to deploy, we're building understanding.

The key is resisting the urge to overpromise. Every hackathon produces an AI demo that can summarize documents or answer questions. The hard part is making it work reliably, securely, and equitably at scale. It's turning that demo into something that a regional grant reviewer can actually trust and use every day.

The Path Forward

As I continue working on enterprise system modernization, I'm excited about what's possible to do in some distant future—but in the careful, incremental progress we can make today. Every microservice we modernize, every API we secure, every data pipeline we optimize is laying groundwork for responsible AI adoption.

Sometimes that groundwork looks like a technical spike document filled with questions about webhook endpoints and OAuth scopes. Sometimes it's a meeting where we realize the "simple" AI integration actually requires coordinating across three teams and two agencies. Sometimes it's discovering that before we can use AI to help review grants, we need to solve basic problems like "how do we reliably map a document back to its grant record?"

This is the work that doesn't make headlines but makes transformation possible. It's the difference between a proof-of-concept and a system that serves real people with real needs.

The public sector doesn't need AI evangelists; it needs builders who understand that transformation happens one secure API call at a time. It needs teams who can balance innovation with compliance, excitement with skepticism, and possibilities with current realities.

For those of us in the trenches of GovTech modernization, the AI opportunity isn't about riding the hype wave. It's about doing the patient work of building systems that are secure, scalable, and truly serve the people who depend on them. That's not always glamorous, but from where I sit—debugging Spring Boot services and designing OAuth flows—it's some of the most important work we can do.

Tobi Afolayan is a Backend Systems Engineer at Systone, currently working on enterprise system modernization initiatives. He brings perspectives from his work at Microsoft and his passion for building technology that enhances rather than replaces human capability.