Rethinking GPUs for the AI Era: From Atoms to Agents

Raja Koduri, Founder and CEO of Oxmiq Labs, recently sat down with Salah Nasri on the Semiconductor Leadership Podcast to unpack a question the AI industry is only beginning to reckon with: whether the entire GPU stack needs to be rebuilt from the ground up for the agentic era.

The Economics Have Become the Real Constraint

A single hyperscale AI data center now costs approximately $50 billion. The silicon budget alone for the most popular GPUs runs $30-35 billion per gigawatt, with energy, power delivery, networking, and cooling adding another $10-15 billion on top. As AI demand accelerates, these costs are only rising, meaning the industry can't build fast enough to keep pace with demand. Koduri's framing throughout the conversation was blunt: bandwidth, not compute, is the true bottleneck in modern AI systems, and it's a constraint no standard or abstraction layer can fully solve.

Given that reality, Koduri argued that utilization, getting more useful work out of every transistor, will determine who wins and loses over the next five years.

OxCore: One Core Instead of Three

At the center of Oxmiq's platform is what Koduri calls "the new computing core," OxCore, which encapsulates scalar, vector, and matrix computing into a single unified architecture. Today, most systems treat these as separate abstractions, CPU-style scalar processing, GPU-style parallel vector processing, and TPU-style matrix math, each with its own programming model. Koduri described OxCore as a single core unifying CPU, GPU, and TPU into one architecture, a design meant to improve utilization while simplifying how developers reason about execution.

On chiplets, an area Koduri has hands-on experience with going back to AMD's HBM1 through Intel's 47-chiplet Data Center GPU Ponte Vecchio, he offered a dose of realism: "Everyone who is super excited about chiplets are the ones who haven't done them yet."

Agents Talking Directly to Silicon

The most striking part of the conversation was Koduri's vision for how AI agents, rather than human programmers, might eventually interact with hardware. "They don't need to talk through Python, C, all these intermediate languages. We created them for humans to program. But when it's an agent generating work, there will be new, more efficient forms of communication, where the agent can talk to what I call nano-agents in silicon directly," he said. He pushed the idea further: "You can express an entire inference model in a single page of math equations. Why am I breaking that down into tens of thousands of lines of code, through all the layers of the stack, just to ask the atoms to wiggle and perform a math computation? What if the future hardware just talks math?"

Oxmiq's long-term architectural bet is that these software layers will thin over time as systems evolve to express intent more directly to hardware, fewer translations meaning lower latency, less wasted energy, and higher efficiency. Asked for the one thing listeners should take away, Koduri's answer was simple: "Think agents to atoms."

The ARM-for-GPUs Play

Unlike traditional fabless companies that design and sell their own chips, Oxmiq licenses its IP so others can build chips tailored to their own needs. As Koduri put it: "There is ARM for CPUs. But there is not ARM for GPUs. Anyone can license our IP and build a chip. That's the problem we're trying to solve." He noted that no single chip can satisfy every workload, pointing out that even Apple's iPhone constraints differ from other smartphones' power, thermal, and sensor requirements, and that "even Nvidia can't cover the entire range." Licensing lets partners build silicon tuned to their own requirements while OxCore scales across edge, datacenter, robotics, and automotive use cases on a common architecture and software ecosystem.

Oxmiq Labs emerged from stealth in August 2025 after two years of intensive IP development, with Koduri assembling a team of GPU and AI architects carrying over 500 years of combined experience, hundreds of patents, and a collective track record of more than $100 billion in revenue generated at prior companies. The company's founding thesis centers on GPUs as the cornerstone of a computing shift toward multimodal experiences spanning text, audio, video, images, and 3D environments, functionality that fixed-function AI accelerators can't match given the general-purpose flexibility GPUs provide.

Koduri also co-founded Mihira two years earlier alongside Shobu Yarlagadda and SS Rajamouli, and has since stepped back from day-to-day operations there to focus full-time on Oxmiq, though he continues as a strategic advisor to Mihira Visual Labs, with Oxmiq holding a minority stake and supporting Mihira's cinematic AI platform through its GPU IP.