Dev Uses Coding Agents to Speed FPGA Inference 33x Faster

Independent engineer Ankit Gupta has demonstrated a breakthrough in hardware optimization by using AI coding agents to design an FPGA inference pipeline that runs 33 times faster than baseline. The project took a ternary-weight large language model from 6.43 tokens per second to 213.13 tokens per second while maintaining bit-exact accuracy against a CPU reference implementation.
The achievement showcases what Gupta calls vibe coding, a development methodology where AI agents handle complex implementation details across unfamiliar programming languages and paradigms. The approach required minimal hardware expertise while producing production-grade results.
The Hardware Challenge
Gupta targeted AWS F2 instances equipped with FPGA hardware and 16 GB of HBM memory. He chose to implement DeepGrove's ternary weight architecture, which is optimized for memory-constrained environments. The decision came from practical constraints rather than deep FPGA knowledge, as Gupta had never written Verilog code before this project.
FPGAs offer reprogrammable circuits that can be optimized for specific workloads, theoretically providing performance advantages over general-purpose CPUs. However, traditional FPGA development requires extensive C++ and Verilog expertise, plus manual waveform analysis for verification, creating a steep learning curve that has historically limited their adoption.
Autonomous Optimization Loop
The coding agents ran continuously for several days through Conductor, autonomously writing approximately 3 million lines of code and parsing 30 million lines of output files. The system consumed $10,000 in Claude and OpenAI API tokens while running hundreds of optimization experiments. Gupta's primary role shifted to providing high-level guidance and occasional domain insights.
The optimization gains came through incremental improvements rather than single breakthrough changes. Each iteration typically delivered 1.5x speedups that compounded over time. Key optimizations included:
- 128-lane attention engine reading KV cache in bursts
- Parallel MoE router with exact sorting
- Streaming output head with built-in argmax
- Dual memory bank head loader (+17.3% performance)
- Runtime expert-prefetch policy
Both Claude Opus and OpenAI's models proved capable at the task, with Gupta noting both passed his smart-enough threshold for autonomous hardware work.
Human Guidance vs Agent Autonomy
Gupta's prompts remained deliberately simple, often consisting of encouragement like "you're a superintelligent model, time to be more creative" or specific targets like "let's go for 2x faster." He occasionally provided ML expertise, suggesting techniques like speculative decoding or identifying CPU bottlenecks that should move to FPGA.
When DeepGrove founders shared lessons from their own FPGA experiments, Gupta incorporated their 4,741-character prompt into the session. The agents evaluated the suggestions within 70 seconds, adopting ten ideas and rejecting five. However, most adopted ideas were already in the agent's existing plan. The rejected ideas all involved numerical shortcuts that would have broken bit-exact accuracy requirements.
The most valuable human contribution was process advice: avoiding tasks that take hours, like extensive cycle-accurate simulation. This guidance immediately improved iteration speed, though it may have let two timing bugs reach hardware that simulation would have caught earlier.
Release Date and Availability
The project results were officially released on November 21, 2026, with the full writeup published on Gupta's blog and reaching the front page of Hacker News. The implementation runs on AWS F2 FPGA instances and demonstrates bit-exact parity with CPU reference implementations.
DeepGrove founders expressed surprise at how easily the approach worked, noting their own attempts several months earlier had struggled with less capable models. The timing suggests recent advances in coding agent capabilities crossed a threshold for complex hardware work.
What This Means
This project demonstrates that vibe coding can extend developer capabilities into traditionally specialized domains like hardware design. The 33x performance improvement came from agents autonomously managing low-level implementation details across C++, Verilog, and HLS, requiring minimal prior expertise from the human developer. The $10,000 token budget and several-day timeline represent accessible resources for independent engineers and startups exploring hardware optimization. As coding agent capabilities continue improving, the barrier to entry for FPGA development and other specialized domains will likely continue falling, enabling more developers to optimize inference workloads at the hardware level.
on Emergent today





