OpenFlyV2 Runs a Vision-Language-Action Model Onboard Drones

OpenFlyV2 has officially launched as a unified vision-language-action model that runs entirely onboard autonomous quadrotors, processing semantic understanding, spatial reasoning, and continuous flight control at 10 Hz on an NVIDIA Jetson Orin NX. Officially released on October 30, 2026, the model achieved 48% zero-shot navigation success across four real-world environments despite training exclusively on simulated data with no real aerial footage.
Unified Architecture for Autonomous Flight
The model architecture integrates vision, language, and action generation into a single shared representation. According to the paper published on Hugging Face, OpenFlyV2 uses "a shared vision-language representation to support semantic understanding, language-grounded perception, spatial reasoning, and continuous aerial control within a single model." A compact action expert converts this multimodal context into actionable flight commands, generating 12-waypoint action chunks every tenth of a second.
The system incorporates motion-aware temporal memory and action priors to reduce computational overhead during closed-loop execution. This design enables the model to fit entirely on edge hardware, specifically the NVIDIA Jetson Orin NX, without requiring cloud connectivity or external processing resources.
Sim-to-Real Transfer Without Aerial Footage
OpenFlyV2 trained on approximately 130,000 simulation samples: roughly 70,000 navigation trajectories and 60,000 vision-language instruction-response pairs. No real-world aerial data entered the training pipeline. The model then transferred zero-shot from simulation to physical quadrotor flight, achieving 48% navigation success across four distinct real environments.
The abstract reports the 48% figure as an aggregate metric without publishing per-environment breakdowns or failure analysis. This sim-to-real gap represents a significant challenge in robotics AI, where models trained in controlled simulation often struggle with real-world sensor noise, lighting variations, and environmental complexity.
Real-Time Onboard Inference at 10 Hz
The model runs inference at 10 Hz directly on the quadrotor's embedded compute module. Each cycle produces a sequence of 12 waypoints that guide the drone's next movements. This real-time performance eliminates latency from network transmission and enables autonomous operation in GPS-denied or communication-constrained environments.
- Inference frequency: 10 Hz on NVIDIA Jetson Orin NX
- Action chunk size: 12 waypoints per inference cycle
- Training data: 130,000 simulated samples, zero real aerial footage
- Zero-shot success rate: 48% across four physical environments
Edge AI for Robotics Applications
OpenFlyV2 joins a growing category of edge-deployed AI models that process complex multimodal inputs on resource-constrained hardware. The model's compact design and efficient inference pipeline demonstrate how vision-language-action architectures can scale down to embedded platforms. This approach contrasts with cloud-dependent systems that require constant network connectivity and introduce communication latency.
The release arrives alongside related edge AI developments, including efforts to deploy transformer world models on even smaller devices like the Raspberry Pi Pico 2 W. These parallel advances signal a broader shift toward onboard intelligence for robotics, where models must operate autonomously without reliance on external compute infrastructure.
What This Means
OpenFlyV2 demonstrates that unified vision-language-action models can operate in real-time on edge hardware for autonomous robotics tasks. The 48% zero-shot success rate shows both the potential and current limitations of sim-to-real transfer, indicating room for improvement as training methods and simulation fidelity advance. For robotics researchers and engineers, the model provides a reference architecture for integrating semantic understanding with continuous control on compute-constrained platforms. The fully onboard approach eliminates network dependencies, opening pathways for autonomous systems in communication-denied environments.
on Emergent today
.avif)




