In this initiative, we conducted a fully non-visual Beyond Visual Line of Sight (NVLoS/BVLoS) flight of a racing drone using a 5G experimental mobile network station. By combining low-latency IP video transmission via Shunkei VTX with the low-latency, high-bandwidth characteristics of 5G, we achieved the world's first successful fully non-visual racing drone flight using angular velocity control.
The drone was piloted from a classroom at Keio University Shonan Fujisawa Campus while flying outdoors. We evaluated the latency breakdown and measured 13.9 ms for downlink control latency, 41.7 ms for uplink Glass-to-Glass latency, and 67 ms in total.
Stick-to-Motion Latency
13.9ms
Glass-to-Glass Latency (UL)
41.7ms
Total Round-trip Latency
67.0ms
Overview
Drone racing is a competition where pilots fly high-speed courses while watching FPV video. Speeds can exceed 150 km/h, so latency in both video and control directly affects safety and race performance. Conventional systems mainly use low-latency analog video at 5.8 GHz and RC links at 2.4 GHz.
However, this approach has issues such as image degradation caused by diffraction and multipath, limited scalability for concurrent users, and operations that are basically restricted to VLoS.
With 4G and 5G now widely available, we aimed to prove that racing drones can be piloted from any internet-connected location by using IP networks.
At the same time, there was little practical knowledge of IP control protocols that satisfy racing-grade ultra-low latency, or of how to measure latency breakdown precisely. In this work, we designed and implemented a low-latency IP piloting system and verified full-IP control and video transmission with Shunkei VTX.
General Racing Drone Video
Example of high-speed FPV racing drone flight
Cases of Drone IP Piloting
IP piloting for drones has been attempted in many contexts, and some companies have progressed to PoC stages. DJI also enables network-based control for Matrice 300 and 400 series using a 4G Dongle [1].
SoftBank developed Cloud GCS [2] for waypoint and mission management over networks. Similar cases are reported in UTM [3]. In Japan, major carriers including NTT Docomo and KDDI provide SIM services for drones [4,5], and further expansion is expected.
The Difficulties of Racing Drones
Most existing solutions are intended for inspection and surveying, where mission and waypoint control of largely autonomous drones is the premise. They do not assume fully manual remote piloting like racing drones require.
Racing drones generally do not use GPS for position hold and cannot stay still on their own. They drift due to wind, tiny propeller balance differences, frame distortion, and gyro drift. Continuous corrective input is required, making low-latency control and video transmission essential.
Standard drones often use ANGLE mode, where aircraft tilt corresponds to stick deflection and returns to level when the stick is released. Racing drones mainly use ACRO mode, where angle integrates over stick deflection duration. This enables aggressive trajectories needed for speed.
In ACRO flight, stick deflection time is critical. Over-input quickly leads to excessive tilt and crashes. For this reason as well, racing drones are among the most latency-sensitive applications.
IP Piloting of Drones
There is no IP control protocol specifically designed for racing drones. Existing telemetry protocols such as MAVLink (ArduPilot ecosystem) and MSP (Betaflight ecosystem) can carry control signals, but they are not optimized for this use.
To achieve higher transmission frequencies in the kHz class, we implemented a proprietary protocol with a 4-byte header and 8-byte payload (12 bytes total), carrying control signals equivalent to 14 channels. It includes timestamps to detect delay fluctuations and packet re-ordering in mobile networks.
To transmit controller signals remotely over IP, we implemented IP-to-S.BUS conversion. A standard S.BUS receiver is connected to Raspberry Pi, which outputs S.BUS based on network-received control signals and forwards commands to the FC. We also implemented automatic fallback to local RC on network disconnection.
Drone Video Transmission
Although analog transmission is common, this experiment used Shunkei VTX for low-latency IP video transmission. Video was transmitted over the 5G network.
Architecture
Mobile networks can be interrupted. If control relies only on IP, this immediately causes loss of control and is unsafe. We therefore implemented failover that switches to a standard transmitter immediately when the IP path is disconnected.
For video as well as control, we prepared both Shunkei VTX-based transmission and conventional analog transmission.
Betaflight was used as the drone FC firmware.
Network
The network was built on a 5G experimental station at Keio University Shonan Fujisawa Campus.
Piloting Observation
This shows full-IP piloting and video transmission from indoors and out of sight. The drone was flown manually in angular velocity control mode without relying on GPS position hold. This is difficult in high-latency environments, and was made possible by a low-latency feedback loop across camera, network, and control.
Latency Evaluation
Total latency, including control transmission to the drone, 5G video transmission, display on the pilot monitor, and actuator delay, was 67.0 ms.
The latency breakdown and measurement methods are as follows.
Using ShunChecker-Remote, we measured one-way Glass-to-Glass latency between the indoor piloting point and outdoor flight area. Measurements at multiple points were approximately 53.2-53.3 ms.
We also measured control-side latency (Stick-to-Motion), from stick input to motor current flow. The UL/DL asymmetry is considered to come from mobile-network timeslot differences.
In this experiment, S.BUS was used. Because S.BUS transmits at 140 Hz, it adds about 3.5 ms on average. Using a higher-rate protocol such as ELRS (up to 1 kHz) could reduce this to around 0.5 ms and shorten IP control latency by about 3 ms.
| Category | Latency |
|---|---|
| Glass-to-Glass Video Latency (excluding network) | 41.7ms |
| UL Network Latency (one-way, wireless segment, including Linux IP stack processing) | 11.5ms |
| IP Control Latency (Stick to Current) | 9.18ms |
| DL Network Latency | 4.69ms |
| Total | 67.1ms |
Conclusion
By combining Shunkei VTX video transmission with a newly implemented IP control protocol for racing drones, we achieved BVLoS flight with angular velocity control over a full-IP 5G network.
Latency evaluation showed 13.9 ms IP control latency and 53.2 ms IP video latency, for 67 ms total. Conventional IP piloting often has 100-200 ms video latency alone, plus control latency. Shunkei VTX and low-latency 5G enabled this latency-sensitive racing application.
Shunkei will continue creating value through low-latency technology.
This experiment was conducted with permission from the Tokyo Regional Civil Aviation Bureau and in collaboration with Keio University.