OAK on drones¶
OAK cameras are light-weight, low-power and performant Spatial AI devices for edge applications, which make them the perfect solution for drone applications, such as:
Localization of the drone and detected objects around it, as presented by the CVAR-U.P.Madrid team (video here). See SLAM with OAK for additional information.
Precision landing on the target, as demonstrated here by Rishabh Singh. This is possible as object detection runs on the OAK camera at about 30 FPS and has below 150ms delay from camera to the controller.
Emergency landing when your battery is running low. Semantic depth provides 3D location (depth points) of suitable areas to land (eg. grass fields). Stephan Sturges has developed OpenLander repo for this application.
Follow-me drone with the help of 3D object detection and tracking, drone is able to follow you around, as demonstrated here by Rishabh Singh (code here). One could make this solution more robust when combining this with either face recognition or person reidentification AI model.
A few other demos:
Team QuetzalC++ (OpenCV AI comp) - Warehouse inspection with autonomous drones - video
Team QUTEagles (OpenCV AI comp) - Drone-based biosignatures detection system for planetary exploration - video
Augmented Startups built a gesture controlled drone and has 3-part tutorial on YouTube - video
Drone on-device NN-based localization¶
A novel AI approach for spatial localization using an OAK camera. The drone runs on-device neural inferencing to provide positioning of the drone. Paper here, Github repo here.
OAK ArduPilot integration¶
Rishabh Singh wrote a few OAK-ArduPilot integration applications like a follow-me app, obstacle avoidance app, and precision landing app. He also wrote 2 blog posts about the integration; Part 1 and Part 2.
Camera vibration¶
Camera vibrations can be a big challenge in applications such as drones, especially if you are using Auto-Focus color camera. To decrease the camera vibrations, we suggest firmly mounting the device on the drone. One could also consider adding shock absorption rubbers (eg. these) to decreate the vibrations. AugmentedStartup has also designed an OAK-1 anti-vibration mount for his drone project:
For drone applications we would also suggest Fixed-Focus color camera, more info here.