SLAM with OAK¶
On-board localization (VIO) and SLAM (Simultaneous Localization And Mapping) on current OAK cameras (RVC2) aren’t yet supported.
Our upcoming Series 3 OAK cameras with RVC3 have Quad-core ARM A53 1.5GHz integrated into the VPU. There will be an open-source SLAM implementation on the RVC3. Users are be able to run custom containarized apps on the ARM, which will allow other companies (which specialize in VIO/SLAM) to port their software stacks to our cameras and license it.
Several SLAM and localization projects that support OAK-D cameras:
ORB SLAM3 with an OAK-D and ROS1 by
@nimdaRTAB-Map recently (PR here) added support for depthai and OAK cameras (via ROS)
SpectacularAI’s SLAM with OAK-D - Free for non-commercial use
You can also check out our #slam channel on our Discord server, where there are tons of
useful information on how to perform SLAM with OAK cameras.
Syncing frames and IMU messages¶
For VIO/SLAM solutions, you would want to sync IMU messages with the middle of the exposure. For exposure timings and timestamps, see Frame capture graphs for details. See here for IMU/frame syncing demo.
Some more advance algorithms weight multiple IMU messages (before/after exposure) and interpolate the final value.
Got questions?
We’re always happy to help with development or other questions you might have.
