On-board localization (VIO) and SLAM (Simultaneous Localization And Mapping) on current OAK cameras (RVC2) aren’t yet supported.

Our upcoming Series 3 OAK cameras with RVC3 have Quad-core ARM A53 1.5GHz integrated into the VPU. There will be an open-source SLAM implementation on the RVC3. Users are be able to run custom containarized apps on the ARM, which will allow other companies (which specialize in VIO/SLAM) to port their software stacks to our cameras and license it.

Several SLAM and localization projects that support OAK-D cameras:

You can also check out our #slam channel on our Discord server, where there are tons of useful information on how to perform SLAM with OAK cameras.

Syncing frames and IMU messages

For VIO/SLAM solutions, you would want to sync IMU messages with the middle of the exposure. For exposure timings and timestamps, see Frame capture graphs for details. See here for IMU/frame syncing demo.

Some more advance algorithms weight multiple IMU messages (before/after exposure) and interpolate the final value.

Got questions?

We’re always happy to help with development or other questions you might have.