YoloSpatialDetectionNetwork

Spatial detection for the Yolo NN. It is similar to a combination of the YoloDetectionNetwork and SpatialLocationCalculator.

How to place it

pipeline = dai.Pipeline()
yoloSpatial = pipeline.createYoloSpatialDetectionNetwork()
dai::Pipeline pipeline;
auto yoloSpatial = pipeline.create<dai::node::YoloSpatialDetectionNetwork>();

Inputs and Outputs

               ┌───────────────────┐
input          │                   │       passthrough
──────────────►│-------------------├─────────────────►
               │     Yolo          │               out
               │     Spatial       ├─────────────────►
               │     Detection     │boundingBoxMapping
               │     Network       ├─────────────────►
inputDepth     │                   │  passthroughDepth
──────────────►│-------------------├─────────────────►
               └───────────────────┘

Message types

Usage

pipeline = dai.Pipeline()
yoloSpatial = pipeline.createYoloSpatialDetectionNetwork()
yoloSpatial.setBlobPath(nnBlobPath)

# Spatial detection specific parameters
yoloSpatial.setConfidenceThreshold(0.5)
yoloSpatial.input.setBlocking(False)
yoloSpatial.setBoundingBoxScaleFactor(0.5)
yoloSpatial.setDepthLowerThreshold(100) # Min 10 centimeters
yoloSpatial.setDepthUpperThreshold(5000) # Max 5 meters

# Yolo specific parameters
yoloSpatial.setNumClasses(80)
yoloSpatial.setCoordinateSize(4)
yoloSpatial.setAnchors(np.array([10,14, 23,27, 37,58, 81,82, 135,169, 344,319]))
yoloSpatial.setAnchorMasks({ "side26": np.array([1,2,3]), "side13": np.array([3,4,5]) })
yoloSpatial.setIouThreshold(0.5)
dai::Pipeline pipeline;
auto yoloSpatial = pipeline.create<dai::node::YoloSpatialDetectionNetwork>();
yoloSpatial->setBlobPath(nnBlobPath);

// Spatial detection specific parameters
yoloSpatial->setConfidenceThreshold(0.5f);
yoloSpatial->input.setBlocking(false);
yoloSpatial->setBoundingBoxScaleFactor(0.5);
yoloSpatial->setDepthLowerThreshold(100); // Min 10 centimeters
yoloSpatial->setDepthUpperThreshold(5000); // Max 5 meters

// yolo specific parameters
yoloSpatial->setNumClasses(80);
yoloSpatial->setCoordinateSize(4);
yoloSpatial->setAnchors({10, 14, 23, 27, 37, 58, 81, 82, 135, 169, 344, 319});
yoloSpatial->setAnchorMasks({{"side13", {3, 4, 5}}, {"side26", {1, 2, 3}}});
yoloSpatial->setIouThreshold(0.5f);

Examples of functionality

Reference

class depthai.YoloSpatialDetectionNetwork

YoloSpatialDetectionNetwork node. (tiny)Yolov3/v4 based network with spatial location data.

class Connection

Connection between an Input and Output

class Id

Node identificator. Unique for every node on a single Pipeline

Properties

alias of depthai.SpatialDetectionNetworkProperties

property boundingBoxMapping

Outputs mapping of detected bounding boxes relative to depth map

Suitable for when displaying remapped bounding boxes on depth frame

getAssets(self: depthai.Node) → List[depthai.Asset]

Retrieves all nodes assets

getInputs(self: depthai.Node) → List[dai::Node::Input]

Retrieves all nodes inputs

getName(self: depthai.Node)str

Retrieves nodes name

getNumInferenceThreads(self: depthai.NeuralNetwork)int

How many inference threads will be used to run the network

Returns

Number of threads, 0, 1 or 2. Zero means AUTO

getOutputs(self: depthai.Node) → List[dai::Node::Output]

Retrieves all nodes outputs

property id

Id of node

property input

Input message with data to be infered upon Default queue is blocking with size 5

property inputDepth

Input message with depth data used to retrieve spatial information about detected object Default queue is non-blocking with size 4

property out

Outputs ImgDetections message that carries parsed detection results.

property passthrough

Passthrough message on which the inference was performed.

Suitable for when input queue is set to non-blocking behavior.

property passthroughDepth

Passthrough message for depth frame on which the spatial location calculation was performed.

Suitable for when input queue is set to non-blocking behavior.

setAnchorMasks(self: depthai.YoloSpatialDetectionNetwork, anchorMasks: Dict[str, List[int]])None

Set anchor masks

setAnchors(self: depthai.YoloSpatialDetectionNetwork, anchors: List[float])None

Set anchors

setBlobPath(self: depthai.NeuralNetwork, path: str)None

Load network blob into assets and use once pipeline is started.

Throws if file doesn’t exist or isn’t a valid network blob.

Parameter path:

Path to network blob

setBoundingBoxScaleFactor(self: depthai.SpatialDetectionNetwork, scaleFactor: float)None

Specifies scale factor for detected bounding boxes.

Parameter scaleFactor:

Scale factor must be in the interval (0,1].

setConfidenceThreshold(self: depthai.DetectionNetwork, thresh: float)None

Specifies confidence threshold at which to filter the rest of the detections.

Parameter thresh:

Detection confidence must be greater than specified threshold to be added to the list

setCoordinateSize(self: depthai.YoloSpatialDetectionNetwork, coordinates: int)None

Set coordianate size

setDepthLowerThreshold(self: depthai.SpatialDetectionNetwork, lowerThreshold: int)None

Specifies lower threshold in milimeters for depth values which will used to calculate spatial data

Parameter lowerThreshold:

LowerThreshold must be in the interval [0,upperThreshold] and less than upperThreshold.

setDepthUpperThreshold(self: depthai.SpatialDetectionNetwork, upperThreshold: int)None

Specifies upper threshold in milimeters for depth values which will used to calculate spatial data

Parameter upperThreshold:

UpperThreshold must be in the interval (lowerThreshold,65535].

setIouThreshold(self: depthai.YoloSpatialDetectionNetwork, thresh: float)None

Set Iou threshold

setNumClasses(self: depthai.YoloSpatialDetectionNetwork, numClasses: int)None

Set num classes

setNumInferenceThreads(self: depthai.NeuralNetwork, numThreads: int)None

How many threads should the node use to run the network.

Parameter numThreads:

Number of threads to dedicate to this node

setNumNCEPerInferenceThread(self: depthai.NeuralNetwork, numNCEPerThread: int)None

How many Neural Compute Engines should a single thread use for inference

Parameter numNCEPerThread:

Number of NCE per thread

setNumPoolFrames(self: depthai.NeuralNetwork, numFrames: int)None

Specifies how many frames will be avilable in the pool

Parameter numFrames:

How many frames will pool have

class dai::node::YoloSpatialDetectionNetwork : public dai::node::SpatialDetectionNetwork

YoloSpatialDetectionNetwork node. (tiny)Yolov3/v4 based network with spatial location data.

Public Functions

YoloSpatialDetectionNetwork(const std::shared_ptr<PipelineImpl> &par, int64_t nodeId)
void setNumClasses(const int numClasses)

Set num classes.

void setCoordinateSize(const int coordinates)

Set coordianate size.

void setAnchors(std::vector<float> anchors)

Set anchors.

void setAnchorMasks(std::map<std::string, std::vector<int>> anchorMasks)

Set anchor masks.

void setIouThreshold(float thresh)

Set Iou threshold.

Got questions?

We’re always happy to help with code or other questions you might have.