# xfeat Python API: `depthai_nodes.node.parsers.xfeat` ## Classes ### XFeatBaseParser Base parser class for parsing the output of the XFeat model. It is the parent class of the XFeatMonoParser and XFeatStereoParser classes. Raises * `ValueError`: If the number of output layers is not 3. * `ValueError`: If the original image size is not specified. * `ValueError`: If the input image size is not specified. * `ValueError`: If the maximum number of keypoints is not specified. * `ValueError`: If the output layer containing features is not specified. * `ValueError`: If the output layer containing keypoints is not specified. * `ValueError`: If the output layer containing heatmaps is not specified. #### Methods ##### init ```python def __init__(output_layer_feats: str = '', output_layer_keypoints: str = '', output_layer_heatmaps: str = '', original_size: tuple[float, float] = None, input_size: tuple[float, float] = (640, 352), max_keypoints: int = 4096): ``` Store tensor names and image geometry for XFeat decoding. Parameters * `output_layer_feats` (`str`): Feature-descriptor layer name. * `output_layer_keypoints` (`str`): Keypoint-logit layer name. * `output_layer_heatmaps` (`str`): Reliability heatmap layer name. * `original_size` (`tuple[float, float]`): Source image size as `(width, height)`. * `input_size` (`tuple[float, float]`): Model input size as `(width, height)`. * `max_keypoints` (`int`): Maximum number of feature points to keep. ##### build ```python def build(head_config: dict[str, Any]) -> XFeatBaseParser: ``` Configures the parser. Parameters * `head_config` (`dict[str, Any]`): The head configuration for the parser. Returns * `XFeatBaseParser`: The parser object with the head configuration set. ##### extractTensors ```python def extractTensors(output: dai.NNData) -> tuple[np.ndarray, np.ndarray, np.ndarray]: ``` Extracts the tensors from the output. It returns the features, keypoints, and heatmaps. It also handles the reshaping of the tensors by requesting the NCHW storage order. Parameters * `output` (`dai.NNData`): Output from the Neural Network node. Returns * `tuple[np.ndarray, np.ndarray, np.ndarray]`: Tuple of features, keypoints, and heatmaps. ##### reference_input.setter ```python def reference_input.setter(reference_input: dai.Node.Input | None): ``` Replace the stored XFeat input port. Parameters * `reference_input` (`dai.Node.Input | None`): Input port for the corresponding neural-network stream. ##### setInputSize ```python def setInputSize(input_size: tuple[int, int]): ``` Sets the input image size. Parameters * `input_size` (`tuple[int, int]`): Input image size. ##### setMaxKeypoints ```python def setMaxKeypoints(max_keypoints: int): ``` Sets the maximum number of keypoints to keep. Parameters * `max_keypoints` (`int`): Maximum number of keypoints. ##### setOriginalSize ```python def setOriginalSize(original_size: tuple[int, int]): ``` Sets the original image size. Parameters * `original_size` (`tuple[int, int]`): Original image size. ##### setOutputLayerFeats ```python def setOutputLayerFeats(output_layer_feats: str): ``` Sets the output layer containing features. Parameters * `output_layer_feats` (`str`): Name of the output layer containing features. ##### setOutputLayerHeatmaps ```python def setOutputLayerHeatmaps(output_layer_heatmaps: str): ``` Sets the output layer containing heatmaps. Parameters * `output_layer_heatmaps` (`str`): Name of the output layer containing heatmaps. ##### setOutputLayerKeypoints ```python def setOutputLayerKeypoints(output_layer_keypoints: str): ``` Sets the output layer containing keypoints. Parameters * `output_layer_keypoints` (`str`): Name of the output layer containing keypoints. ##### target_input.setter ```python def target_input.setter(target_input: dai.Node.Input | None): ``` Replace the stored XFeat input port. Parameters * `target_input` (`dai.Node.Input | None`): Input port for the corresponding neural-network stream. ##### validateParams ```python def validateParams(self): ``` Validates the parameters. #### Attributes ##### input ##### input_size Input image size. ##### max_keypoints Maximum number of keypoints to keep. ##### original_size Original image size. ##### output_layer_feats Name of the output layer containing features. ##### output_layer_heatmaps Name of the output layer containing heatmaps. ##### output_layer_keypoints Name of the output layer containing keypoints. ##### reference_input Returns the reference input. ##### target_input Returns the target input. ### XFeatMonoParser Parser class for parsing the output of the XFeat model. It can be used for parsing the output from one source (e.g. one camera). The reference frame can be set with trigger method. > **Note** > Emits `dai.TrackedFeatures` messages. TrackedFeatures message containing matched keypoints with the same ID. Raises * `ValueError`: If the original image size is not specified. * `ValueError`: If the input image size is not specified. * `ValueError`: If the maximum number of keypoints is not specified. * `ValueError`: If the output layer containing features is not specified. * `ValueError`: If the output layer containing keypoints is not specified. * `ValueError`: If the output layer containing heatmaps is not specified. #### Methods ##### init ```python def __init__(output_layer_feats: str = 'feats', output_layer_keypoints: str = 'keypoints', output_layer_heatmaps: str = 'heatmaps', original_size: tuple[float, float] = None, input_size: tuple[float, float] = (640, 352), max_keypoints: int = 4096): ``` Initializes the XFeatParser node. Parameters * `output_layer_feats` (`str`): Name of the output layer containing features. * `output_layer_keypoints` (`str`): Name of the output layer containing keypoints. * `output_layer_heatmaps` (`str`): Name of the output layer containing heatmaps. * `original_size` (`tuple[float, float]`): Original image size. * `input_size` (`tuple[float, float]`): Input image size. * `max_keypoints` (`int`): Maximum number of keypoints to keep. ##### compute ```python def compute(feats: np.ndarray, keypoints: np.ndarray, heatmaps: np.ndarray, *, resize_rate_w: float, resize_rate_h: float) -> dict[str, Any] | None: ``` Compute features for the first batch item. Parameters * `feats` (`np.ndarray`): Batched dense feature-descriptor tensor. * `keypoints` (`np.ndarray`): Model keypoint tensor. * `heatmaps` (`np.ndarray`): Model heatmap tensor. * `resize_rate_w` (`float`): Horizontal scale factor mapping model coordinates to the source image. * `resize_rate_h` (`float`): Vertical scale factor mapping model coordinates to the source image. Returns * `dict[str, Any] | None`: A dictionary containing keypoints, scores, and descriptors for the first image, or `None` when no candidates are found. ##### emit ```python def emit(output: dai.NNData, result: dict[str, Any] | None): ``` Match features against the stored reference and emit tracked points. > **Note** > Emits an empty message if no current features or no reference is available. Copies current timestamps and sequence number. A pending trigger replaces the stored reference with a non-None result after matching. Parameters * `output` (`dai.NNData`): Neural network output carrying tensors and source timestamps, sequence number, and optional image transformation. * `result` (`dict[str, Any] | None`): Current computed keypoints, scores, and descriptors, or `None`. ##### extract ```python def extract(output: dai.NNData) -> tuple[np.ndarray, np.ndarray, np.ndarray]: ``` Extract feature, keypoint, and heatmap tensors from one network result. Parameters * `output` (`dai.NNData`): Neural network output carrying tensors and source timestamps, sequence number, and optional image transformation. Returns * `tuple[np.ndarray, np.ndarray, np.ndarray]`: A tuple of feature descriptors, keypoint logits, and heatmaps, as produced by `extractTensors()`. ##### run ```python def run(self): ``` Read queued network outputs, parse them, and emit results while running. The pipeline invokes this processing loop. It exits when the input queue closes or the node stops. ##### setTrigger ```python def setTrigger(self): ``` Sets the trigger to set the reference frame. #### Attributes ##### input_size Input image size. ##### max_keypoints Maximum number of keypoints to keep. ##### original_size Original image size. ##### output_layer_feats Name of the output layer containing features. ##### output_layer_heatmaps Name of the output layer containing heatmaps. ##### output_layer_keypoints Name of the output layer containing keypoints. ##### previous_results Previous results from the model. Previous results are used to match keypoints between two frames. ##### trigger Trigger to set the reference frame. ### XFeatStereoParser Parser class for parsing the output of the XFeat model. It can be used for parsing the output from two sources (e.g. two cameras - left and right). > **Note** > Emits `dai.TrackedFeatures` messages. TrackedFeatures message containing matched keypoints with the same ID. Raises * `ValueError`: If the original image size is not specified. * `ValueError`: If the input image size is not specified. * `ValueError`: If the maximum number of keypoints is not specified. * `ValueError`: If the output layer containing features is not specified. * `ValueError`: If the output layer containing keypoints is not specified. * `ValueError`: If the output layer containing heatmaps is not specified. #### Methods ##### init ```python def __init__(output_layer_feats: str = 'feats', output_layer_keypoints: str = 'keypoints', output_layer_heatmaps: str = 'heatmaps', original_size: tuple[float, float] = None, input_size: tuple[float, float] = (640, 352), max_keypoints: int = 4096): ``` Initializes the XFeatParser node. Parameters * `output_layer_feats` (`str`): Name of the output layer containing features. * `output_layer_keypoints` (`str`): Name of the output layer containing keypoints. * `output_layer_heatmaps` (`str`): Name of the output layer containing heatmaps. * `original_size` (`tuple[float, float]`): Original image size. * `input_size` (`tuple[float, float]`): Input image size. * `max_keypoints` (`int`): Maximum number of keypoints to keep. ##### compute ```python def compute(reference_tensors: tuple[np.ndarray, np.ndarray, np.ndarray], target_tensors: tuple[np.ndarray, np.ndarray, np.ndarray], *, resize_rate_w: float, resize_rate_h: float) -> dict[str, tuple[np.ndarray, np.ndarray] | str | None]: ``` Compute and match features between reference and target tensors. Parameters * `reference_tensors` (`tuple[np.ndarray, np.ndarray, np.ndarray]`): Reference feature, keypoint, and heatmap tensors. * `target_tensors` (`tuple[np.ndarray, np.ndarray, np.ndarray]`): Target feature, keypoint, and heatmap tensors. * `resize_rate_w` (`float`): Horizontal scale factor applied to keypoint coordinates. * `resize_rate_h` (`float`): Vertical scale factor applied to keypoint coordinates. Returns * `dict[str, tuple[np.ndarray, np.ndarray] | str | None]`: Dictionary with `status` (`matched`, `reference_missing`, or `target_missing`) and `match_result` (paired point arrays, or `None`). ##### emit ```python def emit(reference_output: dai.NNData, target_output: dai.NNData, result: dict[str, tuple[np.ndarray, np.ndarray] | str | None]): ``` Emit matched reference and target points, or an empty feature message. Parameters * `reference_output` (`dai.NNData`): Supplies the sequence number and, if reference features are missing, timestamps. * `target_output` (`dai.NNData`): Supplies timestamps for matches and missing target features. * `result` (`dict[str, tuple[np.ndarray, np.ndarray] | str | None]`): Status and paired points returned by `compute()`. ##### extract ```python def extract(reference_output: dai.NNData, target_output: dai.NNData) -> tuple[tuple[np.ndarray, np.ndarray, np.ndarray], tuple[np.ndarray, np.ndarray, np.ndarray]]: ``` Extract matching tensors from both network results. Parameters * `reference_output` (`dai.NNData`): Reference-image network output. * `target_output` (`dai.NNData`): Target-image network output. Returns * `tuple[tuple[np.ndarray, np.ndarray, np.ndarray], tuple[np.ndarray, np.ndarray, np.ndarray]]`: Reference and target tuples, each containing features, keypoint logits, and heatmaps. ##### run ```python def run(self): ``` Read queued network outputs, parse them, and emit results while running. The pipeline invokes this processing loop. It exits when the input queue closes or the node stops. #### Attributes ##### input_size Input image size. ##### max_keypoints Maximum number of keypoints to keep. ##### original_size Original image size. ##### out Parser sends the processed network results to this output in a form of DepthAI message. It is a linking point from which the processed network results are retrieved. ##### output_layer_feats Name of the output layer from which the features are extracted. ##### output_layer_heatmaps Name of the output layer from which the heatmaps are extracted. ##### output_layer_keypoints Name of the output layer from which the keypoints are extracted. ##### reference_input Node's input. It is a linking point to which the Neural Network's output is linked. It accepts the output of the Neural Network node. ##### target_input Node's input. It is a linking point to which the Neural Network's output is linked. It accepts the output of the Neural Network node.