# fastsam Python API: `depthai_nodes.node.parsers.utils.fastsam` ## Functions ### adjust_bboxes_to_image_border ```python def adjust_bboxes_to_image_border(boxes: np.ndarray, image_shape: tuple[int, int], threshold: int = 20) -> np.ndarray: ``` Source: [https://github.com/ultralytics/ultralytics/blob/8094df3c474fe83d874fd83da19f704205e1eef3/ultralytics/models/fastsam/utils.py#L6](https://github.com/ultralytics/ultralytics/blob/8094df3c474fe83d874fd83da19f704205e1eef3/ultralytics/models/fastsam/utils.py#L6) (Ultralytics) Adjust bounding boxes to stick to image border if they are within a certain threshold. Parameters * `boxes` (`np.ndarray`): Bounding boxes * `image_shape` (`tuple[int, int]`): Image shape * `threshold` (`int`): Pixel threshold Returns * `np.ndarray`: Adjusted bounding boxes ### bbox_iou ```python def bbox_iou(box1: np.ndarray, boxes: np.ndarray, iou_thres: float = 0.9, image_shape: tuple[int, int] = (640, 640), raw_output: bool = False) -> np.ndarray: ``` Source: [https://github.com/ultralytics/ultralytics/blob/8094df3c474fe83d874fd83da19f704205e1eef3/ultralytics/models/fastsam/utils.py#L30](https://github.com/ultralytics/ultralytics/blob/8094df3c474fe83d874fd83da19f704205e1eef3/ultralytics/models/fastsam/utils.py#L30) (Ultralytics - rewritten to numpy) Compute the Intersection-Over-Union of a bounding box with respect to an array of other bounding boxes. Parameters * `box1` (`np.ndarray`): Array of shape (4, ) representing a single bounding box. * `boxes` (`np.ndarray`): Array of shape (n, 4) representing multiple bounding boxes. * `iou_thres` (`float`): IoU threshold * `image_shape` (`tuple[int, int]`): Image shape (height, width) * `raw_output` (`bool`): If True, return the raw IoU values instead of the indices Returns * `np.ndarray`: Indices of boxes with IoU > thres, or the raw IoU values if raw_output is True ### box_prompt ```python def box_prompt(masks: np.ndarray, bbox: tuple[int, int, int, int], orig_shape: tuple[int, int]) -> np.ndarray: ``` Modifies the bounding box properties and calculates IoU between masks and bounding box. Source: [https://github.com/ultralytics/ultralytics/blob/8094df3c474fe83d874fd83da19f704205e1eef3/ultralytics/models/fastsam/prompt.py#L284](https://github.com/ultralytics/ultralytics/blob/8094df3c474fe83d874fd83da19f704205e1eef3/ultralytics/models/fastsam/prompt.py#L284) Modified so it uses numpy instead of torch. Parameters * `masks` (`np.ndarray`): The resulting masks of the FastSAM model * `bbox` (`tuple[int, int, int, int]`): The prompt bounding box coordinates * `orig_shape` (`tuple[int, int]`): The original shape of the image Returns * `np.ndarray`: The modified masks ### build_mask_coeffs ```python def build_mask_coeffs(parsed_results: np.ndarray, masks_outputs_values: list[np.ndarray], protos_len: int) -> np.ndarray: ``` Gather mask coefficients for all detections, grouped by head. Parameters * `parsed_results` (`np.ndarray`): FastSAM decoded outputs * `masks_outputs_values` (`list[np.ndarray]`): Model mask outputs * `protos_len` (`int`): Number of protos Returns * `np.ndarray` ### compute_fastsam_mask ```python def compute_fastsam_mask(outputs_values: list[np.ndarray], masks_outputs_values: list[np.ndarray], protos_output: np.ndarray, protos_len: int, *, conf_threshold: float, n_classes: int, iou_threshold: float, mask_conf: float, prompt: str, points: tuple[int, int] | None, point_label: int | None, bbox: tuple[int, int, int, int] | None) -> tuple[np.ndarray, int]: ``` Decode FastSAM outputs and apply the configured mask prompt. Parameters * `outputs_values` (`list[np.ndarray]`): Detection tensors ordered by output head. * `masks_outputs_values` (`list[np.ndarray]`): Mask coefficient tensors ordered to match the detection heads. * `protos_output` (`np.ndarray`): Batched prototype tensor with shape `(1, channels, height, width)`. * `protos_len` (`int`): Number of prototype channels used by each mask coefficient vector. * `conf_threshold` (`float`): Minimum detection confidence used to filter candidates. * `n_classes` (`int`): Number of object classes encoded in the detection tensors. * `iou_threshold` (`float`): Intersection-over-union threshold for non-maximum suppression. * `mask_conf` (`float`): Probability threshold used to binarize mask logits. * `prompt` (`str`): Mask selection mode: `"everything"`, `"bbox"`, or `"point"`. * `points` (`tuple[int, int] | None`): Prompt point in image pixel coordinates for point selection. * `point_label` (`int | None`): Point-prompt label used to include or exclude matching masks. * `bbox` (`tuple[int, int, int, int] | None`): Bounding-box prompt in image pixel coordinates. Returns * `tuple[np.ndarray, int]`: A pair of the merged instance mask and the number of selected masks. No selected masks produces a background mask with value -1 and count 0. ### decode_fastsam_output ```python def decode_fastsam_output(outputs: list[np.ndarray], strides: list[int], anchors: list[np.ndarray | None], img_shape: tuple[int, int], conf_thres: float = 0.5, iou_thres: float = 0.45, num_classes: int = 1) -> np.ndarray: ``` Decode the output of the FastSAM model. Parameters * `outputs` (`list[np.ndarray]`): List of FastSAM outputs * `strides` (`list[int]`): List of strides * `anchors` (`list[np.ndarray | None]`): List of anchors * `img_shape` (`tuple[int, int]`): Image shape * `conf_thres` (`float`): Confidence threshold * `iou_thres` (`float`): IoU threshold * `num_classes` (`int`): Number of classes Returns * `np.ndarray`: NMS output ### format_results ```python def format_results(bboxes: np.ndarray, masks: np.ndarray, filter: int = 0) -> list[dict[str, Any]]: ``` Formats detection results into list of annotations each containing ID, segmentation, bounding box, score and area. Source: [https://github.com/ultralytics/ultralytics/blob/8094df3c474fe83d874fd83da19f704205e1eef3/ultralytics/models/fastsam/prompt.py#L59](https://github.com/ultralytics/ultralytics/blob/8094df3c474fe83d874fd83da19f704205e1eef3/ultralytics/models/fastsam/prompt.py#L59) Parameters * `bboxes` (`np.ndarray`): The bounding boxes of the detected objects * `masks` (`np.ndarray`): The masks of the detected objects * `filter` (`int`): The filter value Returns * `list[dict[str, Any]]`: The formatted annotations ### merge_masks ```python def merge_masks(masks: np.ndarray) -> np.ndarray: ``` Merge masks to a 2D array where each object is represented by a unique label. Parameters * `masks` (`np.ndarray`): 3D array of masks Returns * `np.ndarray`: 2D array of masks ### point_prompt ```python def point_prompt(bboxes: np.ndarray, masks: np.ndarray, points: list[tuple[int, int]], pointlabel: list[int], orig_shape: tuple[int, int]) -> np.ndarray: ``` Adjusts points on detected masks based on user input and returns the modified results. Source: [https://github.com/ultralytics/ultralytics/blob/8094df3c474fe83d874fd83da19f704205e1eef3/ultralytics/models/fastsam/prompt.py#L317](https://github.com/ultralytics/ultralytics/blob/8094df3c474fe83d874fd83da19f704205e1eef3/ultralytics/models/fastsam/prompt.py#L317) Modified so it uses numpy instead of torch. Parameters * `bboxes` (`np.ndarray`): The bounding boxes of the detected objects * `masks` (`np.ndarray`): The masks of the detected objects * `points` (`list[tuple[int, int]]`): The points to adjust * `pointlabel` (`list[int]`): The point labels * `orig_shape` (`tuple[int, int]`): The original shape of the image Returns * `np.ndarray`: The modified masks ### process_masks ```python def process_masks(parsed_results: np.ndarray, mask_coeffs: np.ndarray, protos: np.ndarray, orig_shape: tuple[int, int], mask_conf: float) -> np.ndarray: ``` Process output into full-size masks for all detections. Parameters * `parsed_results` (`np.ndarray`): FastSAM decoded outputs * `mask_coeffs` (`np.ndarray`): Mask coefficients * `protos` (`np.ndarray`): Protos from model output * `orig_shape` (`tuple[int, int]`): Input shape of the model * `mask_conf` (`float`): Mask confidence Returns * `np.ndarray`