# cutmix Python API: `luxonis_ml.data.augmentations.custom.cutmix` ## Classes ### CutMix Batch-based augmentation that patches one image into another. CutMix samples a rectangular region from the second image and pastes it into the first image. The rectangle area is derived from a mixing coefficient sampled from Beta(α, α). If the images have different sizes, the second image and its labels are resized to match the first image before the patch is copied. Bounding boxes from the first image are kept if their visible area (original area minus the intersection with the patch) is at least `bbox_min_visibility` times their original area; `occluded_bbox_strategy` decides whether the survivors keep their full extent or are clipped to the part the patch leaves alone. Bounding boxes from the second image are clipped to the patch region. Keypoints from the first image that fall inside the patch are marked invisible, while keypoints from the second image outside the patch are marked invisible. > **See Also** > [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://arxiv.org/abs/1905.04899). #### Methods ##### init ```python def __init__(alpha: float = 1.0, keep_aspect_ratio: bool = True, bbox_min_visibility: float = 0.5, occluded_bbox_strategy: Literal['keep', 'clip'] = 'keep', p: float = 0.5): ``` Create a CutMix augmentation. Parameters * `alpha` (`float`): Positive shape parameter for the symmetric Beta(α, α) distribution that determines the patch size. `1.0` samples uniformly; values below `1.0` favor very small or very large patches, while values above `1.0` favor mid-sized patches. * `keep_aspect_ratio` (`bool`): Whether to preserve the second image's aspect ratio when resizing. * `bbox_min_visibility` (`float`): Minimum fraction of a first-image bounding box that must remain visible (outside the patch) for the box to be kept. Must lie in [0, 1]. * `occluded_bbox_strategy` (`Literal['keep', 'clip']`): What to do with the first-image boxes the patch partly covers but does not hide past `bbox_min_visibility`. * `"keep"` leaves them at their original extent, on the grounds that the object still occupies the whole box and is merely covered up. This is what the reference implementations do. * `"clip"` shrinks them to the largest part of themselves the patch does not reach, so that no box edge sits on top of the patch. A patch cutting a box in two keeps the larger of the two remaining sides. Which box survives is decided by `bbox_min_visibility` either way; this only changes the extent of the survivors. Note that `"clip"` can leave a visible keypoint outside its own, now smaller, box. * `p` (`float`): Probability of applying the transform. Raises * `ValueError`: If `alpha` is not a finite positive number, `bbox_min_visibility` is outside [0, 1], or `occluded_bbox_strategy` is not one of `"keep"` and `"clip"`. ##### apply ```python def apply(image_batch: list[np.ndarray], image_shapes: list[tuple[int, int]], x1: int, y1: int, x2: int, y2: int, **_) -> np.ndarray: ``` Apply CutMix to a batch of images. Parameters * `image_batch` (`list[np.ndarray]`): Images to transform. Each image should be of shape (H, W, C) or (H, W). * `image_shapes` (`list[tuple[int, int]]`): Shapes of the original images. * `x1` (`int`): Left edge of the patch in the first image. * `y1` (`int`): Top edge of the patch in the first image. * `x2` (`int`): Right edge of the patch in the first image. * `y2` (`int`): Bottom edge of the patch in the first image. * `**_` Returns * `np.ndarray`: A single image of shape (Hout, Wout, C) or (Hout, Wout) where the rectangle [y1 : y2, x1 : x2] is replaced with the corresponding region of the (resized) second image. ##### apply_to_array ```python def apply_to_array(array_batch: list[np.ndarray], x1: int, y1: int, x2: int, y2: int, **_) -> np.ndarray: ``` Apply CutMix to arbitrary arrays. ##### apply_to_bboxes ```python def apply_to_bboxes(bboxes_batch: list[np.ndarray], image_shapes: list[tuple[int, int]], x1: int, y1: int, x2: int, y2: int, **_) -> np.ndarray: ``` Apply CutMix to bounding boxes. Bounding boxes from the first image are kept when their visible area (original area minus intersection with the patch) is at least `bbox_min_visibility` of the original area. Bounding boxes from the second image are clipped to the patch region. Boxes that do not survive are collapsed to zero area rather than removed, so that the returned rows stay aligned with the instance masks, keypoints and metadata concatenated alongside them. [BatchCompose](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/data/augmentations/batch_compose.md) discards the collapsed rows together with their associated labels once the batch has been merged. Parameters * `bboxes_batch` (`list[np.ndarray]`): Bounding boxes to transform, in normalized Albumentations format. * `image_shapes` (`list[tuple[int, int]]`): Original image shapes. * `x1` (`int`): Left edge of the patch in the first image. * `y1` (`int`): Top edge of the patch in the first image. * `x2` (`int`): Right edge of the patch in the first image. * `y2` (`int`): Bottom edge of the patch in the first image. * `**_` Returns * `np.ndarray`: Concatenated bounding boxes from both images. ##### apply_to_classification ```python def apply_to_classification(classification_batch: list[np.ndarray], x1: int, y1: int, x2: int, y2: int, **_) -> np.ndarray: ``` Apply CutMix to classification labels. ##### apply_to_instance_mask ```python def apply_to_instance_mask(masks_batch: list[np.ndarray], image_shapes: list[tuple[int, int]], x1: int, y1: int, x2: int, y2: int, **_) -> np.ndarray: ``` Apply CutMix to instance segmentation masks. Parameters * `masks_batch` (`list[np.ndarray]`): Masks to transform. Each mask should be of shape (H, W, N), where N is the number of instances. * `image_shapes` (`list[tuple[int, int]]`): Shapes of the original images. * `x1` (`int`): Left edge of the patch in the first image. * `y1` (`int`): Top edge of the patch in the first image. * `x2` (`int`): Right edge of the patch in the first image. * `y2` (`int`): Bottom edge of the patch in the first image. * `**_` Returns * `np.ndarray`: Concatenated instance masks of shape (Hout, Wout, N1 + N2). First-image instances have their patch region cleared; second-image instances have everything outside the patch cleared. ##### apply_to_keypoints ```python def apply_to_keypoints(keypoints_batch: list[np.ndarray], image_shapes: list[tuple[int, int]], x1: int, y1: int, x2: int, y2: int, **_) -> np.ndarray: ``` Apply CutMix to keypoints. Keypoints from the first image that fall inside the patch are marked invisible; keypoints from the second image outside the patch are marked invisible. Parameters * `keypoints_batch` (`list[np.ndarray]`): Keypoints to transform. * `image_shapes` (`list[tuple[int, int]]`): Original image shapes. * `x1` (`int`): Left edge of the patch in the first image. * `y1` (`int`): Top edge of the patch in the first image. * `x2` (`int`): Right edge of the patch in the first image. * `y2` (`int`): Bottom edge of the patch in the first image. * `**_` Returns * `np.ndarray`: Concatenated keypoints from both images with visibility flags updated according to their position relative to the patch. ##### apply_to_mask ```python def apply_to_mask(masks_batch: list[np.ndarray], image_shapes: list[tuple[int, int]], x1: int, y1: int, x2: int, y2: int, **_) -> np.ndarray: ``` Apply CutMix to semantic segmentation masks. Parameters * `masks_batch` (`list[np.ndarray]`): Masks to transform. Each mask should be of shape (H, W, C) or (H, W). * `image_shapes` (`list[tuple[int, int]]`): Shapes of the original images. * `x1` (`int`): Left edge of the patch in the first image. * `y1` (`int`): Top edge of the patch in the first image. * `x2` (`int`): Right edge of the patch in the first image. * `y2` (`int`): Bottom edge of the patch in the first image. * `**_` Returns * `np.ndarray`: A single semantic segmentation mask of shape (Hout, Wout, C) where the patch region comes from the second (resized) mask and the rest comes from the first mask. ##### apply_to_metadata ```python def apply_to_metadata(metadata_batch: list[np.ndarray], x1: int, y1: int, x2: int, y2: int, **_) -> np.ndarray: ``` Apply CutMix to metadata arrays. ##### get_params ```python def get_params(self) -> dict[str, Any]: ``` Sample the CutMix mixing coefficient. ##### get_params_dependent_on_data ```python def get_params_dependent_on_data(params: dict[str, Any], data: dict[str, Any]) -> dict[str, Any]: ``` Sample a CutMix rectangle from the first image shape.