# composite_scheduler

Python API: `luxonis_train.schedulers.composite_scheduler`

The schedulers that step the member schedulers of a
[CompositeOptimizer](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/optimizers/composite_optimizer.md)
together, and a helper that changes a base learning rate.

## Classes

### CompositeLRScheduler

One scheduler that steps the member schedulers of a
[CompositeOptimizer](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/optimizers/composite_optimizer.md).

Each member scheduler belongs to one inner optimizer of the composite, so its `base_lrs` match the groups of that optimizer.
Lightning needs `scheduler.optimizer` to be the optimizer that `configure_optimizers` returns. This class gives Lightning one
scheduler whose `optimizer` is the composite.

The constructor does not call `LRScheduler.__init__`. The members already made their initial step and patched the `step` counters
of their optimizers.

> **Example**
> ```pycon
>>> from torch import nn
>>> from torch.optim import SGD
>>> from torch.optim.lr_scheduler import StepLR
>>> first = SGD(nn.Linear(2, 2).parameters(), lr=0.1)
>>> second = SGD(nn.Linear(2, 2).parameters(), lr=1.0)
>>> composite = CompositeOptimizer([first, second])
>>> scheduler = CompositeLRScheduler(
...     composite,
...     [StepLR(first, 1, gamma=0.5), StepLR(second, 1, gamma=0.1)],
... )
>>> composite.step()
>>> scheduler.step()
>>> [round(lr, 3) for lr in scheduler.get_last_lr()]
[0.05, 0.1]
```

#### Methods

##### init

```python
def __init__(composite: CompositeOptimizer, members: Sequence[LRScheduler]):
```

Wrap the member schedulers.

The scheduler sets `last_epoch` to `0`.

Parameters

 * `composite` (`CompositeOptimizer`): The optimizer that Lightning receives. The scheduler stores it as `optimizer`.
 * `members` (`Sequence[LRScheduler]`): The member schedulers. Each one belongs to an inner optimizer of `composite`.

##### get_last_lr

```python
def get_last_lr(self) -> list[float | Tensor]:
```

Return the last learning rates of all member schedulers.

Returns

 * `list[float | Tensor]`: The `get_last_lr()` values of the members, joined in member order. The list has one value for each group of each member optimizer.

##### load_state_dict

```python
def load_state_dict(state_dict: dict[str, Any]):
```

Load a state into the scheduler and its members.

The method restores `last_epoch` and loads each entry of `"members"` into the member at the same position.

Parameters

 * `state_dict` (`dict[str, Any]`): A state that the [CompositeLRScheduler.state_dict](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/schedulers/composite_scheduler.md) method returned.

Raises

 * `ValueError`: If `"version"` is not `1`. Also if the number of member states differs from the number of members.

##### state_dict

```python
def state_dict(self) -> dict[str, Any]:
```

Return the state of the scheduler and its members.

Returns

 * `dict[str, Any]`: A dictionary with these keys. * `"version"` is `1`.
    * `"last_epoch"` holds the `last_epoch` counter.
    * `"members"` holds the `state_dict()` of each member.

##### step

```python
def step(epoch: int | None = None):
```

Step every member scheduler once.

The method adds `1` to `last_epoch`, and then calls `step()` of each member in order.

Parameters

 * `epoch` (`int | None`): The epoch that Lightning can pass. The method ignores it.

#### Attributes

##### last_epoch

##### members

The member schedulers, in the order of the constructor.

##### optimizer

### CompositeReduceLROnPlateau

One `ReduceLROnPlateau` that steps the plateau member schedulers of a [CompositeOptimizer](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/optimizers/composite_optimizer.md).

The class does the job of [CompositeLRScheduler](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/schedulers/composite_scheduler.md) for members that monitor a value. Each member is a `ReduceLROnPlateau` over one inner optimizer of the composite. Lightning passes the monitored value as the first argument of [step](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/schedulers/composite_scheduler.md).

The constructor does not call `ReduceLROnPlateau.__init__`. The members hold the plateau settings, such as `mode` and `patience`.

#### Methods

##### init

```python
def __init__(composite: CompositeOptimizer, members: Sequence[ReduceLROnPlateau]):
```

Wrap the member schedulers.

Parameters

 * `composite` (`CompositeOptimizer`): The optimizer that Lightning receives. The scheduler stores it as `optimizer`.
 * `members` (`Sequence[ReduceLROnPlateau]`): The member schedulers. Each one belongs to an inner optimizer of `composite`.

##### load_state_dict

```python
def load_state_dict(state_dict: dict[str, Any]):
```

Load a state into the member schedulers.

The method loads each entry of `"members"` into the member at the same position.

Parameters

 * `state_dict` (`dict[str, Any]`): A state that the [CompositeReduceLROnPlateau.state_dict](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/schedulers/composite_scheduler.md) method returned.

Raises

 * `ValueError`: If `"version"` is not `1`. Also if the number of member states differs from the number of members.

##### state_dict

```python
def state_dict(self) -> dict[str, Any]:
```

Return the states of the member schedulers.

Returns

 * `dict[str, Any]`: A dictionary with the keys `"version"`, which is `1`, and `"members"`, which holds the `state_dict()` of each member.

##### step

```python
def step(metrics: Any, epoch: int | None = None):
```

Pass the monitored value to every member scheduler.

Each member decides on its own whether to reduce the learning rate of its optimizer.

Parameters

 * `metrics` (`Any`): The monitored value, such as the validation loss.
 * `epoch` (`int | None`): The epoch that Lightning can pass. The method ignores it.

#### Attributes

##### members

The member schedulers, in the order of the constructor.

##### optimizer

## Functions

### rebase_scheduler_lr

```python
def rebase_scheduler_lr(scheduler: LRScheduler | ReduceLROnPlateau, index: int, lr: float):
```

Set the base learning rate of one parameter group in a scheduler.

The function sets `base_lrs[index]` of `scheduler`. For a `SequentialLR` or a `ChainedScheduler`, it sets the entry in each child scheduler, and in the children of a child. A scheduler without `base_lrs`, such as a `ReduceLROnPlateau`, does not change. The function does not change the `lr` or the `initial_lr` of the group. [TrainingPlanRuntime.set_group_base_lr](https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-train/luxonis-train-api-reference/lightning/training_plan.md) sets them.

> **Example**
> ```pycon
>>> from torch import nn
>>> from torch.optim import SGD
>>> from torch.optim.lr_scheduler import StepLR
>>> optimizer = SGD(nn.Linear(2, 2).parameters(), lr=0.1)
>>> scheduler = StepLR(optimizer, step_size=1)
>>> rebase_scheduler_lr(scheduler, 0, 0.5)
>>> scheduler.base_lrs
[0.5]
>>> optimizer.param_groups[0]["lr"]
0.1
```

Parameters

 * `scheduler` (`LRScheduler | ReduceLROnPlateau`): The scheduler.
 * `index` (`int`): The index of the parameter group in the optimizer of the scheduler.
 * `lr` (`float`): The new base learning rate.