# benchmark

Python API: `modelconverter.platforms.rvc4.benchmark`

Throughput and latency measurements on RVC4 devices.

An RVC4 model can be measured in two ways, and
[RVC4Benchmark](https://docs.luxonis.com/software-v3/ai-inference/conversion/rvc-conversion/offline/modelconverter/modelconverter-api-reference/platforms/rvc4/benchmark.md)
covers both: a DepthAI pipeline that feeds a neural network node as fast as it will accept data, and SNPE's own
`snpe-parallel-run`, which is executed on the device over ADB or SSH against inputs staged there beforehand. Power, DSP, memory
and CPU usage can be sampled alongside either of them.

## Classes

### InputSpec

Description of one model input to generate benchmark data for.

The shape is required: a random tensor cannot be made for an input whose shape is unknown.

#### Attributes

##### data_type

Data type of the input tensor.

##### name

Name of the input tensor.

##### shape

Shape of the input tensor.

### RVC4Benchmark

Benchmark of an NN archive, a `.dlc` or a HubAI slug on RVC4.

Which of the two backends is used is decided by the `dai_benchmark` option: a DepthAI pipeline, which also reports per-inference
latency but only accepts a `.tar.xz` archive or a slug, or `snpe-parallel-run` run over the device handler, which only reports
throughput and takes a bare `.dlc` as well. Both need the input tensor specifications, read from the DLC with `snpe-dlc-info` and
falling back to the NN archive when that is unavailable.

#### Methods

##### benchmark

```python
def benchmark(configuration: Configuration) -> Result:
```

Run one benchmark of the model on an RVC4 device.

Connects to the device, starts the device monitor if it was asked for, and hands the run to the DepthAI or the SNPE backend.
Whatever the SNPE backend put on the device is removed again afterwards.

> **Warning**
> `configuration` is modified in place: the keys the chosen backend does not take are removed from it, and `device_ip` is filled in with the address the device was found at.

Parameters

 * `configuration` (`Configuration`): Options for this run.

Returns

 * `Result`: The measured `fps` and `latency`, together with the device monitor's readings when it was running.

#### Attributes

##### all_configurations

Return every SNPE profile at one and at two threads.

##### default_configuration

Default configuration for RVC4 benchmarking.

 * `Options:`: profile: The SNPE profile to use for inference. runtime: The SNPE runtime to use for inference. num_images: The
   number of images to use for inference. dai_benchmark: Whether to use the DepthAI for benchmarking. repetitions: The number of
   repetitions to perform (dai-benchmark only, ignored if benchmark_time is set). benchmark_time: Duration in seconds for
   time-based benchmarking (overrides repetitions). num_threads: The number of threads to use for inference (dai-benchmark only).
   num_messages: The number of messages to use for inference (dai-benchmark only). device_ip: Address of the device to benchmark
   on, or None to use the first one found. device_id: Device ID or ADB serial of the device to benchmark on, or None to use the
   first one found. device_monitor: Whether to sample power, DSP, memory and CPU usage alongside the benchmark.

## Attributes

### PROFILES

### RUNTIMES