Benchmark Client

Benchmark client

orion.benchmark.benchmark_client.get_or_create_benchmark(storage, name, algorithms=None, targets=None, executor=None)[source]

Create or get a benchmark object.

Parameters
storage: BaseStorageProtocol

Instance of the storage to use

name: str

Name of the benchmark

algorithms: list, optional

Algorithms used for benchmark, each algorithm can be a string or dict.

targets: list, optional

Targets for the benchmark, each target will be a dict with two keys.

assess: list

Assessment objects

task: list

Task objects

executor: `orion.executor.base.BaseExecutor`, optional

Executor to run the benchmark experiments

Returns
An instance of orion.benchmark.Benchmark