Benchmark Client

Benchmark client

orion.benchmark.benchmark_client.get_or_create_benchmark(name, algorithms=None, targets=None, storage=None, debug=False)[source]

Create or get a benchmark object.

Parameters:
name: str

Name of the benchmark

algorithms: list, optional

Algorithms used for benchmark, each algorithm can be a string or dict.

targets: list, optional

Targets for the benchmark, each target will be a dict with two keys.

assess: list

Assessment objects

task: list

Task objects

storage: dict, optional

Configuration of the storage backend.

debug: bool, optional

If using in debug mode, the storage config is overrided with legacy:EphemeralDB. Defaults to False.

Returns:
An instance of orion.benchmark.Benchmark