Base

Generic Storage Protocol

Implement a generic protocol to allow Orion to communicate using different storage backend.

Storage protocol is a generic way of allowing Orion to interface with different storage. MongoDB, track, cometML, MLFLow, etc…

Examples

>>> storage_factory.create('track', uri='file://orion_test.json')
>>> storage_factory.create('legacy', experiment=...)

class orion.storage.base.BaseStorageProtocol

Implement a generic protocol to allow Orion to communicate using different storage backend

Methods

acquire_algorithm_lock(experiment[, ...])	Acquire lock on algorithm in storage
count_broken_trials(experiment)	Count the number of broken trials
count_completed_trials(experiment)	Count the number of completed trials
create_benchmark(config)	Insert a new benchmark inside the database
create_experiment(config)	Insert a new experiment inside the database
delete_algorithm_lock([experiment, uid])	Delete experiment algorithm lock from the storage
delete_experiment([experiment, uid])	Delete matching experiments from the database
delete_trials([experiment, uid, where])	Delete matching trials from the database
fetch_benchmark(query[, selection])	Fetch all benchmarks that match the query
fetch_experiments(query[, selection])	Fetch all experiments that match the query
fetch_lost_trials(experiment)	Fetch all trials that have a heartbeat older than some given time delta (2 minutes by default)
fetch_noncompleted_trials(experiment)	Fetch all non completed trials
fetch_pending_trials(experiment)	Fetch all trials that are available to be executed by a worker, this includes new, suspended and interrupted trials
fetch_trials([experiment, uid, where])	Fetch all the trials of an experiment in the database
fetch_trials_by_status(experiment, status)	Fetch all trials with the given status
get_algorithm_lock_info([experiment, uid])	Load algorithm lock info
get_trial([trial, uid, experiment_uid])	Fetch a single trial
initialize_algorithm_lock(experiment_id, ...)	Initialize algorithm lock for given experiment
push_trial_results(trial)	Push the trial's results to the database
register_trial(trial)	Create a new trial to be executed
release_algorithm_lock([experiment, uid, ...])	Release the algorithm lock
reserve_trial(experiment)	Select a pending trial and reserve it for the worker
retrieve_result(trial, *args, **kwargs)	Fetch the result from a given medium (file, db, socket, etc..) for a given trial and insert it into the trial object
set_trial_status(trial, status[, heartbeat, was])	Update the trial status and the heartbeat
update_experiment([experiment, uid, where])	Update the fields of a given experiment
update_heartbeat(trial)	Update trial's heartbeat
update_trial([trial, uid, experiment_uid, where])	Update fields of a given trial
update_trials([experiment, uid, where])	Update trials of a given experiment matching a query

acquire_algorithm_lock(experiment: Experiment, timeout: int = 600, retry_interval: int = 1) → Generator[LockedAlgorithmState, None, None]

Acquire lock on algorithm in storage

This method is a contextmanager and should be called using the with-clause.

Parameters

experiment: Experiment

experiment object to retrieve from the storage

timeout: int, optional

Timeout for the acquisition of the lock. If the lock is not obtained before timeout, then LockAcquisitionTimeout is raised. The timeout is only for the acquisition of the lock. Once the lock is obtained, it is valid until the context manager is closed. Default: 600.

retry_interval: int, optional

Sleep time between each attempts at acquiring the lock. Default: 1

Raises

orion.storage.base.LockAcquisitionTimeout

The lock could not be obtained in less than timeout seconds.

count_broken_trials(experiment: Experiment) → int

Count the number of broken trials

count_completed_trials(experiment: Experiment) → int

Count the number of completed trials

create_benchmark(config: dict)

Insert a new benchmark inside the database

create_experiment(config: ExperimentConfig)

Insert a new experiment inside the database

delete_algorithm_lock(experiment: Experiment | None = None, uid: str | int | None = None) → Literal[0, 1]

Delete experiment algorithm lock from the storage

Parameters

experiment: Experiment, optional

experiment object to retrieve from the database

uid: str or int, optional

experiment id used to retrieve the trial object

Returns

Number of algorithm lock deleted. Should 1 if successful, 0 is failed.

Raises

UndefinedCall

if both experiment and uid are not set

AssertionError

if both experiment and uid are provided and they do not match

delete_experiment(experiment: Experiment | None = None, uid: str | int | None = None)

Delete matching experiments from the database

Parameters

experiment: Experiment, optional

experiment object to retrieve from the database

uid: str or int, optional

experiment id used to retrieve the trial object

Returns

Number of experiments deleted.

Raises

UndefinedCall

if both experiment and uid are not set

AssertionError

if both experiment and uid are provided and they do not match

delete_trials(experiment: Experiment | None = None, uid: str | int | None = None, where: dict | None = None) → int

Delete matching trials from the database

Parameters

experiment: Experiment, optional

experiment object to retrieve from the database

uid: str or int, optional

experiment id used to retrieve the trial object

where: Optional[dict]

constraint trials must respect

Returns

Number of trials deleted.

Raises

UndefinedCall

if both experiment and uid are not set

AssertionError

if both experiment and uid are provided and they do not match

fetch_benchmark(query: dict, selection: dict | None = None)

Fetch all benchmarks that match the query

fetch_experiments(query: dict, selection: dict | None = None) → list[ExperimentConfig]

Fetch all experiments that match the query

fetch_lost_trials(experiment: Experiment) → list[Trial]

Fetch all trials that have a heartbeat older than some given time delta (2 minutes by default)

fetch_noncompleted_trials(experiment: Experiment) → list[Trial]

Fetch all non completed trials

fetch_pending_trials(experiment: Experiment) → list[Trial]

Fetch all trials that are available to be executed by a worker, this includes new, suspended and interrupted trials

fetch_trials(experiment: Experiment | None = None, uid: str | int | None = None, where: dict | None = None) → list[Trial] | None

Fetch all the trials of an experiment in the database

Parameters

experiment: Experiment, optional

experiment object to retrieve from the database

uid: str or int, optional

experiment id used to retrieve the trial object

where: Optional[dict]

constraint trials must respect

Returns

return none if the experiment is not found,

Raises

UndefinedCall

if both experiment and uid are not set

AssertionError

if both experiment and uid are provided and they do not match

fetch_trials_by_status(experiment: Experiment, status: str) → list[Trial]

Fetch all trials with the given status

get_algorithm_lock_info(experiment: Experiment | None = None, uid: str | int | None = None) → LockedAlgorithmState

Load algorithm lock info

Parameters

experiment: Experiment, optional

experiment object to retrieve from the database

uid: str or int, optional

experiment id used to retrieve the trial object.

Returns

orion.storage.base.LockedAlgorithmState

The locked state of the algorithm. Note that the lock is not acquired by the process calling get_algorithm_lock_info and the value of LockedAlgorithmState.locked may not be valid if another process is running and could acquire the lock concurrently.

get_trial(trial: Trial | None = None, uid: str | None = None, experiment_uid: str | int | None = None) → Trial | None

Fetch a single trial

Parameters

trial: Trial, optional

trial object to retrieve from the database

uid: str, optional

trial id used to retrieve the trial object

experiment_uid: str or int, optional

experiment id used to retrieve the trial object

Returns

return None if the trial is not found,

Raises

UndefinedCall

if both trial and uid are not set

AssertionError

if both trial and uid are provided and they do not match

initialize_algorithm_lock(experiment_id: int | str, algorithm_config: dict)

Initialize algorithm lock for given experiment

Parameters

experiment_id: int or str

ID of the experiment in storage.

algorithm_config: dict

Configuration of the algorithm.

push_trial_results(trial: Trial)

Push the trial’s results to the database

register_trial(trial: Trial)

Create a new trial to be executed

release_algorithm_lock(experiment: Experiment | None = None, uid: str | int | None = None, new_state: dict | None = None)

Release the algorithm lock

Parameters

experiment: Experiment, optional

experiment object to retrieve from the database

uid: str or int, optional

experiment id used to retrieve the trial object.

new_state: dict, optional

The new state of the algorithm that should be saved in the lock object. If None, the previous state is preserved in the lock object in storage.

reserve_trial(experiment: Experiment) → Trial | None

Select a pending trial and reserve it for the worker

Returns

Returns the reserved trial or None if no trials were found

retrieve_result(trial: Trial, *args, **kwargs) → Trial

Fetch the result from a given medium (file, db, socket, etc..) for a given trial and insert it into the trial object

set_trial_status(trial: Trial, status: str, heartbeat: datetime | None = None, was: str | None = None)

Update the trial status and the heartbeat

Parameters

trial: `Trial` object

Trial object to update in the database.

status: str

Status to be set to the trial

heartbeat: datetime, optional

New heartbeat to update simultaneously with status

was: str, optional

The status the trial should be set to in the database. If None, current trial.status will be used. This is used to ensure coherence in the database, protecting against race conditions for instance.

Raises

FailedUpdate

The exception is raised if the status of the trial object does not match the status in the database

update_experiment(experiment: Experiment | None = None, uid: str | int | None = None, where: dict | None = None, **kwargs: Unpack[PartialExperimentConfig]) → bool

Update the fields of a given experiment

Parameters

experiment: Experiment, optional

experiment object to retrieve from the database

uid: str or int, optional

experiment id used to retrieve the trial object

where: Optional[dict]

constraint experiment must respect

**kwargs: dict

a dictionary of fields to update

Returns

returns true if the underlying storage was updated

Raises

UndefinedCall

if both experiment and uid are not set

AssertionError

if both experiment and uid are provided and they do not match

update_heartbeat(trial: Trial)

Update trial’s heartbeat

update_trial(trial: Trial | None = None, uid: str | int | None = None, experiment_uid: str | int | None = None, where: dict | None = None, **kwargs)

Update fields of a given trial

Parameters

trial: Trial, optional

trial object to update in the database

uid: str, optional

id of the trial to update in the database

experiment_uid: str or int, optional

experiment id of the trial to update in the database

where: Optional[dict]

constraint trials must respect. Note: useful to handle race conditions.

**kwargs: dict

a dictionary of fields to update

Raises

UndefinedCall

if both trial and uid are not set

AssertionError

if both trial and uid are provided and they do not match

update_trials(experiment: Experiment | None = None, uid: str | int | None = None, where: dict | None = None, **kwargs)

Update trials of a given experiment matching a query

Parameters

experiment: Experiment, optional

experiment object to retrieve from the database

uid: str or int, optional

experiment id used to retrieve the trial object

where: Optional[dict]

constraint trials must respect

**kwargs: dict

a dictionary of fields to update

Raises

UndefinedCall

if both experiment and uid are not set

AssertionError

if both experiment and uid are provided and they do not match

exception orion.storage.base.FailedUpdate

Exception raised when we are unable to update a trial’ status

exception orion.storage.base.LockAcquisitionTimeout

Raised when the lock acquisition timeout (not lock is granted).

class orion.storage.base.LockedAlgorithmState(state: dict, configuration: dict, locked: bool = True)

Locked state of the algorithm from the storage.

This class helps handle setting the state of the algorithm or resetting it in case the execution crashes during the lock.

Parameters

state: dict

Dictionary representing the state of the algorithm.

configuration: dict

Configuration of the locked algorithm.

locked: bool

Whether the algorithm is locked or not. Default: True

Attributes

state

State of the algorithm

Methods

reset()	Set back algorithm state to original state found in storage.
set_state(state)	Update the state of the algorithm that should be saved back in storage.

reset() → None

Set back algorithm state to original state found in storage.

set_state(state: dict) → None

Update the state of the algorithm that should be saved back in storage.

property state: dict

State of the algorithm

exception orion.storage.base.MissingArguments

Raised when calling a function without the minimal set of parameters

class orion.storage.base.ReadOnlyStorageProtocol(protocol)

Read-only interface from a storage protocol.

See also

BaseStorageProtocol

orion.storage.base.get_trial_uid_and_exp(trial: Trial | None = None, uid: str | None = None, experiment_uid: str | int | None = None)

Return trial and experiment uid either from trial or directly uids.

Parameters

trial: Trial, optional

Object with .id attribute

uid: str, optional

str id representation of the trial

experiment_uid: str or int, optional

Id of the experiment.

Returns

(trial uid, experiment uid)

Raises

UndefinedCall

if both trial and (uid or experiment_uid) are not set

AssertionError

if both trial and (uid or experiment_uid) are provided and they do not match

orion.storage.base.get_uid(item: Experiment | Trial | None = None, uid: str | int | None = None, force_uid: bool = True)

Return uid either from item or directly uid.

Parameters

item: Experiment or Trial, optional

Object with .id attribute

uid: str or int, optional

str id representation

force_uid: bool, optional

If True, at least one of item or uid must be passed.

Raises

UndefinedCall

if both item and uid are not set and force_uid is True

AssertionError

if both item and uid are provided and they do not match

orion.storage.base.setup_storage(storage=None, debug=False)

Create the storage instance from a configuration.

Parameters

config: dict, optional

Configuration for the storage backend. If not defined, global configuration is used.

debug: bool, optional

If using in debug mode, the storage config is overridden with legacy:EphemeralDB. Defaults to False.