Base¶
orion.storage.base
– Generic Storage Protocol¶
-
class
orion.storage.base.
BaseStorageProtocol
[source]¶ Implement a generic protocol to allow Orion to communicate using different storage backend
Attributes: - instance
Methods
count_broken_trials
(experiment)Count the number of broken trials count_completed_trials
(experiment)Count the number of completed trials create_experiment
(config)Insert a new experiment inside the database fetch_experiments
(query[, selection])Fetch all experiments that match the query fetch_lost_trials
(experiment)Fetch all trials that have a heartbeat older than some given time delta (2 minutes by default) fetch_noncompleted_trials
(experiment)Fetch all non completed trials fetch_pending_trials
(experiment)Fetch all trials that are available to be executed by a worker, this includes new, suspended and interrupted trials fetch_trials
([experiment, uid])Fetch all the trials of an experiment in the database fetch_trials_by_status
(experiment, status)Fetch all trials with the given status get_trial
([trial, uid])Fetch a single trial push_trial_results
(trial)Push the trial’s results to the database register_lie
(trial)Register a fake trial created by the strategist. register_trial
(trial)Create a new trial to be executed reserve_trial
(experiment)Select a pending trial and reserve it for the worker retrieve_result
(trial, *args, **kwargs)Fetch the result from a given medium (file, db, socket, etc..) for a given trial and insert it into the trial object set_trial_status
(trial, status[, heartbeat])Update the trial status and the heartbeat update_experiment
([experiment, uid, where])Update a the fields of a given trials update_heartbeat
(trial)Update trial’s heartbeat -
fetch_lost_trials
(experiment)[source]¶ Fetch all trials that have a heartbeat older than some given time delta (2 minutes by default)
-
fetch_pending_trials
(experiment)[source]¶ Fetch all trials that are available to be executed by a worker, this includes new, suspended and interrupted trials
-
fetch_trials
(experiment=None, uid=None)[source]¶ Fetch all the trials of an experiment in the database
Parameters: - experiment: Experiment, optional
experiment object to retrieve from the database
- uid: str, optional
experiment id used to retrieve the trial object
Returns: - return none if the experiment is not found,
Raises: - UndefinedCall
if both experiment and uid are not set
- AssertionError
if both experiment and uid are provided and they do not match
-
get_trial
(trial=None, uid=None)[source]¶ Fetch a single trial
Parameters: - trial: Trial, optional
trial object to retrieve from the database
- uid: str, optional
trial id used to retrieve the trial object
Returns: - return none if the trial is not found,
Raises: - UndefinedCall
if both trial and uid are not set
- AssertionError
if both trial and uid are provided and they do not match
-
register_lie
(trial)[source]¶ Register a fake trial created by the strategist.
The main difference between fake trial and orignal ones is the addition of a fake objective result, and status being set to completed. The id of the fake trial is different than the id of the original trial, but the original id can be computed using the hashcode on parameters of the fake trial. See mod:
orion.core.worker.strategy
for more information and the Strategist object and generation of fake trials.Parameters: - trial: `Trial` object
Fake trial to register in the database
-
reserve_trial
(experiment)[source]¶ Select a pending trial and reserve it for the worker
Returns: - Returns the reserved trial or None if no trials were found
-
retrieve_result
(trial, *args, **kwargs)[source]¶ Fetch the result from a given medium (file, db, socket, etc..) for a given trial and insert it into the trial object
-
set_trial_status
(trial, status, heartbeat=None)[source]¶ Update the trial status and the heartbeat
Raises: - FailedUpdate
The exception is raised if the status of the trial object does not match the status in the database
-
update_experiment
(experiment=None, uid=None, where=None, **kwargs)[source]¶ Update a the fields of a given trials
Parameters: - experiment: Experiment, optional
experiment object to retrieve from the database
- uid: str, optional
experiment id used to retrieve the trial object
- where: Optional[dict]
constraint experiment must respect
- **kwargs: dict
a dictionary of fields to update
Returns: - returns true if the underlying storage was updated
Raises: - UndefinedCall
if both experiment and uid are not set
- AssertionError
if both experiment and uid are provided and they do not match
-
exception
orion.storage.base.
FailedUpdate
[source]¶ Exception raised when we are unable to update a trial’ status
-
exception
orion.storage.base.
MissingArguments
[source]¶ Raised when calling a function without the minimal set of parameters
-
class
orion.storage.base.
ReadOnlyStorageProtocol
(protocol)[source]¶ Read-only interface from a storage protocol.
See also
orion.core.storage.BaseStorageProtocol
-
class
orion.storage.base.
Storage
[source]¶ Storage protocol is a generic way of allowing Orion to interface with different storage. MongoDB, track, cometML, MLFLow, etc…
Notes
When retrieving an already initialized Storage object you should use
get_storage
. Storage() should only be used for initialization purposes asget_storage
raises more granular error messages.Examples
>>> Storage('track', uri='file://orion_test.json') >>> Storage('legacy', experiment=...)
Attributes: - instance
Methods
count_broken_trials
(experiment)Count the number of broken trials count_completed_trials
(experiment)Count the number of completed trials create_experiment
(config)Insert a new experiment inside the database fetch_experiments
(query[, selection])Fetch all experiments that match the query fetch_lost_trials
(experiment)Fetch all trials that have a heartbeat older than some given time delta (2 minutes by default) fetch_noncompleted_trials
(experiment)Fetch all non completed trials fetch_pending_trials
(experiment)Fetch all trials that are available to be executed by a worker, this includes new, suspended and interrupted trials fetch_trials
([experiment, uid])Fetch all the trials of an experiment in the database fetch_trials_by_status
(experiment, status)Fetch all trials with the given status get_trial
([trial, uid])Fetch a single trial push_trial_results
(trial)Push the trial’s results to the database register_lie
(trial)Register a fake trial created by the strategist. register_trial
(trial)Create a new trial to be executed reserve_trial
(experiment)Select a pending trial and reserve it for the worker retrieve_result
(trial, *args, **kwargs)Fetch the result from a given medium (file, db, socket, etc..) for a given trial and insert it into the trial object set_trial_status
(trial, status[, heartbeat])Update the trial status and the heartbeat update_experiment
([experiment, uid, where])Update a the fields of a given trials update_heartbeat
(trial)Update trial’s heartbeat
-
orion.storage.base.
get_storage
()[source]¶ Return current storage
This is a wrapper around the Storage Singleton object to provide better error message when it is used without being initialized.
Raises: - RuntimeError
If the underlying storage was not initialized prior to calling this function
Notes
To initialize the underlying storage you must first call Storage(…) with the appropriate arguments for the chosen backend
-
orion.storage.base.
setup_storage
(storage=None, debug=False)[source]¶ Create the storage instance from a configuration.
Parameters: - config: dict, optional
Configuration for the storage backend. If not defined, global configuration is used.
- debug: bool, optional
If using in debug mode, the storage config is overrided with legacy:EphemeralDB. Defaults to False.