Trial

Container class for Trial entity

Describe a particular training run, parameters and results.

exception orion.core.worker.trial.AlreadyReleased

Raised when a trial gets released twice

__weakref__

list of weak references to the object (if defined)

class orion.core.worker.trial.Trial(**kwargs)

Represents an entry in database/trials collection.

Attributes

experiment: str

Unique identifier for the experiment that produced this trial. Same as an Experiment._id.

id_override: str

Trial id returned by the database. It should be unique for a given set of parameters

heartbeat: datetime.datetime

Last time trial was identified as being alive.

status: str

Indicates how this trial is currently being used. Can take the following values:

• ‘new’Denotes a fresh set of parameters suggested by an algorithm,

  not yet tried out.

• ‘reserved’Indicates that this trial is currently being evaluated by

  a worker process, it was a ‘new’ trial that got selected.

• ‘suspended’Means that an algorithm decided to stop the evaluation of

  a ‘reserved’ trial prematurely.

• ‘completed’is the status of a previously ‘reserved’ trial that

  successfully got evaluated. Trial.results must contain the evaluation.

• ‘interrupted’Indicates trials that are stopped from being evaluated

  by external actors (e.g. cluster timeout, KeyboardInterrupt, killing of the worker process).

• ‘broken’Indicates a trial that was not successfully evaluated for not

  expected reason.

worker: str

Corresponds to worker’s unique id that handled this trial.

submit_time: `datetime.datetime`

When was this trial suggested?

start_time: `datetime.datetime`

When was this trial first reserved?

end_time: `datetime.datetime`

When was this trial evaluated successfully?

results: list of `Trial.Result`

List of evaluated metrics for this particular set of params. One and only one of them is necessarily an objective function value. The other are constraints, the value of an expression desired to be larger/equal to 0.

params: dict of params

Dict of suggested values for the Experiment parameter space. Consists a sample to be evaluated.

Methods

Param(**kwargs)	Types for a Param can be either an integer (discrete value), floating precision numerical or a categorical expression (e.g.
Result(**kwargs)	Types for a Result can be either an evaluation of an 'objective' function or of an 'constraint' expression.
Value(**kwargs)	Container for a value object.
branch([status, params])	Copy the trial and modify given attributes
build(trial_entries)	Builder method for a list of trials.
compute_trial_hash(trial[, ignore_fidelity, ...])	Generate a unique param md5sum hash for a given Trial
format_params(params[, sep, ignore_fidelity])	Represent with a string the parameters contained in this Trial object.
format_values(values[, sep])	Represent with a string the given values.
params_repr([sep, ignore_fidelity])	Represent with a string the parameters contained in this Trial object.
to_dict()	Needed to be able to convert Trial to dict form.

get_working_dir

class Param(**kwargs)

Types for a Param can be either an integer (discrete value), floating precision numerical or a categorical expression (e.g. a string).

class Result(**kwargs)

Types for a Result can be either an evaluation of an ‘objective’ function or of an ‘constraint’ expression.

class Value(**kwargs)

Container for a value object.

Attributes

name: str

A possible named for the quality that this is quantifying.

type: str

An identifier with semantic importance for Oríon. See Param.type and Result.type.

value: str or numerical

value suggested for this dimension of the parameter space.

Methods

to_dict()

Needed to be able to convert Value to dict form.

__eq__(other)

Test equality based on self.to_dict()

__hash__ = None

__init__(**kwargs)

See attributes of Value for possible argument for kwargs.

__repr__()

Represent partially with a string.

__str__()

Represent partially with a string.

to_dict()

Needed to be able to convert Value to dict form.

property type

For meaning of property type, see Value.type.

__eq__(other)

Whether two trials are equal is based on id alone.

This includes params, experiment, parent and lie. All other attributes of the trials are ignored when comparing them.

__hash__()

Return the hashname for this trial

__init__(**kwargs)

See attributes of Trial for meaning and possible arguments for kwargs.

__repr__()

Represent partially with a string.

__str__()

Represent partially with a string.

branch(status='new', params=None)

Copy the trial and modify given attributes

The status attributes will be reset as if trial was new.

Parameters

status: str, optional

The status of the new trial. Defaults to ‘new’.

params: dict, optional

Some parameters to update. A subset of params may be passed. Passing non-existing params in current trial will lead to a ValueError. Defaults to None.

Raises

ValueError

If some parameters are not present in current trial.

AttributeError

If some attribute does not exist in Trial objects.

classmethod build(trial_entries)

Builder method for a list of trials.

Parameters

trial_entries – List of trial representation in dictionary form, as expected to be saved in a database.

Returns

a list of corresponding Trial objects.

static compute_trial_hash(trial: Trial, ignore_fidelity=False, ignore_experiment=None, ignore_lie=False, ignore_parent=False)

Generate a unique param md5sum hash for a given Trial

property constraints

Return this trial’s constraints

Returns

A list of Trial.Result of type ‘constraint’

property duration

Return trial duration as a timedelta() object

property execution_interval

Return execution interval, or None if unavailable

property exp_working_dir

Return the current working directory of the experiment.

static format_params(params, sep=',', ignore_fidelity=False)

Represent with a string the parameters contained in this Trial object.

static format_values(values, sep=',')

Represent with a string the given values.

property full_name

Generate a unique name using the full definition of parameters.

property gradient

Return this trial’s gradient value if it is evaluated, else None.

Return type

Trial.Result

property hash_name

Generate a unique name with an md5sum hash for this Trial.

Note

Two trials that have the same params must have the same hash_name.

property hash_params

Generate a unique param md5sum hash for this Trial.

Note

The params contributing to the hash do not include the fidelity.

property id

Return hash_name which is also the database key id.

property legacy_id

Backward compatible id

Deprecated and will be removed in v0.4.0.

This is equivalent to Trial.id prior to v0.2.5.

property lie

Return this trial’s fake objective value if it was set, else None.

Return type

Trial.Result

property objective

Return this trial’s objective value if it is evaluated, else None.

Return type

Trial.Result

property params

Parameters of the trial

params_repr(sep=',', ignore_fidelity=False)

Represent with a string the parameters contained in this Trial object.

property results

List of results of the trial

property statistics

Return this trial’s statistics

Returns

A list of Trial.Result de type ‘statistic’

property status

For meaning of property type, see Trial.status.

to_dict()

Needed to be able to convert Trial to dict form.

property working_dir

Return the current working directory of the trial.

orion.core.worker.trial.validate_status(status)

Verify if given status is valid. Can be one of new, reserved, suspended, completed, interrupted, or broken.