# Base definition of algorithms¶

class orion.algo.base.BaseAlgorithm(space: Space, **kwargs)[source]

Base class describing what an algorithm can do.

Parameters
space`orion.algo.space.Space`

Definition of a problem’s parameter space.

kwargsdict

Tunable elements of a particular algorithm, a dictionary from hyperparameter names to values.

Notes

We are using the No Free Lunch theorem’s [R367dceedad15-1]_[R367dceedad15-3]_ formulation of an `BaseAlgorithm`. We treat it as a part of a procedure which in each iteration suggests a sample of the parameter space of the problem as a candidate solution and observes the results of its evaluation.

Developer Note: Each algorithm’s complete specification, i.e. implementation of its methods and parameters of its own, lies in a separate concrete algorithm class, which must be an immediate subclass of `BaseAlgorithm`. [The reason for this is current implementation of `orion.core.utils.Factory` metaclass which uses BaseAlgorithm.__subclasses__().] Second, one must declare an algorithm’s own parameters (tunable elements which could be set by configuration). This is done by passing them to BaseAlgorithm.__init__() by calling Python’s super with a Space object as a positional argument plus algorithm’s own parameters as keyword arguments. The keys of the keyword arguments passed to BaseAlgorithm.__init__() are interpreted as the algorithm’s parameter names. So for example, a subclass could be as simple as this (regarding the logistics, not an actual algorithm’s implementation):

References

1

D. H. Wolpert and W. G. Macready, “No Free Lunch Theorems for Optimization,” IEEE Transactions on Evolutionary Computation, vol. 1, no. 1, pp. 67–82, Apr. 1997.

2

W. G. Macready and D. H. Wolpert, “What Makes An Optimization Problem Hard?,” Complexity, vol. 1, no. 5, pp. 40–46, 1996.

3

D. H. Wolpert and W. G. Macready, “No Free Lunch Theorems for Search,” Technical Report SFI-TR-95-02-010, Santa Fe Institute, 1995.

Examples

``` 1from orion.algo.base import BaseAlgorithm
2from orion.algo.space import (Integer, Space)
3
4class MySimpleAlgo(BaseAlgorithm):
5
6    def __init__(self, space, multiplier=1, another_param="a string param"):
7        super().__init__(space, multiplier=multiplier, another_param=another_param)
8
9    def suggest(self, num=1):
10        print(self.another_param)
11        return list(map(lambda x: tuple(map(lambda y: self.multiplier * y, x)),
12                        self.space.sample(num)))
13
14    def observe(self, points, results):
15        pass
16
17dim = Integer('named_param', 'norm', 3, 2, shape=(2, 3))
18s = Space()
19s.register(dim)
20
21algo = MySimpleAlgo(s, 2, "I am just sampling!")
22algo.suggest()
```
Attributes
`configuration`

Return tunable elements of this algorithm in a dictionary form appropriate for saving.

`fidelity_index`

Returns the name of the first fidelity dimension if there is one, otherwise None.

`has_completed_max_trials`

Returns True if the algorithm has a `max_trials` attribute, and has completed more trials than its value.

`is_done`

Whether the algorithm is done and will not make further suggestions.

max_trials
`n_observed`

Number of completed trials observed by the algorithm.

`n_suggested`

Number of trials suggested by the algorithm

requires_dist
requires_shape
requires_type
`space`

Domain of problem associated with this algorithm’s instance.

`state_dict`

Return a state dict that can be used to reset the state of the algorithm.

`unwrapped`

Return the algorithm without transforms

Methods

 `get_id`(trial[, ignore_fidelity, ignore_parent]) Return unique hash for a trials based on params `has_observed`(trial) Whether the algorithm has observed a given point objective. `has_suggested`(trial) Whether the algorithm has suggested a given point. Returns True if the algorithm has more trials in its registry than the number of possible values in the search space. `judge`(trial, measurements) Inform an algorithm about online measurements of a running trial. `observe`(trials) Observe the results of the evaluation of the trials in the process defined in user's script. `register`(trial) Save the trial as one suggested or observed by the algorithm. `score`(trial) Allow algorithm to evaluate trial based on a prediction about this parameter set's performance. `seed_rng`(seed) Seed the state of the random number generator. `set_state`(state_dict) Reset the state of the algorithm based on the given state_dict `should_suspend`(trial) Allow algorithm to decide whether a particular running trial is still worth to complete its evaluation, based on information provided by the `judge` method. `suggest`(num) Suggest a num of new sets of parameters.
property configuration: dict[str, Any]

Return tunable elements of this algorithm in a dictionary form appropriate for saving.

By default, returns a dictionary containing the attributes of self which are also constructor arguments.

property fidelity_index: str | None

Returns the name of the first fidelity dimension if there is one, otherwise None.

get_id(trial: Trial, ignore_fidelity: bool = False, ignore_parent: bool = False) str[source]

Return unique hash for a trials based on params

The trial is assumed to be in the optimization space of the algorithm.

Parameters
trialTrial

trial from a `orion.algo.space.Space`.

ignore_fidelity: bool, optional

If True, the fidelity dimension is ignored when computing a unique hash for the trial. Defaults to False.

ignore_parent: bool, optional

If True, the parent id is ignored when computing a unique hash for the trial. Defaults to False.

property has_completed_max_trials: bool

Returns True if the algorithm has a `max_trials` attribute, and has completed more trials than its value.

has_observed(trial: Trial) bool[source]

Whether the algorithm has observed a given point objective.

This only counts observed completed trials.

Parameters
trial: ``orion.core.worker.trial.Trial``

Trial object to retrieve from the database

Returns
bool

True if the trial’s objective was observed by the algo, False otherwise.

has_suggested(trial: Trial) bool[source]

Whether the algorithm has suggested a given point.

Parameters
trial: ``orion.core.worker.trial.Trial``

Trial from a `orion.algo.space.Space`.

Returns
bool

True if the trial was suggested by the algo, False otherwise.

has_suggested_all_possible_values() bool[source]

Returns True if the algorithm has more trials in its registry than the number of possible values in the search space.

If there is a fidelity dimension in the search space, only the trials with the maximum fidelity value are counted.

property is_done: bool

Whether the algorithm is done and will not make further suggestions.

Return True, if an algorithm holds that there can be no further improvement. By default, the cardinality of the specified search space will be used to check if all possible sets of parameters has been tried.

judge(trial: Trial, measurements: Any) [source]

Inform an algorithm about online measurements of a running trial.

This method is to be used as a callback in a client-server communication between user’s script and a orion’s worker using a `BaseAlgorithm`. Data returned from this method must be serializable and will be used as a response to the running environment. Default response is None.

Parameters
trial: ``orion.core.worker.trial.Trial``

Trial object to retrieve from the database

Returns
None or a serializable dictionary containing named data

Notes

Calling algorithm to `judge` a point based on its online measurements will effectively change a state in the algorithm (like a reinforcement learning agent’s hidden state or an automatic early stopping mechanism’s regression), which it may change the value of the property `should_suspend`.

property n_observed: int

Number of completed trials observed by the algorithm.

property n_suggested: int

Number of trials suggested by the algorithm

observe(trials: ) None[source]

Observe the results of the evaluation of the trials in the process defined in user’s script.

Parameters
trials: list of ``orion.core.worker.trial.Trial``

Trials from a `orion.algo.space.Space`.

register(trial: Trial) None[source]

Save the trial as one suggested or observed by the algorithm.

Parameters
trial: ``orion.core.worker.trial.Trial``

a Trial from self.space.

score(trial: Trial) [source]

Allow algorithm to evaluate trial based on a prediction about this parameter set’s performance.

By default, return the same score any parameter (no preference).

Parameters
trial: ``orion.core.worker.trial.Trial``

Trial object to retrieve from the database

Returns
A subjective measure of expected performance.
seed_rng(seed)[source]

Seed the state of the random number generator.

Parameters

seed – Integer seed for the random number generator.

Note

This methods does nothing if the algorithm is deterministic.

set_state(state_dict: dict)[source]

Reset the state of the algorithm based on the given state_dict

Parameters

state_dict – Dictionary representing state of an algorithm

should_suspend(trial: Trial) bool[source]

Allow algorithm to decide whether a particular running trial is still worth to complete its evaluation, based on information provided by the `judge` method.

property space: Space

Domain of problem associated with this algorithm’s instance.

property state_dict

Return a state dict that can be used to reset the state of the algorithm.

abstract suggest(num: int) [source]

Suggest a num of new sets of parameters.

Parameters
num: int

Number of points to suggest. The algorithm may return less than the number of points requested.

Returns
list of trials or None

A list of trials representing values suggested by the algorithm. The algorithm may opt out if it cannot make a good suggestion at the moment (it may be waiting for other trials to complete), in which case it will return None.

Notes

New parameters must be compliant with the problem’s domain `orion.algo.space.Space`.

IMPORTANT: Algorithms must call self.register(trial) for every trial that is returned by this method. This is important for the algorithm to be able to keep track of the trials it has suggested/observed, and for the auto-generated unit-tests to pass.

property unwrapped

Return the algorithm without transforms