Base definition of algorithms¶
orion.algo.base
– What is a search algorithm, optimizer of a process¶

class
orion.algo.base.
BaseAlgorithm
(space, **kwargs)[source]¶ Base class describing what an algorithm can do.
Notes
We are using the No Free Lunch theorem’s [R367dceedad151]_[R367dceedad153]_ formulation of an
BaseAlgorithm
. We treat it as a part of a procedure which in each iteration suggests a sample of the parameter space of the problem as a candidate solution and observes the results of its evaluation.Developer Note: Each algorithm’s complete specification, i.e. implementation of its methods and parameters of its own, lies in a separate concrete algorithm class, which must be an immediate subclass of
BaseAlgorithm
. [The reason for this is current implementation ofFactory
metaclass which uses BaseAlgorithm.__subclasses__().] Second, one must declare an algorithm’s own parameters (tunable elements which could be set by configuration). This is done by passing them toBaseAlgorithm.__init__
by calling Python’s super with a Space object as a positional argument plus algorithm’s own parameters as keyword arguments. The keys of the keyword arguments passed toBaseAlgorithm.__init__
are interpreted as the algorithm’s parameter names. So for example, a subclass could be as simple as this (regarding the logistics, not an actual algorithm’s implementation):References
[1] D. H. Wolpert and W. G. Macready, “No Free Lunch Theorems for Optimization,” IEEE Transactions on Evolutionary Computation, vol. 1, no. 1, pp. 67–82, Apr. 1997. [2] W. G. Macready and D. H. Wolpert, “What Makes An Optimization Problem Hard?,” Complexity, vol. 1, no. 5, pp. 40–46, 1996. [3] D. H. Wolpert and W. G. Macready, “No Free Lunch Theorems for Search,” Technical Report SFITR9502010, Santa Fe Institute, 1995. Examples
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
from orion.algo.base import BaseAlgorithm from orion.algo.space import (Integer, Space) class MySimpleAlgo(BaseAlgorithm): def __init__(self, space, multiplier=1, another_param="a string param"): super().__init__(space, multiplier=multiplier, another_param=another_param) def suggest(self, num=1): print(self.another_param) return list(map(lambda x: tuple(map(lambda y: self.multiplier * y, x)), self.space.sample(num))) def observe(self, points, results): pass dim = Integer('named_param', 'norm', 3, 2, shape=(2, 3)) s = Space() s.register(dim) algo = MySimpleAlgo(s, 2, "I am just sampling!") algo.suggest()
Attributes: configuration
Return tunable elements of this algorithm in a dictionary form appropriate for saving.
is_done
Return True, if an algorithm holds that there can be no further improvement.
should_suspend
Allow algorithm to decide whether a particular running trial is still worth to complete its evaluation, based on information provided by the
judge
method.space
Domain of problem associated with this algorithm’s instance.
state_dict
Return a state dict that can be used to reset the state of the algorithm.
Methods
judge
(self, point, measurements)Inform an algorithm about online measurements of a running trial. observe
(self, points, results)Observe the results of the evaluation of the points in the process defined in user’s script. score
(self, point)Allow algorithm to evaluate point based on a prediction about this parameter set’s performance. seed_rng
(self, seed)Seed the state of the random number generator. set_state
(self, state_dict)Reset the state of the algorithm based on the given state_dict suggest
(self[, num])Suggest a num of new sets of parameters. 
configuration
¶ Return tunable elements of this algorithm in a dictionary form appropriate for saving.

is_done
¶ Return True, if an algorithm holds that there can be no further improvement.

judge
(self, point, measurements)[source]¶ Inform an algorithm about online measurements of a running trial.
Parameters: point – A tuple which specifies the values of the (hyper)parameters used to execute user’s script with. This method is to be used as a callback in a clientserver communication between user’s script and a orion’s worker using a
BaseAlgorithm
. Data returned from this method must be serializable and will be used as a response to the running environment. Default response is None.Note
Calling algorithm to
judge
a point based on its online measurements will effectively change a state in the algorithm (like a reinforcement learning agent’s hidden state or an automatic early stopping mechanism’s regression), which it may change the value of the propertyshould_suspend
.Returns: None or a serializable dictionary containing named data

observe
(self, points, results)[source]¶ Observe the results of the evaluation of the points in the process defined in user’s script.
Parameters:  points : list of tuples of arraylikes
Points from a
orion.algo.space.Space
. Evaluated problem parameters by a consumer. results : list of dicts
Contains the result of an evaluation; partial information about the blackbox function at each point in params.

score
(self, point)[source]¶ Allow algorithm to evaluate point based on a prediction about this parameter set’s performance.
By default, return the same score any parameter (no preference).
Returns: A subjective measure of expected perfomance. Return type: float

seed_rng
(self, seed)[source]¶ Seed the state of the random number generator.
Parameters: seed – Integer seed for the random number generator. Note
This methods does nothing if the algorithm is deterministic.

set_state
(self, state_dict)[source]¶ Reset the state of the algorithm based on the given state_dict
Parameters: state_dict – Dictionary representing state of an algorithm

should_suspend
¶ Allow algorithm to decide whether a particular running trial is still worth to complete its evaluation, based on information provided by the
judge
method.

space
¶ Domain of problem associated with this algorithm’s instance.

state_dict
¶ Return a state dict that can be used to reset the state of the algorithm.

suggest
(self, num=1)[source]¶ Suggest a num of new sets of parameters.
Parameters:  num: int, optional
Number of points to suggest. Defaults to 1.
Returns:  list of points or None
A list of lists representing points suggested by the algorithm. The algorithm may opt out if it cannot make a good suggestion at the moment (it may be waiting for other trials to complete), in which case it will return None.
Notes
New parameters must be compliant with the problem’s domain
orion.algo.space.Space
.

class
orion.algo.base.
OptimizationAlgorithm
(space, **kwargs)[source]¶ Class used to inject dependency on an algorithm implementation.
See also
orion.core.utils.Factory
metaclass andBaseAlgorithm
interface.Attributes: configuration
Return tunable elements of this algorithm in a dictionary form appropriate for saving.
is_done
Return True, if an algorithm holds that there can be no further improvement.
should_suspend
Allow algorithm to decide whether a particular running trial is still worth to complete its evaluation, based on information provided by the
judge
method.space
Domain of problem associated with this algorithm’s instance.
state_dict
Return a state dict that can be used to reset the state of the algorithm.
Methods
judge
(self, point, measurements)Inform an algorithm about online measurements of a running trial. observe
(self, points, results)Observe the results of the evaluation of the points in the process defined in user’s script. score
(self, point)Allow algorithm to evaluate point based on a prediction about this parameter set’s performance. seed_rng
(self, seed)Seed the state of the random number generator. set_state
(self, state_dict)Reset the state of the algorithm based on the given state_dict suggest
(self[, num])Suggest a num of new sets of parameters.