Space search

Search space of optimization problems

Classes for representing the search space of an optimization problem.

There are 3 classes representing possible parameter types. All of them subclass the base class Dimension:

• Real

• Integer

• Categorical

These are instantiated to declare a problem’s parameter space. Oríon registers them in a ordered dictionary, Space, which describes how the parameters should be in order for orion.algo.base.AbstractAlgorithm implementations to communicate with orion.core.

Parameter values recorded in orion.core.worker.trial.Trial objects must be and are in concordance with orion.algo.space objects. These objects will be defined by orion.core using the user script’s configuration file.

Prior distributions, contained in Dimension classes, are based on scipy.stats distributions and should be configured as noted in the scipy documentation for each specific implementation of a random variable type, unless noted otherwise!

class orion.algo.space.Categorical(name, categories, **kwargs)

Search space dimension that can take on categorical values.

Parameters

namestr

See Parameters of Dimension.__init__().

categoriesdict or other iterable

A dictionary would associate categories to probabilities, else it assumes to be drawn uniformly from the iterable.

kwargsdict

See Parameters of Dimension.__init__() for general.

Attributes

cardinality

Return the number of all the possible values from Categorical Dimension

get_prior

Return the priors

prior_name

Return the name of the prior

Methods

cast(point)	Cast a point to some category
get_cardinality(shape, categories)	Return the number of all the possible points based and shape and categories
get_prior_string()	Build the string corresponding to current prior
interval([alpha])	Return a tuple of possible values that this categorical dimension can take.
sample([n_samples, seed])	Draw random samples from prior.

property cardinality

Return the number of all the possible values from Categorical Dimension

cast(point)

Cast a point to some category

Casted point will stay a list or a numpy array depending on the given point’s type.

Raises

ValueError

If one of the category in point is not present in current Categorical Dimension.

static get_cardinality(shape, categories)

Return the number of all the possible points based and shape and categories

property get_prior

Return the priors

get_prior_string()

Build the string corresponding to current prior

interval(alpha=1.0)

Return a tuple of possible values that this categorical dimension can take.

property prior_name

Return the name of the prior

sample(n_samples=1, seed=None)

Draw random samples from prior.

See also

Dimension.sample

class orion.algo.space.Dimension(name, prior, *args, **kwargs)

Base class for search space dimensions.

Attributes

namestr

See Dimension attributes.

typestr

See Dimension attributes.

priorscipy.stats.distributions.rv_generic

A distribution over the original dimension.

shapetuple

Return the shape of dimension.

Methods

cast(point)	Cast a point to dimension's type
get_prior_string()	Build the string corresponding to current prior
get_string()	Build the string corresponding to current dimension
interval([alpha])	Return a tuple containing lower and upper bound for parameters.
sample([n_samples, seed])	Draw random samples from prior.
validate()	Validate dimension arguments

property cardinality

Return the number of all the possible points from Dimension. The default value is numpy.inf.

cast(point)

Cast a point to dimension’s type

If casted point will stay a list or a numpy array depending on the given point’s type.

property default_value

Return the default value for this dimensions

get_prior_string()

Build the string corresponding to current prior

get_string()

Build the string corresponding to current dimension

interval(alpha=1.0)

Return a tuple containing lower and upper bound for parameters.

If parameters are drawn from an ‘open’ supported random variable, then it will be attempted to calculate the interval from which a variable is alpha-likely to be drawn from.

property name

See Dimension attributes.

property prior_name

Return the name of the prior

sample(n_samples=1, seed=None)

Draw random samples from prior.

Parameters

n_samplesint, optional

The number of samples to be drawn. Default is 1 sample.

seedNone | int | numpy.random.RandomState instance, optional

This parameter defines the RandomState object to use for drawing random variates. If None (or np.random), the global np.random state is used. If integer, it is used to seed a RandomState instance just for the call of this function. Default is None.

Set random state to something other than None for reproducible results.

.. warning:: Setting `seed` with an integer will cause the same ndarray

to be sampled if n_samples > 0. Set seed with a numpy.random.RandomState to carry on the changes in random state across many samples.

property shape

Return the shape of dimension.

property type

See Dimension attributes.

validate()

Validate dimension arguments

class orion.algo.space.Fidelity(name, low, high, base=2)

Fidelity Dimension for representing multi-fidelity.

Fidelity dimensions are not optimized by the algorithms. If it supports multi-fidelity, the algorithm will select a fidelity level for which it will sample hyper-parameter values to explore a low fidelity space. This class is used as a place-holder so that algorithms can discern fidelity dimensions from hyper-parameter dimensions.

Parameters

namestr

Name of the dimension

low: int

Minimum of the fidelity interval.

high: int

Maximum of the fidelity interval.

base: int

Base logarithm of the fidelity dimension.

Attributes

namestr

See Dimension attributes.

default_value: int

Maximum of the fidelity interval.

Methods

cast([point])	Do not do anything.
get_cardinality(shape, interval)	Return cardinality of Fidelity dimension, leave it to 1 as Fidelity dimension does not contribute to cardinality in a fixed way now.
get_prior_string()	Build the string corresponding to current prior
interval([alpha])	Do not do anything.
sample([n_samples, seed])	Do not do anything.
validate()	Do not do anything.

property cardinality

Return cardinality of Fidelity dimension, leave it to 1 as Fidelity dimension does not contribute to cardinality in a fixed way now.

cast(point=0)

Do not do anything.

property default_value

Return high

static get_cardinality(shape, interval)

Return cardinality of Fidelity dimension, leave it to 1 as Fidelity dimension does not contribute to cardinality in a fixed way now.

get_prior_string()

Build the string corresponding to current prior

interval(alpha=1.0)

Do not do anything.

sample(n_samples=1, seed=None)

Do not do anything.

validate()

Do not do anything.

class orion.algo.space.Integer(name, prior, *args, **kwargs)

Search space dimension representing integer values.

Parameters

name: str

prior: str

See Parameters of Dimension.__init__().

args: list

kwargs: dict

See Parameters of Dimension.__init__() for general.

Notes

Real kwargs (extra)

low: float

Lower bound (inclusive), optional; default -numpy.inf.

high: float:

Upper bound (inclusive), optional; default numpy.inf.

precision: int

Precision, optional; default 4.

shape: tuple

Defines how many dimensions are packed in this Dimension. Describes the shape of the corresponding tensor.

Attributes

cardinality

Return the number of all the possible points from Integer Dimension

prior_name

Return the name of the prior

Methods

cast(point)	Cast a point to int
get_cardinality(shape, interval, *args)	Return the number of all the possible points based and shape and interval
get_prior_string()	Build the string corresponding to current prior

property cardinality

Return the number of all the possible points from Integer Dimension

cast(point)

Cast a point to int

If casted point will stay a list or a numpy array depending on the given point’s type.

static get_cardinality(shape, interval, *args)

Return the number of all the possible points based and shape and interval

get_prior_string()

Build the string corresponding to current prior

property prior_name

Return the name of the prior

class orion.algo.space.Real(name, prior, *args, **kwargs)

Search space dimension that can take on any real value.

Parameters

name: str

prior: str

See Parameters of Dimension.__init__().

args: list

kwargs: dict

See Parameters of Dimension.__init__() for general.

Notes

Real kwargs (extra)

low: float

Lower bound (inclusive), optional; default -numpy.inf.

high: float:

Upper bound (inclusive), optional; default numpy.inf. The upper bound must be inclusive because of rounding errors during optimization which may cause values to round exactly to the upper bound.

precision: int

Precision, optional; default 4.

shape: tuple

Defines how many dimensions are packed in this Dimension. Describes the shape of the corresponding tensor.

Attributes

cardinality

Return the number of all the possible points from Integer Dimension

Methods

cast(point)	Cast a point to float
get_cardinality(shape, interval, precision, ...)	Return the number of all the possible points based and shape and interval
get_prior_string()	Build the string corresponding to current prior
interval([alpha])	Return a tuple containing lower and upper bound for parameters.
sample([n_samples, seed])	Draw random samples from prior.

property cardinality

Return the number of all the possible points from Integer Dimension

cast(point)

Cast a point to float

If casted point will stay a list or a numpy array depending on the given point’s type.

static get_cardinality(shape, interval, precision, prior_name)

Return the number of all the possible points based and shape and interval

get_prior_string()

Build the string corresponding to current prior

interval(alpha=1.0)

Return a tuple containing lower and upper bound for parameters.

If parameters are drawn from an ‘open’ supported random variable, then it will be attempted to calculate the interval from which a variable is alpha-likely to be drawn from.

Note

Both lower and upper bounds are inclusive.

sample(n_samples=1, seed=None)

Draw random samples from prior.

See also

Dimension.sample

class orion.algo.space.Space

Represents the search space.

It is a sorted dictionary which contains Dimension objects. The dimensions are sorted based on their names.

Attributes

cardinality

Return the number of all all possible sets of samples in the space

configuration

Return a dictionary of priors.

Methods

assert_contains(trial)	Same as __contains__ but instead of return true or false it will raise an exception with the exact causes of the mismatch.
contains	alias of Dimension
interval([alpha])	Return a list with the intervals for each contained dimension.
items()	Return items sorted according to keys
keys()	Return sorted keys
register(dimension)	Register a new dimension to Space.
sample([n_samples, seed])	Draw random samples from this space.
values()	Return values sorted according to keys

assert_contains(trial)

Same as __contains__ but instead of return true or false it will raise an exception with the exact causes of the mismatch.

Raises

ValueError if the trial has parameters that are not contained by the space.

property cardinality

Return the number of all all possible sets of samples in the space

property configuration

Return a dictionary of priors.

contains

alias of Dimension

interval(alpha=1.0)

Return a list with the intervals for each contained dimension.

items()

Return items sorted according to keys

keys()

Return sorted keys

register(dimension)

Register a new dimension to Space.

sample(n_samples=1, seed=None)

Draw random samples from this space.

Parameters

n_samplesint, optional

The number of samples to be drawn. Default is 1 sample.

seedNone | int | numpy.random.RandomState instance, optional

This parameter defines the RandomState object to use for drawing random variates. If None (or np.random), the global np.random state is used. If integer, it is used to seed a RandomState instance just for the call of this function. Default is None.

Set random state to something other than None for reproducible results.

Returns

trials: list of orion.core.worker.trial.Trial

Each element is a separate sample of this space, a trial containing values associated with the corresponding dimension.

values()

Return values sorted according to keys

class orion.algo.space.SpaceConverter(*args, **kwds)

SpaceConverter iterates over an Orion search space. This can be used to implement new features for orion.algo.space.Space outside of Orion’s code base.

Methods

categorical(dim)	Called by categorical dimension
convert_dimension(dimension)	Call the dimension conversion handler
dimension(dim)	Called when the dimension does not have a decicated handler
fidelity(dim)	Called by fidelity dimension
integer(dim)	Called by integer dimension
real(dim)	Called by real dimension
space(space)	Iterate through a research space and visit each dimensions

categorical(dim: Categorical) → T

Called by categorical dimension

convert_dimension(dimension: Dimension) → T

Call the dimension conversion handler

dimension(dim: Dimension) → T

Called when the dimension does not have a decicated handler

fidelity(dim: Fidelity) → T

Called by fidelity dimension

integer(dim: Integer) → T

Called by integer dimension

real(dim: Real) → T

Called by real dimension

space(space: Space) → None

Iterate through a research space and visit each dimensions

orion.algo.space.check_random_state(seed)

Return numpy global rng or RandomState if seed is specified

orion.algo.space.to_orionspace(space: Any) → Space

Convert a third party search space into an Orion compatible space

Raises

NotImplementedError if no conversion was registered