Base utils

Base tools to compute diverse analysis

orion.analysis.base.average(trials, group_by='order', key='best', return_var=False)[source]

Compute the average of some trial attribute.

By default it will compute the average objective at each time step across multiple experiments.

Parameters
trials: DataFrame

A dataframe of trials containing, at least, the columns ‘best’ and ‘order’.

group_by: str, optional

The attribute to use to group trials for the average. By default it group trials by order (ex: all first trials across experiments.)

key: str, optional

One attribute or a list of attributes split by ‘,’ to average. Defaults to ‘best’ as returned by orion.analysis.regret.

return_var: bool, optional

If True, and a column ‘{key}_var’ where ‘{key}’ is the value of the argument key. Defaults to False.

Returns
A dataframe with columns ‘order’, ‘{key}_mean’ and ‘{key}_var’.
orion.analysis.base.flatten_numpy(trials_array, flattened_space)[source]

Flatten dimensions

orion.analysis.base.flatten_params(space, params=None)[source]

Return the params of the corresponding flat space

If no params are passed, returns all flattened params. If params are passed, returns the corresponding flattened params.

Parameters
space: Space object

A space object from an experiment.

params: list of str, optional

The parameters to select from the search space. If the flattened search space contains flattened params such as (‘y’ -> ‘y[0]’, ‘y[1]’), passing ‘y’ in the list of params will returned the flattened version [‘y[0]’, ‘y[1]’]

Raises
ValueError

If one of the parameter names passed is not in the flattened space.

Examples

If space has x~uniform(0, 1) and y~uniform(0, 1, shape=(1, 2)). >>> flatten_params(space) [‘x’, ‘y[0,0]’, ‘y[0,1]’] >>> flatten_params(space, params=[‘x’]) [‘x’] >>> flatten_params(space, params=[‘x’, ‘y’]) [‘x’, ‘y[0,0]’, ‘y[0,1]’] >>> flatten_params(space, params=[‘x’, ‘y[0,1]’]) [‘x’, ‘y[0,1]’] >>> flatten_params(space, params=[‘y[0,1]’, ‘x’]) [‘x’, ‘y[0,1]’]

orion.analysis.base.ranking(trials, group_by='order', key='best')[source]

Compute the ranking of some trial attribute.

By default it will compute the ranking with respect to objectives at each time step across multiple experiments.

Parameters
trials: DataFrame

A dataframe of trials containing, at least, the columns ‘best’ and ‘order’.

group_by: str, optional

The attribute to use to group trials for the ranking. By default it group trials by order (ex: all first trials across experiments.)

key: str, optional

The attribute to use for the ranking. Defaults to ‘best’ as returned by orion.analysis.regret.

Returns
A copy of the original dataframe with a new column ‘rank’ for the rankings.
orion.analysis.base.to_numpy(trials, space)[source]

Convert trials in DataFrame to Numpy array of (params + objective)

orion.analysis.base.train_regressor(regressor_name, data, **kwargs)[source]

Train regressor model

Parameters
model: str

Name of the regression model to use. Can be one of - AdaBoostRegressor - BaggingRegressor - ExtraTreesRegressor - GradientBoostingRegressor - RandomForestRegressor (Default)

trials: DataFrame or dict

A dataframe of trials containing, at least, the columns ‘objective’ and ‘id’. Or a dict equivalent.

**kwargs

Arguments for the regressor model.