Regret

Provide tools to calculate regret

orion.analysis.regret_utils.get_regrets_idx(objectives)[source]

Return the indices corresponding to the cumulative minimum

orion.analysis.regret_utils.regret(trials, names=('best', 'best_id'))[source]

Calculates the regret for a collection of orion.core.worker.trial.Trial. The regret is calculated sequentially from the order of the collection.

Parameters
trials: DataFrame or dict

A dataframe of trials containing, at least, the columns ‘objective’ and ‘id’. Or a dict equivalent.

names:

A tuple containing the names of the columns. Default is (‘best’, ‘best-id’).

Returns
A copy of the original dataframe with two new columns containing respectively the best value
so far and its trial id.