Base definition of a Profet task

Base class for Tasks that are generated using the Profet algorithm.

For more information on Profet, see original paper at https://arxiv.org/abs/1905.12982.

Klein, Aaron, Zhenwen Dai, Frank Hutter, Neil Lawrence, and Javier Gonzalez. “Meta-surrogate benchmarking for hyperparameter optimization.” Advances in Neural Information Processing Systems 32 (2019): 6270-6280.

class orion.benchmark.task.profet.profet_task.ProfetTask(max_trials: int = 100, input_dir: Union[Path, str] = 'profet_data', checkpoint_dir: Optional[Union[Path, str]] = None, model_config: Optional[MetaModelConfig] = None, device: Optional[Union[str, Any]] = None, with_grad: bool = False)

Base class for Tasks that are generated using the Profet algorithm.

For more information on Profet, see original paper at https://arxiv.org/abs/1905.12982.

Klein, Aaron, Zhenwen Dai, Frank Hutter, Neil Lawrence, and Javier Gonzalez. “Meta-surrogate benchmarking for hyperparameter optimization.” Advances in Neural Information Processing Systems 32 (2019): 6270-6280.

Parameters

max_trialsint, optional

Max number of trials to run, by default 100

input_dirUnion[Path, str], optional

Input directory containing the data used to train the meta-model, by default None.

checkpoint_dirUnion[Path, str], optional

Directory used to save/load trained meta-models, by default None.

model_configMetaModelConfig, optional

Configuration options for the training of the meta-model, by default None

devicestr, optional

The device to use for training, by default None.

with_gradbool, optional

Whether the task should also return the gradients of the objective function with respect to the inputs. Defaults to False.

Attributes

configuration

Return the configuration of the task.

space

Methods

ModelConfig	alias of MetaModelConfig
call(**kwargs)	Get the value of the sampled objective function at the given point (hyper-parameters).

ModelConfig

alias of MetaModelConfig

call(**kwargs) → List[Dict]

Get the value of the sampled objective function at the given point (hyper-parameters).

If self.with_grad is set, also returns the gradient of the objective function with respect to the inputs.

Parameters

**kwargs

Dictionary of hyper-parameters.

Returns

List[Dict]

Result dictionaries: objective and optionally gradient.

Raises

ValueError

If the input isn’t of a supported type.

property configuration

Return the configuration of the task.

orion.benchmark.task.profet.profet_task.make_reproducible(seed: int)

Makes the random operations within a block of code reproducible for a given seed.