Experiment Builder

orion.core.io.experiment_builder – Create experiment from user options

The instantiation of an Experiment is not a trivial process when the user request an experiment with specific options. One can easily create a new experiment with ExperimentView(‘some_experiment_name’), but the configuration of a _writable_ experiment is less straighforward. This is because there is many sources of configuration and they have a strict hierarchy. From the more global to the more specific, there is:

1. Global configuration:

Defined by orion.core.DEF_CONFIG_FILES_PATHS. Can be scattered in user file system, defaults could look like:

• /some/path/to/.virtualenvs/orion/share/orion.core
• /etc/xdg/xdg-ubuntu/orion.core
• /home/${USER}/.config/orion.core

Note that some variables have default value even if user do not defined them in global configuration:

• max_trials = orion.core.io.resolve_config.DEF_CMD_MAX_TRIALS
• pool_size = orion.core.io.resolve_config.DEF_CMD_POOL_SIZE
• algorithms = random
• Database specific:
  • database.name = ‘orion’
  • database.type = ‘MongoDB’
  • database.host = ${HOST}

2. Oríon specific environment variables:

   Environment variables which can override global configuration

   • Database specific:
     • ORION_DB_NAME
     • ORION_DB_TYPE
     • ORION_DB_ADDRESS

3. Experiment configuration inside the database

Configuration of the experiment if present in the database. Making this part of the configuration of the experiment makes it possible for the user to execute an experiment by only specifying partial configuration. The rest of the configuration is fetched from the database.

For example, a user could:

1. Rerun the same experiment

Only providing the name is sufficient to rebuild the entire configuration of the experiment.

2. Make a modification to an existing experiment

The user can provide the name of the existing experiment and only provide the changes to apply on it. Here is an minimal example where we fully initialize a first experiment with a config file and then branch from it with minimal information.

# Initialize root experiment
orion init_only --config previous_exeriment.yaml ./userscript -x~'uniform(0, 10)'
# Branch a new experiment
orion hunt -n previous_experiment ./userscript -x~'uniform(0, 100)'

4. Configuration file

This configuration file is meant to overwrite the configuration coming from the database. If this configuration file was interpreted as part of the global configuration, a user could only modify an experiment using command line arguments.

5. Command-line arguments

Those are the arguments provided to orion for any method (hunt, insert, etc). It includes the argument to orion itself as well as the user’s script name and its arguments.

class orion.core.io.experiment_builder.ExperimentBuilder

Builder for orion.core.worker.experiment.Experiment and orion.core.worker.experiment.ExperimentView

See also

orion.core.io.experiment_builder for more information on the process of building experiments.

orion.core.worker.experiment.Experiment orion.core.worker.experiment.ExperimentView

Methods

build_from(self, cmdargs[, handle_racecondition])	Build a fully configured (and writable) experiment based on full configuration.
build_from_config(self, config)	Build a fully configured (and writable) experiment based on full configuration.
build_view_from(self, cmdargs)	Build an experiment view based on full configuration.
fetch_config_from_db(self, cmdargs)	Get dictionary of options from experiment found in the database
fetch_default_options(self)	Get dictionary of default options
fetch_env_vars(self)	Get dictionary of environment variables specific to Oríon
fetch_file_config(self, cmdargs)	Get dictionary of options from configuration file provided in command-line
fetch_full_config(self, cmdargs[, use_db])	Get dictionary of the full configuration of the experiment.
fetch_metadata(self, cmdargs)	Infer rest information about the process + versioning
setup_storage(self, config)	Create the storage instance from a configuration.

build_from(self, cmdargs, handle_racecondition=True)

Build a fully configured (and writable) experiment based on full configuration.

See also

orion.core.io.experiment_builder for more information on the hierarchy of configurations.

orion.core.worker.experiment.Experiment for more information on the experiment object.

build_from_config(self, config)

Build a fully configured (and writable) experiment based on full configuration.

See also

orion.core.io.experiment_builder for more information on the hierarchy of configurations.

orion.core.worker.experiment.Experiment for more information on the experiment object.

build_view_from(self, cmdargs)

Build an experiment view based on full configuration.

See also

orion.core.io.experiment_builder for more information on the hierarchy of configurations.

orion.core.worker.experiment.ExperimentView for more information on the experiment view object.

fetch_config_from_db(self, cmdargs)

Get dictionary of options from experiment found in the database

Notes

This method builds an experiment view in the background to fetch the configuration from the database.

fetch_default_options(self)

Get dictionary of default options

fetch_env_vars(self)

Get dictionary of environment variables specific to Oríon

fetch_file_config(self, cmdargs)

Get dictionary of options from configuration file provided in command-line

fetch_full_config(self, cmdargs, use_db=True)

Get dictionary of the full configuration of the experiment.

See also

orion.core.io.experiment_builder for more information on the hierarchy of configurations.

Parameters:

cmdargs:

use_db: bool

Use experiment configuration found in database if True. Defaults to True.

Notes

This method builds an experiment view in the background to fetch the configuration from the database.

fetch_metadata(self, cmdargs)

Infer rest information about the process + versioning

setup_storage(self, config)

Create the storage instance from a configuration.

Parameters:

config: dict

Configuration for the database.