Conflicts detection and resolutions for creation of adapters

Description and resolution of configuration conflicts

Conflicts between a parent experiment and a child configuration exist in many different forms. This module provides the function detect_conflicts to automatically detect them. Any conflict type which inherits from class Conflict is used to detect corresponding conflicts. These conflicts can than be used to generate resolutions which will generate adapters to make trials of both experiments compatible when applicable.

Conflicts objects know how to resolve themselves but may lack information for doing so. For instance ExperimentNameConflict knows it may only be resolved with an ExperimentNameResolution, but it cannot do so unless it is given a new name to instantiate the resolution.

Conflict objects may build different resolutions based on the input given. For instance MissingDimensionConflict may instantiate a RenameDimensionResolution if a NewDimensionConflict is passed to try_resolve, otherwise the resolution will be RemoveDimensionResolution.

In short, conflict knows:

  1. How to detect themselves in pair old_config, new_config (Conflict.detect())

  2. How to resolve themselves (but may lack information for doing so) (conflict.try_resolve())

  3. How to build a diff for user interface (conflict.diff)

  4. How to build a string to represent themselves in user interface (repr(conflict))

  5. How to find resolutions markers and their corresponding arguments (conflict.get_marked_arguments)

while resolution knows:

  1. How to create adapters (resolution.get_adapters())

  2. How to find marked arguments for themselves (determining if this resolution was marked by user) (resolution.find_marked_argument())

  3. How to revert themselves and resetting the corresponding conflicts (resolution.revert())

  4. How to validate themselves on instantiation (resolution.validate())

  5. How to build a string to represent themselves in user interface (repr(resolution)) (note: this string is the one a user would use to mark the resolution in command line or in configuration file)

The class Conflicts is provided for convenience. It provides interface to register, fetch or deprecate (remove) conflicts. Additionaly, it provides a helper method with wraps try_resolve of the Conflict objects, handling invalid resolution errors or additional new conflicts created by resolutions. For instance, a RenameDimensionResolution may create a new ChangedDimensionConflict if the new name is associated to a different prior than the one of the old name.

class orion.core.evc.conflicts.AlgorithmConflict(old_config, new_config)[source]

Representation of an algorithm configuration conflict

Attributes
diff

Produce human-readable differences

Methods

AlgorithmResolution(conflict)

Representation of an algorithn configuration resolution

detect(old_config, new_config[, …])

Detect if algorithm definition in new_config differs from old_config :param branching_config:

try_resolve()

Try to create a resolution AlgorithmResolution

class AlgorithmResolution(conflict)[source]

Representation of an algorithn configuration resolution

Methods

get_adapters()

Return AlgorithmChange adapter

get_adapters()[source]

Return AlgorithmChange adapter

classmethod detect(old_config, new_config, branching_config=None)[source]

Detect if algorithm definition in new_config differs from old_config :param branching_config:

property diff

Produce human-readable differences

try_resolve()[source]

Try to create a resolution AlgorithmResolution

class orion.core.evc.conflicts.ChangedDimensionConflict(old_config, new_config, dimension, old_prior, new_prior)[source]

Representation of a changed prior conflict

Attributes
diff

Produce human-readable differences

Methods

ChangeDimensionResolution(conflict)

Representation of a changed prior resolution

detect(old_config, new_config[, …])

Detect all changed dimensions in new_config based on old_config :param branching_config:

try_resolve()

Try to create a resolution ChangeDimensionResolution

class ChangeDimensionResolution(conflict)[source]

Representation of a changed prior resolution

Attributes
prefix

Build the new prior string, including the default value

Methods

get_adapters()

Return DimensionPriorChange adapter

get_adapters()[source]

Return DimensionPriorChange adapter

property prefix

Build the new prior string, including the default value

classmethod detect(old_config, new_config, branching_config=None)[source]

Detect all changed dimensions in new_config based on old_config :param branching_config:

property diff

Produce human-readable differences

try_resolve()[source]

Try to create a resolution ChangeDimensionResolution

class orion.core.evc.conflicts.CodeConflict(old_config, new_config)[source]

Representation of code change conflict

Attributes
diff

Produce human-readable differences

Methods

CodeResolution(conflict, change_type)

Representation of an code change resolution

detect(old_config, new_config[, …])

Detect if commit hash in new_config differs from old_config :param branching_config:

get_marked_arguments(conflicts[, …])

Find and return marked arguments for code change conflict

try_resolve([change_type])

Try to create a resolution CodeResolution

class CodeResolution(conflict, change_type)[source]

Representation of an code change resolution

Attributes
conflict: `orion.core.evc.conflicts.Conflict`

The conflict which is resolved by this resolution.

change_type: string

One of the types defined in orion.core.evc.adapters.CodeChange.types.

.. seealso ::

orion.core.evc.conflicts.Resolution

Methods

get_adapters()

Return CodeChange adapter

get_adapters()[source]

Return CodeChange adapter

classmethod detect(old_config, new_config, branching_config=None)[source]

Detect if commit hash in new_config differs from old_config :param branching_config:

property diff

Produce human-readable differences

get_marked_arguments(conflicts, code_change_type=None, **branching_kwargs)[source]

Find and return marked arguments for code change conflict

try_resolve(change_type=None)[source]

Try to create a resolution CodeResolution

Parameters
change_type: None or string

One of the types defined in orion.core.evc.adapters.CodeChange.types.

Raises
ValueError

If change_type is not in orion.core.evc.adapters.CodeChange.types.

class orion.core.evc.conflicts.CommandLineConflict(old_config, new_config)[source]

Representation of commandline change conflict

Attributes
diff

Produce human-readable differences

Methods

CommandLineResolution(conflict, change_type)

Representation of an commandline change resolution

detect(old_config, new_config[, …])

Detect if command line call in new_config differs from old_config :param branching_config:

get_marked_arguments(conflicts[, …])

Find and return marked arguments for cli change conflict

get_nameless_args(config[, …])

Get user’s commandline arguments which are not dimension definitions

try_resolve([change_type])

Try to create a resolution CommandLineResolution

class CommandLineResolution(conflict, change_type)[source]

Representation of an commandline change resolution

Attributes
conflict: `orion.core.evc.conflicts.Conflict`

The conflict which is resolved by this resolution.

change_type: string

One of the types defined in orion.core.evc.adapters.CommandLineChange.types.

.. seealso ::

orion.core.evc.conflicts.Resolution

Methods

get_adapters()

Return CommandLineChange adapter

get_adapters()[source]

Return CommandLineChange adapter

classmethod detect(old_config, new_config, branching_config=None)[source]

Detect if command line call in new_config differs from old_config :param branching_config:

property diff

Produce human-readable differences

get_marked_arguments(conflicts, cli_change_type=None, **branching_kwargs)[source]

Find and return marked arguments for cli change conflict

classmethod get_nameless_args(config, user_script_config=None, non_monitored_arguments=None, **kwargs)[source]

Get user’s commandline arguments which are not dimension definitions

try_resolve(change_type=None)[source]

Try to create a resolution CommandLineResolution

Parameters
change_type: None or string

One of the types defined in orion.core.evc.adapters.CommandLineChange.types.

Raises
ValueError

If change_type is not in orion.core.evc.adapters.CommandLineChange.types.

class orion.core.evc.conflicts.Conflict(old_config, new_config)[source]

Representation of a conflict between two configurations

This object is used to embody a conflict during a branching event and provides means to resolve itself and to represent itself in user interface.

A conflict must provide implementations of:

  1. detect() – How it is detected in a pair (old_config, new_config).

  2. try_resolve() – How to resolve itself.

  3. __repr__() – How to represent itself in user interface.

Additionaly, it may also provide implementations of:

  1. diff() – How to compute diff string.

  2. get_marked_arguments() – How to find resolutions markers and their corresponding arguments in new_config.

Attributes
old_config: dict

Configuration of the parent experiment

new_config: dict

Configuration of the child experiment

resolution: None or `orion.core.evc.conflicts.Resolution`

None if not resolved or a Resolution object. Note that deprecated conflicts may be marked as resolved with _is_resolved = True even though resolution is None.

Methods

detect(old_config, new_config[, …])

Detect all conflicts in given pair (old_config, new_config) and return a list of them :param branching_config:

get_marked_arguments(conflicts, …)

Return arguments from marked resolutions in new configuration

try_resolve()

Try to create a resolution

classmethod detect(old_config, new_config, branching_config=None)[source]

Detect all conflicts in given pair (old_config, new_config) and return a list of them :param branching_config:

property diff

Produce human-readable differences

Returns
None or str

Returns None if the conflict cannot produce diffs, otherwise it returns the diff as a string (possibly multi-line).

get_marked_arguments(conflicts, **branching_kwargs)[source]

Return arguments from marked resolutions in new configuration

Some conflicts may be passed arguments with their marker to automate conflict resolution. For instance, a renaming resolution requires the name of a new dimension. In such case, the conflict for missing_dim would find missing_dim~>new_dim in the user command line of configuration arguments, fetch new_dim from conflicts and return the dictionary of arguments {‘new_dimension_conflict’: new_dimension_conflict}.

Parameters
conflicts: `orion.core.evc.conflicts.Conflicts`

Handler of the list of conflicts.

Returns
dict

Marked arguments for conflict.try_resolve(), which may latter be passed as **kwargs.

property is_resolved

Return True if conflict is set as resolved or if it has a resolution

abstract try_resolve()[source]

Try to create a resolution

Conflict is then marked as resolved and its attribute resolution now points to the resolution.

Returns
None or orion.core.evc.conflicts.Resolution

Returns None if the conflict is already resolved, otherwise it returns a resolution object if it is successful.

Raises
ValueError

If the resolution cannot be created without arguments or if the arguments passed are not valid. This is specific to each child of Conflict

class orion.core.evc.conflicts.Conflicts[source]

Handler of a list of conflicts

Registers, deprecate, resolve and fetch conflicts. Revert and fetch corresponding resolutions.

The helper method try_resolve wraps Conflict.try_resolve objects, handling invalid resolution errors messages and adding to its list additional new conflicts created by resolutions.

Attributes
conflicts: list of `Conflict`

List of conflicts which may be resolved or not.

Methods

deprecate(conflicts)

Remove given conflicts from the internal list of conflicts

get([types, dimension_name, callback])

Fetch conflicts

get_remaining([types, dimension_name, callback])

Fetch non resolved conflicts

get_resolutions([types, dimension_name, …])

Fetch resolutions

get_resolved([types, dimension_name, callback])

Fetch resolved conflicts

register(conflict)

Add a new conflict to the list of conflicts

revert(resolution_or_name)

Revert a resolution and deprecate conflicts if applicable

try_resolve(conflict, *args, **kwargs)

Wrap call to conflict.try_resolve

property are_resolved

Return True if all the current conflicts have been resolved

deprecate(conflicts)[source]

Remove given conflicts from the internal list of conflicts

get(types=(), dimension_name=None, callback=None)[source]

Fetch conflicts

Parameters
types: tuple of Conflict types

List of conflict types to fetch

dimension_name: None or string

name of a dimension to fetch. If not None, will raise an error if no conflict found.

callback: None or callable object

If not None, only conflict for which the callback return True will be returned by get()

Raises
ValueError

If argument dimension_name is not None and no conflict is found.

get_remaining(types=(), dimension_name=None, callback=None)[source]

Fetch non resolved conflicts

Note

See orion.core.evc.conflicts.Conflicts.get() for more information.

get_resolutions(types=(), dimension_name=None, callback=None)[source]

Fetch resolutions

Iterate over resolved conflicts and return their resolutions

Note

Some resolutions resolve many conflicts. This method only returns unique resolutions.

Note

See orion.core.evc.conflicts.Conflicts.get() for more information.

get_resolved(types=(), dimension_name=None, callback=None)[source]

Fetch resolved conflicts

Note

See orion.core.evc.conflicts.Conflicts.get() for more information.

register(conflict)[source]

Add a new conflict to the list of conflicts

revert(resolution_or_name)[source]

Revert a resolution and deprecate conflicts if applicable

try_resolve(conflict, *args, **kwargs)[source]

Wrap call to conflict.try_resolve

Catch errors on conflict.try_resolve and print traceback if argument silence_errors is False.

Resolutions may generate side-effect conflicts. In such case, they are added to interval’s list of conflicts.

Parameters
conflict: `orion.ore.evc.conflicts.Conflict`

Conflict object to call try_resolve.

silence_errors: bool

If True, errors raised on execution of conflict.try_resolve will be catched and silenced. If False, errors will be catched and traceback will be printed before methods return None. Defaults to False

*args:

Arguments to pass to conflict.try_resolve

**kwargs:

Keyword arguments to pass to conflict.try_resolve

class orion.core.evc.conflicts.ExperimentNameConflict(old_config, new_config)[source]

Representation of experiment name conflict

Attributes
diff

Produce no diff

version

Retrieve version of configuration

Methods

ExperimentNameResolution(conflict, new_name)

Representation of an experiment name resolution

detect(old_config, new_config[, …])

Return experiment name conflict no matter what

get_marked_arguments(conflicts, …)

Find and return marked arguments for experiment name conflict

try_resolve([new_name])

Try to create a resolution ExperimentNameResolution

class ExperimentNameResolution(conflict, new_name)[source]

Representation of an experiment name resolution

Attributes
conflict: `orion.core.evc.conflicts.Conflict`

The conflict which is resolved by this resolution.

new_name: string

A new name for the branching experiment.

Methods

get_adapters()

Return no adapters, trials need to adaptation to new experiment name

revert()

Reset conflict set experiment name back to old one in new configuration

get_adapters()[source]

Return no adapters, trials need to adaptation to new experiment name

property is_marked

Return True every time since the –branch-from argument is not used when incrementing version of an experiment.

revert()[source]

Reset conflict set experiment name back to old one in new configuration

classmethod detect(old_config, new_config, branching_config=None)[source]

Return experiment name conflict no matter what

Branching event cannot be triggered experiment name is not the same. :param branching_config:

property diff

Produce no diff

get_marked_arguments(conflicts, **branching_kwargs)[source]

Find and return marked arguments for experiment name conflict

try_resolve(new_name=None)[source]

Try to create a resolution ExperimentNameResolution

Parameters
new_name: None or string

A new name for the branching experiment. A ValueError is raised if name is already in database.

Raises
ValueError

If name already exists in database for current version.

property version

Retrieve version of configuration

class orion.core.evc.conflicts.MissingDimensionConflict(old_config, new_config, dimension, prior)[source]

Representation of a new dimension conflict

Attributes
dimension: `orion.algo.space.Dimension`

Dimension object which is defined in new_config but not in old_config.

prior: string

String representing the prior of the dimension

.. seealso ::

orion.core.evc.conflicts.Conflict

Methods

RemoveDimensionResolution(conflict[, …])

Representation of a remove dimension resolution

RenameDimensionResolution(conflict, …)

Representation of a rename dimension resolution

detect(old_config, new_config[, …])

Detect all missing dimensions in new_config based on old_config :param branching_config:

get_marked_arguments(conflicts, …)

Find and return marked arguments for remove or rename resolution

get_marked_remove_arguments(conflicts)

Find and return marked arguments for remove resolution

get_marked_rename_arguments(conflicts)

Find and return marked arguments for rename resolution

try_resolve([new_dimension_conflict, …])

Try to create a resolution RenameDimensionResolution of RemoveDimensionResolution

class RemoveDimensionResolution(conflict, default_value=None)[source]

Representation of a remove dimension resolution

Attributes
default_value: object

Default value for the missing dimension.

.. seealso ::

orion.core.evc.conflicts.Resolution

Methods

get_adapters()

Return DimensionDeletion adapter

get_adapters()[source]

Return DimensionDeletion adapter

property prefix

Build the new prior string, including the default value

class RenameDimensionResolution(conflict, new_dimension_conflict)[source]

Representation of a rename dimension resolution

Attributes
new_dimension_conflict: `orion.core.evc.conflicts.NewDimensionConflict`

New dimension to rename to.

.. seealso ::

orion.core.evc.conflicts.Resolution

Methods

get_adapters()

Return DimensionRenaming adapter

revert()

Reset conflict as well as side-effect conflicts and return the latter for deprecation

get_adapters()[source]

Return DimensionRenaming adapter

property prefix

Build the new prior string, including the default value

revert()[source]

Reset conflict as well as side-effect conflicts and return the latter for deprecation

classmethod detect(old_config, new_config, branching_config=None)[source]

Detect all missing dimensions in new_config based on old_config :param branching_config:

property diff

Produce human-readable differences

get_marked_arguments(conflicts, **branching_kwargs)[source]

Find and return marked arguments for remove or rename resolution

get_marked_remove_arguments(conflicts)[source]

Find and return marked arguments for remove resolution

get_marked_rename_arguments(conflicts)[source]

Find and return marked arguments for rename resolution

try_resolve(new_dimension_conflict=None, default_value=None)[source]

Try to create a resolution RenameDimensionResolution of RemoveDimensionResolution

Parameters
new_dimension_conflict: None or `orion.core.evc.conflicts.NewDimensionConflict`

Dimension used for a rename resolution. If None, a remove resolution will be created instead.

default_value: object

Default value for the missing dimension. Defaults to Dimension.NO_DEFAULT_VALUE. If Dimension.NO_DEFAULT_VALUE, default_value from corresponding dimension will be used. This argument is ignored if new_dimension_conflict is not None.

Raises
ValueError

If default_value is invalid for the corresponding dimension.

class orion.core.evc.conflicts.NewDimensionConflict(old_config, new_config, dimension, prior)[source]

Representation of a new dimension conflict

Attributes
dimension: `orion.algo.space.Dimension`

Dimension object which is defined in new_config but not in old_config.

prior: string

String representing the prior of the dimension

.. seealso ::

orion.core.evc.conflicts.Conflict

Methods

AddDimensionResolution(conflict[, default_value])

Representation of a new dimension resolution

detect(old_config, new_config[, …])

Detect all new dimensions in new_config based on old_config :param branching_config:

try_resolve([default_value])

Try to create a resolution AddDimensionResolution

class AddDimensionResolution(conflict, default_value=None)[source]

Representation of a new dimension resolution

Attributes
default_value: object

Default value for the new dimension.

.. seealso ::

orion.core.evc.conflicts.Resolution

Methods

get_adapters()

Return DimensionAddition adapter

get_adapters()[source]

Return DimensionAddition adapter

property new_prior

Build the new prior string, including the default value

property prefix

Build the prefix including the marker

classmethod detect(old_config, new_config, branching_config=None)[source]

Detect all new dimensions in new_config based on old_config :param branching_config:

property diff

Produce human-readable differences

try_resolve(default_value=None)[source]

Try to create a resolution AddDimensionResolution

Parameters
default_value: object

Default value for the new dimension. Defaults to Dimension.NO_DEFAULT_VALUE.

Raises
ValueError

If default_value is invalid for the corresponding dimension.

class orion.core.evc.conflicts.OrionVersionConflict(old_config, new_config)[source]

Representation of conflict due to different Orion versions

Attributes
diff

Produce human-readable differences

Methods

OrionVersionResolution(conflict)

Representation of an orion version resolution

detect(old_config, new_config[, …])

Detect if orion versions differs

try_resolve()

Try to create a resolution OrionVersionResolution

class OrionVersionResolution(conflict)[source]

Representation of an orion version resolution

Methods

get_adapters()

Return OrionVersionChange adapter

get_adapters()[source]

Return OrionVersionChange adapter

classmethod detect(old_config, new_config, branching_config=None)[source]

Detect if orion versions differs

property diff

Produce human-readable differences

try_resolve()[source]

Try to create a resolution OrionVersionResolution

class orion.core.evc.conflicts.Resolution(conflict)[source]

Representation of a resolution for a conflict between two configurations

This object is used to embody a resolution of a conflict during a branching event and provides means to validate itself, produce side-effect conflicts, detect corresponding user markers, produce corresponding adapters and represent itself in user interface.

The string representing a resolution is precisely what a user should type in command-line to resolve automatically a conflict.

A resolution must provide implementations of:

  1. get_adapters() – How to adapt trials from the two experiments.

  2. __repr__() – How to represent itself in user interface. Note: this should correspond to

    what user should enter in command-line for automatic resolution.

Additionaly, it may also provide implementations of:

  1. revert() – How to revert the resolution and reset corresponding conflicts

  2. _validate() – How to validate if arguments for the resolution are valid.

  3. find_marked_argument() – How to find marked arguments in commandline call or script

    config

Note that resolutions do not modify the configuration, with the exception of experiment name resolution, hence there is no support for diffs inside resolutions. The only diffs are between the two configurations, hence they are defined inside the conflicts.

Attributes
conflict: `orion.core.evc.conflicts.Conflict`

The conflict which is resolved by this resolution.

new_conflicts: list of `orion.core.evc.conflicts.Conflict`

The side-effect conflicts cause by this resolution.

MARKER: None or string

The special marker if resolution is intended for dimension conflicts, otherwise None

ARGUMENT: None or string

The command-line argument if the resolution is not intended for dimension conflicts.

Methods

find_marked_argument()

Find commandline argument on configuration argument which marks this type of resolution for automatic resolution

get_adapters()

Return adapters corresponding to the resolution

namespace()

Return namespace corresponding to self.ARGUMENT

revert()

Reset conflict as well as side-effect conflicts and return the latter for deprecation

validate(*args, **kwargs)

Wrap validate method to revert resolution on invalid arguments

find_marked_argument()[source]

Find commandline argument on configuration argument which marks this type of resolution for automatic resolution

abstract get_adapters()[source]

Return adapters corresponding to the resolution

property is_marked

If this resolution is specifically marked in commandline arguments or configuration arguments

classmethod namespace()[source]

Return namespace corresponding to self.ARGUMENT

ARGUMENT is a command line argument, thus something in the style of –code-change-type. When arguments are passed they are saved in namespace in the style of code_change_type. This property converts command-line style to namespace style.

revert()[source]

Reset conflict as well as side-effect conflicts and return the latter for deprecation

validate(*args, **kwargs)[source]

Wrap validate method to revert resolution on invalid arguments

class orion.core.evc.conflicts.ScriptConfigConflict(old_config, new_config)[source]

Representation of script configuration change conflict

Attributes
diff

Produce human-readable differences

Methods

ScriptConfigResolution(conflict, change_type)

Representation of a script configuration change resolution

detect(old_config, new_config[, …])

Detect if user’s script’s config file in new_config differs from old_config :param branching_config:

get_marked_arguments(conflicts[, …])

Find and return marked arguments for user’s script’s config change conflict

get_nameless_config(config[, user_script_config])

Get configuration dict of user’s script without dimension definitions

try_resolve([change_type])

Try to create a resolution ScriptConfigResolution

class ScriptConfigResolution(conflict, change_type)[source]

Representation of a script configuration change resolution

Attributes
conflict: `orion.core.evc.conflicts.Conflict`

The conflict which is resolved by this resolution.

change_type: string

One of the types defined in orion.core.evc.adapters.ScriptConfighange.types.

.. seealso ::

orion.core.evc.conflicts.Resolution

Methods

get_adapters()

Return ScriptdConfigChange adapter

get_adapters()[source]

Return ScriptdConfigChange adapter

classmethod detect(old_config, new_config, branching_config=None)[source]

Detect if user’s script’s config file in new_config differs from old_config :param branching_config:

property diff

Produce human-readable differences

get_marked_arguments(conflicts, config_change_type=None, **branching_kwargs)[source]

Find and return marked arguments for user’s script’s config change conflict

classmethod get_nameless_config(config, user_script_config=None, **branching_kwargs)[source]

Get configuration dict of user’s script without dimension definitions

try_resolve(change_type=None)[source]

Try to create a resolution ScriptConfigResolution

Parameters
change_type: None or string

One of the types defined in orion.core.evc.adapters.ScriptConfigChange.types.

Raises
ValueError

If change_type is not in orion.core.evc.adapters.ScriptConfigChange.types.

orion.core.evc.conflicts.detect_conflicts(old_config, new_config, branching=None)[source]

Generate a Conflicts object with all conflicts found in pair (old_config, new_config)