Conveys a dense overview of the trial objectives in a multi-dimensional space. Helps identifying trends of best or worst hyperparameter values.
The parallel coordinates plot decomposes a search space of n dimensions into n axis so that the entire space can be visualized simultaneously. Each dimension is represented as a vertical axis and trials are represented as lines crossing each axis at the corresponding value of the hyperparameters. There is no obvious optimal ordering for the vertical axis, and you will often find that changing the order helps better understanding the data. Additionaly, the lines are plotted with graded colors based on the objective. The gradation is shown in a color bar on the right of the plot. Note that the objectives are added as the last axis is the plot as well.
- orion.plotting.base.parallel_coordinates(experiment, with_evc_tree=True, order=None, **kwargs)
Make a Parallel Coordinates Plot to visualize the effect of the hyperparameters on the objective.
- experiment: ExperimentClient or Experiment
The orion object containing the experiment data
- with_evc_tree: bool, optional
Fetch all trials from the EVC tree. Default: True
- order: list of str or None
Indicates the order of columns in the parallel coordinate plot. By default the columns are sorted alphabetically with the exception of the first column which is reserved for a fidelity dimension is there is one in the search space.
- kwargs: dict
All other plotting keyword arguments to be passed to plotly.express.line.
If no experiment is provided.
The parallel coordinates plot can be executed directly from the
plot.parallel_coordinates() as shown in the example below.
from orion.client import get_experiment # Specify the database where the experiments are stored. We use a local PickleDB here. storage = dict(type="legacy", database=dict(type="pickleddb", host="../db.pkl")) # Load the data for the specified experiment experiment = get_experiment("2-dim-exp", storage=storage) fig = experiment.plot.parallel_coordinates() fig
In this basic example the parallel coordinates plot is marginally useful as there are only
2 dimensions. It is possible however to identify the best performing values of
learning_rate. The GIF below demonstrates how to select subsets of the
axis to highlight the trials that corresponds to the best objectives.
Hover is not supported by plotly at the moment. Feature request can be tracked here.
Lets now load the results from tutorial Checkpointing trials for an example with a larger search space.
As you can see, the large number of trials until trained for a few epochs is cluttering the entire plot. You can first select the trials with 120 epochs to clear the plot. Once that is done, We can see that gamma and momentum had limited influence. Good trials can be found for almost any values of gamma and momentum. On the other hand, learning rate and weight decay are clearly more optimal in lower values. You can try re-ordering the columns as shown in the animation below to see the connections between one hyperparameter and the objective.
We can also select a subset of hyperparameters to help with the visualization.
Logarithmic scales are not supported yet. Contributions are welcome. :) See issue.
Dimension with shape¶
If some dimensions have a shape larger than 1, they will be flattened so that each subdimension can be represented in the parallel coordinates plot.
In the example above, the dimension
learning_rate~loguniform(1e-5, 1e-2, shape=3)
is flattened and represented with
learning_rate[i]. If the shape would be or more dimensions
(3, 2)), the indices would be
learning_rate[i,j] with i=0..2 and j=0..1.
The flattened hyperparameters can be fully selected with
Or a subset of the flattened hyperparameters can be selected with
Parallel coordinates plots can also render categorical dimensions, in which case the categories are shown in an arbitrary order on the axis.
Finally we save the image to serve as a thumbnail for this example. See the guide How to save for more information on image saving.
Total running time of the script: ( 0 minutes 1.848 seconds)