Each experiment can have a different hyperparameter configuration. Because there are many hyperparameters and many different configurations for our experiments, we manage our hyperparameters and arguments with hydra. They also have nice tutorials!
Hydra works with configuration files. We store our configurations in conf
. The following screenshot shows an overview of our configurations.
Default parameters
Hydra loads the hyperparameters from main.yaml
, which specifies the most important default parameters. When we run python src/main.py
, hydra automatically adds all the default parameters from this config. For example, the default environment is specified in main.yaml
with env: 'FetchReach-v2'
. main.yaml
also contains good explanations for the most important arguments!
Override arguments
Let’s say we want to choose a different environment. Then we can specify it by setting the env
argument like this: python src/main.py env=FetchPush-v2
. Note that we do not need any quotes like ""
or ''
for the environment name.
Add arguments
If the argument is not in the default arguments, we can add it by writing a +
before it. For example, env_kwargs.reward_type
is not in the default arguments. We can change the reward type of the environment to dense
(default is sparse
) with +env_kwargs.reward_type=dense
.
Override or add arguments
With ++
, you can override arguments or add them if they’re not in the default arguments. E.g. ++n_epochs=3
.
Hierarchical configurations
The advantage of hydra is that we do not have to type out all the hyperparameters every run, but can reuse them with hierarchical configurations. For example, we have a configuration file for each algorithm. This contains the default parameters for the algorithm. Hierarchical arguments are referenced with a .
, e.g. the learning rate of the algorithm can be set with ++algorithm.learning_rate=0.007
.
Performance
The performance
folder contains configurations for hyperparameter optimization and performance testing. They have to be run with the --multirun
flag. E.g. to run performance testing for the FetchReach-v2
environment with SAC & HER, run python src/main.py +performance=FetchReach/sac_her-opti --multirun
. More on this in the Hyperparameter optimization tutorial.
Smoke test config
conf/performance/smoke_test.yaml
is the configuration for smoke tests.