syne_tune.optimizer.schedulers.synchronous.hyperband_impl module

class syne_tune.optimizer.schedulers.synchronous.hyperband_impl.SynchronousGeometricHyperbandScheduler(config_space, **kwargs)[source]

Bases: SynchronousHyperbandScheduler

Special case of SynchronousHyperbandScheduler with rung system defined by geometric sequences (see SynchronousHyperbandRungSystem.geometric()). This is the most frequently used case.

Parameters:

config_space (Dict[str, Any]) – Configuration space for trial evaluation function
metric (str) – Name of metric to optimize, key in result’s obtained via on_trial_result()
grace_period (int, optional) – Smallest (resource) rung level. Must be positive int. Defaults to 1
reduction_factor (float, optional) – Approximate ratio of successive rung levels. Must be >= 2. Defaults to 3
brackets (int, optional) – Number of brackets to be used. The default is to use the maximum number of brackets per iteration. Pass 1 for successive halving.
searcher (str, optional) – Selects searcher. Passed to searcher_factory(). Defaults to “random”
search_options (Dict[str, Any], optional) – Passed to searcher_factory().
mode (str, optional) – Mode to use for the metric given, can be “min” (default) or “max”
points_to_evaluate (List[dict], optional) – List of configurations to be evaluated initially (in that order). Each config in the list can be partially specified, or even be an empty dict. For each hyperparameter not specified, the default value is determined using a midpoint heuristic. If None (default), this is mapped to [dict()], a single default config determined by the midpoint heuristic. If [] (empty list), no initial configurations are specified.
random_seed (int, optional) – Master random seed. Generators used in the scheduler or searcher are seeded using RandomSeedGenerator. If not given, the master random seed is drawn at random here.
max_resource_level (int, optional) – Largest rung level, corresponds to max_t in FIFOScheduler. Must be positive int larger than grace_period. If this is not given, it is inferred like in FIFOScheduler. In particular, it is not needed if max_resource_attr is given.
max_resource_attr (str, optional) – Key name in config for fixed attribute containing the maximum resource. If given, trials need not be stopped, which can run more efficiently.
resource_attr (str, optional) – Name of resource attribute in results obtained via ``on_trial_result(). The type of resource must be int. Default to “epoch”
searcher_data (str, optional) –
Relevant only if a model-based searcher is used. Example: For NN tuning and resource_attr == "epoch", we receive a result for each epoch, but not all epoch values are also rung levels. searcher_data determines which of these results are passed to the searcher. As a rule, the more data the searcher receives, the better its fit, but also the more expensive get_config may become. Choices:
- ”rungs” (default): Only results at rung levels. Cheapest
- ”all”: All results. Most expensive
Note: For a Gaussian additive learning curve surrogate model, this has to be set to “all”.

class syne_tune.optimizer.schedulers.synchronous.hyperband_impl.GeometricDifferentialEvolutionHyperbandScheduler(config_space, **kwargs)[source]

Bases: DifferentialEvolutionHyperbandScheduler

Special case of DifferentialEvolutionHyperbandScheduler with rung system defined by geometric sequences. This is the most frequently used case.

Parameters:

config_space (Dict[str, Any]) – Configuration space for trial evaluation function
grace_period (int, optional) – Smallest (resource) rung level. Must be positive int. Defaults to 1
reduction_factor (float, optional) – Approximate ratio of successive rung levels. Must be >= 2. Defaults to 3
brackets (int, optional) – Number of brackets to be used. The default is to use the maximum number of brackets per iteration. Pass 1 for successive halving.
metric (str) – Name of metric to optimize, key in result’s obtained via on_trial_result()
searcher (str, optional) – Selects searcher. Passed to searcher_factory().. If searcher == "random_encoded" (default), the encoded configs are sampled directly, each entry independently from U([0, 1]). This distribution has higher entropy than for “random” if there are discrete hyperparameters in config_space. Note that points_to_evaluate is still used in this case.
search_options (Dict[str, Any], optional) – Passed to searcher_factory().
mode (str, optional) – Mode to use for the metric given, can be “min” (default) or “max”
points_to_evaluate (List[dict], optional) – List of configurations to be evaluated initially (in that order). Each config in the list can be partially specified, or even be an empty dict. For each hyperparameter not specified, the default value is determined using a midpoint heuristic. If None (default), this is mapped to [dict()], a single default config determined by the midpoint heuristic. If [] (empty list), no initial configurations are specified.
random_seed (int, optional) – Master random seed. Generators used in the scheduler or searcher are seeded using RandomSeedGenerator. If not given, the master random seed is drawn at random here.
max_resource_level (int, optional) – Largest rung level, corresponds to max_t in FIFOScheduler. Must be positive int larger than grace_period. If this is not given, it is inferred like in FIFOScheduler. In particular, it is not needed if max_resource_attr is given.
max_resource_attr (str, optional) – Key name in config for fixed attribute containing the maximum resource. If given, trials need not be stopped, which can run more efficiently.
resource_attr (str, optional) – Name of resource attribute in results obtained via on_trial_result(). The type of resource must be int. Default to “epoch”
mutation_factor (float, optional) – In \((0, 1]\). Factor \(F\) used in the rand/1 mutation operation of DE. Default to 0.5
crossover_probability (float, optional) – In \((0, 1)\). Probability \(p\) used in crossover operation (child entries are chosen with probability \(p\)). Defaults to 0.5
support_pause_resume (bool, optional) – If True, _suggest() supports pause and resume in the first bracket (this is the default). If the objective supports checkpointing, this is made use of. Defaults to True. Note: The resumed trial still gets assigned a new trial_id, but it starts from the earlier checkpoint.