syne_tune.optimizer.schedulers.synchronous.hyperband module

class syne_tune.optimizer.schedulers.synchronous.hyperband.SynchronousHyperbandCommon(config_space, **kwargs)[source]

Bases: TrialSchedulerWithSearcher, MultiFidelitySchedulerMixin

Common code for _create_internal() in SynchronousHyperbandScheduler and DifferentialEvolutionHyperbandScheduler

property searcher: BaseSearcher | None
property resource_attr: str
Returns:

Name of resource attribute in reported results

property max_resource_level: int
Returns:

Maximum resource level

property searcher_data: str
Returns:

Relevant only if a model-based searcher is used. Example: For NN tuning and resource_attr == "epoch", we receive a result for each epoch, but not all epoch values are also rung levels. searcher_data determines which of these results are passed to the searcher. As a rule, the more data the searcher receives, the better its fit, but also the more expensive get_config() may become. Choices:

  • ”rungs”: Only results at rung levels. Cheapest

  • ”all”: All results. Most expensive

  • ”rungs_and_last”: Results at rung levels plus last recent one. Not available for all multi-fidelity schedulers

class syne_tune.optimizer.schedulers.synchronous.hyperband.SynchronousHyperbandScheduler(config_space, bracket_rungs, **kwargs)[source]

Bases: SynchronousHyperbandCommon, DefaultRemoveCheckpointsSchedulerMixin

Synchronous Hyperband. Compared to HyperbandScheduler, this is also scheduling jobs asynchronously, but decision-making is synchronized, in that trials are only promoted to the next milestone once the rung they are currently paused at, is completely occupied.

Our implementation never delays scheduling of a job. If the currently active bracket does not accept jobs, we assign the job to a later bracket. This means that at any point in time, several brackets can be active, but jobs are preferentially assigned to the first one (the “primary” active bracket).

Parameters:

  • config_space (Dict[str, Any]) – Configuration space for trial evaluation function

  • bracket_rungs (List[List[Tuple[int, int]]]) – Determines rung level systems for each bracket, see SynchronousHyperbandBracketManager

  • metric (str) – Name of metric to optimize, key in result’s obtained via on_trial_result()

  • searcher (str, optional) – Searcher for get_config decisions. Passed to searcher_factory() along with search_options and extra information. Supported values: SUPPORTED_SEARCHERS_HYPERBAND. Defaults to “random” (i.e., random search)

  • search_options (Dict[str, Any], optional) – Passed to searcher_factory().

  • mode (str, optional) – Mode to use for the metric given, can be “min” (default) or “max”

  • points_to_evaluate (List[dict], optional) – List of configurations to be evaluated initially (in that order). Each config in the list can be partially specified, or even be an empty dict. For each hyperparameter not specified, the default value is determined using a midpoint heuristic. If None (default), this is mapped to [dict()], a single default config determined by the midpoint heuristic. If [] (empty list), no initial configurations are specified.

  • random_seed (int, optional) – Master random seed. Generators used in the scheduler or searcher are seeded using RandomSeedGenerator. If not given, the master random seed is drawn at random here.

  • max_resource_attr (str, optional) – Key name in config for fixed attribute containing the maximum resource. If given, trials need not be stopped, which can run more efficiently.

  • max_resource_level (int, optional) – Largest rung level, corresponds to max_t in FIFOScheduler. Must be positive int larger than grace_period. If this is not given, it is inferred like in FIFOScheduler. In particular, it is not needed if max_resource_attr is given.

  • resource_attr (str, optional) – Name of resource attribute in results obtained via ``on_trial_result(). The type of resource must be int. Default to “epoch”

  • searcher_data (str, optional) –

    Relevant only if a model-based searcher is used. Example: For NN tuning and resource_attr == "epoch", we receive a result for each epoch, but not all epoch values are also rung levels. searcher_data determines which of these results are passed to the searcher. As a rule, the more data the searcher receives, the better its fit, but also the more expensive get_config may become. Choices:

    • ”rungs” (default): Only results at rung levels. Cheapest

    • ”all”: All results. Most expensive

    Note: For a Gaussian additive learning curve surrogate model, this has to be set to “all”.

property rung_levels: List[int]
Returns:

Rung levels (positive int; increasing), may or may not include max_resource_level

property num_brackets: int
Returns:

Number of brackets (i.e., rung level systems). If the scheduler does not use brackets, it has to return 1

on_trial_result(trial, result)[source]

Called on each intermediate result reported by a trial.

At this point, the trial scheduler can make a decision by returning one of SchedulerDecision.CONTINUE, SchedulerDecision.PAUSE, or SchedulerDecision.STOP. This will only be called when the trial is currently running.

Parameters:

  • trial (Trial) – Trial for which results are reported

  • result (Dict[str, Any]) – Result dictionary

Return type:

str

Returns:

Decision what to do with the trial

on_trial_error(trial)[source]

Given the trial is currently pending, we send a result at its milestone for metric value NaN. Such trials are ranked after all others and will most likely not be promoted.

metric_names()[source]
Return type:

List[str]

Returns:

List of metric names. The first one is the target metric optimized over, unless the scheduler is a genuine multi-objective metric (for example, for sampling the Pareto front)

metric_mode()[source]
Return type:

str

Returns:

“min” if target metric is minimized, otherwise “max”. Here, “min” should be the default. For a genuine multi-objective scheduler, a list of modes is returned

trials_checkpoints_can_be_removed()[source]

Supports the general case (see header comment). This method returns IDs of paused trials for which checkpoints can safely be removed. These trials either cannot be resumed anymore, or it is very unlikely they will be resumed. Any trial ID needs to be returned only once, not over and over. If a trial gets stopped (by returning SchedulerDecision.STOP in on_trial_result()), its checkpoint is removed anyway, so its ID does not have to be returned here.

Return type:

List[int]

Returns:

IDs of paused trials for which checkpoints can be removed