syne_tune.optimizer.schedulers.searchers.bore.multi_fidelity_bore module

class syne_tune.optimizer.schedulers.searchers.bore.multi_fidelity_bore.MultiFidelityBore(config_space, metric, points_to_evaluate=None, allow_duplicates=None, mode=None, gamma=None, calibrate=None, classifier=None, acq_optimizer=None, feval_acq=None, random_prob=None, init_random=None, classifier_kwargs=None, resource_attr='epoch', **kwargs)

Bases: Bore

Adapts BORE (Tiao et al.) for the multi-fidelity Hyperband setting following BOHB (Falkner et al.). Once we collected enough data points on the smallest resource level, we fit a probabilistic classifier and sample from it until we have a sufficient amount of data points for the next higher resource level. We then refit the classifier on the data of this resource level. These steps are iterated until we reach the highest resource level. References:

BORE: Bayesian Optimization by Density-Ratio Estimation,

Tiao, Louis C and Klein, Aaron and Seeger, Matthias W and Bonilla, Edwin V. and Archambeau, Cedric and Ramos, Fabio

Proceedings of the 38th International Conference on Machine Learning

and

BOHB: Robust and Efficient Hyperparameter Optimization at Scale

S. Falkner and A. Klein and F. Hutter

Proceedings of the 35th International Conference on Machine Learning

Additional arguments on top of parent class Bore:

Parameters:

resource_attr (str) – Name of resource attribute. Defaults to “epoch”

configure_scheduler(scheduler)

Some searchers need to obtain information from the scheduler they are used with, in order to configure themselves. This method has to be called before the searcher can be used.

Parameters:

scheduler (TrialScheduler) – Scheduler the searcher is used with.