syne_tune.optimizer.schedulers.searchers.kde.kde_searcher module
- class syne_tune.optimizer.schedulers.searchers.kde.kde_searcher.KernelDensityEstimator(config_space, metric, points_to_evaluate=None, allow_duplicates=None, mode=None, num_min_data_points=None, top_n_percent=None, min_bandwidth=None, num_candidates=None, bandwidth_factor=None, random_fraction=None, **kwargs)[source]
Bases:
StochasticAndFilterDuplicatesSearcher
Fits two kernel density estimators (KDE) to model the density of the top N configurations as well as the density of the configurations that are not among the top N, respectively. New configurations are sampled by optimizing the ratio of these two densities. KDE as model for Bayesian optimization has been originally proposed by Bergstra et al. Compared to their original implementation TPE, we use multi-variate instead of univariate KDE, as proposed by Falkner et al. Code is based on the implementation by Falkner et al: https://github.com/automl/HpBandSter/tree/master/hpbandster
Algorithms for Hyper-Parameter OptimizationJ. Bergstra and R. Bardenet and Y. Bengio and B. K{‘e}glProceedings of the 24th International Conference on Advances in Neural Information Processing Systemsand
BOHB: Robust and Efficient Hyperparameter Optimization at ScaleS. Falkner and A. Klein and F. HutterProceedings of the 35th International Conference on Machine LearningNote:
restrict_configurations
is not supported here, this would require reimplementing the selection of configs in_get_config()
.Additional arguments on top of parent class
StochasticAndFilterDuplicatesSearcher
:- Parameters:
mode (
Optional
[str
]) – Mode to use for the metric given, can be “min” or “max”. Is obtained from scheduler inconfigure_scheduler()
. Defaults to “min”num_min_data_points (
Optional
[int
]) – Minimum number of data points that we use to fit the KDEs. As long as less observations have been received inupdate()
, randomly drawn configurations are returned inget_config()
. If set toNone
, we set this to the number of hyperparameters. Defaults toNone
.top_n_percent (
Optional
[int
]) – Determines how many datapoints we use to fit the first KDE model for modeling the well performing configurations. Defaults to 15min_bandwidth (
Optional
[float
]) – The minimum bandwidth for the KDE models. Defaults to 1e-3num_candidates (
Optional
[int
]) – Number of candidates that are sampled to optimize the acquisition function. Defaults to 64bandwidth_factor (
Optional
[int
]) – We sample continuous hyperparameter from a truncated Normal. This factor is multiplied to the bandwidth to define the standard deviation of this truncated Normal. Defaults to 3random_fraction (
Optional
[float
]) – Defines the fraction of configurations that are drawn uniformly at random instead of sampling from the model. Defaults to 0.33
- configure_scheduler(scheduler)[source]
Some searchers need to obtain information from the scheduler they are used with, in order to configure themselves. This method has to be called before the searcher can be used.
- Parameters:
scheduler (
TrialScheduler
) – Scheduler the searcher is used with.
- clone_from_state(state)[source]
Together with
get_state()
, this is needed in order to store and re-create the mutable state of the searcher.Given state as returned by
get_state()
, this method combines the non-pickle-able part of the immutable state from self with state and returns the corresponding searcher clone. Afterwards,self
is not used anymore.- Parameters:
state (
Dict
[str
,Any
]) – See above- Returns:
New searcher object