syne_tune.optimizer.schedulers.transfer_learning.quantile_based.quantile_based_searcher module

syne_tune.optimizer.schedulers.transfer_learning.quantile_based.quantile_based_searcher.extract_input_output(transfer_learning_evaluations, normalization, random_state)

syne_tune.optimizer.schedulers.transfer_learning.quantile_based.quantile_based_searcher.fit_model(config_space, transfer_learning_evaluations, normalization, max_fit_samples, random_state, model=XGBRegressor(base_score=None, booster=None, callbacks=None, colsample_bylevel=None, colsample_bynode=None, colsample_bytree=None, device=None, early_stopping_rounds=None, enable_categorical=False, eval_metric=None, feature_types=None, gamma=None, grow_policy=None, importance_type=None, interaction_constraints=None, learning_rate=None, max_bin=None, max_cat_threshold=None, max_cat_to_onehot=None, max_delta_step=None, max_depth=None, max_leaves=None, min_child_weight=None, missing=nan, monotone_constraints=None, multi_strategy=None, n_estimators=None, n_jobs=None, num_parallel_tree=None, random_state=None, ...))

syne_tune.optimizer.schedulers.transfer_learning.quantile_based.quantile_based_searcher.eval_model(model_pipeline, X, y)

syne_tune.optimizer.schedulers.transfer_learning.quantile_based.quantile_based_searcher.subsample(X, y, max_samples=10000, random_state=None)

Subsample both X and y with max_samples elements. If max_samples is not set then X and y are returned as such and if it is set, the index of X is reset. :rtype: Tuple[DataFrame, array] :return: (X, y) with max_samples sampled elements.

class syne_tune.optimizer.schedulers.transfer_learning.quantile_based.quantile_based_searcher.QuantileBasedSurrogateSearcher(config_space, metric, transfer_learning_evaluations, mode=None, max_fit_samples=100000, normalization='gaussian', **kwargs)

Bases: StochasticSearcher

Implements the transfer-learning method:

A Quantile-based Approach for Hyperparameter Transfer Learning.

David Salinas, Huibin Shen, Valerio Perrone.

ICML 2020.

This is the Copula Thompson Sampling approach described in the paper where a surrogate is fitted on the transfer learning data to predict mean/variance of configuration performance given a hyperparameter. The surrogate is then sampled from and the best configurations are returned as next candidate to evaluate.

Additional arguments on top of parent class StochasticSearcher:

Parameters:

• mode (Optional[str]) – Whether to minimize or maximize, default to “min”.

• transfer_learning_evaluations (Dict[str, TransferLearningTaskEvaluations]) – Dictionary from task name to offline evaluations.

• max_fit_samples (int) – Maximum number to use when fitting the method. Defaults to 100000

• normalization (str) – Default to “gaussian” which first computes the rank and then applies Gaussian inverse CDF. “standard” applies just standard normalization (remove mean and divide by variance) but can perform significantly worse.

clone_from_state(state)

Together with get_state(), this is needed in order to store and re-create the mutable state of the searcher.

Given state as returned by get_state(), this method combines the non-pickle-able part of the immutable state from self with state and returns the corresponding searcher clone. Afterwards, self is not used anymore.

Parameters:

state (Dict[str, Any]) – See above

Returns:

New searcher object

get_config(**kwargs)

Suggest a new configuration.

Note: Query _next_initial_config() for initial configs to return first.

Parameters:

kwargs – Extra information may be passed from scheduler to searcher

Return type:

Optional[dict]

Returns:

New configuration. The searcher may return None if a new configuration cannot be suggested. In this case, the tuning will stop. This happens if searchers never suggest the same config more than once, and all configs in the (finite) search space are exhausted.