syne_tune.optimizer.schedulers.searchers.botorch.botorch_searcher module

class syne_tune.optimizer.schedulers.searchers.botorch.botorch_searcher.BoTorchSearcher(config_space, points_to_evaluate=None, num_init_random=3, no_fantasizing=False, max_num_observations=200, input_warping=True, random_seed=None)

Bases: SingleObjectiveBaseSearcher

A searcher that suggest configurations using BOTORCH to build GP surrogate and optimize acquisition function.

qExpectedImprovement is used for the acquisition function, given that it supports pending evaluations.

Additional arguments on top of parent class StochasticAndFilterDuplicatesSearcher:

Parameters:

• mode – “min” (default) or “max”

• num_init_random (int) – get_config() returns randomly drawn configurations until at least init_random observations have been recorded in update(). After that, the BOTorch algorithm is used. Defaults to 3

• no_fantasizing (bool) – If True, fantasizing is not done and pending evaluations are ignored. This may lead to loss of diversity in decisions. Defaults to False

• max_num_observations (Optional[int]) – Maximum number of observation to use when fitting the GP. If the number of observations gets larger than this number, then data is subsampled. If None, then all data is used to fit the GP. Defaults to 200

• input_warping (bool) – Whether to apply input warping when fitting the GP. Defaults to True

on_trial_complete(trial_id, config, metric)

Inform searcher about result

The scheduler passes every result. If update == True, the searcher should update its surrogate model (if any), otherwise result is an intermediate result not modelled.

The default implementation calls _update() if update == True. It can be overwritten by searchers which also react to intermediate results.

Parameters:

• trial_id (int) – See on_trial_result()

• config (Dict[str, Any]) – See on_trial_result()

• metric (float) – See on_trial_result()

num_suggestions()

suggest()

Suggest a new configuration.

Note: Query _next_points_to_evaluate() for initial configs to return first.

Parameters:

kwargs – Extra information may be passed from scheduler to searcher

Return type:

Optional[dict]

Returns:

New configuration. The searcher may return None if a new configuration cannot be suggested. In this case, the tuning will stop. This happens if searchers never suggest the same config more than once, and all configs in the (finite) search space are exhausted.

evaluation_failed(trial_id)

cleanup_pending(trial_id)

dataset_size()

objectives()