syne_tune.optimizer.schedulers.searchers.bayesopt.tuning_algorithms.bo_algorithm_components module

class syne_tune.optimizer.schedulers.searchers.bayesopt.tuning_algorithms.bo_algorithm_components.IndependentThompsonSampling(predictor=None, active_metric=None, random_state=None)[source]

Bases: ScoringFunction

Note: This is not Thompson sampling, but rather a variant called “independent Thompson sampling”, where means and variances are drawn from the marginal rather than the joint distribution. This is cheap, but incorrect. In fact, the larger the number of candidates, the more likely the winning configuration is arising from pure chance.

Parameters:

  • predictor (Union[Predictor, Dict[str, Predictor], None]) – Surrogate predictor for statistics of predictive distribution

  • random_state (Optional[RandomState]) – PRN generator

score(candidates, predictor=None)[source]
Parameters:

  • candidates (Iterable[Dict[str, Union[int, float, str]]]) – Configurations for which scores are to be computed

  • predictor (Optional[Predictor]) – Overrides default predictor

Return type:

List[float]

Returns:

List of score values, length of candidates

class syne_tune.optimizer.schedulers.searchers.bayesopt.tuning_algorithms.bo_algorithm_components.LBFGSOptimizeAcquisition(hp_ranges, predictor, acquisition_class, active_metric=None)[source]

Bases: LocalOptimizer

optimize(candidate, predictor=None)[source]

Run local optimization, starting from candidate

Parameters:

  • candidate (Dict[str, Union[int, float, str]]) – Starting point

  • predictor (Union[Predictor, Dict[str, Predictor], None]) – Overrides self.predictor

Return type:

Dict[str, Union[int, float, str]]

Returns:

Configuration found by local optimization

class syne_tune.optimizer.schedulers.searchers.bayesopt.tuning_algorithms.bo_algorithm_components.NoOptimization(*args, **kwargs)[source]

Bases: LocalOptimizer

optimize(candidate, predictor=None)[source]

Run local optimization, starting from candidate

Parameters:

  • candidate (Dict[str, Union[int, float, str]]) – Starting point

  • predictor (Optional[Predictor]) – Overrides self.predictor

Return type:

Dict[str, Union[int, float, str]]

Returns:

Configuration found by local optimization

class syne_tune.optimizer.schedulers.searchers.bayesopt.tuning_algorithms.bo_algorithm_components.RandomStatefulCandidateGenerator(hp_ranges, random_state)[source]

Bases: CandidateGenerator

This generator maintains a random state, so if generate_candidates() is called several times, different sequences are returned.

Parameters:

  • hp_ranges (HyperparameterRanges) – Feature generator for configurations

  • random_state (RandomState) – PRN generator

generate_candidates()[source]
Return type:

Iterator[Dict[str, Union[int, float, str]]]

generate_candidates_en_bulk(num_cands, exclusion_list=None)[source]
Parameters:

  • num_cands (int) – Number of candidates to generate

  • exclusion_list – If given, these candidates must not be returned

Return type:

List[Dict[str, Union[int, float, str]]]

Returns:

List of num_cands candidates. If exclusion_list is given, the number of candidates returned can be < num_cands

syne_tune.optimizer.schedulers.searchers.bayesopt.tuning_algorithms.bo_algorithm_components.generate_unique_candidates(candidates_generator, num_candidates, exclusion_candidates)[source]
Return type:

List[Dict[str, Union[int, float, str]]]

class syne_tune.optimizer.schedulers.searchers.bayesopt.tuning_algorithms.bo_algorithm_components.RandomFromSetCandidateGenerator(base_set, random_state, ext_config=None)[source]

Bases: CandidateGenerator

In this generator, candidates are sampled from a given set.

Parameters:

  • base_set (List[Dict[str, Union[int, float, str]]]) – Set of all configurations to sample from

  • random_state (RandomState) – PRN generator

  • ext_config (Optional[Dict[str, Union[int, float, str]]]) – If given, each configuration is updated with this dictionary before being returned

generate_candidates()[source]
Return type:

Iterator[Dict[str, Union[int, float, str]]]

generate_candidates_en_bulk(num_cands, exclusion_list=None)[source]
Parameters:

  • num_cands (int) – Number of candidates to generate

  • exclusion_list – If given, these candidates must not be returned

Return type:

List[Dict[str, Union[int, float, str]]]

Returns:

List of num_cands candidates. If exclusion_list is given, the number of candidates returned can be < num_cands

class syne_tune.optimizer.schedulers.searchers.bayesopt.tuning_algorithms.bo_algorithm_components.DuplicateDetector[source]

Bases: object

contains(existing_candidates, new_candidate)[source]
Return type:

bool

class syne_tune.optimizer.schedulers.searchers.bayesopt.tuning_algorithms.bo_algorithm_components.DuplicateDetectorNoDetection[source]

Bases: DuplicateDetector

contains(existing_candidates, new_candidate)[source]
Return type:

bool

class syne_tune.optimizer.schedulers.searchers.bayesopt.tuning_algorithms.bo_algorithm_components.DuplicateDetectorIdentical[source]

Bases: DuplicateDetector

contains(existing_candidates, new_candidate)[source]
Return type:

bool