syne_tune.optimizer.schedulers.searchers.bayesopt.tuning_algorithms.bo_algorithm module

class syne_tune.optimizer.schedulers.searchers.bayesopt.tuning_algorithms.bo_algorithm.BayesianOptimizationAlgorithm(initial_candidates_generator, initial_candidates_scorer, num_initial_candidates, local_optimizer, pending_candidate_state_transformer, exclusion_candidates, num_requested_candidates, greedy_batch_selection, duplicate_detector, num_initial_candidates_for_batch=None, sample_unique_candidates=False, debug_log=None)[source]

Bases: NextCandidatesAlgorithm

Core logic of the Bayesian optimization algorithm

Parameters:

  • initial_candidates_generator (CandidateGenerator) – generator of candidates

  • initial_scoring_function – scoring function used to rank the initial candidates. Note: If a batch is selected in one go (num_requested_candidates > 1, greedy_batch_selection == False), this function should encourage diversity among its top scorers. In general, greedy batch selection is recommended.

  • num_initial_candidates (int) – how many initial candidates to generate, if possible

  • local_optimizer (LocalOptimizer) – local optimizer which starts from score minimizer. If a batch is selected in one go (not greedily), then local optimizations are started from the top num_requested_candidates ranked candidates (after scoring)

  • pending_candidate_state_transformer (Optional[ModelStateTransformer]) – Once a candidate is selected, it becomes pending, and the state is transformed by appending information. This is done by the transformer. This is object is needed only if next_candidates() goes through more than one outer iterations (i.e., if greedy_batch_selection == True and num_requested_candidates > 1. Otherwise, None can be passed here. Note: Model updates (by the state transformer) for batch candidates beyond the first do not involve fitting hyperparameters, so they are usually cheap.

  • exclusion_candidates (ExclusionList) – Set of candidates that should not be returned, because they are already labeled, currently pending, or have failed

  • num_requested_candidates (int) – number of candidates to return

  • greedy_batch_selection (bool) – If True and num_requested_candidates > 1, we generate, order, and locally optimize for each single candidate to be selected. Otherwise, this is done just once, and num_requested_candidates are extracted in one go. Note: If this is True, pending_candidate_state_transformer is needed.

  • duplicate_detector (DuplicateDetector) – used to make sure no candidates equal to already evaluated ones is returned

  • num_initial_candidates_for_batch (Optional[int]) – This is used only if num_requested_candidates > 1 and greedy_batch_selection == True. In this case, num_initial_candidates_for_batch overrides num_initial_candidates when selecting all but the first candidate for the batch. Typically, num_initial_candidates is larger than num_initial_candidates_for_batch in this case, which speeds up selecting large batches, but still select the first candidate thoroughly

  • sample_unique_candidates (bool) – If True, we check that initial candidates sampled at random are unique and disjoint from the exclusion list. This can be expensive. Defaults to False

  • debug_log (Optional[DebugLogPrinter]) – If a DebugLogPrinter object is passed here, it is used to write log messages

initial_candidates_generator: CandidateGenerator
initial_candidates_scorer: ScoringFunction
num_initial_candidates: int
local_optimizer: LocalOptimizer
pending_candidate_state_transformer: Optional[ModelStateTransformer]
exclusion_candidates: ExclusionList
num_requested_candidates: int
greedy_batch_selection: bool
duplicate_detector: DuplicateDetector
num_initial_candidates_for_batch: Optional[int] = None
sample_unique_candidates: bool = False
debug_log: Optional[DebugLogPrinter] = None
next_candidates()[source]
Return type:

List[Dict[str, Union[int, float, str]]]