benchmarking.utils.searcher_state_callback module

class benchmarking.utils.searcher_state_callback.StoreSearcherStatesCallback

Bases: TunerCallback

Stores list of searcher states alongside a tuning run. The list is extended by a new state whenever the TuningJobState has changed compared to the last recently added one.

This callback is useful to create meaningful unit tests, by sampling a given searcher alongside a realistic experiment.

Works only for ModelBasedSearcher searchers. For other searchers, nothing is stored.

on_tuning_start(tuner)

Called at start of tuning loop

Parameters:

tuner (Tuner) – Tuner object

on_trial_result(trial, status, result, decision)

Called when a new result (reported by a trial) is observed

The arguments here are inputs or outputs of scheduler.on_trial_result (called just before).

Parameters:

• trial (Trial) – Trial whose report has been received

• status (str) – Status of trial before scheduler.on_trial_result has been called

• result (Dict) – Result dict received

• decision (str) – Decision returned by scheduler.on_trial_result

property states

searcher_state_as_code(pos, add_info=False)