syne_tune.blackbox_repository.conversion_scripts.scripts.yahpo_import module

Wrap Surrogates from YAHPO Gym - An Efficient Multi-Objective Multi-Fidelity Benchmark for Hyperparameter Optimization Florian Pfisterer, Lennart Schneider, Julia Moosbauer, Martin Binder, Bernd Bischl

syne_tune.blackbox_repository.conversion_scripts.scripts.yahpo_import.download(target_path, version)

class syne_tune.blackbox_repository.conversion_scripts.scripts.yahpo_import.BlackBoxYAHPO(benchmark, fidelities=None)

Bases: Blackbox

A wrapper that allows putting a ‘YAHPO’ BenchmarkInstance into a Blackbox.

If fidelities is given, it restricts fidelity_values to these values. The sequence must be positive int and increasing. This works only if there is a single fidelity attribute with integer values (but note that for some specific YAHPO benchmarks, a fractional fidelity is transformed to an integer one).

Even though YAHPO interpolates between fidelities, it can make sense to restrict them to the values which have really been acquired in the data. Note that this restricts multi-fidelity schedulers like HyperbandScheduler, in that all their rungs levels have to be fidelity values.

For example, for YAHPO iaml, the fidelity trainsize has been acquired at [0.05, 0.1, 0.2, 0.4, 0.6, 0.8, 1], this is transformed to [1, 2, 4, 8, 12, 16, 20]. By default, the fidelity is represented by cs.randint(1, 20), but if fidelities is passed, it uses cs.ordinal(fidelities).

Parameters:

• benchmark (BenchmarkSet) – YAHPO BenchmarkSet

• fidelities (Optional[List[int]]) – See above

set_instance(instance)

Set an instance for the underlying YAHPO Benchmark.

property instances: array

property fidelity_values: array

Returns:

Fidelity values; or None if the blackbox has none

property time_attribute: str

Name of the time column

syne_tune.blackbox_repository.conversion_scripts.scripts.yahpo_import.cs_to_synetune(config_space)

Convert ConfigSpace.ConfigSpace to a synetune configspace.

TODO cover all possible hyperparameters of ConfigSpace.ConfigSpace, right now we only convert the one we need.

syne_tune.blackbox_repository.conversion_scripts.scripts.yahpo_import.instantiate_yahpo(scenario, check=False, fidelities=None)

Instantiates a dict of BlackBoxYAHPO, one entry for each instance.

Parameters:

• scenario (str) –

• check (bool) – If False, objective_function of the blackbox does not check whether the input configuration is valid. This is faster, but calls fail silently if configurations are invalid.

Returns:

syne_tune.blackbox_repository.conversion_scripts.scripts.yahpo_import.serialize_yahpo(scenario, target_path, version='1.0')

class syne_tune.blackbox_repository.conversion_scripts.scripts.yahpo_import.YAHPORecipe(name)

Bases: BlackboxRecipe