syne_tune.blackbox_repository.blackbox_tabular module

class syne_tune.blackbox_repository.blackbox_tabular.BlackboxTabular(hyperparameters, configuration_space, fidelity_space, objectives_evaluations, fidelity_values=None, objectives_names=None)

Bases: Blackbox

Blackbox that contains tabular evaluations (e.g. all hyperparameters evaluated on all fidelities). We use a separate class than BlackboxOffline, as performance improvement can be made by avoiding to repeat hyperparameters and by storing all evaluations in a single table.

Additional arguments on top of parent class Blackbox:

Parameters:

• hyperparameters (DataFrame) – dataframe of hyperparameters, shape (num_evals, num_hps), columns must match hyperparameter names of configuration_space

• objectives_evaluations (array) – values of recorded objectives, must have shape (num_evals, num_seeds, num_fidelities, num_objectives)

• fidelity_values (Optional[array]) – values of the num_fidelities fidelities, default to [1, ..., num_fidelities]

property fidelity_values: array

Returns:

Fidelity values; or None if the blackbox has none

hyperparameter_objectives_values(predict_curves=False)

If predict_curves is False, the shape of X is (num_evals * num_seeds * num_fidelities, num_hps + 1), the shape of y is (num_evals * num_seeds * num_fidelities, num_objectives). This can be reshaped to (num_fidelities, num_seeds, num_evals, *). The final column of X is the fidelity value (only a single fidelity attribute is supported).

If predict_curves is True, the shape of X is (num_evals * num_seeds, num_hps), the shape of y is (num_evals * num_seeds, num_fidelities * num_objectives). The latter can be reshaped to (num_seeds, num_evals, num_fidelities, num_objectives).

Parameters:

predict_curves (bool) – See above. Default is False

Return type:

Tuple[DataFrame, DataFrame]

Returns:

Dataframes corresponding to X and y

rename_objectives(objective_name_mapping)

Parameters:

objective_name_mapping (Dict[str, str]) – dictionary from old objective name to new one, old objective name must be present in the blackbox

Return type:

BlackboxTabular

Returns:

a blackbox with as many objectives as objective_name_mapping

all_configurations()

This method is useful in order to set restrict_configurations in StochasticAndFilterDuplicatesSearcher or GPFIFOSearcher, which restricts the searcher to only return configurations in this set. This allows you to use a tabular blackbox without a surrogate.

Return type:

List[Dict[str, Any]]

Returns:

List of all hyperparameter configurations for which objective values can be returned

syne_tune.blackbox_repository.blackbox_tabular.serialize(bb_dict, path, metadata=None)

syne_tune.blackbox_repository.blackbox_tabular.deserialize(path)

Deserialize blackboxes contained in a path that were saved with serialize() above.

TODO: the API is currently dissonant with serialize(), deserialize() for BlackboxOffline as serialize is a member function there. A possible way to unify is to have serialize also be a free function for BlackboxOffline.

Parameters:

path (str) – a path that contains blackboxes that were saved with serialize()

Return type:

Dict[str, BlackboxTabular]

Returns:

a dictionary from task name to blackbox