syne_tune.blackbox_repository.repository module

syne_tune.blackbox_repository.repository.blackbox_list()

Return type:

List[str]

Returns:

list of blackboxes available

syne_tune.blackbox_repository.repository.load_blackbox(name, skip_if_present=True, s3_root=None, generate_if_not_found=True, yahpo_kwargs=None, ignore_hash=True)

Parameters:

• name (str) –

  name of a blackbox present in the repository, see blackbox_list() to get list of available blackboxes. Syne Tune currently provides the following blackboxes evaluations:

  • ”nasbench201”: 15625 multi-fidelity configurations of computer vision architectures evaluated on 3 datasets. NAS-Bench-201: Extending the scope of reproducible neural architecture search. Dong, X. and Yang, Y. 2020.

  • ”fcnet”: 62208 multi-fidelity configurations of MLP evaluated on 4 datasets. Tabular benchmarks for joint architecture and hyperparameter optimization. Klein, A. and Hutter, F. 2019.

  • ”lcbench”: 2000 multi-fidelity Pytorch model configurations evaluated on many datasets. Reference: Auto-PyTorch: Multi-Fidelity MetaLearning for Efficient and Robust AutoDL. Lucas Zimmer, Marius Lindauer, Frank Hutter. 2020.

  • ”icml-deepar”: 2420 single-fidelity configurations of DeepAR forecasting algorithm evaluated on 10 datasets. A quantile-based approach for hyperparameter transfer learning. Salinas, D., Shen, H., and Perrone, V. 2021.

  • ”icml-xgboost”: 5O00 single-fidelity configurations of XGBoost evaluated on 9 datasets. A quantile-based approach for hyperparameter transfer learning. Salinas, D., Shen, H., and Perrone, V. 2021.

  • ”yahpo-*”: Number of different benchmarks from YAHPO Gym. Note that these blackboxes come with surrogates already, so no need to wrap them into SurrogateBlackbox

• skip_if_present (bool) – skip the download if the file locally exists

• s3_root (Optional[str]) – S3 root directory for blackbox repository. Defaults to S3 bucket name of SageMaker session

• generate_if_not_found (bool) – If the blackbox file is not present locally or on S3, should it be generated using its conversion script?

• yahpo_kwargs (Optional[dict]) – For a YAHPO blackbox (name == "yahpo-*"), these are additional arguments to instantiate_yahpo

• ignore_hash (bool) – do not check if hash of currently stored files matches the pre-computed hash. Be careful with this option. If hashes do not match, results might not be reproducible.

Return type:

Union[Dict[str, Blackbox], Blackbox]

Returns:

blackbox with the given name, download it if not present.

syne_tune.blackbox_repository.repository.check_blackbox_local_files(tgt_folder)

checks whether the file of the blackbox name are present in repository_path

Return type:

bool