syne_tune.config_space module

class syne_tune.config_space.Domain

Bases: object

Base class to specify a type and valid range to sample parameters from.

This base class is implemented by parameter spaces, like float ranges (Float), integer ranges (Integer), or categorical variables (Categorical). The Domain object contains information about valid values (e.g. minimum and maximum values), and exposes methods that allow specification of specific samplers (e.g. uniform() or loguniform()).

sampler = None

default_sampler_cls = None

property value_type

Returns:

Type of values (one of str, float, int)

cast(value)

Parameters:

value – Value top cast

Returns:

value cast to domain. For a finite domain, this can involve rounding

set_sampler(sampler, allow_override=False)

get_sampler()

Return type:

Sampler

sample(spec=None, size=1, random_state=None)

Parameters:

• spec (Union[List[dict], dict, None]) – Passed to sampler

• size (int) – Number of values to sample, defaults to 1

• random_state (Optional[RandomState]) – PRN generator

Return type:

Union[Any, List[Any]]

Returns:

Single value (size == 1) or list (size > 1)

is_grid()

Return type:

bool

is_function()

Return type:

bool

is_valid(value)

Parameters:

value (Any) – Value to test

Returns:

Is value a valid value in domain?

property domain_str

match_string(value)

Returns string representation of value (which must be of domain type) which is to match configurations for (approximate) equality. For discrete types (e.g., Integer, Categorical), this matches for exact equality.

Parameters:

value (Any) – Value of domain type (use cast() to be safe)

Return type:

str

Returns:

String representation useful for matching

class syne_tune.config_space.Sampler

Bases: object

sample(domain, spec=None, size=1, random_state=None)

class syne_tune.config_space.BaseSampler

Bases: Sampler

class syne_tune.config_space.Uniform

Bases: Sampler

class syne_tune.config_space.LogUniform(base=2.718281828459045)

Bases: Sampler

Note: We keep the argument base for compatibility with Ray Tune. Since base has no effect on the distribution, we don’t use it internally.

class syne_tune.config_space.Normal(mean=0.0, sd=0.0)

Bases: Sampler

class syne_tune.config_space.Grid

Bases: Sampler

Dummy sampler used for grid search

sample(domain, spec=None, size=1, random_state=None)

class syne_tune.config_space.Float(lower, upper)

Bases: Domain

Continuous value in closed interval [lower, upper].

Parameters:

• lower (float) – Lower bound (included)

• upper (float) – Upper bound (included)

default_sampler_cls

alias of _Uniform

property value_type

Returns:

Type of values (one of str, float, int)

uniform()

loguniform()

reverseloguniform()

normal(mean=0.0, sd=1.0)

quantized(q)

is_valid(value)

Parameters:

value (float) – Value to test

Returns:

Is value a valid value in domain?

property domain_str

match_string(value)

Returns string representation of value (which must be of domain type) which is to match configurations for (approximate) equality. For discrete types (e.g., Integer, Categorical), this matches for exact equality.

Parameters:

value – Value of domain type (use cast() to be safe)

Return type:

str

Returns:

String representation useful for matching

class syne_tune.config_space.Integer(lower, upper)

Bases: Domain

Integer value in closed interval [lower, upper]. Note that upper is included.

Parameters:

• lower (int) – Lower bound (included)

• upper (int) – Upper bound (included)

default_sampler_cls

alias of _Uniform

property value_type

Returns:

Type of values (one of str, float, int)

cast(value)

Parameters:

value – Value top cast

Returns:

value cast to domain. For a finite domain, this can involve rounding

quantized(q)

uniform()

loguniform()

is_valid(value)

Parameters:

value (int) – Value to test

Returns:

Is value a valid value in domain?

property domain_str

match_string(value)

Returns string representation of value (which must be of domain type) which is to match configurations for (approximate) equality. For discrete types (e.g., Integer, Categorical), this matches for exact equality.

Parameters:

value – Value of domain type (use cast() to be safe)

Return type:

str

Returns:

String representation useful for matching

class syne_tune.config_space.Categorical(categories)

Bases: Domain

Value from finite set, whose values do not have a total ordering. For values with an ordering, use Ordinal.

Parameters:

categories (Sequence) – Finite sequence, all entries must have same type

default_sampler_cls

alias of _Uniform

uniform()

grid()

is_valid(value)

Parameters:

value (Any) – Value to test

Returns:

Is value a valid value in domain?

property value_type

Returns:

Type of values (one of str, float, int)

property domain_str

cast(value)

Parameters:

value – Value top cast

Returns:

value cast to domain. For a finite domain, this can involve rounding

match_string(value)

Returns string representation of value (which must be of domain type) which is to match configurations for (approximate) equality. For discrete types (e.g., Integer, Categorical), this matches for exact equality.

Parameters:

value – Value of domain type (use cast() to be safe)

Return type:

str

Returns:

String representation useful for matching

class syne_tune.config_space.Ordinal(categories)

Bases: Categorical

Represents an ordered set. As far as random sampling is concerned, this type is equivalent to Categorical, but when used in methods that require encodings (or distances), nearby values have closer encodings.

Parameters:

categories (Sequence) – Finite sequence, all entries must have same type

class syne_tune.config_space.OrdinalNearestNeighbor(categories, log_scale=False)

Bases: Ordinal

Different type for ordered set of numerical values (int or float). Essentially, the finite set is represented by a real-valued interval containing all values, and random sampling draws a value from this interval and rounds it to the nearest value in categories. If log_scale is True, all of this happens in log scale. Unless values are equidistant, this is different from Ordinal.

Parameters:

• categories (Sequence) – Finite sequence, must be strictly increasing, value type must be float or int. If log_scale=True, values must be positive

• log_scale (bool) – Encoding and NN matching in log domain?

property lower_int: float | None

property upper_int: float | None

property categories_int: ndarray | None

cast_int(value_int)

cast(value)

Parameters:

value – Value top cast

Returns:

value cast to domain. For a finite domain, this can involve rounding

set_sampler(sampler, allow_override=False)

get_sampler()

sample(spec=None, size=1, random_state=None)

Parameters:

• spec (Union[List[dict], dict, None]) – Passed to sampler

• size (int) – Number of values to sample, defaults to 1

• random_state (Optional[RandomState]) – PRN generator

Return type:

Union[Any, List[Any]]

Returns:

Single value (size == 1) or list (size > 1)

class syne_tune.config_space.FiniteRange(lower, upper, size, log_scale=False, cast_int=False)

Bases: Domain

Represents a finite range [lower, ..., upper] with size values equally spaced in linear or log domain. If cast_int, the value type is int (rounding after the transform).

Parameters:

• lower (float) – Lower bound (included)

• upper (float) – Upper bound (included)

• size (int) – Number of values

• log_scale (bool) – Equal spacing in log domain?

• cast_int (bool) – Value type is int (float otherwise)

property values

property value_type

Returns:

Type of values (one of str, float, int)

cast(value)

Parameters:

value – Value top cast

Returns:

value cast to domain. For a finite domain, this can involve rounding

set_sampler(sampler, allow_override=False)

get_sampler()

sample(spec=None, size=1, random_state=None)

Parameters:

• spec (Union[List[dict], dict, None]) – Passed to sampler

• size (int) – Number of values to sample, defaults to 1

• random_state (Optional[RandomState]) – PRN generator

Return type:

Union[Any, List[Any]]

Returns:

Single value (size == 1) or list (size > 1)

property domain_str

match_string(value)

Returns string representation of value (which must be of domain type) which is to match configurations for (approximate) equality. For discrete types (e.g., Integer, Categorical), this matches for exact equality.

Parameters:

value – Value of domain type (use cast() to be safe)

Return type:

str

Returns:

String representation useful for matching

syne_tune.config_space.uniform(lower, upper)

Uniform float value between lower and upper

Parameters:

• lower (float) – Lower bound (included)

• upper (float) – Upper bound (included)

Returns:

Float object

syne_tune.config_space.loguniform(lower, upper)

Log-uniform float value between lower and upper

Sampling is done as exp(x), where x is uniform between log(lower) and log(upper).

Parameters:

• lower (float) – Lower bound (included; positive)

• upper (float) – Upper bound (included; positive)

Returns:

Float object

syne_tune.config_space.randint(lower, upper)

Uniform integer between lower and upper

lower and upper are inclusive. This is a difference to Ray Tune, where upper is exclusive.

Parameters:

• lower (int) – Lower bound (included)

• upper (int) – Upper bound (included)

:return Integer object

syne_tune.config_space.lograndint(lower, upper)

Log-uniform integer between lower and upper

lower and upper are inclusive. Note: Ray Tune has an argument base here, but since this does not affect the distribution, we drop it.

Parameters:

• lower (int) – Lower bound (included)

• upper (int) – Upper bound (included)

:return Integer object

syne_tune.config_space.choice(categories)

Uniform over list of categories

Parameters:

categories (list) – Sequence of values, all entries must have the same type

Returns:

Categorical object

syne_tune.config_space.ordinal(categories, kind=None)

Ordinal value from list categories. Different variants are selected by kind.

For kind == "equal", sampling is the same as for choice, and the internal encoding is by int (first value maps to 0, second to 1, …).

For kind == "nn", the finite set is represented by a real-valued interval containing all values, and random sampling draws a value from this interval and rounds it to the nearest value in categories. This behaves like a finite version of uniform or randint. For kind == "nn-log", nearest neighbour rounding happens in log space, which behaves like a finite version of loguniform`() or lograndint`(). You can also use the synonym logordinal(). For this type, values in categories must be int or float and strictly increasing, and also positive if kind == "nn-log".

Parameters:

• categories (list) – Sequence of values, all entries must have the same type

• kind (Optional[str]) – Can be “equal”, “nn”, “nn-log”

Returns:

Ordinal or OrdinalNearestNeighbor object

syne_tune.config_space.logordinal(categories)

Corresponds to ordinal() with kind="nn-log", so that nearest neighbour mapping happens in log scale. Values in categories must be int or float, strictly increasing, and positive.

Parameters:

categories (list) – Sequence of values, strictly increasing, of type float or int, all positive

Returns:

OrdinalNearestNeighbor object

syne_tune.config_space.finrange(lower, upper, size, cast_int=False)

Finite range [lower, ..., upper] with size entries, which are equally spaced. Finite alternative to uniform().

Parameters:

• lower (float) – Smallest feasible value

• upper (float) – Largest feasible value

• size (int) – Size of (finite) domain, must be >= 2

• cast_int (bool) – Values rounded and cast to int?

Returns:

FiniteRange object

syne_tune.config_space.logfinrange(lower, upper, size, cast_int=False)

Finite range [lower, ..., upper] with size entries, which are equally spaced in the log domain. Finite alternative to loguniform().

Parameters:

• lower (float) – Smallest feasible value (positive)

• upper (float) – Largest feasible value (positive)

• size (int) – Size of (finite) domain, must be >= 2

• cast_int (bool) – Values rounded and cast to int?

Returns:

FiniteRange object

syne_tune.config_space.is_log_space(domain)

Parameters:

domain (Domain) – Hyperparameter type

Return type:

bool

Returns:

Logarithmic encoding?

syne_tune.config_space.is_reverse_log_space(domain)

Return type:

bool

syne_tune.config_space.is_uniform_space(domain)

Parameters:

domain (Domain) – Hyperparameter type

Return type:

bool

Returns:

Linear (uniform) encoding?

syne_tune.config_space.add_to_argparse(parser, config_space)

Use this to prepare argument parser in endpoint script, for the non-fixed parameters in config_space.

Parameters:

• parser (ArgumentParser) – argparse.ArgumentParser object

• config_space (Dict[str, Any]) – Configuration space (modified)

syne_tune.config_space.cast_config_values(config, config_space)

Returns config with keys, values of config, but values are cast to their specific types.

Parameters:

• config (Dict[str, Any]) – Config whose values are to be cast

• config_space (Dict[str, Any]) – Configuration space

Return type:

Dict[str, Any]

Returns:

New config with values cast to correct types

syne_tune.config_space.non_constant_hyperparameter_keys(config_space)

Parameters:

config_space (Dict[str, Any]) – Configuration space

Return type:

List[str]

Returns:

Keys corresponding to (non-fixed) hyperparameters

syne_tune.config_space.config_space_size(config_space, upper_limit=1048576)

Counts the number of distinct configurations in the configuration space config_space. If this is infinite (due to real-valued parameters) or larger than upper_limit, None is returned.

Parameters:

• config_space (Dict[str, Any]) – Configuration space

• upper_limit (int) – See above. Defaults to 2**20

Return type:

Optional[int]

Returns:

Number of distinct configurations; or None if infinite or more than upper_limit

syne_tune.config_space.config_to_match_string(config, config_space, keys)

Maps configuration to a match string, which can be used to compare configs for (approximate) equality. Only keys in keys are used, in that ordering.

Parameters:

• config (Dict[str, Any]) – Configuration to be encoded in match string

• config_space (Dict[str, Any]) – Configuration space

• keys (List[str]) – Keys of parameters to be encoded

Return type:

str

Returns:

Match string

syne_tune.config_space.to_dict(x)

We assume that for each Domain subclass, the __init__() kwargs are also members, and all other members start with _.

Parameters:

x (Domain) – Domain object

Return type:

Dict[str, Any]

Returns:

Representation as dict

syne_tune.config_space.from_dict(d)

Parameters:

d (Dict[str, Any]) – Representation of Domain object as dict

Return type:

Domain

Returns:

Decoded Domain object

syne_tune.config_space.config_space_to_json_dict(config_space)

Converts config_space into a dictionary that can be saved as a json file.

Parameters:

config_space (Dict[str, Union[Domain, int, float, str]]) – Configuration space

Return type:

Dict[str, Union[int, float, str]]

Returns:

JSON-serializable dictionary representing config_space

syne_tune.config_space.config_space_from_json_dict(config_space_dict)

Converts the given dictionary into a Syne Tune search space.

Reverse of config_space_to_json_dict().

Parameters:

config_space_dict (Dict[str, Union[int, float, str]]) – JSON-serializable dict, as output by config_space_to_json_dict()

Return type:

Dict[str, Union[Domain, int, float, str]]

Returns:

Configuration space corresponding to config_space_dict

syne_tune.config_space.restrict_domain(numerical_domain, lower, upper)

Restricts a numerical domain to be in the range [lower, upper]

Parameters:

• numerical_domain (Domain) – Numerical domain

• lower (float) – Lower bound

• upper (float) – Upper bound

Return type:

Domain

Returns:

Restricted domain

class syne_tune.config_space.Quantized(sampler, q)

Bases: Sampler

get_sampler()

sample(domain, spec=None, size=1, random_state=None)

syne_tune.config_space.quniform(lower, upper, q)

Sample a quantized float value uniformly between lower and upper.

Sampling from tune.uniform(1, 10) is equivalent to sampling from np.random.uniform(1, 10))

The value will be quantized, i.e. rounded to an integer increment of q. Quantization makes the upper bound inclusive.

syne_tune.config_space.reverseloguniform(lower, upper)

Values 0 <= x < 1, internally represented as -log(1 - x)

Paam lower:

Lower boundary of the output interval (e.g. 0.99)

Parameters:

upper (float) – Upper boundary of the output interval (e.g. 0.9999)

Returns:

Float object

syne_tune.config_space.qloguniform(lower, upper, q)

Sugar for sampling in different orders of magnitude.

The value will be quantized, i.e. rounded to an integer increment of q. Quantization makes the upper bound inclusive.

Parameters:

• lower (float) – Lower boundary of the output interval (e.g. 1e-4)

• upper (float) – Upper boundary of the output interval (e.g. 1e-2)

• q (float) – Quantization number. The result will be rounded to an integer increment of this value.

syne_tune.config_space.qrandint(lower, upper, q=1)

Sample an integer value uniformly between lower and upper.

lower is inclusive, upper is also inclusive (!).

The value will be quantized, i.e. rounded to an integer increment of q. Quantization makes the upper bound inclusive.

syne_tune.config_space.qlograndint(lower, upper, q)

Sample an integer value log-uniformly between lower and upper

lower is inclusive, upper is also inclusive (!).

The value will be quantized, i.e. rounded to an integer increment of q. Quantization makes the upper bound inclusive.