syne_tune.optimizer.schedulers.searchers.bayesopt.gpautograd.warping module

class syne_tune.optimizer.schedulers.searchers.bayesopt.gpautograd.warping.Warping(dimension, coordinate_range=None, encoding_type='logarithm', **kwargs)[source]

Bases: MeanFunction

Warping transform on contiguous range of feature \(x\). Each warped coordinate has two independent warping parameters.

If \(x = [x_1, \dots, x_d]\) and coordinate_range = (l, r), the warping transform operates on \([x_l, \dots, x_{r-1}]\). The default for coordinate_range is the full range, and we must have l < r. The block is the identity on all remaining coordinates. Input coordinates are assumed to lie in \([0, 1]\). The warping transform on each coordinate is due to Kumaraswamy:

\[warp(x_j) = 1 - (1 - r(x_j)^{a_j})^{b_j}.\]

Here, \(r(x_j)\) linearly maps \([0, 1]\) to \([\epsilon, 1 - \epsilon]\) for a small \(\epsilon > 0\), which avoids numerical issues when taking derivatives.

Parameters:

  • dimension (int) – Dimension \(d\) of input

  • coordinate_range (Optional[Tuple[int, int]]) – Range (l, r), see above. Default is (0, dimension), so the full range

  • encoding_type (str) – Encoding type

forward(x)[source]

Actual computation of the warping transformation (see details above)

Parameters:

x – Input data, shape (n, d)

param_encoding_pairs()[source]
Returns list of tuples

(param_internal, encoding)

over all Gluon parameters maintained here.

Returns:

List [(param_internal, encoding)]

get_params()[source]
Return type:

Dict[str, Any]

Returns:

Dictionary with hyperparameter values

set_params(param_dict)[source]
Parameters:

param_dict (Dict[str, Any]) – Dictionary with new hyperparameter values

Returns:

syne_tune.optimizer.schedulers.searchers.bayesopt.gpautograd.warping.warpings_for_hyperparameters(hp_ranges)[source]

It is custom to warp hyperparameters which are not categorical. This function creates warpings based on your configuration space.

Parameters:

hp_ranges (HyperparameterRanges) – Encoding of configuration space

Return type:

List[Warping]

Returns:

To be used as warpings in WarpedKernel

syne_tune.optimizer.schedulers.searchers.bayesopt.gpautograd.warping.kernel_with_warping(kernel, hp_ranges)[source]

Note that the coordinates corresponding to categorical parameters are not warped.

Parameters:
Return type:

KernelFunction

Returns:

Kernel with warping

class syne_tune.optimizer.schedulers.searchers.bayesopt.gpautograd.warping.WarpedKernel(kernel, warpings, **kwargs)[source]

Bases: KernelFunction

Block that composes warping with an arbitrary kernel. We allow for a list of warping transforms, so that a non-contiguous set of input coordinates can be warped.

It is custom to warp hyperparameters which are not categorical. You can use kernel_with_warping() to furnish a kernel with warping for all non-categorical hyperparameters.

Parameters:

  • kernel (KernelFunction) – Kernel \(k(x, x')\)

  • warpings (List[Warping]) – List of warping transforms, which are applied sequentially. Ranges of different entries should be non-overlapping, this is not checked.

forward(X1, X2)[source]

Overrides to implement forward computation using NDArray. Only accepts positional arguments. Parameters ———- *args : list of NDArray

Input tensors.

diagonal(X)[source]
Parameters:

X – Input data, shape (n, d)

Returns:

Diagonal of \(k(X, X)\), shape (n,)

diagonal_depends_on_X()[source]

For stationary kernels, diagonal does not depend on X

Returns:

Does diagonal() depend on X?

param_encoding_pairs()[source]
Returns list of tuples

(param_internal, encoding)

over all Gluon parameters maintained here.

Returns:

List [(param_internal, encoding)]

get_params()[source]
Return type:

Dict[str, Any]

Returns:

Dictionary with hyperparameter values

set_params(param_dict)[source]
Parameters:

param_dict (Dict[str, Any]) – Dictionary with new hyperparameter values

Returns: