syne_tune.optimizer.schedulers.multiobjective.moasha module

class syne_tune.optimizer.schedulers.multiobjective.moasha.MOASHA(config_space, metrics, mode=None, time_attr='training_iteration', multiobjective_priority=None, max_t=100, grace_period=1, reduction_factor=3, brackets=1)

Bases: TrialScheduler

Implements MultiObjective Asynchronous Successive HAlving with different multiobjective sort options. References:

A multi-objective perspective on jointly tuning hardware and hyperparameters

David Salinas, Valerio Perrone, Cedric Archambeau and Olivier Cruchant

NAS workshop, ICLR2021.

and

Multi-objective multi-fidelity hyperparameter optimization with application to fairness

Robin Schmucker, Michele Donini, Valerio Perrone, Cédric Archambeau

Parameters:

• config_space (Dict[str, Any]) – Configuration space

• metrics (List[str]) – List of metric names MOASHA optimizes over

• mode (Union[str, List[str], None]) – One of {"min", "max"} or a list of these values (same size as metrics). Determines whether objectives are minimized or maximized. Defaults to “min”

• time_attr (str) – A training result attr to use for comparing time. Note that you can pass in something non-temporal such as training_iteration as a measure of progress, the only requirement is that the attribute should increase monotonically. Defaults to “training_iteration”

• multiobjective_priority (Optional[MOPriority]) – The multiobjective priority that is used to sort multiobjective candidates. We support several choices such as non-dominated sort or linear scalarization, default is non-dominated sort.

• max_t (int) – max time units per trial. Trials will be stopped after max_t time units (determined by time_attr) have passed. Defaults to 100

• grace_period (int) – Only stop trials at least this old in time. The units are the same as the attribute named by time_attr. Defaults to 1

• reduction_factor (float) – Used to set halving rate and amount. This is simply a unit-less scalar. Defaults to 3

• brackets (int) – Number of brackets. Each bracket has a different grace_period and number of rung levels. Defaults to 1

metric_names()

Return type:

List[str]

Returns:

List of metric names. The first one is the target metric optimized over, unless the scheduler is a genuine multi-objective metric (for example, for sampling the Pareto front)

metric_mode()

Return type:

str

Returns:

“min” if target metric is minimized, otherwise “max”. Here, “min” should be the default. For a genuine multi-objective scheduler, a list of modes is returned

on_trial_add(trial)

Called when a new trial is added to the trial runner.

Additions are normally triggered by suggest.

Parameters:

trial (Trial) – Trial to be added

on_trial_result(trial, result)

Called on each intermediate result reported by a trial.

At this point, the trial scheduler can make a decision by returning one of SchedulerDecision.CONTINUE, SchedulerDecision.PAUSE, or SchedulerDecision.STOP. This will only be called when the trial is currently running.

Parameters:

• trial (Trial) – Trial for which results are reported

• result (Dict[str, Any]) – Result dictionary

Return type:

str

Returns:

Decision what to do with the trial

on_trial_complete(trial, result)

Notification for the completion of trial.

Note that on_trial_result() is called with the same result before. However, if the scheduler only uses one final report from each trial, it may ignore on_trial_result() and just use result here.

Parameters:

• trial (Trial) – Trial which is completing

• result (Dict[str, Any]) – Result dictionary

on_trial_remove(trial)

Called to remove trial.

This is called when the trial is in PAUSED or PENDING state. Otherwise, call on_trial_complete().

Parameters:

trial (Trial) – Trial to be removed

is_multiobjective_scheduler()

Return True if a scheduler is multi-objective.

Return type:

bool