tempor.automl.tuner module¶

Module containing the interface for, and the implemented hyperparameter tuners.

class tempor.automl.tuner.EvaluationCallback(*args, **kwargs)[source]¶

Bases: Protocol

Evaluation callback callable.

Should take in Type[BasePredictor] as first argument, then the dataset (PredictiveDataset), hyperparameters, returning a float score.

Signature like: (estimator: Type[BasePredictor], dataset: PredictiveDataset, *args: Any, **kwargs: Any) -> float.

class tempor.automl.tuner.BaseTuner(study_name: str, direction: Literal[minimize] | Literal[maximize], **kwargs: Any)[source]¶

Bases: ABC

Base hyperparameter tuner from which tuner implementations should derive. Defines the initializer and the tune method.

Parameters:¶

study_name : str¶: Study name.
direction : metric_typing.MetricDirection¶: Optimization direction ("minimize" or "maximize").
**kwargs : Any: Currently unused.

abstract tune(estimator: type[BasePredictor] | PipelineSelector, dataset: PredictiveDataset, evaluation_callback: EvaluationCallback, override_hp_space: list[Params] | None = None, compute_baseline_score: bool = True, **kwargs: Any) → tuple[list[float], list[dict]][source]¶

Run the hyperparameter tuner and return scores and chosen hyperparameters.

Parameters:¶

estimator : AutoMLCompatibleEstimator¶: Estimator class, or PipelineSelector, whose hyperparameters will be tuned.
dataset : PredictiveDataset¶: Dataset to use.
evaluation_callback : EvaluationCallback¶: Evaluation callback which will take in the estimator class, hyperparameters, and return a score.
override_hp_space : Optional[List[Params]]¶: If this is not None, hyperparameters will be sampled from this list, rather than from those defined in the hyperparameter_space method of the estimator. Defaults to None.
compute_baseline_score : bool, optional¶: If True, a trial will be run with default parameters (hyperparameters passed to __init__ as an empty dictionary). This will be returned as the zeroth item in scores and params. If False, this will be skipped. Defaults to True.
**kwargs : Any: Currently unused.

Returns:¶

(scores, params) tuple, containing a list of scores for the tuning runs and a list of dictionaries containing the parameters for each corresponding tuning run.

Return type:¶

Tuple[List[float], List[Dict]]

class tempor.automl.tuner.OptunaTuner(study_name: str, direction: Literal[minimize] | Literal[maximize], *, study_sampler: BaseSampler, study_storage: str | BaseStorage | None = None, study_pruner: BasePruner | None = None, study_load_if_exists: bool = False, **kwargs: Any)[source]¶

Bases: BaseTuner

Hyper parameter tuning (optimization) helper for an optuna.study.Study using any optuna.sampler.BaseSampler.

Parameters:¶

study_name : str¶: Study name.
direction : metric_typing.MetricDirection¶: Optimization direction ("minimize" or "maximize").
study_sampler : optuna.samplers.BaseSampler¶: An optuna sampler (passed to optuna.create_study).
study_storage : Optional[Union[str, optuna.storages.BaseStorage]], optional¶: An optuna storage object (passed to optuna.create_study). Defaults to None.
study_pruner : Optional[optuna.pruners.BasePruner], optional¶: An optuna pruner (passed to optuna.create_study). Defaults to None.
study_load_if_exists : bool, optional¶: The load_if_exists parameter (passed to optuna.create_study). Defaults to False.
**kwargs : Any: Currently unused.

create_study() → Study[source]¶

Create an optuna.Study to be used for tuning. Sets the self.study attribute.

Returns:¶: The created study.
Return type:¶: optuna.Study

tune(estimator: type[BasePredictor] | PipelineSelector, dataset: PredictiveDataset, evaluation_callback: EvaluationCallback, override_hp_space: list[Params] | None = None, compute_baseline_score: bool = True, optimize_kwargs: dict[str, Any] | None = None, **kwargs: Any) → tuple[list[float], list[dict]][source]¶

Run the hyperparameter tuner and return scores and chosen hyperparameters.

Parameters:¶

estimator : AutoMLCompatibleEstimator¶: Estimator class, or PipelineSelector, whose hyperparameters will be tuned.
dataset : PredictiveDataset¶: Dataset to use.
evaluation_callback : EvaluationCallback¶: Evaluation callback which will take in the estimator class, hyperparameters, and return a score.
override_hp_space : Optional[List[Params]]¶: If this is not None, hyperparameters will be sampled from this list, rather than from those defined in the hyperparameter_space method of the estimator. Defaults to None.
compute_baseline_score : bool, optional¶: If True, a trial will be run with default parameters (hyperparameters passed to __init__ as an empty dictionary). This will be returned as the zeroth item in scores and params. If False, this will be skipped. Defaults to True.
optimize_kwargs : Optional[Dict[str, Any]], optional¶: Keyword arguments to pass to study.optimize. Defaults to None.
**kwargs : Any: Currently unused.

Returns:¶

(scores, params) tuple, containing a list of scores for the tuning runs and a list of dictionaries containing the parameters for each corresponding tuning run.

Return type:¶

Tuple[List[float], List[Dict]]