tempor.models.mlp module¶

Model implementations related to MLP / fully connected neural networks.

class tempor.models.mlp.LinearLayer(n_units_in: int, n_units_out: int, dropout: float = 0.0, batch_norm: bool = False, nonlin: Literal[none] | Literal[elu] | Literal[relu] | Literal[leaky_relu] | Literal[selu] | Literal[tanh] | Literal[sigmoid] | Literal[softmax] | Literal[gumbel_softmax] | None = 'relu', device: Any = device(type='cpu'))[source]¶

Bases: Module

Linear layer with optional dropout, batch norm, and nonlinearity.

Parameters:¶

n_units_in : int¶

Number of input units.

n_units_out : int¶

Number of output units.

dropout : float, optional¶

Dropout. Defaults to 0.0.

batch_norm : bool, optional¶

Batch normalization. Defaults to False.

nonlin : Optional[Nonlin], optional¶

Nonlinearity (activation function). Defaults to "relu".

device : Any, optional¶

Device. Defaults to constants.DEVICE.

forward(X: Tensor) → Tensor[source]¶

Forward pass.

training : bool¶

class tempor.models.mlp.ResidualLayer(n_units_in: int, n_units_out: int, dropout: float = 0.0, batch_norm: bool = False, nonlin: Literal[none] | Literal[elu] | Literal[relu] | Literal[leaky_relu] | Literal[selu] | Literal[tanh] | Literal[sigmoid] | Literal[softmax] | Literal[gumbel_softmax] | None = 'relu', device: Any = device(type='cpu'))[source]¶

Bases: LinearLayer

Residual layer.

Parameters:¶

n_units_in : int¶

Number of input units.

n_units_out : int¶

Number of output units.

dropout : float, optional¶

Dropout. Defaults to 0.0.

batch_norm : bool, optional¶

Batch normalization. Defaults to False.

nonlin : Optional[Nonlin], optional¶

Nonlinearity (activation function). Defaults to "relu".

device : Any, optional¶

Device. Defaults to constants.DEVICE.

forward(X: Tensor) → Tensor[source]¶

Forward pass.

training : bool¶

class tempor.models.mlp.MultiActivationHead(activations: list[tuple[Module, int]], device: Any = device(type='cpu'))[source]¶

Bases: Module

Final layer with multiple activations. Useful for tabular data. The different activations are applied to different sub-lengths of the X tensor in the forward pass, on its final dimension. Hence the X tensor must have a shape of (..., sum(activation_lengths)).

Parameters:¶

activations : List[Tuple[nn.Module, int]]¶

List of tuples of activations and their lengths, [(activation, activation_length), ...].

device : Any, optional¶

Device. Defaults to constants.DEVICE.

forward(X: Tensor) → Tensor[source]¶

Forward pass.

training : bool¶

class tempor.models.mlp.MLP(task_type: Literal[classification] | Literal[regression], n_units_in: int, n_units_out: int, n_layers_hidden: int = 1, n_units_hidden: int = 100, nonlin: Literal[none] | Literal[elu] | Literal[relu] | Literal[leaky_relu] | Literal[selu] | Literal[tanh] | Literal[sigmoid] | Literal[softmax] | Literal[gumbel_softmax] = 'relu', nonlin_out: list[tuple[Literal[none] | Literal[elu] | Literal[relu] | Literal[leaky_relu] | Literal[selu] | Literal[tanh] | Literal[sigmoid] | Literal[softmax] | Literal[gumbel_softmax], int]] | None = None, lr: float = 0.001, weight_decay: float = 0.001, opt_betas: tuple[float, float] = (0.9, 0.999), n_iter: int = 1000, batch_size: int = 500, n_iter_print: int = 100, random_state: int = 0, patience: int = 10, n_iter_min: int = 100, dropout: float = 0.1, clipping_value: int = 1, batch_norm: bool = False, early_stopping: bool = True, residual: bool = False, loss: Callable | None = None, device: Any = device(type='cpu'))[source]¶

Bases: Module

Fully connected or residual neural nets for classification and regression.

Parameters:¶

task_type : constants.ModelTaskType¶

The type of the problem. Available options: ModelTaskType.

n_units_in : int¶

Number of features.

n_units_out : int¶

Number of outputs.

n_layers_hidden : int, optional¶

Number of hidden layers. Defaults to 1.

n_units_hidden : int, optional¶

Number of hidden units in each layer. Defaults to 100.

nonlin : Nonlin, optional¶

Nonlinearity to use in NN. Available options: Nonlin. Defaults to "relu".

nonlin_out : Optional[List[Tuple[Nonlin, int]]], optional¶

List of activations for the output. Example [("tanh", 1), ("softmax", 3)] - means the output layer will apply "tanh" for the first unit, and "softmax" for the following 3 units in the output. Defaults to None.

lr : float, optional¶

Learning rate. Defaults to 1e-3.

weight_decay : float, optional¶

l2 (ridge) penalty for the weights. Defaults to 1e-3.

opt_betas : Tuple[float, float], optional¶

Optimizer betas. Defaults to (0.9, 0.999).

n_iter : int, optional¶

Maximum number of iterations. Defaults to 1000.

batch_size : int, optional¶

Batch size. Defaults to 500.

n_iter_print : int, optional¶

Number of iterations after which to print updates and check the validation loss. Defaults to 100.

random_state : int, optional¶

Random seed. Defaults to 0.

patience : int, optional¶

Number of iterations to wait before early stopping after decrease in validation loss. Defaults to 10.

n_iter_min : int, optional¶

Minimum number of iterations to go through before starting early stopping. Defaults to 100.

dropout : float, optional¶

Dropout value. If 0, the dropout is not used. Defaults to 0.1.

clipping_value : int, optional¶

Gradients clipping value. Defaults to 1.

batch_norm : bool, optional¶

Enable/disable batch normalization. Defaults to False.

early_stopping : bool, optional¶

Enable/disable early stopping. Defaults to True.

residual : bool, optional¶

Add residuals. Defaults to False.

loss : Optional[Callable], optional¶

Optional custom loss function. If None, the loss is torch.nn.CrossEntropyLoss for classification tasks, or torch.nn.MSELoss for regression. Defaults to None.

device : Any, optional¶

PyTorch device to use. Defaults to DEVICE.

fit(X: ndarray, y: ndarray) → MLP[source]¶

Fit (train) the model.

predict_proba(X: ndarray) → ndarray[source]¶

Predict probabilities for classification tasks.

Parameters:¶

X : np.ndarray¶

Input data.

Returns:¶

Predicted probabilities.

Return type:¶

np.ndarray

predict(X: ndarray) → ndarray[source]¶

Make predictions.

score(X: ndarray, y: ndarray) → float[source]¶

Compute the default score of the model. See source code.

forward(X: Tensor) → Tensor[source]¶

Forward pass.

training : bool¶