tempor.models.transformer module¶

Model implementations for Transformers.

class tempor.models.transformer.Permute(*dims: Any)[source]¶

Bases: Module

Permute dimensions of a tensor with torch.Tensor.permute.

Parameters:¶

class tempor.models.transformer.Transpose(*dims: Any, contiguous: bool = False)[source]¶

Bases: Module

Transpose dimensions of a tensor with torch.Tensor.transpose.

Parameters:¶

*dims : Any: Dimensions to transpose.
contiguous : bool, optional¶: Whether to call contiguous() on the output. Defaults to False.

class tempor.models.transformer.TransformerModel(n_units_in: int, n_units_hidden: int = 64, n_head: int = 1, d_ffn: int = 128, dropout: float = 0.1, activation: str = 'relu', n_layers_hidden: int = 1, device: Any = device(type='cpu'))[source]¶

Bases: Module

Transformer model.

Parameters:¶

n_units_in : int¶: The number of features (a.k.a. variables, dimensions, channels) in the time series dataset.
n_units_hidden : int, optional¶: Total dimension of the model. Defaults to 64.
n_head : int, optional¶: Parallel attention heads. Defaults to 1.
d_ffn : int, optional¶: The dimension of the feedforward network model. Defaults to 128.
dropout : float, optional¶: Dropout value passed to TransformerEncoderLayer s. Defaults to 0.1.
activation : str, optional¶: The activation function of intermediate layer, "relu" or "gelu". Defaults to "relu".
n_layers_hidden : int, optional¶: The number of sub-encoder-layers in the encoder. Defaults to 1.
device : Any, optional¶: PyTorch device. Defaults to DEVICE.