xlstm_jax.models.xlstm_pytorch.components.ln

Classes

LayerNorm	LayerNorm but with an optional bias.
MultiHeadLayerNorm	LayerNorm but with an optional bias.

Module Contents

class xlstm_jax.models.xlstm_pytorch.components.ln.LayerNorm(ndim=-1, weight=True, bias=False, eps=1e-05, residual_weight=True)

Bases: torch.nn.Module

LayerNorm but with an optional bias.

PyTorch doesn’t support simply bias=False.

Parameters:

• ndim (int)

• weight (bool)

• bias (bool)

• eps (float)

• residual_weight (bool)

weight

bias = None

eps = 1e-05

residual_weight = True

ndim = -1

property weight_proxy: torch.Tensor

Return type:

torch.Tensor

forward(input)

Parameters:

input (torch.Tensor)

Return type:

torch.Tensor

reset_parameters()

class xlstm_jax.models.xlstm_pytorch.components.ln.MultiHeadLayerNorm(ndim=-1, weight=True, bias=False, eps=1e-05, residual_weight=True)

Bases: LayerNorm

LayerNorm but with an optional bias.

PyTorch doesn’t support simply bias=False.

Parameters:

• ndim (int)

• weight (bool)

• bias (bool)

• eps (float)

• residual_weight (bool)

forward(input)

Parameters:

input (torch.Tensor)

Return type:

torch.Tensor

weight

bias = None

eps = 1e-05

residual_weight = True

ndim = -1

property weight_proxy: torch.Tensor

Return type:

torch.Tensor

reset_parameters()