exordium.video.deep.swint¶

Swin Transformer model implementation.

Attributes¶

model_name

Mapping of Swin Transformer variant names to their weight file names.

Classes¶

`SwinTWrapper`	Swin Transformer feature extractor wrapper.
`Mlp`	Multi-layer perceptron module.
`WindowAttention`	Window-based multi-head self-attention with relative position bias.
`SwinTransformerBlock`	Swin Transformer Block.
`PatchMerging`	Patch Merging Layer.
`BasicLayer`	A basic Swin Transformer layer for one stage.
`PatchEmbed`	Image to Patch Embedding.
`SwinTransformer`	Swin Transformer hierarchical vision transformer.

Functions¶

`window_partition`(x, window_size)	Partition input tensor into non-overlapping windows.
`window_reverse`(windows, window_size, H, W)	Reverse window partitioning to reconstruct the full image.
`swin_transformer_tiny`([pretrained])	Create a tiny Swin Transformer model.
`swin_transformer_small`([pretrained])	Create a small Swin Transformer model.
`swin_transformer_base`([pretrained])	Create a base Swin Transformer model.

Module Contents¶

exordium.video.deep.swint.model_name¶: Mapping of Swin Transformer variant names to their weight file names.

class exordium.video.deep.swint.SwinTWrapper(arch: str = 'tiny', pretrained: bool = True, device_id: int | None = None)¶

Bases: exordium.video.deep.base.VisualModelWrapper

Swin Transformer feature extractor wrapper.

Loads a pretrained Swin Transformer backbone and extracts global average-pooled patch features, producing a (B, D) array.

Feature dimensions by architecture:

"tiny" / "small": 768-d
"base": 1024-d

Parameters:

arch – One of "tiny", "small", "base". Defaults to "tiny".
pretrained – Load ImageNet-pretrained weights. Defaults to True.
device_id – GPU device index. None or negative uses CPU.

preprocess(frames) → torch.Tensor¶

Resize, centre-crop and normalise to ImageNet stats.

Parameters:: frames – Any input supported by _to_uint8_tensor().
Returns:: Float tensor of shape (B, 3, 224, 224) on self.device.

inference(tensor: torch.Tensor) → torch.Tensor¶

Swin Transformer forward pass with global average pooling.

The backbone forward returns patch tokens of shape (B, N, C). This method averages over the patch dimension to produce (B, C) embeddings.

Parameters:: tensor – Preprocessed tensor of shape (B, 3, 224, 224) on self.device.
Returns:: Feature tensor of shape (B, D).

class exordium.video.deep.swint.Mlp(in_features, hidden_features=None, out_features=None, act_layer=nn.GELU, drop=0.0)¶

Bases: torch.nn.Module

Multi-layer perceptron module.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(x)¶

Forward pass through MLP.

Parameters:: x – Input tensor.
Returns:: Output tensor.

exordium.video.deep.swint.window_partition(x, window_size)¶

Partition input tensor into non-overlapping windows.

Parameters:

x – Input tensor of shape (B, H, W, C).
window_size – Window size.

Returns:

Partitioned windows of shape (num_windows*B, window_size, window_size, C).

exordium.video.deep.swint.window_reverse(windows, window_size, H, W)¶

Reverse window partitioning to reconstruct the full image.

Parameters:

windows – Partitioned windows of shape (num_windows*B, window_size, window_size, C).
window_size – Window size.
H – Height of image.
W – Width of image.

Returns:

Reconstructed image of shape (B, H, W, C).

class exordium.video.deep.swint.WindowAttention(dim, window_size, num_heads, qkv_bias=True, qk_scale=None, attn_drop=0.0, proj_drop=0.0)¶

Bases: torch.nn.Module

Window-based multi-head self-attention with relative position bias.

Supports both shifted and non-shifted window attention mechanisms.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(x, mask=None)¶

Forward pass of window attention.

Parameters:

x – Input features with shape of (num_windows*B, N, C).
mask – (0/-inf) mask with shape of (num_windows, Wh*Ww, Wh*Ww) or None.

extra_repr() → str¶

Return string representation with layer parameters.

Returns:: String with dimension, window size and num heads.

flops(N)¶

Calculate FLOPs for this attention module.

Parameters:: N – Token length.
Returns:: Number of floating point operations.

class exordium.video.deep.swint.SwinTransformerBlock(dim, input_resolution, num_heads, window_size=7, shift_size=0, mlp_ratio=4.0, qkv_bias=True, qk_scale=None, drop=0.0, attn_drop=0.0, drop_path=0.0, act_layer=nn.GELU, norm_layer=nn.LayerNorm)¶

Bases: torch.nn.Module

Swin Transformer Block.

Parameters:

dim (int) – Number of input channels.
input_resolution (tuple[int]) – Input resulotion.
num_heads (int) – Number of attention heads.
window_size (int) – Window size.
shift_size (int) – Shift size for SW-MSA.
mlp_ratio (float) – Ratio of mlp hidden dim to embedding dim.
qkv_bias (bool, optional) – If True, add a learnable bias to query, key, value. Default: True
qk_scale (float | None, optional) – Override default qk scale of head_dim ** -0.5 if set.
drop (float, optional) – Dropout rate. Default: 0.0
attn_drop (float, optional) – Attention dropout rate. Default: 0.0
drop_path (float, optional) – Stochastic depth rate. Default: 0.0
act_layer (nn.Module, optional) – Activation layer. Default: nn.GELU
norm_layer (nn.Module, optional) – Normalization layer. Default: nn.LayerNorm

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(x)¶

Forward pass of Swin Transformer block.

Parameters:: x – Input tensor.
Returns:: Output tensor.

extra_repr() → str¶

Return string representation with layer parameters.

Returns:: String with layer parameters.

flops()¶

Calculate FLOPs for this layer.

Returns:: Number of floating point operations.

class exordium.video.deep.swint.PatchMerging(input_resolution, dim, norm_layer=nn.LayerNorm)¶

Bases: torch.nn.Module

Patch Merging Layer.

Parameters:

input_resolution (tuple[int]) – Resolution of input feature.
dim (int) – Number of input channels.
norm_layer (nn.Module, optional) – Normalization layer. Default: nn.LayerNorm

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(x)¶

Forward pass merging patches.

Parameters:: x – Input tensor of shape (B, H*W, C).
Returns:: Merged tensor of shape (B, H/2*W/2, 2*C).

extra_repr() → str¶

Return string representation with layer parameters.

Returns:: String with input resolution and dimension.

flops()¶

Calculate FLOPs for this layer.

Returns:: Number of floating point operations.

class exordium.video.deep.swint.BasicLayer(dim, input_resolution, depth, num_heads, window_size, mlp_ratio=4.0, qkv_bias=True, qk_scale=None, drop=0.0, attn_drop=0.0, drop_path=0.0, norm_layer=nn.LayerNorm, downsample=None, use_checkpoint=False)¶

Bases: torch.nn.Module

A basic Swin Transformer layer for one stage.

Parameters:

dim (int) – Number of input channels.
input_resolution (tuple[int]) – Input resolution.
depth (int) – Number of blocks.
num_heads (int) – Number of attention heads.
window_size (int) – Local window size.
mlp_ratio (float) – Ratio of mlp hidden dim to embedding dim.
qkv_bias (bool, optional) – If True, add a learnable bias to query, key, value. Default: True
qk_scale (float | None, optional) – Override default qk scale of head_dim ** -0.5 if set.
drop (float, optional) – Dropout rate. Default: 0.0
attn_drop (float, optional) – Attention dropout rate. Default: 0.0
drop_path (float | tuple[float], optional) – Stochastic depth rate. Default: 0.0
norm_layer (nn.Module, optional) – Normalization layer. Default: nn.LayerNorm
downsample (nn.Module | None, optional) – Downsample layer at the end of the layer. Default: None
use_checkpoint (bool) – Whether to use checkpointing to save memory. Default: False.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(x)¶

Forward pass through basic layer.

Parameters:: x – Input tensor.
Returns:: Output tensor.

extra_repr() → str¶

Return string representation with layer parameters.

Returns:: String with dimension, resolution and depth.

flops()¶

Calculate FLOPs for this layer.

Returns:: Number of floating point operations.

class exordium.video.deep.swint.PatchEmbed(img_size=224, patch_size=4, in_chans=3, embed_dim=96, norm_layer=None)¶

Bases: torch.nn.Module

Image to Patch Embedding.

Parameters:

img_size (int) – Image size. Default: 224.
patch_size (int) – Patch token size. Default: 4.
in_chans (int) – Number of input image channels. Default: 3.
embed_dim (int) – Number of linear projection output channels. Default: 96.
norm_layer (nn.Module, optional) – Normalization layer. Default: None

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(x)¶

Convert image to patch embeddings.

Parameters:: x – Input tensor of shape (B, C, H, W).
Returns:: Patch embeddings of shape (B, num_patches, embed_dim).

flops()¶

Calculate FLOPs for patch embedding.

Returns:: Number of floating point operations.

class exordium.video.deep.swint.SwinTransformer(img_size=224, patch_size=4, in_chans=3, num_classes=1000, embed_dim=96, depths=[2, 2, 6, 2], num_heads=[3, 6, 12, 24], window_size=7, mlp_ratio=4.0, qkv_bias=True, qk_scale=None, drop_rate=0.0, attn_drop_rate=0.0, drop_path_rate=0.1, norm_layer=nn.LayerNorm, ape=False, patch_norm=True, use_checkpoint=False, **_kwargs)¶

Bases: torch.nn.Module

Swin Transformer hierarchical vision transformer.

A PyTorch implementation of: Swin Transformer: Hierarchical Vision Transformer using Shifted Windows https://arxiv.org/pdf/2103.14030

Parameters:

img_size (int | tuple(int)) – Input image size. Default 224
patch_size (int | tuple(int)) – Patch size. Default: 4
in_chans (int) – Number of input image channels. Default: 3
num_classes (int) – Number of classes for classification head. Default: 1000
embed_dim (int) – Patch embedding dimension. Default: 96
depths (tuple(int)) – Depth of each Swin Transformer layer.
num_heads (tuple(int)) – Number of attention heads in different layers.
window_size (int) – Window size. Default: 7
mlp_ratio (float) – Ratio of mlp hidden dim to embedding dim. Default: 4
qkv_bias (bool) – If True, add a learnable bias to query, key, value. Default: True
qk_scale (float) – Override default qk scale of head_dim ** -0.5 if set. Default: None
drop_rate (float) – Dropout rate. Default: 0
attn_drop_rate (float) – Attention dropout rate. Default: 0
drop_path_rate (float) – Stochastic depth rate. Default: 0.1
norm_layer (nn.Module) – Normalization layer. Default: nn.LayerNorm.
ape (bool) – If True, add absolute position embedding to the patch embedding. Default: False
patch_norm (bool) – If True, add normalization after patch embedding. Default: True
use_checkpoint (bool) – Whether to use checkpointing to save memory. Default: False

Initialize internal Module state, shared by both nn.Module and ScriptModule.

no_weight_decay()¶

Return set of parameter names that should not have weight decay applied.

Returns:: Set of parameter names.

no_weight_decay_keywords()¶

Return keywords for parameters that should not have weight decay applied.

Returns:: Set of keywords.

forward_features(x)¶

Extract features using transformer layers.

Parameters:: x – Input tensor.
Returns:: Extracted features.

forward(x)¶

Forward pass through Swin Transformer.

Parameters:: x – Input tensor.
Returns:: Output tensor.

flops()¶

Calculate FLOPs for the model.

Returns:: Number of floating point operations.

exordium.video.deep.swint.swin_transformer_tiny(pretrained=True, **kwargs)¶

Create a tiny Swin Transformer model.

Parameters:

pretrained – If True, load ImageNet pre-trained weights. Defaults to True.
**kwargs – Additional arguments for SwinTransformer.

Returns:

Swin Transformer model.

exordium.video.deep.swint.swin_transformer_small(pretrained=True, **kwargs)¶

Create a small Swin Transformer model.

Parameters:

pretrained – If True, load ImageNet pre-trained weights. Defaults to True.
**kwargs – Additional arguments for SwinTransformer.

Returns:

Swin Transformer model.

exordium.video.deep.swint.swin_transformer_base(pretrained=True, **kwargs)¶

Create a base Swin Transformer model.

Parameters:

pretrained – If True, load ImageNet pre-trained weights. Defaults to True.
**kwargs – Additional arguments for SwinTransformer.

Returns:

Swin Transformer model.