fovi.probes

class fovi.probes.LinearProbe(num_features, mlp_coeff=1, num_classes=1000, bottleneck_dim=None, layer_norm=False, dropout=None)[source]

Bases: Module

A LinearProbe module is designed to probe a single layer of a network. It can be combined with other LinearProbes to form a more comprehensive probing system. This module is particularly useful for analyzing the representations learned by a network at different layers.

probe

The probing module, which can be a simple linear layer or a more complex module with a bottleneck layer and layer normalization.

Type:: nn.Module

dropout

Dropout layer to apply dropout to the inputs before probing.

Type:: nn.Module

__init__(num_features, mlp_coeff=1, num_classes=1000, bottleneck_dim=None, layer_norm=False, dropout=None)[source]

Initializes a LinearProbe module.

Parameters:

num_features (int) – The number of features in the input.
mlp_coeff (int, optional) – Coefficient to scale the number of features for the MLP. Defaults to 1.
num_classes (int, optional) – The number of classes for classification. Defaults to 1000.
bottleneck_dim (int or str, optional) – The dimension of the bottleneck layer. If ‘infer’, it will be set to num_features. Defaults to None.
layer_norm (bool, optional) – Whether to use layer normalization. Defaults to False.
dropout (float, optional) – Dropout rate. Defaults to None.

forward(inputs)[source]

Parameters:: inputs (torch.Tensor) – The input tensor to the module.
Returns:: The output of the forward pass.
Return type:: torch.Tensor

class fovi.probes.FoviNetProbe(num_features, fix_agg, num_classes, dropout=None)[source]

Bases: Module

FoviNetProbe is a module designed to apply a probe to the outputs of a network, specifically tailored for FoviNet models.

fix_agg

The aggregation method for fixations.

Type:: str

num_features

The number of features in the input.

Type:: int

num_classes

The number of classes for classification.

Type:: int

dropout

The dropout rate. Defaults to None.

Type:: float, optional

__init__(num_features, fix_agg, num_classes, dropout=None)[source]

Initializes a FoviNetProbe module.

Parameters:

num_features (int) – The number of features in the input.
fix_agg (str) – The aggregation method for fixations.
num_classes (int) – The number of classes for classification.
dropout (float, optional) – The dropout rate. Defaults to None.

_init_fix_agg()[source]: Initializes the fixation aggregation module.

forward(x)[source]

Forward pass of the FoviNetProbe.

Parameters:: x (torch.Tensor) – Tensor of shape (batch, fixations, features).
Returns:: Embeddings of shape (batch, features).
Return type:: torch.Tensor

class fovi.probes.FoviNetProbes(mlp_spec, fix_agg, num_classes, dropout=None)[source]

Bases: Module

FoviNetProbes is a module that applies probes to the output of a FoviNet model. It is designed to extract features from different layers of the model and apply a classification head to each layer output. This allows for the evaluation of the model’s performance at different stages of processing.

mlp_spec

A string representation of the sizes of all layer outputs, joined by hyphens.

Type:: str

fix_agg

The method used for aggregating fixation-level features.

Type:: str

num_classes

The number of classes in the classification task.

Type:: int

dropout

The dropout rate to apply to the probes. Defaults to None.

Type:: float, optional

probes

A list of probes, each applied to a different layer output.

Type:: nn.ModuleList

__init__(mlp_spec, fix_agg, num_classes, dropout=None)[source]

Initializes the FoviNetProbes module.

Parameters:

mlp_spec (str) – A string representation of the sizes of all layer outputs, joined by hyphens.
fix_agg (str) – The method used for aggregating fixation-level features.
num_classes (int) – The number of classes in the classification task.
dropout (float, optional) – The dropout rate to apply to the probes. Defaults to None.

_init_probes()[source]: Initializes the probes for each layer output.

forward(list_outputs, probes_layer_inds=None)[source]

Parameters:

list_outputs (list) – A list of outputs from different layers of the FoviNet model.
probes_layer_inds (list, optional) – Indices of the layers to apply the probes to. Defaults to None.

Returns:

A list of outputs from the probes applied to the specified layers.

Return type:

list