fovi.sensing.samplers

class fovi.sensing.samplers.BaseGridSampler(*args: Any, **kwargs: Any)[source]

Bases: Module

Base class for grid samplers.

Note: objects of the BaseGridSampler family should not be used directly; it is much more convenient to use RetinalTransform, which stores a BaseGridSampler child, due to its handling of fixation parameters.

_transform_fix_grid(img_shape, fix_loc, fixation_size)[source]

Transform fixation grid to image coordinates.

Parameters:

img_shape (tuple) – Image shape (height, width).
fix_loc (torch.Tensor) – Fixation location.
fixation_size (torch.Tensor) – Fixation size.

Returns:

Transformed grid coordinates.

Return type:

torch.Tensor

_prep_grid_for_grid_sample(cartesian_grid)[source]

Prepare grid for torch.nn.functional.grid_sample.

Parameters:: cartesian_grid (torch.Tensor) – (n,2) coordinates, where each 2-vector coordinate is specified in (x,y) in the typical math sense (normalized to [-1,1])
Returns:: (1,1,n,2) coordinates, where each 2-vector coordinate is specified in (-y, x) or (row, col) format for grid_sample (normalized to [-1,1])
Return type:: torch.Tensor

class fovi.sensing.samplers.GridSampler(fov, cmf_a, resolution, device='cuda', dtype=torch.float, mode='nearest', style='isotropic', coords=None)[source]

Bases: BaseGridSampler

Grid sampler for foveated vision using regular grid sampling.

This sampler uses standard grid sampling with nearest neighbor or bilinear interpolation to sample from the foveated sampling grid.

__init__(fov, cmf_a, resolution, device='cuda', dtype=torch.float, mode='nearest', style='isotropic', coords=None)[source]

Initialize the GridSampler.

Parameters:

fov (float) – Field of view diameter in degrees.
cmf_a (float) – A parameter from the CMF: M(r)=1/(r+a). Smaller = stronger foveation.
resolution (int) – Resolution parameter.
device (str, optional) – Device to run on. Defaults to ‘cuda’.
dtype (torch.dtype, optional) – Data type. Defaults to torch.float.
mode (str, optional) – Sampling mode (‘nearest’ or ‘bilinear’). Defaults to ‘nearest’.
style (str, optional) – Sampling style. Defaults to ‘isotropic’.
coords (SamplingCoords, optional) – Pre-computed sampling coordinates. Defaults to None.

forward(img, fix_loc=None, fixation_size=None, return_coords=False)[source]

Forward pass for grid sampling.

Parameters:

img (torch.Tensor) – Input image tensor.
fix_loc (torch.Tensor, optional) – Fixation location. Defaults to None.
fixation_size (torch.Tensor, optional) – Fixation size. Defaults to None.
return_coords (bool, optional) – Whether to return sampling coordinates. Defaults to False.

Returns:

Sampled image tensor.

Return type:

torch.Tensor

all_coords(device=None)[source]

Get all sampling coordinates.

Parameters:: device (str, optional) – Device to place coordinates on. Defaults to None.
Returns:: All sampling coordinates.
Return type:: torch.Tensor

class fovi.sensing.samplers.KNNGridSampler(fov, cmf_a, resolution, res_mult=3, cmf_a_mult=1, fixation_size=3000, k=None, style='isotropic', sample_cortex=True, dtype=torch.float, device='cuda')[source]

Bases: BaseGridSampler

K-Nearest Neighbors grid sampler for foveated vision.

This sampler uses KNN-based sampling to perform local average pooling over a high-res sensor array into a lower-res sensor array, with the same CorticalSensorManifold. - highres_coords: akin to photoreceptors: there are more of them - coords: akin to retinal ganglion cells: there are less of them, and they integrate over a local pool of photoreceptors (highres_coords)

__init__(fov, cmf_a, resolution, res_mult=3, cmf_a_mult=1, fixation_size=3000, k=None, style='isotropic', sample_cortex=True, dtype=torch.float, device='cuda')[source]

Initialize the KNNGridSampler.

Parameters:

fov (float) – Field of view diameter in degrees.
cmf_a (float) – A parameter from the CMF: M(r)=1/(r+a). Smaller = stronger foveation.
resolution (int) – Resolution parameter.
res_mult (int, optional) – Resolution multiplier for photoreceptor layer vs. rgc layer. Defaults to 3.
cmf_a_mult (int, optional) – CMF_a multiplier for photoreceptor layer vs. rgc layer. Defaults to 1.
fixation_size (int, optional) – Fixation size in pixels. Defaults to 3000.
k (int, optional) – Number of nearest neighbors. Defaults to None.
style (str, optional) – Sampling style. Defaults to ‘isotropic’.
sample_cortex (bool, optional) – Whether to sample from cortex. Defaults to True.
dtype (torch.dtype, optional) – Data type. Defaults to torch.float.
device (str, optional) – Device to run on. Defaults to ‘cuda’.

forward(img, fix_loc=None, fixation_size=None)[source]

Forward pass for KNN grid sampling.

Parameters:

img (torch.Tensor) – Input image tensor.
fix_loc (torch.Tensor, optional) – Fixation location. Defaults to None.
fixation_size (torch.Tensor, optional) – Fixation size. Defaults to None.

Returns:

Pooled samples from KNN grid sampling.

Return type:

torch.Tensor

all_coords(device=None)[source]

Get all sampling coordinates.

Parameters:: device (str, optional) – Device to place coordinates on. Defaults to None.
Returns:: Cartesian, polar, and plotting coordinates.
Return type:: tuple

class fovi.sensing.samplers.GaussianKNNGridSampler(*args, **kwargs)[source]

Bases: KNNGridSampler

K-Nearest Neighbors grid sampler with Gaussian-weighted pooling.

Similar to KNNGridSampler, but uses Gaussian-weighted pooling rather than simple averaging. The Gaussian weighting gives higher weight to photoreceptors that are closer to the center of each retinal ganglion cell’s receptive field, providing a more biologically plausible pooling mechanism.

Inherits all attributes and methods from KNNGridSampler, with the pooler replaced by a Gaussian-weighted version.

__init__(*args, **kwargs)[source]

Initialize the GaussianKNNGridSampler.

Parameters:

*args – Variable length argument list passed to KNNGridSampler. See KNNGridSampler.__init__ for details on positional arguments: fov, cmf_a, resolution, etc.
**kwargs – Arbitrary keyword arguments passed to KNNGridSampler. See KNNGridSampler.__init__ for details on keyword arguments: res_mult, cmf_a_mult, fixation_size, k, style, sample_cortex, dtype, device.