fovi.sensing.retina

class fovi.sensing.retina.RetinalTransform(resolution, start_res=256, fov=16, cmf_a=0.5, style='isotropic', sampler='grid_nn', fixation_size=None, device='cuda', dtype=torch.float, auto_match_cart_resources=True, pre_transforms=None, post_transforms=None, sigma=None, no_color_val=False, **kwargs)[source]

Bases: Module

Implements two computational hallmarks of retinal processing:

  • spatially non-uniform (foveated) spatial sampling of the visual field

  • spatially non-uniform (foveated) color representation

Foveated spatial sampling is based on isotropic cortical magnification of the form CMF=1/(r+a),where:

  • r=polar radius (eccentricity)

  • a is a parameter that controls the degree of foveation. smaller = more foveation. as a->infinity, we get uniform sampling.

  • Equal sampling in cortical space is assumed, and the visual coordinates are computed by back-projection to acquire the foveated sampling grid.

This uses a CorticalSensorManifold module to represent the V1-like manifold of retina-like samples.

Foveated color representation is implemented by modeling hue saturation as a 1D gaussian of the visual field eccentricity, using a GaussianColorDecay module. - parameterized by self.sigma - can be turned off during eval mode with no_color_val=True

__init__(resolution, start_res=256, fov=16, cmf_a=0.5, style='isotropic', sampler='grid_nn', fixation_size=None, device='cuda', dtype=torch.float, auto_match_cart_resources=True, pre_transforms=None, post_transforms=None, sigma=None, no_color_val=False, **kwargs)[source]

Initialize the RetinalTransform module.

Parameters:

  • resolution (int) – Target resolution for the retinal transform.

  • start_res (int, optional) – Starting resolution. Defaults to 256.

  • fov (float, optional) – Field of view diameter in degrees. Defaults to 16.

  • cmf_a (float, optional) – Cortical magnification factor parameter. Defaults to 0.5.

  • style (str, optional) – Sampling style. Defaults to ‘isotropic’.

  • sampler (str, optional) – Sampler type. Defaults to ‘grid_nn’.

  • fixation_size (int, optional) – Fixation size in pixels. Defaults to None.

  • device (str, optional) – Device to use. Defaults to ‘cuda’.

  • dtype (torch.dtype, optional) – Data type. Defaults to torch.float.

  • auto_match_cart_resources (bool, optional) – Whether to auto-match cartesian resources. Defaults to True.

  • pre_transforms (callable, optional) – Pre-processing transforms. Defaults to None.

  • post_transforms (callable, optional) – Post-processing transforms. Defaults to None.

  • sigma (float, optional) – Standard deviation for Gaussian color decay. Defaults to None.

  • no_color_val (bool, optional) – Whether to disable color in eval mode. Defaults to False.

  • **kwargs – Additional arguments passed to warping function.

forward(x, fix_loc, fixation_size=None, **kwargs)[source]

Forward pass of the retinal transform.

Parameters:

  • x (torch.Tensor) – Input tensor.

  • fix_loc (torch.Tensor or tuple) – Fixation location.

  • fixation_size (int, optional) – Fixation size. Defaults to None.

  • **kwargs – Additional arguments.

Returns:

Transformed tensor.

Return type:

torch.Tensor

change_sigma(sigma)[source]

Change the sigma parameter for the Gaussian color decay.

Parameters:

sigma (float) – New sigma value for the Gaussian color decay.

get_warp_params()[source]

Get the warping parameters.

Returns:

(fov, cmf_a, resolution, fixation_size)

Return type:

tuple

_check_fixation_size(fixation_size, batch_size)[source]

Validate and format fixation size for batch processing.

Parameters:

  • fixation_size (int, tuple, or None) – Fixation size specification.

  • batch_size (int) – Number of samples in the batch.

Returns:

Formatted fixation size array of shape (batch_size, 2).

Return type:

np.ndarray

_check_fix_loc(fix_loc, batch_size)[source]

Validate and format fixation location for batch processing.

Parameters:

  • fix_loc (tuple, list, torch.Tensor, or None) – Fixation location specification.

  • batch_size (int) – Number of samples in the batch.

Returns:

Formatted fixation location tensor of shape (batch_size, 2).

Return type:

torch.Tensor

_check_aspect_ratio(aspect_ratio, batch_size)[source]

Validate and format aspect ratio for batch processing.

Parameters:

  • aspect_ratio (float, list, or None) – Aspect ratio specification.

  • batch_size (int) – Number of samples in the batch.

Returns:

Formatted aspect ratio array of shape (batch_size,).

Return type:

np.ndarray

_apply_aspect_ratio(fixation_size, aspect_ratio, batch_size)[source]

Apply aspect ratio to fixation size while preserving area.

Parameters:

  • fixation_size (torch.Tensor) – Current fixation size.

  • aspect_ratio (float) – Aspect ratio to apply.

  • batch_size (int) – Number of samples in the batch.

Returns:

Modified fixation size with applied aspect ratio.

Return type:

np.ndarray

fovi.sensing.retina.min_diff_for_cmf_a(cmf_a, fov, output_res, fixation_size, force_n_points=None, disallow_undersampling=True, force_less_than=False, device='cuda')[source]

Helper function that computes minimum difference between radii for a given cmf_a value.

Parameters:

  • cmf_a (float) – Cortical magnification factor parameter.

  • fov (float) – Field of view diameter in degrees.

  • output_res (int) – Output resolution.

  • fixation_size (int) – Fixation size in pixels.

  • force_n_points (int, optional) – Force number of points. Defaults to None.

  • disallow_undersampling (bool, optional) – Whether to disallow undersampling. Defaults to True.

  • force_less_than (bool, optional) – Whether to force less than. Defaults to False.

  • device (str, optional) – Device to use. Defaults to ‘cuda’.

Returns:

Minimum difference between radii.

Return type:

float

fovi.sensing.retina.get_min_cmf_a(fixation_size, output_res, start_res=5496, fov=65, start_cmf_a=0.15, style='isotropic', maxiters=200, disallow_undersampling=True, use_scaled_fov=True, device='cuda')[source]

Find the minimum cmf_a value that satisfies the constraints.

Parameters:

  • fixation_size (int) – Fixation size in pixels.

  • output_res (int) – Output resolution.

  • start_res (int, optional) – Starting resolution. Defaults to 5496.

  • fov (float, optional) – Field of view diameter in degrees. Defaults to 65.

  • start_cmf_a (float, optional) – Starting cmf_a value. Defaults to 0.15.

  • style (str, optional) – Sampling style. Defaults to ‘isotropic’.

  • maxiters (int, optional) – Maximum iterations for optimization. Defaults to 200.

  • disallow_undersampling (bool, optional) – Whether to disallow undersampling. Defaults to True.

  • use_scaled_fov (bool, optional) – Whether to use scaled FOV. Defaults to True.

Returns:

Minimum cmf_a value, or None if not found.

Return type:

float or None

class fovi.sensing.retina.GaussianColorDecay(sigma)[source]

Bases: Module

Implements foveated color representation using Gaussian decay based on eccentricity.

This module models hue saturation as a 1D Gaussian function of visual field eccentricity, simulating the reduced color sensitivity in the periphery.

__init__(sigma)[source]

Initialize the Gaussian color decay module.

Parameters:

sigma (float) – Standard deviation of the Gaussian decay function.

forward(x, radius)[source]

Apply Gaussian color decay based on eccentricity.

Parameters:

  • x (torch.Tensor) – Input tensor of shape (batch_size, channels, height, width).

  • radius (torch.Tensor) – Eccentricity radius tensor.

Returns:

Color-decayed tensor with same shape as input.

Return type:

torch.Tensor