fovi.sensing.retina

class fovi.sensing.retina.RetinalTransform(resolution, start_res=256, fov=16, cmf_a=0.5, style='isotropic', sampler='grid_nn', fixation_size=None, device='cuda', dtype=torch.float, auto_match_cart_resources=True, pre_transforms=None, post_transforms=None, sigma=None, no_color_val=False, isotropic_plotting_type='v1like', sampler_backend='auto', fov_type='circular', **kwargs)

Bases: Module

Implements two computational hallmarks of retinal processing:

• spatially non-uniform (foveated) spatial sampling of the visual field

• spatially non-uniform (foveated) color representation

Foveated spatial sampling is based on isotropic cortical magnification of the form CMF=1/(r+a),where:

• r=polar radius (eccentricity)

• a is a parameter that controls the degree of foveation. smaller = more foveation. as a->infinity, we get uniform sampling.

• Equal sampling in cortical space is assumed, and the visual coordinates are computed by back-projection to acquire the foveated sampling grid.

This uses a CorticalSensorManifold module to represent the V1-like manifold of retina-like samples.

Foveated color representation is implemented by modeling hue saturation as a 1D gaussian of the visual field eccentricity, using a GaussianColorDecay module. - parameterized by self.sigma - can be turned off during eval mode with no_color_val=True

__init__(resolution, start_res=256, fov=16, cmf_a=0.5, style='isotropic', sampler='grid_nn', fixation_size=None, device='cuda', dtype=torch.float, auto_match_cart_resources=True, pre_transforms=None, post_transforms=None, sigma=None, no_color_val=False, isotropic_plotting_type='v1like', sampler_backend='auto', fov_type='circular', **kwargs)

Initialize the RetinalTransform module.

Parameters:

• resolution (int) – Target resolution for the retinal transform.

• start_res (int, optional) – Starting resolution. Defaults to 256.

• fov (float, optional) – Field of view diameter in degrees. Defaults to 16.

• cmf_a (float, optional) – Cortical magnification factor parameter. Defaults to 0.5.

• style (str, optional) – Sampling style. Defaults to ‘isotropic’.

• sampler (str, optional) – Sampler type. Defaults to ‘grid_nn’.

• fixation_size (int, optional) – Fixation size in pixels. Defaults to None.

• device (str, optional) – Device to use. Defaults to ‘cuda’.

• dtype (torch.dtype, optional) – Data type. Defaults to torch.float.

• auto_match_cart_resources (bool, optional) – Whether to auto-match cartesian resources. Defaults to True.

• pre_transforms (callable, optional) – Pre-processing transforms. Defaults to None.

• post_transforms (callable, optional) – Post-processing transforms. Defaults to None.

• sigma (float, optional) – Standard deviation for Gaussian color decay. Defaults to None.

• no_color_val (bool, optional) – Whether to disable color in eval mode. Defaults to False.

• sampler_backend (str, optional) – Sampler backend (auto, torch, or cuda) for both uint8 and floating inputs. Defaults to auto.

• fov_type (str, optional) – FoV geometry. Defaults to 'circular'.

• **kwargs – Additional arguments passed to warping function.

forward(x, fix_loc, fixation_size=None, **kwargs)

Forward pass of the retinal transform.

Parameters:

• x (torch.Tensor) – NCHW input tensor. Floating inputs retain their existing value-range semantics; uint8 inputs are interpreted as the equivalent unit-range image and converted after sampling whenever semantics permit.

• fix_loc (torch.Tensor or tuple) – Fixation location.

• fixation_size (int, optional) – Fixation size. Defaults to None.

• **kwargs – Additional arguments.

Returns:

Transformed tensor.

Return type:

torch.Tensor

_uint8_to_unit(x)

Convert native uint8-scale values to the current unit-float contract.

This deliberately mirrors the tensor path in ToTorchImage: cast to the target dtype first, then divide by 255. It is representation conversion, not statistical normalization; normalization remains in the supplied transform pipelines.

_announce_pre_transform_execution(fast)

One-time notice per decision, making the semantics/execution distinction explicit: the transforms.where config declares SEMANTICS (pre_warp = augmentations are defined on the full pre-warp image); the implementation is free to choose any execution that preserves those semantics exactly, and semantics-preserving post-warp execution (sample-then-augment) is preferred because it touches ~11x fewer elements. Re-announces only if the decision changes (e.g. transforms were swapped at runtime).

_fast_pre_transforms_supported()

The fast path requires (a) a plain nearest-neighbor GridSampler, so that sampling commutes exactly with pointwise (per-pixel, per-image-parameter) transforms, and (b) pre_transforms composed solely of such pointwise transforms.

Strict allowlist, checked by exact type: the container must be a plain fastT.Compose (other containers such as RandomApply/MultiSample also expose .transforms but have different call semantics), and every member must be exactly one of the verified pointwise classes — no subclasses, whose overridden behavior the fast path’s replicated arithmetic would not reproduce. Anything else (e.g., a future spatial transform in pre_transforms) falls back to the bit-exact reference path rather than silently producing point-cloud-augmented results.

_random_grayscale_points(s4, idx, num_output_channels)

Same arithmetic as fastF.random_grayscale (ITU-R 601-2 luma, index_copy on the selected images), but with the 3-element luma weights cached on-device instead of an H2D copy per call.

_padding_mask_src(x)

(B,1,H,W) ones (batch-expanded view of a cached (1,1,H,W) tensor) used to mark which sampled points fall inside the image (grid_sample zero-pads outside).

_sample_then_pre_transform(x, fix_loc, fixation_size)

Equivalent to self.sampler(self.pre_transforms(x.clone()), ...) for a nearest-neighbor GridSampler and pointwise pre_transforms, but ~10x cheaper: the image is sampled once, and the transforms run on the (B, C, N) point cloud.

Exactness notes (all verified bit-exact against the reference path): - nearest-neighbor sampling selects pixel values, so pointwise ops commute with it; - the contrast-jitter reference mean is a full-image statistic and is still computed

on the (brightness-adjusted) full image, exactly as FT.adjust_contrast does;

• grid_sample zero-pads out-of-bounds points AFTER the transforms in the reference path, so the transformed samples are re-masked with a grid-sampled ones image;

• RNG draws (bernoulli masks + per-image jitter parameters) have identical shapes and order, leaving the CUDA generator stream identical to the reference path.

change_sigma(sigma)

Change the sigma parameter for the Gaussian color decay.

Parameters:

sigma (float) – New sigma value for the Gaussian color decay.

get_warp_params()

Get the warping parameters.

Returns:

(fov, cmf_a, resolution, fixation_size)

Return type:

tuple

_check_fixation_size(fixation_size, batch_size)

Validate and format fixation size for batch processing.

Parameters:

• fixation_size (int, tuple, torch.Tensor, or None) – Fixation size specification.

• batch_size (int) – Number of samples in the batch.

Returns:

Formatted fixation size of shape (batch_size, 2). Tensor inputs stay tensors on their device (no host round-trip / device sync).

Return type:

np.ndarray or torch.Tensor

_check_fix_loc(fix_loc, batch_size)

Validate and format fixation location for batch processing.

Parameters:

• fix_loc (tuple, list, torch.Tensor, or None) – Fixation location specification.

• batch_size (int) – Number of samples in the batch.

Returns:

Formatted fixation location tensor of shape (batch_size, 2).

Return type:

torch.Tensor

_check_aspect_ratio(aspect_ratio, batch_size)

Validate and format aspect ratio for batch processing.

Parameters:

• aspect_ratio (float, list, or None) – Aspect ratio specification.

• batch_size (int) – Number of samples in the batch.

Returns:

Formatted aspect ratio array of shape (batch_size,).

Return type:

np.ndarray

_apply_aspect_ratio(fixation_size, aspect_ratio, batch_size)

Apply aspect ratio to fixation size while preserving area.

Parameters:

• fixation_size (torch.Tensor) – Current fixation size.

• aspect_ratio (float) – Aspect ratio to apply.

• batch_size (int) – Number of samples in the batch.

Returns:

Modified fixation size with applied aspect ratio.

Return type:

np.ndarray

fovi.sensing.retina.min_diff_for_cmf_a(cmf_a, fov, output_res, fixation_size, force_n_points=None, disallow_undersampling=True, force_less_than=False, device='cuda', fov_type='circular')

Helper function that computes minimum difference between radii for a given cmf_a value.

Parameters:

• cmf_a (float) – Cortical magnification factor parameter.

• fov (float) – Field of view diameter in degrees.

• output_res (int) – Output resolution.

• fixation_size (int) – Fixation size in pixels.

• force_n_points (int, optional) – Force number of points. Defaults to None.

• disallow_undersampling (bool, optional) – Whether to disallow undersampling. Defaults to True.

• force_less_than (bool, optional) – Whether to force less than. Defaults to False.

• device (str, optional) – Device to use. Defaults to ‘cuda’.

• fov_type (str, optional) – FoV geometry. Isotropic CMF fitting supports 'circular' and 'square'.

Returns:

Minimum difference between radii.

Return type:

float

fovi.sensing.retina.get_min_cmf_a(fixation_size, output_res, start_res=5496, fov=65, start_cmf_a=0.15, style='isotropic', maxiters=200, disallow_undersampling=True, use_scaled_fov=True, device='cuda', fov_type='circular')

Find the minimum cmf_a value that satisfies the constraints.

Parameters:

• fixation_size (int) – Fixation size in pixels.

• output_res (int) – Output resolution.

• start_res (int, optional) – Starting resolution. Defaults to 5496.

• fov (float, optional) – Field of view diameter in degrees. Defaults to 65.

• start_cmf_a (float, optional) – Starting cmf_a value. Defaults to 0.15.

• style (str, optional) – Sampling style. Defaults to ‘isotropic’.

• maxiters (int, optional) – Maximum iterations for optimization. Defaults to 200.

• disallow_undersampling (bool, optional) – Whether to disallow undersampling. Defaults to True.

• use_scaled_fov (bool, optional) – Whether to use scaled FOV. Defaults to True.

• fov_type (str, optional) – FoV geometry. Defaults to 'circular'.

Returns:

Minimum cmf_a value, or None if not found.

Return type:

float or None

class fovi.sensing.retina.GaussianColorDecay(sigma)

Bases: Module

Implements foveated color representation using Gaussian decay based on eccentricity.

This module models hue saturation as a 1D Gaussian function of visual field eccentricity, simulating the reduced color sensitivity in the periphery.

__init__(sigma)

Initialize the Gaussian color decay module.

Parameters:

sigma (float) – Standard deviation of the Gaussian decay function.

forward(x, radius)

Apply Gaussian color decay based on eccentricity.

Parameters:

• x (torch.Tensor) – Input tensor of shape (batch_size, channels, height, width).

• radius (torch.Tensor) – Eccentricity radius tensor.

Returns:

Color-decayed tensor with same shape as input.

Return type:

torch.Tensor