tramway.inference package¶
tramway.inference.base module¶
-
class
tramway.inference.base.
Local
(index, data, center=None, span=None, boundary=None)¶ Bases:
tramway.core.lazy.Lazy
Spatially local subset of elements (e.g. translocations). Abstract class.
-
index
¶ This cell’s index as referenced in
FiniteElements
.Type: int
-
center
¶ Cell center coordinates.
Type: array-like
-
span
¶ Difference vectors from this cell’s center to adjacent centers.
Type: array-like
-
boundary/hull
Convex hull of the contained locations.
Type: scipy.spatial.qhull.ConvexHull
-
boundary
¶ any, property
Cell boundary/hull.
-
center
-
data
-
dim
¶ int, property
Dimension of the terminal elements.
-
get_localization_error
(_kwargs=None, _default_value=None, _localization_error_is_sigma=False, **kwargs)¶ Return the localization error as \(sigma^2\).
Parameters: - _kwargs (dict) – mutable keyword arguments; ‘localization_error’, ‘sigma’ and ‘sigma2’ are popped out.
- _default_value (float) – default localization error (subject to _localization_error_is_sigma).
- _localization_error_is_sigma (bool) – argument ‘localization_error’ in _kwargs or kwargs is considered as \(sigma\).
Returns: \(sigma^2\).
Return type: float
-
hull
¶ scipy.spatial.qhull.ConvexHull (or any if not convex), property
Alias for boundary.
-
index
-
span
-
tcount
¶ int, property
Total number of terminal elements (e.g. translocations).
-
volume
¶ float, property
Surface area (2D) or volume (3D+) of the convex hull.
Note that the hull should be convex so that the volume can be transparently extracted. In other cases, volume can be manually set just like a normal attribute.
-
-
class
tramway.inference.base.
Distributed
(cells, adjacency, index=None, center=None, span=None, central=None, boundary=None)¶ Bases:
tramway.inference.base.Local
-
central
¶ margin cells are not central.
Type: array of bools
-
adjacency
¶ scipy.sparse.csr_matrix, rw property
Cell adjacency matrix. Row and column indices are to be mapped with indices.
-
any_cell
()¶
-
ccount
¶ int, ro property
Total number of terminal cells. Duplicates are ignored.
-
cells
¶ list or OrderedDict, rw property for
data
Collection of
Local
. Indices may not match with the globalindex
attribute of the elements, but match with attributescentral
,adjacency
anddegree
.
-
central
-
clear_caches
()¶
-
degree
¶ array of ints, ro lazy property
Number of adjacent cells.
-
dim
¶ int, ro property
Dimension of the terminal points.
-
flatten
()¶
-
grad
(i, X, index_map=None, **kwargs)¶ Local spatial gradient.
Sub-classes are free to return multi-component gradients as matrices provided that they exhibit as many columns as there are dimensions in the (trans-)location data.
grad_sum()
is then responsible for summing all the elements of such matrices.Parameters: - i (int) – cell index at which the gradient is evaluated.
- X (numpy.ndarray) – vector of a scalar measurement at every cell.
- index_map (numpy.ndarray) – index map that converts cell indices to indices in X.
Returns: gradient vector with as many elements as spatial dimensions.
Return type: numpy.ndarray
-
grad_sum
(i, grad2, index_map=None)¶ Sum for local penalties (e.g. local gradient vector with squared elements) at a given cell.
Parameters: - i (int) – cell index.
- grad2 (numpy.ndarray) – local penalties (e.g. local gradient vector with squared elements).
- index_map (numpy.ndarray) – index mapping, useful to convert cell indices to positional indices in an optimization array for example.
Returns: weighted sum of the elements of grad2.
Return type: float
-
group
(ngroups=None, max_cell_count=None, cell_centers=None, adjacency_margin=2, connected=False)¶ Make groups of cells.
This builds up an extra hierarchical level. For example if self is a FiniteElements of Cell, then this returns a FiniteElements of FiniteElements (the groups) of Cell.
Several grouping strategies are proposed.
Parameters: - ngroups (int) – number of groups.
- max_cell_count (int) – maximum number of cells per group.
- cell_centers (array-like) – spatial centers of the groups.
Cells are sorted by nearest neighbours.
If not provided,
group()
will use a k-means approach to positionning the centers. - adjacency_margin (int) – groups are dilated to the adjacent cells adjacency_margin times. Defaults to 2.
- connected (bool) – separates the connected components from one another. Conflicts with the other arguments.
Returns: a new object of the same type as self that contains other such objects as cells.
Return type: FiniteElements
-
indices
¶
-
items
()¶
-
keys
()¶
-
local_variation
(i, X, index_map=None, **kwargs)¶ Local spatial variation, gradient-like, aimed at penalizing spatial variations.
See also
delta0()
.As of version 0.3.8: new; called for spatial regularization in stochastic_dv.
As of version 0.4: default implementation becomes
delta0()
.May become the new default for spatial regularization.
Parameters: - i (int) – cell index at which the gradient is evaluated.
- X (numpy.ndarray) – vector of a scalar measurement at every cell.
- index_map (numpy.ndarray) – index map that converts cell indices to indices in X.
Returns: delta vector with as many elements as there are spatial dimensions.
Return type: numpy.ndarray
-
neighbours
(i)¶ Indices of neighbour cells.
Argument:
i (int): cell index.Returns: indices of the neighbour cells of cell i. Return type: numpy.ndarray
-
reverse
¶ dict of ints, ro lazy property
Get “local” indices from global ones.
-
run
(function, *args, **kwargs)¶ Apply a function that takes a group (
FiniteElements
) of terminal cells as input argument, plus args and kwargs, and must return a pandas.DataFrame array with the indices referring to cells.function is called for each group of terminal cells, adjacency margins are removed if any, and the resulting DataFrame are merged into a single DataFrame.
function may instead be applied to self in the case self.cells has been overloaded to exhibit the output features as attributes.
Parameters: - function (callable) – the function to be called on each terminal
FiniteElements
. Its first argument is theFiniteElements
object. It should return maps as aDataFrame
and optionally posteriors asSeries
orDataFrame
. - args (list) – positional arguments for function after the first one.
- kwargs (dict) – keyword arguments for function; the following arguments are popped out of kwargs:
- returns (list) – attributes to be collected from all the individual cells as return values; if defined, the values returned by function are ignored.
- worker_count (int) – number of simultaneously working processing units, if the self.cells .
- profile (bool or str or tuple) – profile each child job if any;
if str, dump the output stats into .prof files;
if tuple, print a report with
print_stats()
and tuple elements as input arguments. - function_worker_count (int) – if worker_count is a reserved keyword argument for function, function_worker_count can be specified and is passed to function with name/keyword worker_count; in this case, worker_count must be a multiple of function_worker_count.
Returns: single merged array of maps. If function returns two output arguments,
run()
also returns a second merged array of posterior probabilities(?).Return type: pandas.DataFrame
- function (callable) – the function to be called on each terminal
-
space_cols
¶ list of ints or strings, ro property
Column indices for coordinates of the terminal points.
-
tcount
¶ int, ro property
Total number of terminal points. Duplicates are ignored.
-
values
()¶
-
-
class
tramway.inference.base.
Cell
(index, data, center=None, span=None, boundary=None)¶ Bases:
tramway.inference.base.Local
Spatially constrained subset of (trans-)locations with associated intermediate calculations.
-
cache
¶ Depending on the inference approach and objective, caching an intermediate result may avoid repeating many times a same computation. Usage of this cache is totally free and comes without support for concurrency.
Type: any
-
cache
-
clear_cache
()¶
-
dim
¶ int, property
Dimension of the terminal elements.
-
fuzzy
¶
-
space_cols
¶ list of ints or strings, lazy
Column indices for coordinates.
-
space_data
¶
-
tcount
¶ int
Number of (trans-)locations.
-
time_col
¶ int or string, lazy
Column index for time.
-
time_data
¶
-
-
class
tramway.inference.base.
Locations
(index, data, center=None, span=None, boundary=None)¶ Bases:
tramway.inference.base.Cell
-
locations
¶ array-like, property
Locations as a matrix of coordinates and times with as many columns as dimensions; this is an alias for
data
.
-
r
¶ array-like, ro property
Location coordinates; xy is an alias
-
t
¶ array-like, ro property
Location timestamps.
-
xy
¶
-
-
class
tramway.inference.base.
Translocations
(index, translocations, center=None, span=None, boundary=None)¶ Bases:
tramway.inference.base.Cell
-
origins
¶ Initial locations (both spatial coordinates and times).
Type: array-like, ro property
-
destinations
¶ Final locations (both spatial coordinates and times).
Type: array-like, ro property
-
destinations
-
dr
¶ array-like, ro property
Translocation displacements in space; dxy is an alias.
-
dt
¶ array-like, ro property
Translocation durations.
-
dxy
¶
-
origins
-
r
¶
-
t
¶ array-like, ro property
Initial timestamps.
-
-
class
tramway.inference.base.
Maps
(maps, mode=None, posteriors=None)¶ Bases:
tramway.core.lazy.Lazy
Basic container for maps, posteriors and the associated input parameters used to generate the maps.
Attributes distributed_translocations, partition_file, tessellation_program, version are deprecated and will be removed.
Functional dependencies:
- setting maps unsets _features
-
defattr
(attr, val)¶
-
features
¶ list or
None
List of different feature names without the coordinate suffix.
For example, if maps has columns diffusivity, force x and force y, features will return diffusivity and force.
-
maps
¶
-
sub
(ix, reindex=False)¶ Sub-map.
Operates like loc on a DataFrame.
Parameters: - ix (slice or array-like) – cell indices or boolean array.
- reindex (bool) – make index range from 0 without missing integer.
Returns: sub-map.
Return type:
-
variables
¶ Alias for features.
-
tramway.inference.base.
FiniteElement
¶ alias of
tramway.inference.base.Cell
-
tramway.inference.base.
FiniteElements
¶ alias of
tramway.inference.base.Distributed
-
tramway.inference.base.
identify_columns
(points, trajectory_col=True)¶ Identify columns by type.
Parameters: - points (array-like) – location coordinates and trajectory index.
- trajectory_col (bool or int or str) – trajectory column index or name,
or
False
orNone
if there is no trajectory information is to be found or extracted, orTrue
to automatically identify such a column.
Returns: (coord_cols, trajectory_col, get_var, get_point)
Return type: tuple
coord_cols is an array of column indices/names other than the trajectory column. If the data contains typical translocation column names such as ‘dx’, ‘dy’, ‘dt’, coord_cols is a tuple with, as first element, an array of location column names, and as second element an array of translocation column names.
trajectory_col is the column index/name that contains tracking information.
get_var is a callable that takes an array like points and column identifiers and returns the corresponding sub-matrix.
get_point is a callable that takes an array like points and row indices and returns the corresponding sub-matrix.
-
tramway.inference.base.
get_locations
(points, index=None, coord_cols=None, get_var=None, get_point=None)¶ Make helpers for manipulating the point data.
Parameters: - points (array-like) – location coordinates and trajectory index.
- index (ndarray or pair of ndarrrays or sparse matrix) – point-cell association (see
CellStats
documentation orcell_index()
).
Returns: (locations, location_cell, get_point)
Return type: tuple
locations are point coordinates (same format as points with the trajectory column removed.
location_cell is either an array of cell indices (same size as locations) or a callable that takes a cell index and returns a boolean array with Trues for locations associated with the specified cell and Falses elsewhere.
get_point is a callable that takes an array like points or locations and row indices and returns the corresponding rows in the same format.
-
tramway.inference.base.
get_translocations
(points, index=None, coord_cols=None, trajectory_col=True, get_var=None, get_point=None)¶ Identify translocations as initial and final points.
Parameters: - points (array-like) – location coordinates and trajectory index.
- index (ndarray or pair of ndarrays or sparse matrix) – point-cell association (see
CellStats
documentation orcell_index()
).
Returns: (initial_point, final_point, initial_cell, final_cell, get_point)
Return type: tuple
initial_point and final_point are point coordinates (same format as points with the trajectory index column removed) respectively of the initial and final displacement locations.
initial_cell and final_cell are either arrays of cell indices (same size as initial_point) or callables that take a cell index and return a boolean array with Trues for displacements/translocations associated with the specified cell and Falses elsewhere.
get_point is a callable that takes an array like points and row indices and returns the corresponding rows in the same format.
-
tramway.inference.base.
distributed
(cells, new_cell=None, new_group=<class 'tramway.inference.base.Distributed'>, fuzzy=None, new_cell_kwargs={}, new_group_kwargs={}, fuzzy_kwargs={}, new=None, include_empty_cells=False, verbose=False)¶ Build a FiniteElements-like object from a
CellStats
object.Cells with no (trans-)locations are discarded in addition to those with null or negative label.
Parameters: - cells (CellStats) – tessellation and partition; must contain (trans-)location data.
- new_cell (callable) – cell constructor; default is
Locations
orTranslocations
depending on the data in cells. - new_group (callable) – constructor for groups of cell; default is
FiniteElements
. - fuzzy (callable) – (trans-)location-cell weighting function; the default for translocations considers the initial cell, no weight.
- new_cell_kwargs (dict) – keyword arguments for new_cell.
- new_group_kwargs (dict) – keyword arguments for new_group.
- fuzzy_kwargs (dict) – keyword arguments for fuzzy.
- new (callable) – legacy argument; use new_group instead.
- include_empty_cells (bool) – do not discard cells with no (trans-)locations.
fuzzy takes a
Tessellation
object, a cell index (int), location coordinates (array-like), cell indices (array-like or callable) and the get_point function returned by get_locations or get_translocations.If the data are translocations, fuzzy takes a pair of arrays as location coordinates (initial and final locations respectively) and a pair of arrays or callables as cell indices (initial and final cells respectively).
fuzzy returns an array of booleans or values of any scalar type that can be evaluated logically. The returned array contains as many elements as input (trans-)locations.
-
class
tramway.inference.base.
TrackedMolecules
(index, translocations, center=None, span=None, boundary=None)¶ Bases:
tramway.inference.base.Translocations
-
n
¶ Trajectory indices.
Type: numpy.ndarray
-
n
-
-
class
tramway.inference.base.
DistributeMerge
(cells, adjacency, min_location_count=None)¶ Bases:
tramway.inference.base.Distributed
-
merged
¶
-
-
exception
tramway.inference.base.
DiffusivityWarning
(diffusivity, lower_bound=None)¶ Bases:
RuntimeWarning
-
exception
tramway.inference.base.
OptimizationWarning
¶ Bases:
RuntimeWarning
-
tramway.inference.base.
smooth_infer_init
(cells, min_diffusivity=None, jeffreys_prior=None, **kwargs)¶ Initialize the diffusivity array for translocations.
This function is a collection of checks and initial computations suitable for infer functions in inference plugins.
Parameters: - cells (FiniteElements) – first input argument of the infer function.
- min_diffusivity (float) – minimum allowed diffusivity value.
- jeffreys_prior (bool) – activate Jeffreys’ prior.
Returns:
- index (
numpy.ndarray
) – cell indices corresponding to arrays n, dt_mean and D_initial. - reverse_index (
numpy.ndarray
) – element indices in arrays index, n, dt_mean, D_initial for each cell index. - n (
numpy.ndarray
) – translocation count for each cell. - dt_mean (
numpy.ndarray
) – mean translocation duration for each cell. - D_initial (
numpy.ndarray
) – initial diffusivity array. - min_diffusivity (
float
orNone
) – minimum diffusivity value; the returned value depends on jeffreys_prior. - D_bounds (
list
) – list of (lower bound, upper bound) couples; suitable as bounds input argument for thescipy.optimize.minimize()
function. - border (
numpy.ndarray
) – MxD boolean matrix with M the number of cells and D the number of space dimensions
tramway.inference.d plugin¶
-
tramway.inference.d.
infer_D
(cells, diffusivity_prior=None, jeffreys_prior=None, min_diffusivity=None, max_iter=None, epsilon=None, rgrad=None, **kwargs)¶
tramway.inference.degraded_d plugin¶
tramway.inference.standard_d plugin¶
-
class
tramway.inference.standard_d.
Result
(result, success, message, ncalls)¶ Bases:
tuple
-
message
¶ Alias for field number 2
-
ncalls
¶ Alias for field number 3
-
result
¶ Alias for field number 0
-
success
¶ Alias for field number 1
-
-
tramway.inference.standard_d.
d_neg_posterior1
(diffusivity, cells, sigma2, diffusivity_prior, jeffreys_prior, dt_mean, min_diffusivity, index, reverse_index, grad_kwargs)¶ Similar to
smooth_d_neg_posterior()
. The smoothing prior features an alternative spatial “gradient” implemented usinglocal_variation()
instead ofgrad()
.
-
tramway.inference.standard_d.
infer_smooth_D
(cells, diffusivity_prior=None, jeffreys_prior=None, min_diffusivity=None, max_iter=None, epsilon=None, rgrad=None, verbose=False, **kwargs)¶
-
tramway.inference.standard_d.
interruptible_minimize
(func, prms, args, **kwargs)¶
-
tramway.inference.standard_d.
smooth_d_neg_posterior
(diffusivity, cells, sigma2, diffusivity_prior, jeffreys_prior, dt_mean, min_diffusivity, index, reverse_index, grad_kwargs)¶ Adapted from InferenceMAP’s dDDPosterior procedure:
for (int a = 0; a < NUMBER_OF_ZONES; a++) { ZONES[a].gradDx = dvGradDx(DD,a); ZONES[a].gradDy = dvGradDy(DD,a); ZONES[a].priorActive = true; } for (int z = 0; z < NUMBER_OF_ZONES; z++) { const double gradDx = ZONES[z].gradDx; const double gradDy = ZONES[z].gradDy; const double D = DD[z]; for (int j = 0; j < ZONES[z].translocations; j++) { const double dt = ZONES[z].dt[j]; const double dx = ZONES[z].dx[j]; const double dy = ZONES[z].dy[j]; const double Dnoise = LOCALIZATION_ERROR*LOCALIZATION_ERROR/dt; result += - log(4.0*PI*(D + Dnoise)*dt) - ( dx*dx + dy*dy)/(4.0*(D+Dnoise)*dt); } if (ZONES[z].priorActive == true) { result -= D_PRIOR*(gradDx*gradDx*ZONES[z].areaX + gradDy*gradDy*ZONES[z].areaY); if (JEFFREYS_PRIOR == 1) { result += 2.0*log(D) - 2.0*log(D*ZONES[z].dtMean + LOCALIZATION_ERROR*LOCALIZATION_ERROR); } } } return -result;
tramway.inference.d_conj_prior plugin¶
-
tramway.inference.d_conj_prior.
infer_d_conj_prior
(cells, alpha=0.95, return_zeta_spurious=True, trust=False, **kwargs)¶ Infer diffusivity MAP and confidence interval using a conjugate prior [Serov et al. 2019]
Parameters: - cells (tramway.inference.base.Distributed) – distributed cells.
- alpha (float) – confidence level for CI estimation.
- return_zeta_spurious (bool) – add variable zeta_spurious to the returned dataframe.
- trust (bool) – if
False
, silently catch any exception raised in get_D_confidence_interval.
Returns: maps including diffusivity MAPs and confidence intervals, and SNR extensions.
Return type: pandas.DataFrame
Other valid input arguments are localization- and gradient-related arguments.
See also
get_D_confidence_interval()
,get_localization_error()
andget_grad_kwargs()
.
-
tramway.inference.d_conj_prior.
infer_d_map_and_ci
(*args, **kwargs)¶ Backward-compatibility alias for
infer_d_conj_prior()
.
tramway.inference.ddrift plugin¶
-
tramway.inference.ddrift.
infer_DD
(cells, diffusivity_prior=None, drift_prior=None, jeffreys_prior=False, min_diffusivity=None, max_iter=None, epsilon=None, rgrad=None, **kwargs)¶
tramway.inference.degraded_ddrift plugin¶
tramway.inference.standard_ddrift plugin¶
-
tramway.inference.standard_ddrift.
dd_neg_posterior1
(x, dd, cells, sigma2, diffusivity_prior, drift_prior, jeffreys_prior, dt_mean, min_diffusivity, index, reverse_index, grad_kwargs)¶
-
tramway.inference.standard_ddrift.
infer_smooth_DD
(cells, diffusivity_prior=None, drift_prior=None, jeffreys_prior=False, min_diffusivity=None, max_iter=None, epsilon=None, rgrad=None, verbose=False, **kwargs)¶
-
tramway.inference.standard_ddrift.
smooth_dd_neg_posterior
(x, dd, cells, sigma2, diffusivity_prior, drift_prior, jeffreys_prior, dt_mean, min_diffusivity, index, reverse_index, grad_kwargs)¶
tramway.inference.density plugin¶
-
tramway.inference.density.
infer_density
(cells, convex_hull='fallback', trust_volume=False, scale=False, min_volume=None, volume=None, **kwargs)¶ If
knn
was defined for the partition, useconvex_hull='always'
.With the Voronoi-based tessellations (random, kmeans, gwr), do not set
trust_volume
toTrue
. With the other tessellations (grid, hexagon, kdtree), usetrust_volume=True
ifconvex_hull
evaluates toTrue
.With the regular tessellations (grid, hexagon),
scale
can beTrue
. With the other tessellations, do not setscale
toTrue
.
tramway.inference.df plugin¶
-
tramway.inference.df.
infer_DF
(cells, diffusivity_prior=None, force_prior=None, jeffreys_prior=False, min_diffusivity=None, max_iter=None, epsilon=None, rgrad=None, **kwargs)¶
tramway.inference.degraded_df plugin¶
tramway.inference.standard_df plugin¶
-
tramway.inference.standard_df.
df_neg_posterior1
(x, df, cells, sigma2, diffusivity_prior, force_prior, jeffreys_prior, dt_mean, min_diffusivity, index, reverse_index, grad_kwargs)¶
-
tramway.inference.standard_df.
infer_smooth_DF
(cells, diffusivity_prior=None, force_prior=None, potential_prior=None, jeffreys_prior=False, min_diffusivity=None, max_iter=None, epsilon=None, rgrad=None, verbose=False, **kwargs)¶ Argument potential_prior is an alias for force_prior which penalizes the large force amplitudes.
-
tramway.inference.standard_df.
smooth_df_neg_posterior
(x, df, cells, sigma2, diffusivity_prior, force_prior, jeffreys_prior, dt_mean, min_diffusivity, index, reverse_index, grad_kwargs)¶
tramway.inference.dv plugin¶
-
class
tramway.inference.dv.
DV
(diffusivity, potential, diffusivity_prior=None, potential_prior=None, minimum_diffusivity=None, positive_diffusivity=None, prior_include=None)¶ Bases:
tramway.core.chain.ChainArray
Convenience class for handling the D+V parameter vector and related priors.
Objects of this class are not supposed to be serialized in .rwa files.
-
D
¶
-
V
¶
-
diffusivity_prior
(j)¶
-
minimum_diffusivity
¶
-
potential_prior
(j)¶
-
prior_include
¶
-
-
tramway.inference.dv.
dv_neg_posterior
(x, dv, cells, sigma2, jeffreys_prior, dt_mean, index, reverse_index, grad_kwargs, y0, verbose, posteriors)¶ Adapted from InferenceMAP’s dvPosterior procedure modified:
for (int a = 0; a < NUMBER_OF_ZONES; a++) { ZONES[a].gradVx = dvGradVx(DV,a); ZONES[a].gradVy = dvGradVy(DV,a); ZONES[a].gradDx = dvGradDx(DV,a); ZONES[a].gradDy = dvGradDy(DV,a); ZONES[a].priorActive = true; } for (int z = 0; z < NUMBER_OF_ZONES; z++) { const double gradVx = ZONES[z].gradVx; const double gradVy = ZONES[z].gradVy; const double gradDx = ZONES[z].gradDx; const double gradDy = ZONES[z].gradDy; const double D = DV[2*z]; for (int j = 0; j < ZONES[z].translocations; j++) { const double dt = ZONES[z].dt[j]; const double dx = ZONES[z].dx[j]; const double dy = ZONES[z].dy[j]; const double Dnoise = LOCALIZATION_ERROR*LOCALIZATION_ERROR/dt; result += - log(4.0*PI*(D + Dnoise)*dt) - ((dx-D*gradVx*dt)*(dx-D*gradVx*dt) + (dy-D*gradVy*dt)*(dy-D*gradVy*dt))/(4.0*(D+Dnoise)*dt); } if (ZONES[z].priorActive == true) { result -= V_PRIOR*(gradVx*gradVx*ZONES[z].areaX + gradVy*gradVy*ZONES[z].areaY); result -= D_PRIOR*(gradDx*gradDx*ZONES[z].areaX + gradDy*gradDy*ZONES[z].areaY); if (JEFFREYS_PRIOR == 1) { result += 2.0*log(D*1.00) - 2.0*log(D*ZONES[z].dtMean + LOCALIZATION_ERROR*LOCALIZATION_ERROR); } }
with
dx-D*gradVx*dt
anddy-D*gradVy*dt
modified asdx+D*gradVx*dt
anddy+D*gradVy*dt
respectively.See also DV.
-
tramway.inference.dv.
dv_neg_posterior1
(x, dv, cells, sigma2, jeffreys_prior, dt_mean, index, reverse_index, grad_kwargs, y0, verbose, posteriors)¶ Similar to
dv_neg_posterior()
. The smoothing priors feature an alternative spatial “gradient” implemented usinglocal_variation()
instead ofgrad()
.
-
tramway.inference.dv.
inferDV
(cells, diffusivity_prior=None, potential_prior=None, jeffreys_prior=False, min_diffusivity=None, max_iter=None, epsilon=None, export_centers=False, verbose=True, compatibility=False, D0=None, V0=None, rgrad=None, **kwargs)¶
tramway.inference.gradient module¶
-
tramway.inference.gradient.
get_grad_kwargs
(_kwargs=None, gradient=None, grad_epsilon=None, grad_selection_angle=None, compatibility=None, grad=None, epsilon=None, **kwargs)¶ Parse
grad1()
keyworded arguments.Parameters: - _kwargs (dict) – mutable keyword arguments; all the keywords below are popped out.
- gradient (str) – either grad1 or gradn.
- grad_epsilon (float) – eps argument for
grad1()
(orneighbours_per_axis()
). - grad_selection_angle (float) – selection_angle argument for
grad1()
(orneighbours_per_axis()
). - compatibility (bool) – backward compatibility with InferenceMAP.
- grad (str) – alias for gradient.
- epsilon (float) – alias for grad_epsilon.
Returns: keyworded arguments to
grad()
.Return type: dict
Note: grad and gradient are currently ignored.
-
tramway.inference.gradient.
neighbours_per_axis
(i, cells, centers=None, eps=None, selection_angle=None)¶ See also
grad1()
.
-
tramway.inference.gradient.
grad1
(cells, i, X, index_map=None, eps=None, selection_angle=None, na=nan)¶ Local gradient by 2 degree polynomial interpolation along each dimension independently.
Considering spatial coordinate \(x\), bin \(i\) and its neighbour bins \(\\mathcal{N}_i\):
\[\begin{split}\left.X'_i\right|_x = \left\{ \begin{array}{ll} \frac{X_i - \overline{X}_{\mathcal{N}_i}}{x_i - \overline{x}_{\mathcal{N}_i}} & \textrm{ if either } \mathcal{N}_i^- \textrm{ or } \mathcal{N}_i^+ \textrm{ is } \emptyset \\ b + 2 c x_{i} & \textrm{ with } \left[ \begin{array}{ccc} 1 & \overline{x}_{\mathcal{N}_i^-} & \overline{x}_{\mathcal{N}_i^-}^2 \\ 1 & x_i & x_i^2 \\ 1 & \overline{x}_{\mathcal{N}_i^+} & \overline{x}_{\mathcal{N}_i^+}^2 \end{array} \right] . \left[ \begin{array}{c} a \\ b \\ c \end{array} \right] = \left[ \begin{array}{c}\overline{X}_{\mathcal{N}_i^-} \\ X_i \\ \overline{X}_{\mathcal{N}_i^+}\end{array} \right] \textrm{ otherwise } \\ \end{array} \right.\end{split}\]Claims cache variable grad1.
Supports the InferenceMAP way of dividing the space in non-overlapping regions (quadrants in 2D) so that each neighbour is involved in the calculation of a single component of the gradient.
This is the default behaviour in cases where the adjacent attribute contains adequate labels, i.e.
-j
andj
for each dimensionj
, representing one side and the other side of cell-i’s center. This is also the default behaviour otherwise, when eps isNone
.If eps is defined, the calculation of a gradient component may recruit all the points minus those at a projected distance smaller than this value.
If selection_angle is defined, neighbours are selected in two symmetric hypercones which top angle is selection_angle times \(\pi\) radians.
If
get_grad_kwargs()
is called (most if not all the inference modes call this function), the default selection behaviour is equal to selection_angle=0.9. Otherwise,grad1()
defaults to selection_angle=0.5 in 2D.See also
neighbours_per_axis.
Parameters: - cells (tramway.inference.base.Distributed) – distributed cells.
- i (int) – cell index at which the gradient is evaluated.
- X (array) – vector of a scalar measurement at every cell.
- index_map (array) – index map that converts cell indices to indices in X; for a given cell, should be constant.
- eps (float) – minimum projected distance from cell-i’s center for neighbours to be considered
in the calculation of the gradient;
if eps is
None
, the space is instead divided in twice as many non-overlapping regions as dimensions, and each neighbour is assigned to a single region (counts against a single dimension). Incompatible with selection_angle. - selection_angle (float) – top angle of the neighbour selection hypercones; should be in the \([0.5, 1.0[\) range. Incompatible with eps.
- na (float) – value for undefined components (no neighbours).
Returns: local gradient vector with as many elements as there are dimensions in the (trans-)location data.
Return type: array
-
tramway.inference.gradient.
gradn
(cells, i, X, index_map=None)¶ Local gradient by multi-dimensional 2 degree polynomial interpolation.
Parameters: - cells (tramway.inference.base.Distributed) – distributed cells.
- i (int) – cell index at which the gradient is evaluated.
- X (array) – vector of a scalar measurement at every cell.
- index_map (array) – index map that converts cell indices to indices in X.
Returns: local gradient vector with as many elements as there are dimensions in the (trans-)location data.
Return type: array
-
tramway.inference.gradient.
delta0
(cells, i, X, index_map=None, **kwargs)¶ Differences with neighbour values:
\[\Delta X_i = \frac{1}{\sqrt{|\mathcal{N}_i|}} \left[ \frac{X_i-X_j}{|| \textbf{x}_i-\textbf{x}_j ||} \right]_{j \in \mathcal{N}_i}\]The above scaling is chosen so that combining the element-wise square of
local_variation()
withgrad_sum()
results in the following scalar penalty:\[\Delta X_i^2 = \frac{1}{ | \mathcal{N}_i | } \sum_{j \in \mathcal{N}_i} \left( \frac{X_i-X_j}{|| \textbf{x}_i-\textbf{x}_j ||} \right)^2\]Claims cache variable ‘delta0’.
Parameters: - cells (tramway.inference.base.Distributed) – distributed cells.
- i (int) – cell index at which the differences are evaluated.
- X (array) – vector of a scalar measurement at every cell.
- index_map (array) – index map that converts cell indices to indices in X.
Returns: difference vector with as many elements as there are neighbours.
Return type: array
-
tramway.inference.gradient.
delta0_without_scaling
(cells, i, X, index_map=None, **kwargs)¶
-
tramway.inference.gradient.
delta1
(cells, i, X, index_map=None, eps=None, selection_angle=None)¶ Local spatial variation.
Similar to grad1. Considering spatial coordinate \(x\), bin \(i\) and its neighbour bins \(\\mathcal{N}_i\):
\[\begin{split}\left.\Delta X_i\right|_x = \left( \begin{array}{ll} \frac{X_i - \overline{X}_{\mathcal{N}_i^-}}{x_i - \overline{x}_{\mathcal{N}_i^-}} & \textrm{ or } 0 \textrm{ if } \mathcal{N}_i^- \textrm{ is } \emptyset \\ \frac{X_i - \overline{X}_{\mathcal{N}_i^+}}{x_i - \overline{x}_{\mathcal{N}_i^+}} & \textrm{ or } 0 \textrm{ if } \mathcal{N}_i^+ \textrm{ is } \emptyset \\ \end{array} \right)\end{split}\]Also claims cache variable grad1 in a compatible way.
Parameters: - i (int) – cell index at which the gradient is evaluated.
- X (numpy.ndarray) – vector of a scalar measurement at every cell.
- index_map (numpy.ndarray) – index map that converts cell indices to indices in X.
Returns: delta vector with as many elements as there are spatial dimensions.
Return type: numpy.ndarray
-
tramway.inference.gradient.
setup_with_grad_arguments
(setup)¶ Add
grad()
related arguments to inference plugin setup.Input argument setup is modified inplace.
-
tramway.inference.gradient.
gradient_map
(cells, feature=None, **kwargs)¶
tramway.inference.optimization module¶
-
class
tramway.inference.optimization.
BFGSResult
(x, H, resolution, niter, f, df, projg, cumtime, err, diagnosis, ncalls)¶ Bases:
tuple
-
H
¶ Alias for field number 1
-
cumtime
¶ Alias for field number 7
-
df
¶ Alias for field number 5
-
diagnosis
¶ Alias for field number 9
-
err
¶ Alias for field number 8
-
f
¶ Alias for field number 4
-
ncalls
¶ Alias for field number 10
-
niter
¶ Alias for field number 3
-
projg
¶ Alias for field number 6
-
resolution
¶ Alias for field number 2
-
x
¶ Alias for field number 0
-
-
tramway.inference.optimization.
minimize_sparse_bfgs
(fun, x0, component, covariate, gradient_subspace, descent_subspace, args=(), bounds=None, _sum=<function sum>, gradient_sum=None, gradient_covariate=None, memory=10, eps=1e-06, ftol=1e-06, gtol=1e-10, low_df_rate=0.9, low_dg_rate=0.9, step_scale=1.0, max_iter=None, regul=None, regul_decay=1e-05, ls_kwargs={}, ls_regul=None, ls_step_max=None, ls_step_max_decay=None, ls_iter_max=None, ls_armijo_max=None, ls_wolfe=None, ls_failure_rate=0.9, fix_ls=None, fix_ls_trigger=5, gradient_initial_step=1e-08, Component=<class 'tramway.inference.optimization.Component'>, independent_components=False, newton=True, verbose=False, diagnosis=None, returns=(), max_runtime=None, update_timeout=None, **kwargs)¶ Let the objective function \(f(x) = \sum_{i \in C} f_{i}(x) \forall x \in \Theta\) be a linear function of sparse components \(f_{i}\) such that \(\forall j \notin G_{i}, \forall x \in \Theta, {\partial f_{i}}{\partial x_{j}}(x) = 0\).
Let the components also covary sparsely: \(\forall i \in C, \exists C_{i} \subset C | i \in C_{i}, \exists D_{i} \subset G_{i}, \forall j \in D_{i}, \forall x \in \Theta, \frac{\\partial f}{\partial x_{j}}(x) = \sum_{i' \in C_{i}} \frac{\partial f_{i'}}{\partial x_{j}}(x)\).
We may additionally need that \(\forall i \in C, D_{i} = \bigcap_{i' \in C_{i}} G_{i'}\), \(\bigcup_{i \in C} D_{i} = J\) and \(D_{i} \cap D_{j} = \emptyset \forall i, j \in C^{2}\) with \(J\) the indices of parameter vector \(x = \lvert x_{j}\rvert_{j \in J}\) (to be checked).
At iteration \(k\), let choose component \(i\) and minimize \(f\) wrt parameters \(\{x_{j} | j \in D_{i}\}\).
Compute gradient \(g_{i}(x) = \lvert\frac{\partial f}{\partial x_{j}}(x)\rvert_{j} = \sum_{i' \in C_{i}} \lvert\frac{\partial f_{i'}}{\partial x_{j}}(x) \rvert_{j}\) with (again) \(g_{i}(x)\rvert_{j} = 0 \forall j \notin G_{i}\).
Perform a Wolfe line search along descent direction restricted to subspace \(D_{i}\). Update \(x\) and compute the gradient again.
The inverse Hessian matrix must also be updated twice: before and after the update.
Parameters: - fun (callable) – takes a component index (int) and the parameter vector (numpy.ndarray) and returns a scalar float.
- x0 (numpy.ndarray) – initial parameter vector.
- component (callable) – takes an iteration index (int) and returns a component index (int); all the components are expected to be drawn exactly once per epoch; if component is an int, will be interpreted as the number of components.
- covariate (callable) – takes a component index i (int) and returns a sequence of indices of the components that covary with i, including i.
- gradient_subspace (callable) – takes a component index (int) and returns a sequence of indices of the parameters spanning the gradient subspace; if None, the gradient ‘subspace’ is the full space.
- descent_subspace (callable) – takes a component index (int) and returns a sequence of indices of the parameters spanning the descent subspace; if None, the descent subspace equals to the gradient subspace.
- args (tuple or list) – sequence of positional arguments to fun.
- bounds (tuple or list) – sequence of pairs of (lower, upper) bounds on the parameters.
- _sum (callable) – takes a list of float values and returns a float.
- gradient_sum (callable) – replacement for _sum in the calculation of the gradient.
- gradient_covariate (callable) – takes a parameter index (int) and returns a sequence of the components affected by this parameter.
- memory (int) – number of memorized pairs of H updates in quasi-Newton mode.
- eps (float) – initial scaling of the descent direction.
- ftol (float) – maximum decrease in the local objective.
- gtol (float) – maximum decrease in the projected gradient.
- low_df_rate (float) – epoch-wise rate of low decrease in local objective (below ftol); at the end of an epoch, if this rate has been reached, then the iteration stops.
- low_dg_rate (float) – epoch-wise rate of low decrease in the projected gradient (below gtol); at the end of an epoch, if this rate has been reached, then the iteration stops.
- step_scale (float) – reduction factor on the line-searched step.
- max_iter (int) – maximum number of iterations.
- regul (float) – regularization trade-off coefficient for the L2-norm of the parameter vector.
- regul_decay (float) – decay parameter for regul.
- ls_kwargs (dict) – keyword arguments to
wolfe_line_search()
. - ls_regul (float) – regularization trade-off coefficient for the L2-norm of the parameter vector update.
- ls_step_max (float) – maximum L-infinite norm of the parameter vector update; if ls_step_max_decay is defined, ls_step_max can be an (initial, final) tuple instead.
- ls_step_max_decay (float) – decay parameter for ls_step_max.
- ls_iter_max (int) – maximum number of linesearch iterations.
- ls_armijo_max (float) – maximum expected decrease in the local objective; throttles the Armijo threshold.
- ls_wolfe (tuple) – (c2, c3) pair for
wolfe_line_search()
. - ls_failure_rate (float) – epoch-wise rate of linesearch failure; at the end of an epoch, if this rate has been reached, then the iteration stops.
- fix_ls (callable) – takes a component index (int) and the parameter vector (numpy.ndarray) to be modified inplace in the case of recurrent linesearch failures for this component.
- fix_ls_trigger (int) – minimum number of iterations with successive linesearch failures on a given component for fix_ls to be triggered.
- Component (type) – component constructor.
- independent_components (bool) – whether to represent the local inverse Hessian submatrix for a component independently of the other components.
- newton (bool) – quasi-Newton (BFGS) mode; if False, use vanilla gradient descent instead.
- verbose (bool or logging.Logger) – verbose mode or logger.
- diagnosis (callable) – function of the iteration number and the current and candidate-new components.
- returns (sequence of str) – any subset of {‘f’, ‘df’, ‘projg’, ‘err’, ‘ncalls’, ‘diagnosis’} or ‘all’.
- xref (numpy.ndarray) – reference final parameter vector; if defined, at each iteration, the L2-norm of the difference between this and the current parameter vector is evaluated and returned as attribute err; note that this computation may add quite some overhead.
Returns: final parameter vector.
Return type: See also
SparseFunction
andwolfe_line_search()
.
-
tramway.inference.optimization.
minimize_sparse_bfgs0
(fun, x0, component, covariate, gradient_subspace, descent_subspace, args=(), bounds=None, _sum=<function sum>, gradient_sum=None, gradient_covariate=None, memory=10, eps=1e-06, ftol=1e-06, gtol=1e-10, low_df_rate=0.9, low_dg_rate=0.9, step_scale=1.0, max_iter=None, regul=None, regul_decay=1e-05, ls_regul=None, ls_step_max=None, ls_step_max_decay=None, ls_iter_max=None, ls_armijo_max=None, ls_wolfe=None, ls_failure_rate=0.9, fix_ls=None, fix_ls_trigger=5, gradient_initial_step=1e-08, independent_components=True, newton=True, verbose=False)¶ Let the objective function \(f(x) = \sum_{i \in C} f_{i}(x) \forall x \in \Theta\) be a linear function of sparse components \(f_{i}\) such that \(\forall j \notin G_{i}, \forall x \in \Theta, {\partial f_{i}}{\partial x_{j}}(x) = 0\).
Let the components also covary sparsely: \(\forall i \in C, \exists C_{i} \subset C | i \in C_{i}, \exists D_{i} \subset G_{i}, \forall j \in D_{i}, \forall x \in \Theta, \frac{\\partial f}{\partial x_{j}}(x) = \sum_{i' \in C_{i}} \frac{\partial f_{i'}}{\partial x_{j}}(x)\).
We may additionally need that \(\forall i \in C, D_{i} = \bigcap_{i' \in C_{i}} G_{i'}\), \(\bigcup_{i \in C} D_{i} = J\) and \(D_{i} \cap D_{j} = \emptyset \\forall i, j \in C^{2}\) with \(J\) the indices of parameter vector \(x = \lvert x_{j}\rvert_{j \in J}\) (to be checked).
At iteration \(k\), let choose component \(i\) and minimize \(f\) wrt parameters \(\{x_{j} | j \in D_{i}\}\).
Compute gradient \(g_{i}(x) = \lvert\frac{\partial f}{\partial x_{j}}(x)\rvert_{j} = \sum_{i' \in C_{i}} \lvert\frac{\partial f_{i'}}{\partial x_{j}}(x) \rvert_{j}\) with (again) \(g_{i}(x)\rvert_{j} = 0 \forall j \notin G_{i}\).
Perform a Wolfe line search along descent direction restricted to subspace \(D_{i}\). Update \(x\) and compute the gradient again.
The inverse Hessian matrix must also be updated twice: before and after the update.
Parameters: - fun (callable) – takes a component index (int) and the parameter vector (numpy.ndarray) and returns a scalar float.
- x0 (numpy.ndarray) – initial parameter vector.
- component (callable) – takes an iteration index (int) and returns a component index (int); all the components are expected to be drawn exactly once per epoch; if component is an int, will be interpreted as the number of components.
- covariate (callable) – takes a component index i (int) and returns a sequence of indices of the components that covary with i, including i.
- gradient_subspace (callable) – takes a component index (int) and returns a sequence of indices of the parameters spanning the gradient subspace; if None, the gradient ‘subspace’ is the full space.
- descent_subspace (callable) – takes a component index (int) and returns a sequence of indices of the parameters spanning the descent subspace; if None, the descent subspace equals to the gradient subspace.
- args (tuple or list) – sequence of positional arguments to fun.
- bounds (tuple or list) – sequence of pairs of (lower, upper) bounds on the parameters.
- _sum (callable) – takes a list of float values and returns a float.
- gradient_sum (callable) – replacement for _sum in the calculation of the gradient.
- gradient_covariate (callable) – takes a parameter index (int) and returns a sequence of the components affected by this parameter.
- memory (int) – number of memorized pairs of H updates in quasi-Newton mode.
- eps (float) – initial scaling of the descent direction.
- ftol (float) – maximum decrease in the local objective.
- gtol (float) – maximum decrease in the projected gradient.
- low_df_rate (float) – epoch-wise rate of low decrease in local objective (below ftol); at the end of an epoch, if this rate has been reached, then the iteration stops.
- low_dg_rate (float) – epoch-wise rate of low decrease in the projected gradient (below gtol); at the end of an epoch, if this rate has been reached, then the iteration stops.
- step_scale (float) – reduction factor on the line-searched step.
- max_iter (int) – maximum number of iterations.
- regul (float) – regularization trade-off coefficient for the L2-norm of the parameter vector.
- regul_decay (float) – decay parameter for regul.
- ls_regul (float) – regularization trade-off coefficient for the L2-norm of the parameter vector update.
- ls_step_max (float) – maximum L-infinite norm of the parameter vector update; if ls_step_max_decay is defined, ls_step_max can be an (initial, final) tuple instead.
- ls_step_max_decay (float) – decay parameter for ls_step_max.
- ls_iter_max (int) – maximum number of linesearch iterations.
- ls_armijo_max (float) – maximum expected decrease in the local objective; throttles the Armijo threshold.
- ls_wolfe (tuple) – (c2, c3) pair for
wolfe_line_search()
. - ls_failure_rate (float) – epoch-wise rate of linesearch failure; at the end of an epoch, if this rate has been reached, then the iteration stops.
- fix_ls (callable) – takes a component index (int) and the parameter vector (numpy.ndarray) to be modified inplace in the case of recurrent linesearch failures for this component.
- fix_ls_trigger (int) – minimum number of iterations with successive linesearch failures on a given component for fix_ls to be triggered.
- independent_components (bool) – whether to represent the local inverse Hessian submatrix for a component independently of the other components.
- newton (bool) – quasi-Newton (BFGS) mode; if False, use vanilla gradient descent instead.
- verbose (bool or logging.Logger) – verbose mode or logger.
Returns: final parameter vector.
Return type: See also
SparseFunction
andwolfe_line_search()
.
-
tramway.inference.optimization.
minimize_sparse_bfgs1
(fun, x0, component, covariate, gradient_subspace, descent_subspace, args=(), bounds=None, _sum=<function sum>, gradient_sum=None, gradient_covariate=None, memory=10, eps=1e-06, ftol=1e-06, gtol=1e-10, low_df_rate=0.9, low_dg_rate=0.9, step_scale=1.0, max_iter=None, regul=None, regul_decay=1e-05, ls_kwargs={}, ls_regul=None, ls_step_max=None, ls_step_max_decay=None, ls_iter_max=None, ls_armijo_max=None, ls_wolfe=None, ls_failure_rate=0.9, fix_ls=None, fix_ls_trigger=5, gradient_initial_step=1e-08, Component=<class 'tramway.inference.optimization.Component'>, independent_components=False, newton=True, verbose=False, diagnosis=None, returns=(), max_runtime=None, update_timeout=None, **kwargs)¶ Let the objective function \(f(x) = \sum_{i \in C} f_{i}(x) \forall x \in \Theta\) be a linear function of sparse components \(f_{i}\) such that \(\forall j \notin G_{i}, \forall x \in \Theta, {\partial f_{i}}{\partial x_{j}}(x) = 0\).
Let the components also covary sparsely: \(\forall i \in C, \exists C_{i} \subset C | i \in C_{i}, \exists D_{i} \subset G_{i}, \forall j \in D_{i}, \forall x \in \Theta, \frac{\\partial f}{\partial x_{j}}(x) = \sum_{i' \in C_{i}} \frac{\partial f_{i'}}{\partial x_{j}}(x)\).
We may additionally need that \(\forall i \in C, D_{i} = \bigcap_{i' \in C_{i}} G_{i'}\), \(\bigcup_{i \in C} D_{i} = J\) and \(D_{i} \cap D_{j} = \emptyset \forall i, j \in C^{2}\) with \(J\) the indices of parameter vector \(x = \lvert x_{j}\rvert_{j \in J}\) (to be checked).
At iteration \(k\), let choose component \(i\) and minimize \(f\) wrt parameters \(\{x_{j} | j \in D_{i}\}\).
Compute gradient \(g_{i}(x) = \lvert\frac{\partial f}{\partial x_{j}}(x)\rvert_{j} = \sum_{i' \in C_{i}} \lvert\frac{\partial f_{i'}}{\partial x_{j}}(x) \rvert_{j}\) with (again) \(g_{i}(x)\rvert_{j} = 0 \forall j \notin G_{i}\).
Perform a Wolfe line search along descent direction restricted to subspace \(D_{i}\). Update \(x\) and compute the gradient again.
The inverse Hessian matrix must also be updated twice: before and after the update.
Parameters: - fun (callable) – takes a component index (int) and the parameter vector (numpy.ndarray) and returns a scalar float.
- x0 (numpy.ndarray) – initial parameter vector.
- component (callable) – takes an iteration index (int) and returns a component index (int); all the components are expected to be drawn exactly once per epoch; if component is an int, will be interpreted as the number of components.
- covariate (callable) – takes a component index i (int) and returns a sequence of indices of the components that covary with i, including i.
- gradient_subspace (callable) – takes a component index (int) and returns a sequence of indices of the parameters spanning the gradient subspace; if None, the gradient ‘subspace’ is the full space.
- descent_subspace (callable) – takes a component index (int) and returns a sequence of indices of the parameters spanning the descent subspace; if None, the descent subspace equals to the gradient subspace.
- args (tuple or list) – sequence of positional arguments to fun.
- bounds (tuple or list) – sequence of pairs of (lower, upper) bounds on the parameters.
- _sum (callable) – takes a list of float values and returns a float.
- gradient_sum (callable) – replacement for _sum in the calculation of the gradient.
- gradient_covariate (callable) – takes a parameter index (int) and returns a sequence of the components affected by this parameter.
- memory (int) – number of memorized pairs of H updates in quasi-Newton mode.
- eps (float) – initial scaling of the descent direction.
- ftol (float) – maximum decrease in the local objective.
- gtol (float) – maximum decrease in the projected gradient.
- low_df_rate (float) – epoch-wise rate of low decrease in local objective (below ftol); at the end of an epoch, if this rate has been reached, then the iteration stops.
- low_dg_rate (float) – epoch-wise rate of low decrease in the projected gradient (below gtol); at the end of an epoch, if this rate has been reached, then the iteration stops.
- step_scale (float) – reduction factor on the line-searched step.
- max_iter (int) – maximum number of iterations.
- regul (float) – regularization trade-off coefficient for the L2-norm of the parameter vector.
- regul_decay (float) – decay parameter for regul.
- ls_kwargs (dict) – keyword arguments to
wolfe_line_search()
. - ls_regul (float) – regularization trade-off coefficient for the L2-norm of the parameter vector update.
- ls_step_max (float) – maximum L-infinite norm of the parameter vector update; if ls_step_max_decay is defined, ls_step_max can be an (initial, final) tuple instead.
- ls_step_max_decay (float) – decay parameter for ls_step_max.
- ls_iter_max (int) – maximum number of linesearch iterations.
- ls_armijo_max (float) – maximum expected decrease in the local objective; throttles the Armijo threshold.
- ls_wolfe (tuple) – (c2, c3) pair for
wolfe_line_search()
. - ls_failure_rate (float) – epoch-wise rate of linesearch failure; at the end of an epoch, if this rate has been reached, then the iteration stops.
- fix_ls (callable) – takes a component index (int) and the parameter vector (numpy.ndarray) to be modified inplace in the case of recurrent linesearch failures for this component.
- fix_ls_trigger (int) – minimum number of iterations with successive linesearch failures on a given component for fix_ls to be triggered.
- Component (type) – component constructor.
- independent_components (bool) – whether to represent the local inverse Hessian submatrix for a component independently of the other components.
- newton (bool) – quasi-Newton (BFGS) mode; if False, use vanilla gradient descent instead.
- verbose (bool or logging.Logger) – verbose mode or logger.
- diagnosis (callable) – function of the iteration number and the current and candidate-new components.
- returns (sequence of str) – any subset of {‘f’, ‘df’, ‘projg’, ‘err’, ‘ncalls’, ‘diagnosis’} or ‘all’.
- xref (numpy.ndarray) – reference final parameter vector; if defined, at each iteration, the L2-norm of the difference between this and the current parameter vector is evaluated and returned as attribute err; note that this computation may add quite some overhead.
Returns: final parameter vector.
Return type: See also
SparseFunction
andwolfe_line_search()
.
-
class
tramway.inference.optimization.
SparseFunction
(x, covariate, gradient_subspace, descent_subspace, eps, fun, _sum, args, regul, bounds, h0)¶ Bases:
tramway.core.parallel.Workspace
Parameter singleton.
-
x
¶ working copy of the parameter vector.
Type: numpy.ndarray
-
covariate
¶ returns the components that covary with the input component (all indices).
Type: callable
-
gradient_subspace
¶ takes a component index and returns the indices of the related parameters.
Type: callable
-
descent_subspace
¶ takes a component index and returns the indices of the related parameters.
Type: callable
-
eps
¶ initial scaling of the descent direction when no curvature information is available.
Type: float
-
fun
¶ local cost function (takes a component index before the parameter vector).
Type: callable
-
_sum
¶ mixture function for local costs.
Type: callable
-
args
¶ positional input arguments to fun.
Type: sequence
-
regul
¶ regularization coefficient on the parameters.
Type: float
-
bounds
¶ (lower, upper) bounds as a couple of lists.
Type: tuple
-
h0
¶ gradient initial step.
Type: float
-
ncalls
¶ number of calls to fun.
Type: int
See also
minimize_sparse_bfgs()
.-
fun
(*args, **kwargs)
-
update
(component)¶
-
x
-
-
tramway.inference.optimization.
wolfe_line_search
(f, x, p, g, subspace=None, args_f=(), args_g=None, args=None, f0=None, g0=None, bounds=None, weight_regul=None, step_regul=None, eta_max=1.0, iter_max=10, step_max=None, c0=0.5, c1=0.0001, c2=0.9, c3=0.9, c4=0.1, c5=1e-10, armijo_max=None, return_resolution=False)¶ Wolfe line search along direction p possibly restricted to a subspace.
If defined, argument subspace is passed to gradient function g as last positional argument, unless already present in args_g.
args is an alias for args_f. If args_g is
None
, then it falls back to args_f.f(x_k + eta_k p_k) <= f(x_k) + c1 eta_k p_k^T g(x_k) # Armijo rule c2 * p_k^T g(x_k) <= -p_k^T g(x_k + eta_k p_k) <= -c3 p_k^T g(x_k) # weak (c2=None) or strong (c2=c3) Wolfe condition c4 eta_k <= eta_{k+1} <= c0 eta_k eta_min = c5 eta_max
-
tramway.inference.optimization.
sparse_grad
(fun, x, active_i, active_j, args=(), _sum=<function sum>, regul=None, bounds=None, h0=1e-08)¶ Compute the derivative of a function.
tramway.inference.stochastic_dv plugin¶
-
class
tramway.inference.stochastic_dv.
LocalDV
(diffusivity, potential, diffusivity_spatial_prior=None, potential_spatial_prior=None, diffusivity_time_prior=None, potential_time_prior=None, minimum_diffusivity=None, positive_diffusivity=None, prior_include=None, regions=None, prior_delay=None, logger=True)¶ Bases:
tramway.inference.dv.DV
-
diffusivity_indices
(cell_ids)¶
-
diffusivity_prior
(i)¶
-
diffusivity_spatial_prior
(i)¶
-
diffusivity_time_prior
(i)¶
-
indices
(cell_ids)¶
-
logger
¶
-
pop_workspace_update
()¶
-
potential_indices
(cell_ids)¶
-
potential_prior
(i)¶
-
potential_spatial_prior
(i)¶
-
potential_time_prior
(i)¶
-
prior_delay
¶
-
push_workspace_update
(update)¶
-
region
(i)¶
-
regions
¶
-
undefined_grad
(i, feature='')¶
-
undefined_time_derivative
(i, feature='')¶
-
verbose
¶
-
-
tramway.inference.stochastic_dv.
infer_stochastic_DV
(cells, diffusivity_spatial_prior=None, potential_spatial_prior=None, diffusivity_time_prior=None, potential_time_prior=None, jeffreys_prior=False, min_diffusivity=None, max_iter=None, compatibility=False, export_centers=False, verbose=True, superlocal=True, stochastic=True, D0=None, V0=None, x0=None, rgrad=None, debug=False, fulltime=False, diffusion_prior=None, diffusion_spatial_prior=None, diffusion_time_prior=None, prior_delay=None, return_struct=False, posterior_max_count=None, diffusivity_prior=None, potential_prior=None, time_prior=None, allow_negative_potential=False, **kwargs)¶ Parameters: - .. –
- time_prior (float or tuple) – one of two regularization coefficients that penalize the temporal derivative of diffusivity and potential energy; the first coefficient applies to diffusivity (and is multiplied by diffusivity_prior) and the second applies to potential energy (and is multiplied by potential_prior).
- diffusivity_spatial_prior/diffusion_spatial_prior – alias for diffusivity_prior.
- diffusivity_time_prior/diffusion_time_prior – penalizes the temporal derivative of diffusivity; this coefficient does NOT multiply with diffusivity_prior.
- potential_spatial_prior – alias for potential_prior.
- potential_time_prior – penalizes the temporal derivative of potential; this coefficient does NOT multiply with potential_prior.
- stochastic (bool) – if
False
, emulate standard DV with sparse gradient. - D0 (float or ndarray) – initial diffusivity value(s).
- V0 (float or ndarray) – initial potential energy value(s).
- x0 (ndarray) – initial parameter vector [diffusivities, potentials].
- rgrad (str) – either ‘grad’/’grad1’, ‘gradn’ (none recommended), ‘delta’/’delta0’
or ‘delta1’;
see the corresponding functions in module
gradient
. - debug (bool or sequence of str) – any subset of {‘ncalls’, ‘f_history’, ‘df_history’, ‘projg_history’, ‘error’, ‘diagnoses’}.
- xref (ndarray) – reference parameter vector [diffusivities, potentials]; required to compute the ‘error’ debug variable.
- diagnosis (callable) – function of the iteration number, current component and candidate updated component; required to compute the ‘diagnoses’ debug variable.
- prior_delay/return_struct/posterior_max_count – all deprecated.
- allow_negative_potential (bool) – do not offset the whole potential map towards all-positive values.
- .. –
See also
minimize_sparse_bfgs()
.
-
tramway.inference.stochastic_dv.
local_dv_neg_posterior
(j, x, dv, cells, sigma2, jeffreys_prior, dt_mean, index, reverse_index, grad_kwargs, posterior_info=None, iter_num=None, verbose=False)¶
-
tramway.inference.stochastic_dv.
lookup_space_cells
(cells)¶
-
tramway.inference.stochastic_dv.
make_regions
(cells, index, reverse_index, size=1)¶
tramway.inference.bayes_factors subpackage¶
-
tramway.inference.bayes_factors.
calculate_bayes_factors
(zeta_ts, zeta_sps, ns, Vs, Vs_pi, loc_error, dim=2, B_threshold=10, verbose=True)¶ Calculate the Bayes factor for a set of bins given a uniform localization error.
Input: zeta_ts — signal-to-noise ratios for the total force = dx_mean / sqrt(var(dx)) in bins. Size: M x 2, zeta_sps — signal-to-noise ratios for the spurious force = grad(D) / sqrt(var(dx)) in bins. Size: M x 2, ns — number of jumps in each bin. Size: M x 1, Vs — jump variance in bins = E((dx - dx_mean) ** 2). Size: M x 1, Vs_pi — jump variance in all other bins relative to the current bin. Size: M x 1, loc_error — localization error. Same units as variance. Set to 0 if localization error can be ignored; dim — dimensionality of the problem; B_threshold — the values of Bayes factor for thresholding.
Output: Bs, forces, min_ns
lg_Bs — log_10 Bayes factor values in the bins. Size: M x 1, forces — Returns 1 if there is strong evidence for the presence of a conservative forces, -1 for strong evidence for a spurious force, and 0 if the is not enough evidence. Size: M x 1; min_ns — minimum number of data points in a bin to support the currently favored model at the required evidence level. Size: M x 1.
Notation: M — number of bins.
-
tramway.inference.bayes_factors.
calculate_bayes_factors_for_one_cell
(cell, loc_error, dim=2, B_threshold=10, verbose=True)¶ Calculate Bayes factors for one cell. The results are saved to cell properties.
tramway.inference.bayes_factors.calculate_bayes_factors module¶
-
exception
tramway.inference.bayes_factors.calculate_bayes_factors.
NaNInputError
¶ Bases:
ValueError
-
tramway.inference.bayes_factors.calculate_bayes_factors.
calculate_bayes_factors
(zeta_ts, zeta_sps, ns, Vs, Vs_pi, loc_error, dim=2, B_threshold=10, verbose=True)¶ Calculate the Bayes factor for a set of bins given a uniform localization error.
Input: zeta_ts — signal-to-noise ratios for the total force = dx_mean / sqrt(var(dx)) in bins. Size: M x 2, zeta_sps — signal-to-noise ratios for the spurious force = grad(D) / sqrt(var(dx)) in bins. Size: M x 2, ns — number of jumps in each bin. Size: M x 1, Vs — jump variance in bins = E((dx - dx_mean) ** 2). Size: M x 1, Vs_pi — jump variance in all other bins relative to the current bin. Size: M x 1, loc_error — localization error. Same units as variance. Set to 0 if localization error can be ignored; dim — dimensionality of the problem; B_threshold — the values of Bayes factor for thresholding.
Output: Bs, forces, min_ns
lg_Bs — log_10 Bayes factor values in the bins. Size: M x 1, forces — Returns 1 if there is strong evidence for the presence of a conservative forces, -1 for strong evidence for a spurious force, and 0 if the is not enough evidence. Size: M x 1; min_ns — minimum number of data points in a bin to support the currently favored model at the required evidence level. Size: M x 1.
Notation: M — number of bins.
-
tramway.inference.bayes_factors.calculate_bayes_factors.
calculate_bayes_factors_for_one_cell
(cell, loc_error, dim=2, B_threshold=10, verbose=True)¶ Calculate Bayes factors for one cell. The results are saved to cell properties.
-
tramway.inference.bayes_factors.calculate_bayes_factors.
calculate_minimal_n
(zeta_t, zeta_sp, n0, V, V_pi, loc_error, dim=2, B_threshold=10)¶ Calculate the minimal number of jumps per bin needed to obtain strong evidence for the active force or the spurious force model.
Input: zeta_t, zeta_sp - – vectors of length 2 for one bin n0 - – initial number of jumps(evidence already available). This way the “next” strong evidence can be found, i.e. the minimal number of data points to support the current conclusion
Output: min_n - – minimal number of jumps to obtain strong evidence for the conservative force model. Return - 1 if unable to find the min_n
-
tramway.inference.bayes_factors.calculate_bayes_factors.
check_dimensionality
(dim)¶
-
tramway.inference.bayes_factors.calculate_bayes_factors.
check_for_nan
(*args)¶ Return ‘nan’ if a nan value is present in any of the variables. Return ‘None’ if any of the variables is None. Return ‘ok’ otherwise
tramway.inference.bayes_factors.calculate_marginalized_integral module¶
-
tramway.inference.bayes_factors.calculate_marginalized_integral.
calculate_any_lambda_integral
(func, break_points=[])¶ DEPRECATED: calculate ratios instead. This function allows to calculate more complicated lambda integrals, such as 1D posteriors in 2D.
Input: func(lambda) - function of lambda to integrate, break_points - special integration points (i.e. zeta_t/zeta_sp), optional.
-
tramway.inference.bayes_factors.calculate_marginalized_integral.
calculate_integral_ratio
(arg_func_up, arg_func_down, pow_up, pow_down, v, rel_loc_error, break_points=[], lamb='marg', rtol=1e-06, atol=1e-32)¶ Calculate the ratio of two similar lambda integrals, each of which has the form int_0^1 dlambda arg_func(lambda)**(-pow) * gammainc(pow, rel_loc_error * arg_func(lambda))
Input: v — zeta-independent term inside the argument functions. Must be the same upstairs and downstairs. In Bayes factor calculations, this is v := (1 + n_pi * V_pi / n / V).
Return: Natural logarithm of the integral ratio
Details: Calculations are performed differently for the case with and without localization error. This avoids overflow errors. In these cases, both the integrated equations and prefactors are different. The integrals are taken over lambda if lamb == ‘marg’, otherwise they are evaluated at the given lambda.
-
tramway.inference.bayes_factors.calculate_marginalized_integral.
calculate_marginalized_integral
(zeta_t, zeta_sp, p, v, E, rel_loc_error, zeta_a=[0, 0], factor_za=0, lamb='int')¶ Calculate the marginalized lambda integral >>> Integrate[gamma_inc[p, arg * rel_loc_error] * arg ** (-p), {lambda, 0, 1}] >>> for the given values of zeta_t and zeta_sp.
Here: arg = (v + factor_za * zeta_a**2 + E * (zeta_t - zeta_a - lambda * zeta_sp)**2); IMPORTANT: gamma_inc — is the normalized lower incomplete gamma function; rel_loc_error = n * V / (4 * sigma_L^2) — inverse relative localization error,
Input: zeta_t and zeta_sp are vectors of length dim (x, y, z). All other parameters are scalars. abs_tol — absolute tolerance for integral calculations. lamb = float in [0, 1] or ‘int’. If a float is given, the integrated function is just evaluated at the given value of lambda, without integrating.
-
tramway.inference.bayes_factors.calculate_marginalized_integral.
sum_series
(term_func, rtol, *args)¶ Sum any series with the first argument of the term_func being the term number. Input: function, return - float
tramway.inference.bayes_factors.calculate_posteriors module¶
The file contains posterior calculation functions.
-
tramway.inference.bayes_factors.calculate_posteriors.
calculate_one_1D_posterior_in_2D
(zeta_a, zeta_t, zeta_sp, n, V, V_pi, loc_error, lamb='marg', axis='x')¶ Calculate the posterior for a projection of zeta_a for a given inference convention for one bin in 2D
Input: lambda: float in [0, 1] or ‘marg’; for different conventions axis: [‘x’, ‘y’]; for which projection to calculate the posterior zeta_a: float; a projection on the selected axis
-
tramway.inference.bayes_factors.calculate_posteriors.
calculate_one_1D_prior_in_2D
(zeta_a, V_pi, sigma2)¶ Calculate the prior for a projection of zeta_a for any convention for one bin in 2D
Input: zeta_a: float; a projection on the selected axis
-
tramway.inference.bayes_factors.calculate_posteriors.
calculate_one_2D_posterior
(zeta_t, zeta_sp, zeta_a, n, V, V_pi, loc_error, dim=2, lamb='marg')¶ Calculate the posterior for zeta_a for a given method for one bin in 2D
Input: lambda — float in [0, 1] or ‘marg’ — for different conventions
-
tramway.inference.bayes_factors.calculate_posteriors.
get_lambda_MAP
(zeta_t, zeta_sp)¶ Calculate the maximum a posteriori value of lambda for given zetas in 1D and 2D
tramway.inference.bayes_factors.convenience_functions module¶
Small functions used throughout the package
-
tramway.inference.bayes_factors.convenience_functions.
n_pi_func
(dim)¶
-
tramway.inference.bayes_factors.convenience_functions.
p
(n, dim)¶
tramway.inference.bayes_factors.find_marginalized_zeta_t_roots module¶
-
tramway.inference.bayes_factors.find_marginalized_zeta_t_roots.
find_marginalized_zeta_t_roots
(zeta_sp_par, n, n_pi, B, u, dim, zeta_t_perp, sigma2)¶ Find marginalized roots zeta_t. I have proven that the min B for the marginalized inference is achieved at zeta_t = zeta_sp/2 (for the parallel component and under the condition that the orthogonal component is 0). sigma2 — localization error. Same units as variance. Set to 0 if localization error can be ignored;
tramway.inference.bayes_factors.get_D_posterior module¶
Functions allowing to calculate the D posterior and MAP(D) as (will be) described in the Ito-Stratonovich article. Provides 3 functions: - get_D_posterior - get_MAP_D - get_D_confidence_interval
If unsure, use get_D_confidence_interval, which provides MAP(D) and a confidence interval for D for the given confidence level. If need the D posterior, use get_D_posterior.
-
tramway.inference.bayes_factors.get_D_posterior.
get_D_confidence_interval
(alpha, n, zeta_t, V, V_pi, dt, sigma2, dim)¶ Returns MAP_D and a confidence interval corresponding to confidence level alpha, i.e. the values of D giving the values [(1-alpha)/2, 1-(1-alpha)/2] for the D posterior integrals
Input: alpha - confidence level, n - number of jumps in bin, zeta_t - signal-to-noise ratio for the total force: <dr>/sqrt(V), V - (biased) variance of jumps in the current bin, V_pi - (biased) variance of jumps in all other bins excluding the current one, dt - time step sigma2 - localization error (in the units of variance), dim - dimensionality of the problem
Output: MAP_D - MAP value of the diffusivity, CI - a confidence interval for the diffusivity, 2x1 np.array
-
tramway.inference.bayes_factors.get_D_posterior.
get_D_posterior
(n, zeta_t, V, V_pi, dt, sigma2, dim, _MAP_D=False)¶ Return the diffusivity posterior as a function. If _MAP_D flag is supplied, returns instead a single MAP(D) value
Input: V - (biased) variance of jumps in the current bin V_pi - (biased) variance of jumps in all other bins excluding the current one dt - time step sigma2 - localization error (in the units of variance) dim - dimensionality of the problem _MAP_D - if True, returns MAP_D instead of the posterior
Output: posterior - function object accepting D as the only argument
-
tramway.inference.bayes_factors.get_D_posterior.
get_MAP_D
(n, zeta_t, V, V_pi, dt, sigma2, dim)¶