algorithms package

Submodules

algorithms.homology module

algorithms.homology.graph_from_tuples(tuples, directed=True, weight_func=<function <lambda>>)[source]

Returns a weighted graph constructed from a set of tuples

Parameters

tuples (iterable) – An iterable of tuples of the form (vertex1, vertex2, weight)
weight_func (function) – A function that takes the weight from a tuple as its input and returns the weight of the edge between vertex1 and vertex2 in the graph, default= f(x)=x
directed (bool) –

Returns

G – A weighted graph with edge weights given by the weight_func of the tuple weights

Return type

networkx.Graph

algorithms.homology.mat_from_graph(G, kernel=<function <lambda>>)[source]

Given a weighted graph create a matrix based on a kernel function of the shortest weighted path distance.

Parameters

G (networkx.Graph) – A weighted graph
kernel (function) – A function of the shortest weighted path distance, default= f(x)=x

Returns

matrix

Return type

numpy.ndarray

Note

This will only be a distance matrix if the kernel function keeps things as distances.

algorithms.homology.ripser_of_distmat(dist_matrix, maxdim=1)[source]

Given a distance matrix compute the persistent homology using ripser.

Parameters

dist_matrix (numpy.ndarray) – A distance matrix
maxdim (int, optional, default=1) – The maximum homology dimension to compute, default=1

Returns

R – A dictionary holding the results of the computation

Return type

dict

Note

The given dist_mat should be a distance matrix, but this is not strictly enforced

algorithms.homology.ripser_of_graph(G, kernel=<function <lambda>>, maxdim=1)[source]

Given a weighted graph compute the persistent homology using ripser.

Parameters

G (networkx.Graph) – A weighted graph
kernel (function) – A function of the shortest weighted path distance, default= f(x)=x
maxdim (int) – The maximum homology dimension to compute, default=1

Returns

R – A dictionary holding the results of the computation

Return type

dict

Note

The given kernel function should produce a distance matrix, but this is not strictly enforced

algorithms.homology.tuples_from_dict(d, layers=None)[source]

Returns a list of tuples representing synapses up to a specified layer

Parameters

d (dict) – A dictionary of synapse weights, keyed by layer and synapse
layers (list, optional, default=None) – If None (default), dictionary entries for all layers will be included in the list of tuples, otherwise, entries from the layers given in the list will be included

Returns

tuples – A list of tuples of the form ((layer1, (neuron1,)), (layer2, (neuron2,)), weight)

Return type

list

Note

Neurons are renamed to be (layer, neuron) since different layers may have the same neuron indices. The parameter nlayers is an inclusive bound.

algorithms.jaccard module

algorithms.jaccard.avg_jaccard(profile1, profile2, neuron=True, layers=None)[source]

Computes the jaccard similarity at each layer using synapse sets (or neuron sets) then averages the values.

Parameters

profile1 (Profile) –
profile2 (Profile) –
neuron (bool, optional, default=False) – Set to true if wish to compute the iou on the neuron sets instead of the synapse sets
layers (list, optional, deafult=None) – Specify a list of layers to calculate similarity over, defaults to all layers of the profile

Returns

Mean Intersection-over-Union (IOU) across layers of synapse (neuron) sets in Profile object.

Return type

float

algorithms.spectral_analysis module

class algorithms.spectral_analysis.SpectralAnalysis(model)[source]

Bases: object

Spectral Analysis is based on methods originating from Random Matrix theory, brought to deep neural networks by Martin and Mahoney. Traditional and Heavy-Tailed Self Regularization in Neural Network Models by Martin and Mahoney Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data by Martin, Peng,and Mahoney

These methods act only on the weights of the Fully Connected and Convolutional layers a deep neural network. Despite this, they have proven effective in predicting 1. Test accuracies with no access to the data distribution on which it was trained OR tested 2. Relative performance between models of similar architecture classes 3. Model and architecture improvements while training

The major improvement we make over the above work is our handling of convolutional layers: our methods are more principled, and over an order of magnitude faster than the code released by the authors in https://github.com/CalculatedContent/WeightWatcher.

Parameters: model (Module) –

implemented_classes

List of classes in PyTorch we can examine, i.e. have implemented spectral distributions

Type: set

model

model to be spectral-analyzed

Type: torch.nn.Module()

fit_power_law(eig_dict=None, plot_alpha=False, plot_eig=False)[source]

Fits the eigenvalue spectrum distribution of the layer weights \(X = W W^T\) with a power-law distribution. Uses the MLE approach from https://arxiv.org/abs/0706.1062.

Parameters

eigdict (Dict[int, Tuple[np.array, float]]) – Optional, useful if pre-computed with .spectral_analysisr() Dictionary with keys of the nth layer proviled, values of (eigenvalues, Q), where eigenvalues are those of the weight matrix for the layer, and Q is the aspect ratio of the matrix.
plot_alpha (bool) – Plot per-layer power-law fit of the eigenvalue spectrum distribution.
plot_eig (bool) – Plot per-layer eigenvalue spectrum distribution

Returns

alpha_dict – Dictionary with keys of the nth layer proviled, values of (alpha, eig_max), where alpha is the power law fit alpha, i.e: :math: rho(lambda) sim lambda^{-alpha}. eig_max is the max eigenvalue.

Return type

Dict[int, Tuple[float, float]]

layer_RMT(alpha_dict=None, verbose=False, plot_alpha=False, plot_eig=False)[source]

Prints the random matrix theory phenomenology of the layer eigenspectrum distribution from \(X = W W^T\). From https://arxiv.org/abs/1901.08276

Parameters

alpha_dict (Dict[int, Tuple[float, float]]) – Optional, useful if pre-computed with .spectral_analysisr() Dictionary with keys of the nth layer proviled, values of (alpha, eig_max)
plot_alpha (bool) – Plot per-layer power-law fit of the eigenvalue spectrum distribution.
plot_eig (bool) – Plot per-layer eigenvalue spectrum distribution
verbose (bool) –

Return type

Tuple[List[str], Dict]

spectral_analysis(plot=False)[source]

Returns a dictionary keyed by the order of the linear and convolutional layers, with the eigenvalues of \(X = W W^T\). Optional plot of the spectrum.

Parameters: plot (bool) – Plot per-layer empirical spectral distribution.
Returns: eigdict – Dictionary with keys of the nth layer proviled, values of (eigenvalues, Q), where eigenvalues are those of the weight matrix for the layer, and Q is the aspect ratio of the matrix.
Return type: Dict[int, Tuple[float, float]]

universal_metric(alpha_dict=None)[source]

Returns the universal capacity metric \(\widehat{\alpha}=\frac{1}{L} \sum_{l} \alpha_{l} \log \lambda_{\max , l}\) from https://arxiv.org/abs/2002.06716

Parameters: alpha_dict (Dict[int, Tuple[float, float]]) – Optional, useful if pre-computed with .spectral_analysisr() Dictionary with keys of the nth layer proviled, values of (alpha, eig_max)
Returns: metric – Universal capacity metric. A useful engineering metric for average case capacity in DNNs, from https://arxiv.org/abs/1901.08278
Return type: float

algorithms package

Submodules

algorithms.homology module

algorithms.jaccard module

algorithms.spectral_analysis module

Module contents