SLiM: Structured Linear Maps

This package provides a suite of structured linear maps which can be used as drop-in replacements for PyTorch’s nn.Linear module. Recent work viewing neural networks from a dynamical systems perspective has introduced a host of parametrizations for the basic linear maps which are subcomponents of neural networks. Such parametrizations may enhance the stability of learning, and embed models with inductive priors that encode domain application knowledge. This package is an effort to collect these all in one place with a common API to facilitate rapid exploration.

Note

We encourage folks to contribute any new structured linear maps as feature requests.

Linear Map	class
Orthogonal	OrthogonalLinear
L0	L0Linear
Butterfly	ButterflyLinear
Spectral	SpectralLinear
Schur Decomposition	SchurDecompositionLinear
Lasso	LassoLinear
Symplectic	SymplecticLinear
AntiSymmetric	SkewSymmetricLinear

Benchmarks

Sequence to Classification:

• MNIST

• XOR

Point to Classification:

• Concentric Ellipses

• Swiss Roll

• Peaks

Sequence to Regression:

• Add

• Multiply

Point to Regression:

• Ackley

Dynamics:

• Building models

Docs:

• Linear
• RNN
• Benchmarks