neuromancer.slim.butterfly.butterfly_multiply module

class neuromancer.slim.butterfly.butterfly_multiply.ButterflyFactorMult(*args, **kwargs)

Bases: Function

static backward(ctx, grad)

Parameters:

grad – (batch_size, 2, n) if real or (batch_size, 2, n, 2) if complex

Returns:

(2, 2, n) if real or (2, 2, n, 2) if complex d_input: (batch_size, 2, n) if real or (batch_size, 2, n, 2) if complex

Return type:

d_twiddle

static forward(ctx, twiddle, input)

Multiply by a single factor. :param twiddle: (2, 2, n) if real or (2, 2, n, 2) if complex :param input: (batch_size, 2, n) if real or (batch_size, 2, n, 2) if complex

Returns:

(batch_size, 2, n) if real or (batch_size, 2, n, 2) if complex

Return type:

output

class neuromancer.slim.butterfly.butterfly_multiply.ButterflyMult(*args, **kwargs)

Bases: Function

static backward(ctx, grad)

Parameters:

• grad – (batch_size, nstack, n) if real or (batch_size, nstack, n, 2) if complex

• twiddle – (nstack, n - 1, 2, 2) if real or (nstack, n - 1, 2, 2, 2) if complex

• backward (output + intermediate values for) – (log n + 1, batch_size, nstack, n) if real or (log n + 1, batch_size, nstack, n, 2) if complex

Returns:

(nstack, n - 1, 2, 2) if real or (nstack, n - 1, 2, 2, 2) if complex d_input: (batch_size, n) if real or (batch_size, n, 2) if complex

Return type:

d_twiddle

static forward(ctx, twiddle, input, increasing_stride=True)

Parameters:

• twiddle – (nstack, n - 1, 2, 2) if real or (nstack, n - 1, 2, 2, 2) if complex

• input – (batch_size, n) if real or (batch_size, n, 2) if complex

• increasing_stride – whether to multiply with increasing stride (e.g. 2, 4, …, n/2) or decreasing stride (e.g., n/2, n/4, …, 2). Note that this only changes the order of multiplication, not how twiddle is stored. In other words, twiddle[@log_stride] always stores the twiddle for @stride.

Returns:

(batch_size, nstack, n) if real or (batch_size, nstack, n, 2) if complex

Return type:

output

class neuromancer.slim.butterfly.butterfly_multiply.ButterflyMultInplace(*args, **kwargs)

Bases: Function

static backward(ctx, grad)

Define a formula for differentiating the operation with backward mode automatic differentiation.

This function is to be overridden by all subclasses. (Defining this function is equivalent to defining the vjp function.)

It must accept a context ctx as the first argument, followed by as many outputs as the forward() returned (None will be passed in for non tensor outputs of the forward function), and it should return as many tensors, as there were inputs to forward(). Each argument is the gradient w.r.t the given output, and each returned value should be the gradient w.r.t. the corresponding input. If an input is not a Tensor or is a Tensor not requiring grads, you can just pass None as a gradient for that input.

The context can be used to retrieve tensors saved during the forward pass. It also has an attribute ctx.needs_input_grad as a tuple of booleans representing whether each input needs gradient. E.g., backward() will have ctx.needs_input_grad[0] = True if the first input to forward() needs gradient computed w.r.t. the output.

static forward(ctx, twiddle, input, increasing_stride=True)

Experimental in-place implementation that does not store intermediate results. Instead, the intermediate results are computed from the output during the backward pass. :param twiddle: (n - 1, 2, 2) if real or (n - 1, 2, 2, 2) if complex :param input: (batch_size, n) if real or (batch_size, n, 2) if complex :param increasing_stride: whether to multiply with increasing stride (e.g. 2, 4, …, n/2) or

decreasing stride (e.g., n/2, n/4, …, 2). Note that this only changes the order of multiplication, not how twiddle is stored. In other words, twiddle[@log_stride] always stores the twiddle for @stride.

Returns:

(batch_size, n) if real or (batch_size, n, 2) if complex

Return type:

output

class neuromancer.slim.butterfly.butterfly_multiply.ButterflyMultUntied(*args, **kwargs)

Bases: Function

static backward(ctx, grad)

Parameters:

• grad – (batch_size, nstack, n) if real or (batch_size, nstack, n, 2) if complex

• twiddle – (nstack, log 2, n / 2, 2, 2) if real or (nstack, log 2, n / 2, 2, 2, 2) if complex

• backward (output + intermediate values for) – (log n + 1, batch_size, nstack, n) if real or (log n + 1, batch_size, nstack, n, 2) if complex

Returns:

(nstack, log 2, n / 2, 2, 2) if real or (nstack, log 2, n / 2, 2, 2, 2) if complex d_input: (batch_size, n) if real or (batch_size, n, 2) if complex

Return type:

d_twiddle

static forward(ctx, twiddle, input, increasing_stride=True)

Parameters:

• twiddle – (nstack, log 2, n / 2, 2, 2) if real or (nstack, log 2, n / 2, 2, 2, 2) if complex

• input – (batch_size, n) if real or (batch_size, n, 2) if complex

• increasing_stride – whether to multiply with increasing stride (e.g. 2, 4, …, n/2) or decreasing stride (e.g., n/2, n/4, …, 2). Note that this only changes the order of multiplication, not how twiddle is stored. In other words, twiddle[@log_stride] always stores the twiddle for @stride.

Returns:

(batch_size, nstack, n) if real or (batch_size, nstack, n, 2) if complex

Return type:

output

neuromancer.slim.butterfly.butterfly_multiply.butterfly_mult(twiddle, input, increasing_stride=True, return_intermediates=False)

Parameters:

• twiddle – (nstack, n - 1, 2, 2) if real or (nstack, n - 1, 2, 2, 2) if complex

• input – (batch_size, n) if real or (batch_size, n, 2) if complex

• increasing_stride – whether to multiply with increasing stride (e.g. 2, 4, …, n/2) or decreasing stride (e.g., n/2, n/4, …, 2). Note that this only changes the order of multiplication, not how twiddle is stored. In other words, twiddle[@log_stride] always stores the twiddle for @stride.

• return_intermediates – whether to return all the intermediate values computed, for debugging

Returns:

(batch_size, nstack, n) if real or (batch_size, nstack, n, 2) if complex

Return type:

output

neuromancer.slim.butterfly.butterfly_multiply.butterfly_mult_factors(twiddle, input, increasing_stride=True, return_intermediates=False)

Implementation that have separate kernels for each factor, for debugging. :param twiddle: (n - 1, 2, 2) if real or (n - 1, 2, 2, 2) if complex :param input: (batch_size, n) if real or (batch_size, n, 2) if complex :param increasing_stride: whether to multiply with increasing stride (e.g. 2, 4, …, n/2) or

decreasing stride (e.g., n/2, n/4, …, 2). Note that this only changes the order of multiplication, not how twiddle is stored. In other words, twiddle[@log_stride] always stores the twiddle for @stride.

Parameters:

return_intermediates – whether to return all the intermediate values computed, for debugging

Returns:

(batch_size, n) if real or (batch_size, n, 2) if complex

Return type:

output

neuromancer.slim.butterfly.butterfly_multiply.butterfly_mult_torch(twiddle, input, increasing_stride=True, return_intermediates=False)

Parameters:

• twiddle – (nstack, n - 1, 2, 2) if real or (nstack, n - 1, 2, 2, 2) if complex

• input – (batch_size, n) if real or (batch_size, n, 2) if complex

• increasing_stride – whether to multiply with increasing stride (e.g. 2, 4, …, n/2) or decreasing stride (e.g., n/2, n/4, …, 2). Note that this only changes the order of multiplication, not how twiddle is stored. In other words, twiddle[@log_stride] always stores the twiddle for @stride.

• return_intermediates – whether to return all the intermediate values computed, for debugging

Returns:

(batch_size, nstack, n) if real or (batch_size, nstack, n, 2) if complex

Return type:

output

neuromancer.slim.butterfly.butterfly_multiply.butterfly_mult_untied(twiddle, input, increasing_stride=True, return_intermediates=False)

Parameters:

• twiddle – (nstack, log n, n / 2, 2, 2) if real or (nstack, log n, n / 2, 2, 2, 2) if complex

• input – (batch_size, n) if real or (batch_size, n, 2) if complex

• increasing_stride – whether to multiply with increasing stride (e.g. 2, 4, …, n/2) or decreasing stride (e.g., n/2, n/4, …, 2). Note that this only changes the order of multiplication, not how twiddle is stored. In other words, twiddle[@log_stride] always stores the twiddle for @stride.

• return_intermediates – whether to return all the intermediate values computed, for debugging

Returns:

(batch_size, nstack, n) if real or (batch_size, nstack, n, 2) if complex

Return type:

output

neuromancer.slim.butterfly.butterfly_multiply.butterfly_mult_untied_torch(twiddle, input, increasing_stride=True, return_intermediates=False)

Parameters:

• twiddle – (nstack, log n, n / 2, 2, 2) if real or (nstack, log n, n / 2, 2, 2, 2) if complex

• input – (batch_size, n) if real or (batch_size, n, 2) if complex

• increasing_stride – whether to multiply with increasing stride (e.g. 2, 4, …, n/2) or decreasing stride (e.g., n/2, n/4, …, 2). Note that this only changes the order of multiplication, not how twiddle is stored. In other words, twiddle[@log_stride] always stores the twiddle for @stride.

• return_intermediates – whether to return all the intermediate values computed, for debugging

Returns:

(batch_size, nstack, n) if real or (batch_size, nstack, n, 2) if complex

Return type:

output