How should I multiply scipy.fftpack output vectors together?

Question

The scipy.fftpack.rfft function returns the DFT as a vector of floats, alternating between the real and complex part. This means to multiply to DFTs together (for convolution) I will have to do the complex multiplication "manually" which seems quite tricky. This must be something people do often - I presume/hope there is a simple trick to do this efficiently that I haven't spotted?

Basically I want to fix this code so that both methods give the same answer:

import numpy as np
import scipy.fftpack as sfft

X = np.random.normal(size = 2000)
Y = np.random.normal(size = 2000)
NZ = np.fft.irfft(np.fft.rfft(Y) * np.fft.rfft(X))
SZ = sfft.irfft(sfft.rfft(Y) * sfft.rfft(X))    # This multiplication is wrong

NZ
array([-43.23961083,  53.62608086,  17.92013729, ..., -16.57605207,
     8.19605764,   5.23929023])
SZ
array([-19.90115323,  16.98680347,  -8.16608202, ..., -47.01643274,
    -3.50572376,  58.1961597 ])

N.B. I am aware that fftpack contains a convolve function, but I only need to fft one half of the transform - my filter can be fft'd once in advance and then used over and over again.

Answer 1

You don't have to flip back to np.float64 and hstack. You can create an empty destination array, the same shape as sfft.rfft(Y) and sfft.rfft(X), then create a np.complex128 view of it and fill this view with the result of the multiplication. This will automatically fill the destination array as wanted.
If I retake your example :

import numpy as np
import scipy.fftpack as sfft

X = np.random.normal(size = 2000)
Y = np.random.normal(size = 2000)
Xf = np.fft.rfft(X)
Xf_cpx = Xf[1:-1].view(np.complex128)
Yf = np.fft.rfft(Y)
Yf_cpx = Yf[1:-1].view(np.complex128)

Zf = np.empty(X.shape)
Zf_cpx = Zf[1:-1].view(np.complex128)

Zf[0] = Xf[0]*Yf[0]

# the [...] is important to use the view as a reference to Zf and not overwrite it
Zf_cpx[...] = Xf_cpx * Yf_cpx 

Zf[-1] = Xf[-1]*Yf[-1]

Z = sfft.irfft.irfft(Zf)

and that's it! You can use a simple if statement if you want your code to be more general and handle odd lengths as explained in Jaime's answer. Here is a function that does what you want:

def rfft_mult(a,b):
    """Multiplies two outputs of scipy.fftpack.rfft"""
    assert a.shape == b.shape
    c = np.empty( a.shape )
    c[...,0] = a[...,0]*b[...,0]
    # To comply with the rfft support of multi dimensional arrays
    ar = a.reshape(-1,a.shape[-1])
    br = b.reshape(-1,b.shape[-1])
    cr = c.reshape(-1,c.shape[-1])
    # Note that we cannot use ellipses to achieve that because of 
    # the way `view` work. If there are many dimensions, one should 
    # consider to manually perform the complex multiplication with slices.
    if c.shape[-1] & 0x1: # if odd
        for i in range(len(ar)):
            ac = ar[i,1:].view(np.complex128)
            bc = br[i,1:].view(np.complex128)
            cc = cr[i,1:].view(np.complex128)
            cc[...] = ac*bc
    else:
        for i in range(len(ar)):
            ac = ar[i,1:-1].view(np.complex128)
            bc = br[i,1:-1].view(np.complex128)
            cc = cr[i,1:-1].view(np.complex128)
            cc[...] = ac*bc
        c[...,-1] = a[...,-1]*b[...,-1]
    return c

Answer 2

You can take a view of a slice of your return array, e.g.:

>>> scipy.fftpack.fft(np.arange(8))
array([ 28.+0.j        ,  -4.+9.65685425j,  -4.+4.j        ,
        -4.+1.65685425j,  -4.+0.j        ,  -4.-1.65685425j,
        -4.-4.j        ,  -4.-9.65685425j])
>>> a = scipy.fftpack.rfft(np.arange(8))
>>> a
array([ 28.        ,  -4.        ,   9.65685425,  -4.        ,
         4.        ,  -4.        ,   1.65685425,  -4.        ])
>>> a.dtype
dtype('float64')
>>> a[1:-1].view(np.complex128) # First and last entries are real
array([-4.+9.65685425j, -4.+4.j        , -4.+1.65685425j])

You will need to handle even or odd sized FFTs differently:

>>> scipy.fftpack.fft(np.arange(7))
array([ 21.0+0.j        ,  -3.5+7.26782489j,  -3.5+2.79115686j,
        -3.5+0.79885216j,  -3.5-0.79885216j,  -3.5-2.79115686j,
        -3.5-7.26782489j])
>>> a = scipy.fftpack.rfft(np.arange(7))
>>> a
array([ 21.        ,  -3.5       ,   7.26782489,  -3.5       ,
         2.79115686,  -3.5       ,   0.79885216])
>>> a.dtype
dtype('float64')
>>> a[1:].view(np.complex128)
array([-3.5+7.26782489j, -3.5+2.79115686j, -3.5+0.79885216j])