"DC blocking" filter using FFT->iFFT?

[Fender19]

We expected that we could easily add a "DC blocking" filter to an existing FFT->iFFT process by simply setting the lowest bin (bin 0, "DC") magnitude to 0.0 - but it doesn't work. Instead of removing DC offset from the signal it creates only a slight roll off (~3dB) while adding THD to the AC frequencies. We don't do anything else to the DC bins.

Why didn't this work as expected?

[Music Engineer]

Are you doing one big FFT on a complete signal or are you doing some sort of STFT process with signal blocks, windowing, etc.? In the latter case, I would assume that the blocking introduces artifacts. Imagine what the process would do to a slowly varying sinusoid whose periodicity is of the order of the block-length (but longer than that). Then each signal block would see a different "DC" value. It would appear as a sort of "undulating DC". Trying to just remove it by zeroing out the DC bin in each block may introduce all sorts of weirdness, I suppose. The effects of trying to remove the "undulating DC" (i.e. very low frequency components) will interfere with the effect of trying to remove the actual, bona fide DC. And there might also be some sort of spectral leakage be going on in the STFT and whatnot. It's all rather messy.

Some background:
In general, the task of producing a time-domain signal from a modified (or arbitrarily generated) spectrogram in such a way, that the produced signal does indeed have the prescribed spectrogram, is a nontrivial task that can only be solved approximately (in the least squares sense) anyway. This is detailed on pages 15 ff here:

http://recherche.ircam.fr/anasyn/roebel ... le/VL2.pdf

The gist of it is that you have to apply your analysis window again as synthesis window and then possibly de-modulate, if your overlapping squared windows do not add up to 1 - that's what the denominator does in eq. 16, if I'm not mistaken. It divides by the sum of overlapping squared windows. The square appears because the window is applied twice. Once in the analysis and then once again in the (re)synthesis.

This might be the relevant original paper:

http://dub.ucsd.edu/CATbox/Reader/GriffinLimMSTFT.pdf

although, I'm not sure about that. I've just glanced over it but it appears to me that Roebel's equation 16 in the lecture notes reproduces equation A1 in the Griffin-Lim paper with y(n) rather inconveniently expressed as inverse discrete time Fourier transform (IDTFT) or something in the latter which makes the formula look way more scary than it needs to be. I could be wrong about that, though. Edit: Yep - I was wrong: After taking a somewhat closer look at the paper, I now think, that eq. 16 in the lecture notes/slides corresponds to eq. 6 in the paper.

But the general lesson to be learned is that naively modifying a spectrogram by, for example, zeroing out the contents of certain bins, might not give you, what you expect. Signal synthesis from arbitrary (or arbitrarily modified) spectrograms is more complicated than that.

[Fender19]

Are you doing one big FFT on a complete signal or are you doing some sort of STFT process with signal blocks, windowing, etc.? In the latter case, I would assume that the blocking introduces artifacts. Imagine what the process would do to a slowly varying sinusoid whose periodicity is of the order of the block-length (but longer than that). Then each signal block would see a different "DC" value. It would appear as a sort of "undulating DC". Trying to just remove it by zeroing out the DC bin in each block may introduce all sorts of weirdness, I suppose.

It's an "overlap and add" FFT->iFFT process. As you suggest, perhaps removing the contribution of the DC bin (by setting it to zero) breaks the "foundation" of the signal - like removing the ground wire in an electronic circuit - thereby creating randomness in the iFFT rather than simply removing any DC offset, IDK. Just trying to understand conceptually what's going on here.

But the general lesson to be learned is that naively modifying a spectrogram by, for example, zeroing out the contents of certain bins, might not give you, what you expect.

Understood. Other High Pass FFT filters we have built used "convolution" approaches using the desired impulse response. Checking now though I see these filters attenuate down to a certain level but do not "block" DC.

So then, is it even possible to create a true DC blocking filter (not just HPF) using an FFT->iFFT? We have experimented with some special "DC blocking" IIR filters but they always seem to introduce transients - in other words, they have side effects as well.

[Music Engineer]

Yeah - in general you may have some "random undulating DC" be going on, i.e. low frequency components - so low that they are not resolved by your FFT and therefore appear as "random" fluctuations primarily in the DC bin (but to a lesser extent in other bins as well). And I would expect that to mess up everything.

Other High Pass FFT filters we have built used "convolution" approaches using the desired impulse response. Checking now though I see these filters attenuate down to a certain level but do not "block" DC.

FFT/iFFT based fast convolution algorithms don't really do any overlapping or windowing (other than the implicit rectangular window, of course), as far as I know. At least, not at the input side. At the output side, there needs to be some overlap though because the output chunks are longer than the input chunks (because they are results of (partial) convolutions of input chunks with impulse-response chunks). So, I would consider that to be a different kind of process compared to "spectrogram editing". If your convolution filters do not fully block DC, I would check the mean value of the impulse response. If I'm not mistaken, that mean value should precisely be the (linear) DC gain. A highpass impulse response should have a mean value of zero. And if you remove the actual (computed) mean value from some given impulse response (i.e. subtract it), then the resulting impulse response should completely block DC when you use it in convolution.

Edit: Or maybe it's the sum rather than the mean...yeah...that actually makes more sense....a filter with impulse response [1, 1, 1] should have a DC gain of 3, for example. So - yeah - maybe try subtracting the sum of all values from your highpass impulse responses and then check again if they now properly block DC. In general, nothing should stop an FIR filter from completely blocking DC. Consider a "differentiator" with impulse response [-1, 1]. It clearly blocks DC completely. This is obvious if you think about DC as a constant stream of 1s and just consider what the filter computes at any given sample instant: It's -1*1 + 1*1 = 0.

is it even possible to create a DC blocking filter using an FFT->iFFT?

Hmmm...I never tried. Of course, if you are just using one big FFT on a complete signal and a rectangular window, then zeroing out the DC bin before doing an iFFT should give you the same signal as you would get when just computing the mean of your signal and then subtracting it. But with blocks and windows, it gets messy and I'd rather use a simple highpass filter for that purpose. Maybe Butterworth, if higher order is desired - or maybe Bessel, if you care about the time-domain signal shape - although I'm currently not sure, if the time-domain preservation properties of Bessel filters apply to its highpass variant as well - I would guess so, but I would verify that. Or, in a non-realtime scenario, I'd probably go with a bidirectional (i.e. zero-phase) filter

[signalsmith]

Why didn't this work as expected?

I think a good perspective would be be viewing the STFT as a downsampled filter bank.

In this view, you're zeroing out the lowest band. However:

• The bands overlap, so that DC component is present in the first few bands, not just index 0
• The downsample produces aliasing within each sub-band.

This aliasing cancels out when using a "perfect reconstruction" setup, but the moment you change anything then it stops cancelling, and this aliasing is present in the result. Using more overlap between blocks improves this, because your downsample has more headroom (here's a plot of pessimistic aliasing with a Kaiser-windowed STFT).

[Music Engineer]

deleted [accidentally quoted instead of edit - damn - this is annoying! ]

[mystran]

We expected that we could easily add a "DC blocking" filter to an existing FFT->iFFT process by simply setting the lowest bin (bin 0, "DC") magnitude to 0.0 - but it doesn't work.

Ofcourse it works perfectly fine... but remember FFT is a transform on periodic signals, so this method only works if you interpret both your input and output as periodic signals.

If your signals are not periodic, then you can't do this "zeroing out bins" but rather need to build a regular FIR kernel and then use convolution, which you certainly can accelerate with FFT.

[mystran]

It's an "overlap and add" FFT->iFFT process. As you suggest, perhaps removing the contribution of the DC bin (by setting it to zero) breaks the "foundation" of the signal - like removing the ground wire in an electronic circuit - thereby creating randomness in the iFFT rather than simply removing any DC offset, IDK. Just trying to understand conceptually what's going on here.

So what is really going on is that FFT computes correlation with sinusoids of the form exp(i*2*pi*n/N) where "n" is the bin and N is the window size. For the DC bin, this becomes exp(0)=1, so the DC bin is simply the average value over the window. If you set the DC bin to zero, then you subtract the average from the window and then there's no DC, because the average is zero.

However.. normally if you do some sort of spectral processing, then you would use overlapping frames and you would apply some sort of window function in order to actually get those frames to join together smoothly... and that's where things go wrong, because windowing in time-domain blurs the spectrum; in a sense it's the usual time-frequency uncertainty thing, we make our frame more compact in time-domain, we end up making it less compact in frequency... and that's where the adjacent bins bleed into the DC bin, so even though there might be exactly zero DC before windowing, there might be some again after windowing... but really even if you think about it purely in terms of "DC = average value" it is easy to see that depending on waveshape, the average value before and after windowing might be different.