Spectrogram VST?

[autloc]

pedantic.^2:

from Discrete-Time Processing of Speech Signals (Deller, Hansen, Proakis)

"...In contemporary signal processing, it is possible to exhibit these changes using a three-dimensional plot of magnitude spectra over time..."

I think the point at issue here is that a spectrogram/waterfall is a function -- there is a unique output y for each time and frequency. This means the data can be stored in a two dimension table, but it does not mean that the curve the data describes is two dimensional.

[texture]

Well, a waterfall is the same info displayed in a 3D box, but it's not actually 3D. A spectrogram is saved in a 2D table

so is each element of that 2D array 0D? Isn't it really an array of blocks? Those blocks essentially being a vector of bits?

[Miles1981]

Is an image 2D or 3D for you ?

[Space Boy]

Is an image 2D or 3D for you ?

MRI or Ultrasound?

[texture]

I don't really care. the point of the matter is that a spectorgram conveys 3 pieces of information that vary with respect to each other, and a spectrum conveys 2. Both of them can be printed on a '2d' x-y plot. Lets just make a compromise and say the spectrogram is 2.5D.

[Space Boy]

I don't really care. the point of the matter is that a spectorgram conveys 3 pieces of information that vary with respect to each other, and a spectrum conveys 2. Both of them can be printed on a '2d' x-y plot. Lets just make a compromise and say the spectrogram is 2.5D.

You may be confusing data representation with information content. A single channel of audio signal is 1D information. Amplitude as a function of time. Time is the dimension. Classically, there are four dimensions x,y,z (spatial) and t (time). For most of us, time only goes in one direction.

Chopping that audio stream into segments, making a spectrum of each segment and stacking them in a way in which the time axis is horizontal, frequency axis is vertical and amplitude is represented by intensity, yields a 2D image. 2D because the information is now represented by 2 dimensions - time (x) and frequency (y). Information content did not change. Remember that each segment, from which a spectrum was calculated, took a finite amount of time.

An alternative way is to make a pseudo 3D plot in which the length of the line plotted at point (x,y) represents the amplitude. This is a 3D representation of the 1D information.

Further still, one could make a movie of the 3D plot changing as a function of time. This would be a 4D representation of 1D information.

As far as the computer displaying this information is concerned, it displays a series of 2D pixel intensity maps (even when displaying a 1D plot or a 4D representation).

So a Spectrogram displayed in the form of a static image is a 2D representation. A 2D image that changes in time is strictly a 3D representation. A Spectrogram displayed in the form of a 3D plot is a 3D representation. And a pseudo 3D plot that changes as a function of time is a 4D representation.

The information content, however, is still only 1D.

2.5D is really difficult to handle.

[texture]

You may be confusing data representation with information content. A single channel of audio signal is 1D information. Amplitude as a function of time.

How is amplitude a function of time, and why is it not a dimension? Time does not dictate what amplitude should be, only when coupled with an object that has other parameters. Something is varying with time, not being varied by time.

yes, you can say that amp = sin(t), but that just describes one functional relationship between time and amplitude. You could do the same with y = sin(x).

I've always considered a dimension to be something that represents a physical quantity, be it length, mass etc. Perhaps I'm wrong.

I do see where you're coming from with saying it is 2D, but that is only from a spatial perspective. I would generally consider an image 2D, although I sometimes wonder if it is strictly correct to do so. It is just something we've been fed as children and not really questioned.

You must admit that a spectrogram is much richer in its information that just a spectrum. The spectrum, whist being a 2d plot, can be represented by a '1d' vector, whith each item in the vector representing a dimension of the plot. The spectrogram requires an extra dimension to be added to the vector in order to represent it.

[Space Boy]

You may be confusing data representation with information content. A single channel of audio signal is 1D information. Amplitude as a function of time.

How is amplitude a function of time, and why is it not a dimension? Time does not dictate what amplitude should be, only when coupled with an object that has other parameters. Something is varying with time, not being varied by time.

yes, you can say that amp = sin(t), but that just describes one functional relationship between time and amplitude. You could do the same with y = sin(x).

I've always considered a dimension to be something that represents a physical quantity, be it length, mass etc. Perhaps I'm wrong.

I do see where you're coming from with saying it is 2D, but that is only from a spatial perspective. I would generally consider an image 2D, although I sometimes wonder if it is strictly correct to do so. It is just something we've been fed as children and not really questioned.

You must admit that a spectrogram is much richer in its information that just a spectrum. The spectrum, whist being a 2d plot, can be represented by a '1d' vector, whith each item in the vector representing a dimension of the plot. The spectrogram requires an extra dimension to be added to the vector in order to represent it.

The critical quality a dimension requires is that it can be represented by a scale. Amplitude can have units of decibels and can be represented on a scale on an x axis. Mass can have units of kg and can be represented on a scale on a y axis. Amplitude is not a dimension and neither is mass, they are quantities. In your amp = sin(t) example, you clearly indicated the relevant dimension, t.

Having said that, you could argue that intensity is scaled from 0 to 255. This is indeed correct, however, knowing the intensity/amplitude alone does not tell me the x and y coordinates in my 2D image. There could be more than one point with the same intensity. Whereas, knowing the x and y coordinate does provide me with the intensity. All points on the dimenisional axes are unique.

Notes of music on a page indicate pitch as a function of time. Let's say the first note in bar 16 is the note C#. If I always play the note in bar 16 it will always be a C#. However, If I now just play a C# (without looking at the music), can you tell me if I was playing the C# in bar 16 or the one in bar 24? Can you tell me if I was even playing a C#?Actually you cannot - you don't have enough information (unless you already heard me play a C#, but then you can only identify the pitch, you still don't know which bar the note came from).

Only when I play all notes in the piece, and you know the piece starts at bar 1, can you figure out that I played a C# in bar 16 and 24. Only by examining the music, as a function of the dimension of time, are you able to correctly identify the notes. So amplitude, pitch, mass etc. tend to be quantities and not dimensions.

A spectrogram mostly shows the correlation of signals in time. I agree that it is easier to see relationships this way. But actually, it is only an attempt to do for our eyes what our ears already do extremely well - in 1D.

[autloc]

The critical quality a dimension requires is that it can be represented by a scale. Amplitude can have units of decibels and can be represented on a scale on an x axis. Mass can have units of kg and can be represented on a scale on a y axis. Amplitude is not a dimension and neither is mass, they are quantities. In your amp = sin(t) example, you clearly indicated the relevant dimension, t.

Scales? Quantities? I think you're searching for an "intuitive" definition of what a dimension is, but clearly it is not the same as my intuitive definition. In any case, it seems like a moot point since a general consensus seems to exist. From wikipedia:

In mathematics, no definition of dimension adequately captures the concept in all situations where we would like to make use of it. Consequently, mathematicians have devised numerous definitions of dimension for different types of spaces. All, however, are ultimately based on the concept of the dimension of Euclidean n-space E^n. The point E^0 is 0-dimensional. The line E^1 is 1-dimensional. The plane E^2 is 2-dimensional. And in general E^n is n-dimensional.

Any point in a spectrogram is specified by three coordinates: time, frequency and intensity. E^3. It may be represented using only 2 spatial dimensions (with colour or intensity indicating the third), but the data itself represents a 3-dimensional surface.

[Space Boy]

Scales? Quantities? I think you're searching for an "intuitive" definition of what a dimension is, but clearly it is not the same as my intuitive definition. In any case, it seems like a moot point since a general consensus seems to exist.


Any point in a spectrogram is specified by three coordinates: time, frequency and intensity. E^3. It may be represented using only 2 spatial dimensions (with colour or intensity indicating the third), but the data itself represents a 3-dimensional surface.

http://reference.allrefer.com/encyclope ... s-phy.html

Dimension, in physics, Physics

Related Category: Physics

dimension, in physics, an expression of the character of a derived quantity in relation to fundamental quantities, without regard for its numerical value. In any system of measurement, such as the metric system, certain quantities are considered fundamental, and all others are considered to be derived from them. Systems in which length (L), time (T), and mass (M) are taken as fundamental quantities are called absolute systems. In an absolute system force is a derived quantity whose dimensions are defined by Newton's second law of motion as ML/T2, in terms of the fundamental quantities. Pressure (force per unit area) then has dimensions M/LT2; work or energy (force times distance) has dimensions ML2/T2; and power (energy per unit time) has dimensions ML2/T3. Additional fundamental quantities are also defined, such as electric charge and luminous intensity. The expression of any particular quantity in terms of fundamental quantities is known as dimensional analysis and often provides physical insight into the results of a mathematical calculation.

No, I'm not searching, nor need to be intuitive (though, I am mostly). There is no need for a concensus vote - it is a well defined concept in Physics. Physics deals with everyday reality. Mathematics has the luxury that it can be abstract.

The Euclidean expression you quote, and the way you use it, is not appropriate for the physical situation we are working with here.

amp = f(t) 1D
amp = f(x,y) 2D representation where x = t and y = f = 1/t

E^3 does not provide a practical, or convenient, means with which to model what is observed in the case of our simple 1D audio.

wikipedia? I just visited that site for the first time. Apparently, it says at the top that the article on dimension is not about what is considered to be the normal usage of the meaning. From wikipedia:

Dimension (from Latin "measured out") is, in essence, the number of degrees of freedom available for movement in a space. (In common usage, the dimensions of an object are the measurements that define its shape and size. That usage is related to, but different from, what this article is about.)

Also less than a complete definition . But now I see that wikepedia deals in 'Open source' definitions (oh no ). Perhaps I'll change it to support my point of view. Anyone can edit it apparently. So, the common concenus may also eventually make time travel possible, and the earth flat!.

Physically, however, the earth will remain spherical (in our frame of reference) and time travel is near to impossible.

Note: Your use of the word coordinates, as in 3, to describe amplitude, time, frequency and intensity, - is incorrect. There are 3 parameters. However, there is 1 quantity, intensity, and only 2 coordinates, time and frequency.

[Miles1981]

texture > do you need something special in your spectrogram ? - size, ... ? -. I'm asking so that my prog will satisfy your needs

[autloc]

Let's talk past each other some more!

I can agree that the data in a spectrogram is fundamentally two dimensional, since the frequency axis merely derived by a linear operation (DFT) performed using the two dimensional time and amplitude time domain values -- so in that sense the extension on to the frequency axis is redundant -- but the curve in the spectrogram visualization is nonetheless geometrically three dimensional (i.e., has three axes). I guess you already said as much when you said "You may be confusing data representation with information content," though to slightly different conclusions.

Anyway, I guess there isn't much point in debating about it anymore. Feel free to add your viewpoint to wikipedia if you so want, though peer review can be pretty frustrating. It may be an "open document", but I think you'll find that its articles on math and physics are generally pretty good. Not so strong on DSP, but even there there are some interesting (if disorganized) articles.

[Space Boy]

Let's talk past each other some more!

I can agree that the data in a spectrogram is fundamentally two dimensional, since the frequency axis merely derived by a linear operation (DFT) performed using the two dimensional time and amplitude time domain values -- so in that sense the extension on to the frequency axis is redundant -- but the curve in the spectrogram visualization is nonetheless geometrically three dimensional (i.e., has three axes). I guess you already said as much when you said "You may be confusing data representation with information content," though to slightly different conclusions.

Anyway, I guess there isn't much point in debating about it anymore. Feel free to add your viewpoint to wikipedia if you so want, though peer review can be pretty frustrating. It may be an "open document", but I think you'll find that its articles on math and physics are generally pretty good. Not so strong on DSP, but even there there are some interesting (if disorganized) articles.

Debate is healthy - but I think we now agree.

If you are trying to solve a problem then, in practise, if the picture of the problem you construct for yourself enables you to solve the problem, then it doesn' really matter what your picture looks like. It only becomes a problem when you want to communicate your picture to somebody else - then you have a debate (or argument).

I have enough text books with definitions that tend to agree. I'll leave my viewpoint out of wikipedia for now.

[texture]

@ Wide Boy:You said that mass is not a dimension, and then posted a quote to prove that amplitude is not a dimension that states that mass is a dimension. Thats a bit of a contradiction of your own terms isn't it? It also mentions mass under the definition of 'dimension' at www.dictionary.com. The meaning of the word does depend upon context I guess. Perhaps it is just that.


@Miles:
Things I'd like to see:
Resizable VST window, with the spectrogram strecthing to fit.
A zoom tool.
Selectable FFT windowing function & FFT size.
Scope in parallel with the spectrogram.
Ability to change the contrast of the display.
Ability to freeze the view.
Ability to record a short section of audio and play back a visualisation of it in slow motion.

Some of these are pretty ambitious, but they'd be cool additions. The main thing I'd like is the resizable view. And/or zoom.

[Miles1981]

Well, That's for the 2.0 version, I'm afraid I want first a simple spectrogram with some other things at its side, but taking the spectrogram from the analyzer is not very difficult.
The ability to record a short version is more for a host, I think, but why not...

Having a resizable VST window is not that simple : the size is fixed by a callback function - getRect - before the VST GUI is shown. I don't know at this time how I will make it variable - but I thought of that too -