Morphological Image Processing?

[AdmiralQuality]

Also, isn't antialiasing like this built into just about every OS already? Particularly for font rendering?

This is more than a simple anti-aliasing.

Sorry, not trying to be difficult. But what then? Most of the Photoshop layer effects are exactly what they say they are (add, subtract, multiply...)

Emboss is just a kernel like this:

0 0 0
0 1 0
-1 0 0

(Obviously that gets a bit different if you want to support other angles and depths, but that's the basic idea.)

Frankly I'd rather have a large "filmstrip" in my GUI than waste a nanosecond of CPU time on unnecessary image processing. Anything that can be "canned" beforehand, I'd can. And you can achieve dropshadow and even lighting effects by designing your images and alpha channel right.

Calculating the effects at run time lets you do more than can be done with pre-rendered bitmaps. You could overlay a control with glowing edges on top of another element that changes its appearance. Or you can change colours dynamically based on context. There are many possibilities.

But also, this will become part of a system that uses just in time compilation that shows you a preview of your code as you are writing it (disclaimer: I'm not writing that part).

See Poly-Ana for controls with glowing edges (the light buttons) overlapping other controls (the knobs that are near them, particularly in the performance section to the left of the keyboard). Also right click on a control and bring up the MIDI-learn, notice how it overlaps other controls yet is semi-transparent AND had a drop shadow? All done with PNGs with transparency.

Rotating color space is easy enough, but I'd do it to the filmstrip once rather than on demand at every idle redraw.

But sure, there are lots of times we'd like to arbitrarily draw content. Analyzer displays in particular.

[thevinn]

Sorry, not trying to be difficult. But what then? Most of the Photoshop layer effects are exactly what they say they are (add, subtract, multiply...)

Emboss is just a kernel like this:

0 0 0
0 1 0
-1 0 0

LOL..that's last century's technology. This is the modern style:

Code: Select all

/** Directional lighting transform.

    Calculates the incident highlights and shadows for each
    point in a discrete height map.
*/
struct LightingTransform
{
  /** Calculate the transform.

      @param shader     A functor called to shade each point.
      @param map        The height map.
      @param depth      Depth of the light source range [0, 10].
      @param angle      Light source angle, in radians.
      @param elevation  Light source elevation, in radians.

      @tparam T The underlying type to use for calculation (float or double)
  */
  template <class T, class Shader, class Map, class R1, class R2, class R3>
  static void calculate (
    Shader shader,
    Map map,
    R1 depth,
    R2 angle,
    R3 elevation)
  {
    Vec3 <T> lightNormal (
       cos (elevation) * cos (angle),
      -cos (elevation) * sin (angle),
       sin (elevation));

    for (int y = 1; y < map.getRows () - 1; ++y)
    {
      for (int x = 1; x < map.getCols () - 1; ++x)
      {
        // Calculate normal from height map.
        Vec3 <T> n (T (map (x+1,y) - map (x-1,y)),
                    T (map (x,y+1) - map (x,y-1)),
                    T (depth));

        // Calculate incident light amount, range [-1, 1]
        T incidentLight = n.getDotProduct (lightNormal) / n.getNormal ();

        shader (x, y, incidentLight);
      }
    }
  }

  /** Shader for highlight and shadows Pixels map.
  */
  struct PixelShader
  {
    PixelShader (Pixels hi, Pixels lo) : m_hi (hi), m_lo (lo)
    {
    }

    template <class T>
    inline void operator() (int x, int y, T incidentLight)
    {
      if (incidentLight >= 0)
        m_hi (x, y) = uint8 (255 * incidentLight + T(.5));
      else
        m_lo (x, y) = uint8 (-255 * incidentLight + T(.5));
    }

  private:
    Pixels::Map2D m_hi;
    Pixels::Map2D m_lo;
  };
};

The height map is the output of the anti-aliased euclidean distance transform.

All done with PNGs with transparency.

In my mind, the pre-rendering of bitmaps for controls is nothing more than a combination of 1) A substitute for missing effects code and 2) A draw-time optimization.

I'm directly addressing #1 by writing the code to implement these effects. And #2 could always be implemented on top of it, just pre-draw the controls at various settings when the application starts up and cache the images.

[AdmiralQuality]

LOL..that's last century's technology. This is the modern style:

My first job was programming binary load-lifters, very similar to your vaporators in most respects!

[HeavensOnEarth]

As a footnote, if you're making a UI for Windows8 Metro, this is helpful:

[url]http://www.colorzilla.com/gradient-editor/[/url]

"Ultimate CSS Gradient Generator - A powerful Photoshop-like CSS gradient editor from ColorZilla. "

[Frank123]

I'm going to do this all at runtime. I don't believe that the CPU impact is significant since this is for controls, which do not update frequently. You wouldn't want to put these layer styles onto a scrolling waveform or a level meter (which updates often).

GPU shaders are not really an option since this code requires a software renderer (to get at the pixels). Besides, that would be a maintenance nightmare.

Like mystran, I also would recommend using fragment shaders:
http://glsl.heroku.com/e#3448.3

Using the smoothstep function, you can paint AA shapes ranging from hard pixelated borders to soft shades which can be useful for glow effects and shadows.

[thevinn]

That's a REALLY COOL site.

[mystran]

For "cheap" emboss/bumpmap, I'd start with a height map (can approximate from luminosity; at least in Gimp you can also use separate image for the height when bumpmapping). Run Sobel-filter on the heightmap to estimate partial derivatives (the 3-10-3 variant usually looks better than the 1-2-1 variant). Then with a heigh-scaling parameter and the partial derivatives you can estimate surface normals. Once you know the normals, you can then use any standard shading model (and whatever type of light).

[thevinn]

For "cheap" emboss/bumpmap, I'd start with a height map (can approximate from luminosity; at least in Gimp you can also use separate image for the height when bumpmapping). Run Sobel-filter on the heightmap to estimate partial derivatives (the 3-10-3 variant usually looks better than the 1-2-1 variant). Then with a heigh-scaling parameter and the partial derivatives you can estimate surface normals. Once you know the normals, you can then use any standard shading model (and whatever type of light).

Well, yeah that's what I'm doing. I posted the code snippet for my LightingTransform that computes the surface normal from the height map and then applies a global light source.

The "hard" part is generating the height map. But I'm doing alright with it now, it's simply the distance transform.

After many hours of analysis it seems that the distance transform is the building block of all Photoshop Layer Style effects. I've implemented the best three of them from the research papers. I modified one of them to work with subpixel precision, and it seems to be working.

I'll post some progress images shortly.

Also, I'd like to reiterate that square convolution matrices are old-school. Photoshop hasn't used them for its internal filters for quite a few years now. CPUs have gotten powerful enough that they can use better techniques. For example, linearly separable convolution with radially symmetric kernels. Or the distance transform instead of the Sobel operator.

[mystran]

No. You don't understand:

Code: Select all

        // Calculate normal from height map. 
        Vec3 <T> n (T (map (x+1,y) - map (x-1,y)), 
                    T (map (x,y+1) - map (x,y-1)), 
                    T (depth)); 

That's your code. That's the part that I'd replace with Sobel to get better normals. Also what you probably should be doing with the partial derivatives is take the two vectors [1,0,dx] and [0,1,dy] and find their cross-product, then normalize that. The "add depth and normalize" trick works but isn't really right.

I don't care how you find your height-map. Any image-derived height-map tends to look like shit anyway.

[thevinn]

No. You don't understand:

Ah! Apparently not...I thought you meant the Sobel matrix applied to the image data. This is what I'm using now:

Code: Select all

Vec3 <T> n (
  T(map(x,y)) - .25*(2 * T(map(x+1,y)) + T(map(x+1,y-1)) + T(map(x+1,y+1))),
  T(map(x,y)) - .25*(2 * T(map(x,y+1)) + T(map(x-1,y+1)) + T(map(x+1,y+1))),
  T(depth));

what you probably should be doing with the partial derivatives is take the two vectors [1,0,dx] and [0,1,dy] and find their cross-product, then normalize that. The "add depth and normalize" trick works but isn't really right.

I can't say I'm completely proficient with vectors. Is this what you mean by add depth and normalize:

Code: Select all

T incidentLight = n.getDotProduct (lightNormal) / n.getNormal ();

?

I don't care how you find your height-map. Any image-derived height-map tends to look like shit anyway.

Well I'm trying to get results that are identical to what is produced in Photoshop. Since an artist will prototype their controls in Photoshop first, I need to get the same results so that the controls are reproducible in code.

[thevinn]

Like mystran, I also would recommend using fragment shaders:
http://glsl.heroku.com/e#3448.3

I think this is something that would be considered in an optimization step. But before doing that it is necessary to have a correct algorithm even if it is slow.

[thevinn]

take the two vectors [1,0,dx] and [0,1,dy] and find their cross-product, then normalize that. The "add depth and normalize" trick works but isn't really right.

I did some searching and I stumbled on "bump mapping"; It seems this is the same algorithm. This helps a lot because now I know what to search for. Everything that I've done has been the result of extensive web searching.

I got subscriptions to the ACM and IEEE digital libraries but their indexes and search engines are pretty bad, so any pointers on what I can search for in order to find more papers that are relevant is really helpful.

[mystran]

Well, what I mean you can do normalize(-dx,-dy,1) but that's not strictly speaking right; the partial derivatives are just that: derivatives relative to the position. You can use them to build two tangent vectors, and the surface normal then is their (normalized) cross-product. It's not a huge deal, but since the cross-product is cheap compared to the normalize I don't see much point not to do it.