VSTGUI 3.6 Windows 7 GDI+ Sluggishness: A Workaround...

[aciddose]

so you're saying it's called from another thread, in fact from an interrupt?

or do you use a polling timer in a thread loop?

or is it a kernel callback?

there are about a hundred ways to set up timers in windows...

if you're using another thread the problem with tearing occurs because two threads might be working on graphics at the same time. your main window dispatch might dispatch a wm_paint in the middle of the timer thread calling idle, which triggered an animation update in a plugin.

in fact, because windows doesn't to my knowledge do any locking at all with these messages, you might be painting into the same pixels in two different places at identical times..

unless you have an example of the "correct" way to do animation ... all the implementations i've ever seen used idle, as to my knowledge that is the only possibility.

your complaints about me suggesting updates from another thread apply to the way you claim you're calling idle...

[nollock]

If you use WM_TIMERs, then they have the nice property that there can only be at most one pending message per timer, and those messages have the lowest possible priority, which means you will never be handling any WM_TIMER messages until you have handled all other messages in the queue. So if you trigger a redraw from the timer, then the WM_PAINT will always get handled first, before you get another WM_TIMER (which will still get queued as soon as the interval has passed again).

That doesn't help because were talking about two separate threads, and separate threads have separate message queues.

Edit: Actually reading further he says he's not doing the painting in a separate thread. I will have to read back and see where i got that impression.

[tony tony chopper]

so you're saying it's called from another thread, in fact from an interrupt?or do you use a polling timer in a thread loop?

I mostly use Windows's "timer queues" (which are threaded), but the idling is -of course- called in the GUI thread.

if you're using another thread the problem with tearing occurs because two threads might be working on graphics at the same time.

I explained what Edison's tearing was about (it's not tearing at all)

unless you have an example of the "correct" way to do animation ... all the implementations i've ever seen used idle, as to my knowledge that is the only possibility.

animation is something else, you would indeed use idle or your own timers (actually, I see that the VST specs don't seem to specify (as usual) if idle is to be called from GUI, even though it's obvious it might not be the case in all hosts, and knowing that many hosts don't follow the idling freq suggested by the SDK, it's better to use your own timers), but in any case still invalidating windows instead of repainting directly. And if you were invalidating windows instead of repainting, it wouldn't matter which thread the invalidation comes from.

[tony tony chopper]

That doesn't help because were talking about two separate threads

Apparently there's no multithreading in the equation (says aciddose). So basically no possible benefit at all in the way he's painting the GUI.

[nollock]

...because the blits are dma to hardware, and they're cached, it's most efficient to draw and blit from a separate thread so you don't end up blocking window messages while you're fooling around drawing something.

Is that not saying blit in one thread and draw in another?

And if you're not doing it in a separate thread, then you will still block the GUI thread with expensive redraws. Granted that should you use WM_TIMER, you may reduce the frequency of such a thing happening a bit. But not by much as windows already throttles WM_PAINT if the system is under heavy load.

Anyways....

[aciddose]

yes of course, i was only saying in theory the most efficient way would be another thread, but then you'd have to use mutex locks of course.

the advantage of using idle for all updates is that the host then "could" control idle frequency if it wanted to. that's better than having a spec where plugins do all sorts of insane stuff like launching 15 timer threads, some using interrupts and accessing memory all over the place.

it's unfortunate you'd have to admit that they never explained idle() in the spec - but like i said i think the way i use it was it's original intention. there is absolutely no disadvantage to using it that way if the host calls idle from the gui thread.

i haven't personally seen a host that doesn't call idle from the gui thread... in fact most seem to do this:

while (active)
{
if (time() - last_time > idle_time)
{
plugins.idle()
}

while (peekmessage())
{
translate();
dispatch();
}
}

so if that's the way a host works, it's easy to modify the timing by just calling idle at different rates. the 2.4 spec says idle needs to happen at a specific rate? i hadn't seen that actually - it's partially good, partially bad. while it's good to have a standard rate, i think that should ultimately be up to the host. after all if another plugin locks the thread you can't guarantee the right idle frequency anyway.

you can make sure not to call idle above the refresh rate, which seems like a good idea. i think windows handles HID events using refresh rate limiting. i haven't read that, but it seems to be the case. i'm not sure where you'd look, maybe in the ddk sections for HID devices.

but the issue they leave out is where to call idle, why to call idle, why idle can be used, why it can't be used and other specifics which would really help with stability between hosts.

of course, i don't think their intention was ever to help host authors write better hosts than cubase - which is their flagship product.

[tony tony chopper]

it's unfortunate you'd have to admit that they never explained idle() in the spec - but like i said i think the way i use it was it's original intention. there is absolutely no disadvantage to using it that way if the host calls idle from the gui thread.

I think the idea was just to spare an expensive timer (don't forget that the SDK is old), as a good timer has to be thread-based. That was probably before Windows got its own internal thread-based system with shared threads.
CreateTimerQueueTimer didn't exist in Win9x, you either had to use a shitty very random timer, or your own thread, or an older timer system that Windows advises not using anymore (can't find the name back, not multimedia timers as those too are Win2k minimum).
Plus the fact that the SDK seems to be more Mac-oriented, as it has events for keyboard as well, which Windows shouldn't need.

[vladg]

Thanks so much, it's something I have to do. BTW shouldn't a lib like VSTGUI take care to set 32 bit PARGB anyway? And if it doesn't why it doesn't offer an option? Or does it?

I'm porting my code from VSTGUI 3.5 to VSTGUI 4.0.1 and also have problems with too slow animation.

Solution #1. Usage of Direct 2D support (VSTGUI_DIRECT2D_SUPPORT) gives extra cool performance on Vista/Seven but it doesn't supported on XP.

Solution #2. Make it like VSTGUI 3.5. VSTGUI 3.5 uses libpng to get pixels and then creates Gdiplus::Bitmap with pixel type PixelFormat32bppPARGB on demand (see CBitmap::getBitmap()). Some users complain about performance but for all my and beta-testers PCs it works perfect.

The reason of poor performance is the fact that a lot of my images has 256-color palette format to make plugin size smaller. And most of the time inside Graphics::DrawImage was spent in these routines (inside GDI+): ConvertBitmapData > EpFormatConverter::Convert > ScanOperation::AlphaMultiply_sRGB.

My solution is to convert all files to PixelFormat32bppPARGB on load.

I fixed vstgui_401\vstgui\lib\platform\win32\gdiplusbitmap.cpp in this way:

Code: Select all

bool GdiplusBitmap::loadFromStream (IStream* stream)
{
    if (bitmap)
        return false;
    bool result = false;
    bitmap = Gdiplus::Bitmap::FromStream (stream, TRUE);
    if (bitmap)
    {
        // ---- my fix ----
        Gdiplus::PixelFormat pf = bitmap->GetPixelFormat();

        if (pf != PixelFormat32bppPARGB) {

            // convert bitmap format to PixelFormat32bppPARGB

            Gdiplus::Bitmap *newBmp = new Gdiplus::Bitmap(bitmap->GetWidth(), bitmap->GetHeight(), PixelFormat32bppPARGB);
            Gdiplus::Graphics *pGraphics = new Gdiplus::Graphics(newBmp);

            Gdiplus::Rect sizeR(0,0, bitmap->GetWidth(), bitmap->GetHeight());

            pGraphics->DrawImage(bitmap, sizeR, 0, 0, (int)bitmap->GetWidth(), (int)bitmap->GetHeight(), Gdiplus::UnitPixel);

            delete pGraphics;
            delete bitmap;

            bitmap = newBmp;

        }
        // ---- end of my fix ----

        size.x = (CCoord)bitmap->GetWidth ();
        size.y = (CCoord)bitmap->GetHeight ();
    }
    return bitmap != 0;
}

Now with VSTSDK 4.0.1 I have the same performance like in VSTSDK 3.5 on XP and much greater performance (due to Direct 2D usage) on Vista/Seven.

Hope someone will find this useful.

[AdmiralQuality]

Good work! You might want to share it with the VSTGUI list. Maybe they'll adopt the change.

[AdmiralQuality]

Hi everyone,

2 weeks ago I upgraded to Windows 7, and until then I wasnt aware of the terrible performance of GDI+ on Windows 7. Since VSTGUI 3.6 uses GDI+, and I'm locked to version 3.6 for a while, I had to find a "workaround"...

So, after some testing, coding, yes I found it! And I want to share this with all the developers here:

You need to modify CBitmap->draw function, so that it does NOT draw with GDI+, but just uses BitBlt/AlphaBlend functions found in Win32. That DOES the trick!

For BitBlt/AlphaBlend you'll need the images as HBITMAP though...

If your bitmap is scaled while drawing though (again for this you need to modify the CBitmap->draw function, VSTGUI 3.6 does not scale images), you'll have to draw the image with GDI+ so it draws the image antialiased.

Once I did this change, my plugin user interface went back to normal...

I hesitate to dredge this one up. Before we begin, I should say I don't need to hear from aciddose or tony tony chopper unless they have a concise and exact answer to my question to kv331, below. I don't care to debate what does or doesn't suck, or who it is I think I'm talking to.

To kv331 specifically, can you expand on exactly what you did to speed up VSTGUI 3.6? I'm struggling right now with some very chunky motion. (You can see this in SCAMP 1.3 Beta 1, currently on our site. Though it's even worse in the plug-in I'm working on right now, I assume due to even larger "filmstrip" image assets.) My symptom is knob motion will be smooth for fractions of a second, then seem to seize up for other fractions of a second, making for jerky, very annoying lags in the movement. It's frustrating, as obviously if it can keep up during the smooth part of the motion, it should be able to all the time.

Sorry, but your explanation of "BitBlt/AlphaBlend functions found in Win32" is a little vague. Exactly what function should be replaced, with what, where?

And no, I'm not doing or desiring any scaling.

This is in Windows of course (running on Win 7), Mac works fine. And yes, GDIPLUS is enabled. (If I disable it I don't get any images displaying at all. Regardless of whether I USE_LIBPNG or not.)

Would really appreciate if you can recall or look up what you did. I really don't want to use VSTGUI 4, I don't like it at all and have enough work to do just adapting my old VSTGUI 3.0 code to 3.6. If I go to 4 I'll be starting over again from scratch.

Thanks!

[aciddose]

He means you need to convert the bitmap data into a format compatible with a HBITMAP handle and pass that to the standard GDI functions BitBlit and AlphaBlend.

Then he mentions the only issue might be if you want to scale images that the old GDI functions do not provide the anti-aliasing and interpolation capabilities of the GDI+ functions.

This might help, my gdi-to-abstract-interface wrapper:
(sorry if certain things don't make sense, this is cut directly out of a complex library that has many additional functions and the weird shit going on here is probably to support all that other stuff.)

Code: Select all

void gdi::Init(void *handle, coord X, coord Y)
{
	BITMAPINFO info;

	memset(&info, 0, sizeof(BITMAPINFO));

	info.bmiHeader.biSize = sizeof(BITMAPINFOHEADER);
	info.bmiHeader.biWidth = width();
	info.bmiHeader.biHeight = height();
	info.bmiHeader.biPlanes = 1;
	info.bmiHeader.biBitCount = 32; 
	info.bmiHeader.biCompression = BI_RGB;
	info.bmiHeader.biSizeImage = info.bmiHeader.biWidth * info.bmiHeader.biHeight * (info.bmiHeader.biBitCount/8) * info.bmiHeader.biPlanes;

	bitmap = CreateDIBSection(0, &info, DIB_RGB_COLORS, &surface, 0, 0); 
	check_for_error(!bitmap);

	destination = GetDC((HWND)hWnd);
	check_for_error(!destination);

	source = CreateCompatibleDC((HDC)destination);
	check_for_error(!source);

	if (GetWindowLong((HWND)hWnd, GWL_EXSTYLE) & WS_EX_LAYERED)
	{
		pm_bitmap = CreateDIBSection(0, &info, DIB_RGB_COLORS, &pm_surface, 0, 0); 
		check_for_error(!pm_bitmap);
		oldobj = SelectObject((HDC)source, (HBITMAP)pm_bitmap);
	} else
	{
		oldobj = SelectObject((HDC)source, (HBITMAP)bitmap);
	}
}

void gdi::Update(coord left, coord top, coord right, coord bottom, bool sync)
{
	if (GetWindowLong((HWND)hWnd, GWL_EXSTYLE) & WS_EX_LAYERED)
	{
		premul(0, 0, width()-1, height()-1);

		RECT WndRect;
		GetWindowRect((HWND)hWnd, &WndRect);
		int w = WndRect.right - WndRect.left;
		int h = WndRect.bottom - WndRect.top;

		POINT ptDst = {WndRect.left, WndRect.top};
		POINT ptSrc = {0, 0};
		SIZE WndSize = {w, h};
		BLENDFUNCTION blendPixelFunction= { AC_SRC_OVER, 0, 255, AC_SRC_ALPHA };

		int err = UpdateLayeredWindow((HWND)hWnd, (HDC)destination, &ptDst, &WndSize, (HDC)source, &ptSrc, 0, &blendPixelFunction, ULW_ALPHA);
		check_for_error(err == 0);
	} else
	{
		int w = right - left + 1;
		int h = bottom - top + 1;
		int err = BitBlt((HDC)destination, left, top, w, h, (HDC)source, left, top, SRCCOPY);
		check_for_error(err == 0);
	}
}

That is pretty much it. So you just need to replace the draw() functions for CBitmap to use this sort of code rather than whatever GDI+ stuff it was using.

[AdmiralQuality]

He means you need to convert the bitmap data into a format compatible with a HBITMAP handle and pass that to the standard GDI functions BitBlit and AlphaBlend.

Then he mentions the only issue might be if you want to scale images that the old GDI functions do not provide the anti-aliasing and interpolation capabilities of the GDI+ functions.

This might help, my gdi-to-abstract-interface wrapper:
(sorry if certain things don't make sense, this is cut directly out of a complex library that has many additional functions and the weird shit going on here is probably to support all that other stuff.)

Code: Select all

void gdi::Init(void *handle, coord X, coord Y)
{
	BITMAPINFO info;

	memset(&info, 0, sizeof(BITMAPINFO));

	info.bmiHeader.biSize = sizeof(BITMAPINFOHEADER);
	info.bmiHeader.biWidth = width();
	info.bmiHeader.biHeight = height();
	info.bmiHeader.biPlanes = 1;
	info.bmiHeader.biBitCount = 32; 
	info.bmiHeader.biCompression = BI_RGB;
	info.bmiHeader.biSizeImage = info.bmiHeader.biWidth * info.bmiHeader.biHeight * (info.bmiHeader.biBitCount/8) * info.bmiHeader.biPlanes;

	bitmap = CreateDIBSection(0, &info, DIB_RGB_COLORS, &surface, 0, 0); 
	check_for_error(!bitmap);

	destination = GetDC((HWND)hWnd);
	check_for_error(!destination);

	source = CreateCompatibleDC((HDC)destination);
	check_for_error(!source);

	if (GetWindowLong((HWND)hWnd, GWL_EXSTYLE) & WS_EX_LAYERED)
	{
		pm_bitmap = CreateDIBSection(0, &info, DIB_RGB_COLORS, &pm_surface, 0, 0); 
		check_for_error(!pm_bitmap);
		oldobj = SelectObject((HDC)source, (HBITMAP)pm_bitmap);
	} else
	{
		oldobj = SelectObject((HDC)source, (HBITMAP)bitmap);
	}
}

void gdi::Update(coord left, coord top, coord right, coord bottom, bool sync)
{
	if (GetWindowLong((HWND)hWnd, GWL_EXSTYLE) & WS_EX_LAYERED)
	{
		premul(0, 0, width()-1, height()-1);

		RECT WndRect;
		GetWindowRect((HWND)hWnd, &WndRect);
		int w = WndRect.right - WndRect.left;
		int h = WndRect.bottom - WndRect.top;

		POINT ptDst = {WndRect.left, WndRect.top};
		POINT ptSrc = {0, 0};
		SIZE WndSize = {w, h};
		BLENDFUNCTION blendPixelFunction= { AC_SRC_OVER, 0, 255, AC_SRC_ALPHA };

		int err = UpdateLayeredWindow((HWND)hWnd, (HDC)destination, &ptDst, &WndSize, (HDC)source, &ptSrc, 0, &blendPixelFunction, ULW_ALPHA);
		check_for_error(err == 0);
	} else
	{
		int w = right - left + 1;
		int h = bottom - top + 1;
		int err = BitBlt((HDC)destination, left, top, w, h, (HDC)source, left, top, SRCCOPY);
		check_for_error(err == 0);
	}
}

That is pretty much it. So you just need to replace the draw() functions for CBitmap to use this sort of code rather than whatever GDI+ stuff it was using.

I knew I'd be able to count on you for yet another opaque and unhelpful answer.

What, convert the bitmaps at load time or every time I draw them (which would seem wasteful)? If at load time, what other repercussions will that have on the rest of the VSTGUI 3.6 system?

Sorry, but I was looking for specifics. Like the line numbers to change. Or at least the function names where changes will be needed. (Like I asked kv331, tell me what you did, not how you did it.)

Man, if I wanted to write this raw stuff I wouldn't be using a GUI framework at all!

Another form of the question... There's this GDIPLUS macro. It seems it's designed to be disabled...

Code: Select all

#if WINDOWS
 #define _WIN32_WINNT 0x0501
 #ifndef GDIPLUS
 #define GDIPLUS		1	// by default we use GDIPlus
 #endif
#endif

That's the only place it's defined. So the only other place it could be defined as 0 (thus skipping this definition) is in one of my header files or a preprocessor definition. And disabling it should also force DYNAMICALPHABLEND on...

Code: Select all

#define DYNAMICALPHABLEND   !GDIPLUS

...which makes it explicitly find an alpha blending function from msimg32.dll...

Code: Select all

#if DYNAMICALPHABLEND
	pfnAlphaBlend = 0;
	pfnTransparentBlt = 0;

	hInstMsimg32dll = LoadLibrary (TEXT("msimg32.dll"));
	if (hInstMsimg32dll)
	{
		pfnAlphaBlend = (PFNALPHABLEND)GetProcAddress (hInstMsimg32dll, "AlphaBlend");

		// Is this win NT or better?
		if (gSystemVersion.dwPlatformId >= VER_PLATFORM_WIN32_NT)
		{
			// Yes, then TransparentBlt doesn't have the memory-leak and can be safely used
			pfnTransparentBlt = (PFNTRANSPARENTBLT)GetProcAddress (hInstMsimg32dll, "TransparentBlt");
		}
	}
	#endif	// DYNAMICALPHABLEND
    
#endif

But I can't get any images to show if I set GDIPLUS=0 in the preproc defs. (This is true whether I set USE_LIBPNG=1 or not. And yes, I'm including the libpng and zlib sources.)

Is this just not completely filled in, or am I doing something wrong?

I would like to just make this work, not rewrite VSTGUI 3.6 (RC2 as found here http://sourceforge.net/projects/vstgui/ ... GUI%203.6/) from the ground up. The names of these definitions are making me think that this functionality is already in there. But again, maybe it's not completely filled in?

[AdmiralQuality]

AH!!! I'm on the case now...

I see why my resources weren't loading with GDIPLUS=0. I need to go back to loading them by resource number rather than by name string. Once I did that the background loaded fine.

Working on that... will let you know how it turns out.

[AdmiralQuality]

That did it!

Here's what I did...

1. Make sure you start with the latest/greatest VSTGUI 3.6 RC2. Found here: http://sourceforge.net/projects/vstgui/ ... GUI%203.6/
2. Set GDIPLUS=0 in the Windows projects' preprocessor definitions.
3. Set USE_LIBPNG=1 in the Windows projects' preprocessor definitions.
4. Include source for libpng and zlib in project. (Which I used to have anyway.)
5. Go back to the "number PNG "path_to_file.png"" format for the image resources in your project.rc file. (I had switched to the "symbol DATA "path_to_file.png"" format that VSTGUI 3.6 supports.)
6. Make sure your resource.h file defines these symbols (I had stopped using it entirely since switching to the new method).
7. In your Editor.cpp file, change all your image loads to "bmpWhatever = new CBitmap (SYMBOL);", using the symbols in your resource.h file.

My knobs are now running super smooth! So, yes, this already is in the latest/greatest VSTGUI 3.6 RC2. You don't need to add or change anything to the VSTGUI code. (That probably wasn't the case when kv333 started this thread, several years ago.)

Yay!