Login / Register 0 items | $0.00 New @ KVR
stratum
KVRAF
 
1837 posts since 29 May, 2012

Postby stratum; Thu Apr 12, 2018 6:19 am DirectX9 surface update not synchronized

Hi

This question would probably be more appropriate for a game development forum but, as plugins often have DirectX code for display, perhaps it's not completely off topic.

I'm having a problem with the code here:
https://gist.github.com/karlgluck/84679 ... d-9-cpp-L8

This is a part of a video rendering app as intended, and I need to do some processing on the hardware decoded image before it is displayed.
Needless to say, it would be better to do all this on the GPU, but it's not always feasible.
So I'm trying to get the image to system memory, process it and send it back to the GPU with a call like UpdateSurface ( https://msdn.microsoft.com/en-us/library/windows/desktop/bb205857(v=vs.85).aspx) after the UnlockRect above, as in

Code: Select all
      HRESULT hr = poffscreensurface->UnlockRect();
      if (copyDataBack && !FAILED(hr))
      {
         hr = pd3d9device->UpdateSurface(poffscreensurface,NULL,
               pcurrentrenderingtarget,NULL);
      }


It works, but display flickers as if there is a synchronization problem.
Any idea about why that happens?

Thanks

p.s. Removing the incorrect D3DLOCK_READONLY flag from the code in github does not fix anything.

p.s.2 Apparently I'm not alone? http://xboxforums.create.msdn.com/forum ... aspx#17202
~stratum~
mystran
KVRAF
 
4979 posts since 11 Feb, 2006, from Helsinki, Finland

Postby mystran; Thu Apr 12, 2018 10:11 am Re: DirectX9 surface update not synchronized

Using DirectX (any version really) in plugins is not a good idea as it doesn't play well when mixed with GDI (which you can't avoid, because it's up to the host), at least on Win7 (no idea about more recent). OpenGL works perfectly fine (not to mention it works on macOS too) so little reason to really bother with DX.

As for your problem.. the first thing I'd check is that you actually get the correct back buffer for each UpdateSurface call, as normally you have multiple back buffers that are used in round-robin fashion. This at least will cause very obvious flicker if you try to do something that assumes there is just one back buffer.
Image <- plugins | forum
stratum
KVRAF
 
1837 posts since 29 May, 2012

Postby stratum; Thu Apr 12, 2018 11:05 am Re: DirectX9 surface update not synchronized

mystran wrote:Using DirectX (any version really) in plugins is not a good idea as it doesn't play well when mixed with GDI (which you can't avoid, because it's up to the host), at least on Win7 (no idea about more recent). OpenGL works perfectly fine (not to mention it works on macOS too) so little reason to really bother with DX.


Thanks for your reply.
While this is not a plugin, the "host" is in this case QT5, which probably uses either OpenGL or some version of DirectX.

mystran wrote:As for your problem.. the first thing I'd check is that you actually get the correct back buffer for each UpdateSurface call, as normally you have multiple back buffers that are used in round-robin fashion. This at least will cause very obvious flicker if you try to do something that assumes there is just one back buffer.


I'm using the same back buffer for rendering text and some other geometric shapes on the decoded image and that works well without flickering. (i.e. it is the same buffer that is being set to device for painting using pdev->SetRenderTarget(0, pSurface) )
But if I use the above code to get a copy of the back buffer to the system memory, modify it and send it back using UpdateSurface, it flickers. Problem is about UpdateSurface because as the image modification I have only used color negation (255-color_value) on the resulting 4-channel image. Every pixel color including those of the geometric shapes drawn either is negated or not. Perhaps I should be careful not to modify the alpha color value, about which I haven't paid attention.

edit: I was wrong about the alpha channel idea. Modifying only the blue channel results a flickering blue surface. So I guess you are correct in some sense about the surface being the wrong one but I couldn't understand why only UpdateSurface causes flickering while the same surface is also the render target for painting.
~stratum~
User avatar
Guillaume Piolat
KVRist
 
175 posts since 21 Sep, 2015, from Grenoble

Postby Guillaume Piolat; Thu Apr 12, 2018 12:47 pm Re: DirectX9 surface update not synchronized

At least on OpenGL getting back pixels from the GPU require flushing of commands and stalls performance, it's kind of a no-no.
It's better explained that I could here: https://stackoverflow.com/questions/244 ... le_rich_qa

You can't escape this with unified memory-mapping (like what exist in say, CUDA) since the pixels will still transit through the PCI-e in reverse fashion.

What I've seen people do is manually double or triple-buffering those surfaces to get the data from 2 (resp. 3) frames before.
VST/AU/AAX: Couture | Panagement | Graillon
mystran
KVRAF
 
4979 posts since 11 Feb, 2006, from Helsinki, Finland

Postby mystran; Thu Apr 12, 2018 2:14 pm Re: DirectX9 surface update not synchronized

stratum wrote:edit: I was wrong about the alpha channel idea. Modifying only the blue channel results a flickering blue surface. So I guess you are correct in some sense about the surface being the wrong one but I couldn't understand why only UpdateSurface causes flickering while the same surface is also the render target for painting.


Hold on.. are you explicitly rendering to this specific surface or are you just rending to "screen" and assuming it's the same buffer? Because rendering to "screen" will automatically use the correct back buffer for the frame. Normally there are at least 2 buffers (ie. one back buffer and one front buffer) no matter how you create the device and unless you're using D3DSWAPEFFECT_COPY (which isn't really ever a good idea as you basically always want _DISCARD) then at every Present() call the buffers rotate. Assuming two buffers, the back buffer becomes the front buffer and the front buffer becomes the back buffer.

So.. check your code: are you fetching the pointer for the back buffer surface on per-frame basis, or are you just getting it once? If you're getting it once, then every other frame it's going to point to the wrong buffer.
Image <- plugins | forum
stratum
KVRAF
 
1837 posts since 29 May, 2012

Postby stratum; Thu Apr 12, 2018 11:27 pm Re: DirectX9 surface update not synchronized

The surface is being obtained via calls IMFSample::GetBufferByIndex and MFGetService, and the original Microsoft example (which is a custom windows media foundation presenter/renderer (what's what they call it ) ) does not even use it afterwards. It just calls IDirect3DSwapChain9::Present, noting that you can also do something "fancier". Following that advice, I have attached a method just before IDirect3DSwapChain9::Present, made that surface the current rendering target by calling IDirect3DDevice9::SetRenderTarget, and painted a few geometric shapes on the image, which worked fine. If I comment out that SetRenderTarget call, the geometric shapes are not visible. All that works fine as long as I do not try to process that data in memory via CPU.

Code: Select all
   

// Get the buffer from the sample.
CHECK_HR(hr = pSample->GetBufferByIndex(0, &pBuffer));

// Get the surface from the buffer.
CHECK_HR(hr = MFGetService(pBuffer, MR_BUFFER_SERVICE, __uuidof(IDirect3DSurface9), (void**)&pSurface));

// Get the swap chain from the surface.
CHECK_HR(hr = pSurface->GetContainer(__uuidof(IDirect3DSwapChain9), (LPVOID*)&pSwapChain));

// This calls SetRenderTarget and performs painting
MyCallbackGoesHere(pSwapChain, pSurface, m_rcDestRect, m_hwnd);

hr = pSwapChain->Present(NULL, &m_rcDestRect, m_hwnd, NULL, 0);

~stratum~
stratum
KVRAF
 
1837 posts since 29 May, 2012

Postby stratum; Tue Apr 17, 2018 6:23 am Re: DirectX9 surface update not synchronized

Copying data to the default swap chain via IDirect3DDevice9Ex::StretchRect and calling IDirect3DDevice9Ex::Present instead of IDirect3DSwapChain9::Present solved the problem.
The downside is that, now the video is always displayed in a predetermined surface size (say 1920x1080, regardless of what the actual image size is)

edit : while the flickering problem is solved, the decoding performance is now pretty bad because 1920x1080 is unnecessarily high for the test video.

edit2: Calling IDirect3DDevice9Ex::Reset with the parameters of the newly constructed swap chain solved the performance problem. :)
~stratum~

Moderator: KVR Moderators (Main)

Return to DSP and Plug-in Development