1. Install/place your VST in your plugins folder
2. Start up Audition & Click Refresh effects list
3. Start the Windows Task Manager (right click taskbar or hit ctrl-alt-del)
4. Click the Processes tab, then Image Name (to sort by name)
5. Find the Audition.exe process
6. Close Audition normally (the X in corner of window).
7. Look at the Task Manager, if you have a "bad" VST the Audition.exe process will not close (end, die, disapear).
8. Note: If Audition is hanging then select it in the Task Manager and click End Process
I have found (in general) that VSTs created from SynthEdit will cause Adobe to hang like this (and also tend to render Audition a lil more unstable).
For a quick verification on this try loading Baxxpander (this fails for me).
Also since I know hink will respond to this I want to totally thank him for the silent tracks multitracking tip (I just found it today, wow cool)