I haven't had this with my UMC22, Traktor A6, or Scarlett 2i2 interfaces. Only this one. Which is a huge letdown for the SSL2.
In this video I'm hitting pause repeatedly to show this. At low buffer size, it makes a grainy lo-fi sound. On 2048 (maximum) buffer, it's like a stuttering 'machine gun' sound. So the buffer size isn't the cause, but it weirdly affects it.
I've tried restarting the PC, reinstalling the SSL2 Asio drivers, changing sample rates (in Ableton and Windows). Tried multiple USB ports on my PC, unplugged other devices... Using a Ryzen 9 3900X with all maximum power settings. I have no clue how to fix this