I tried putting a 5 ohm resistor in series with the usb power. That didn't really help. I guess the problem is that most of the ripple is created by my own circuit, and is not coming from the outside.
Yes, sorry for suggesting this, it was nonsense
It would only help if the ripple were coming from other devices on the bus.
It is also unlikely that an oscillating regulator produces 8kHz, they usually oscillate faster. Your USB microframe theory is more plausible.
Besides, it's not that hard to find LDOs stable with any reasonable ESR.
If there are regulators on board, their output capacitors shouldn't see much ripple and produce much noise, so the problem is the input capacitor.
Since 100μF tant was still not good enough, maybe try a 10μF one rated for higher voltage. It may or may not be quieter. Lower capacity will not violate the USB spec. Increased voltage rating may increase its reliability. Pay particular attention to a transient voltage peak which occurs after the device is connected and all capacitors have just charged up but cable inductance keeps pushing more current into them. This can destroy underrated voltage regulators or capacitors.