My guitar FX is not "high gain", it is digital (analog signal sampled at "normal" gain, digital magic is applied, converted back to analog and sent to the headphones).
I am using is as basic USB audio interface (sometimes with guitar, sometimes like USB audio card without guitar).
It could be very quiet when only headphones and USB to the PC are connected.
To prove it I did my experiment with another USB audio interface (without guitar FX capabilities) and result is the same.
I think my noise comes from my wireless receiver.
It can produce noise even if not connected to the USB charger.
Touching it (or placing hand near it) can reduce the noise.
Moving it across the desk (while connected with USB to the PC) can produce various levels of noise (from none to some).