Based on a quick eyeball of the frequency peaks and the description of the transmitters on a hill across the river, I'd guess you live in the NCR? I do as well, but much further south and so further away from most of the transmitters. My lab is also in the basement, which helps. I have never experienced an issue with noise as bad as yours.
The FM peaks are certainly a smoking gun. There is no question why you pick them up; any conductor will pick up RF radiation to some extent. The spring is very short compared to the ground lead, so it picks up less. Being so close to multi-kW transmitters is certainly a mixed blessing; great for picking up clear radio or TV stations, but bad otherwise. Try holding the ground lead parallel to the probe rather than letting it form a big loop. You could try adding a ferrite on the scope probe lead very near the probe (not on the ground lead!!, but on the coax from probe to scope). As before, this may help or hinder, but is easy to try.
Reducing the sampling speed does not eliminate high frequency noise, as you have found. The scope does not apply a low-pass (anti-aliasing) filter to the signal, and the sample-and-hold prior to the ADC is very fast. It will capture the instantaneous voltage of the waveform at the sample time regardless of the sampling period. This means all HF noise is still there, just aliased to a different frequency.
As for other mitigations, you could try the scope's built in filters: low pass or band stop. In either case, the sampling frequency will need to be high enough that the noise at 87 - 107 MHz appears at that frequency and is not aliased to a lower frequency. So sample rate will generally need to be very high. Filters are effective with single-shot or repetitive captures, as long as you have lots of samples to spare (high sampling rate). Obviously, it will reduce HF content of the signal in addition to noise.
Since the FM radio noise is not coherent with the input signal, using averaging can help dramatically, even with a small number of samples like 4 or 8. This requires a repetitive signal of course, but when that is what you are observing, I recommend averaging at a low rate (4/8) in general. Averaging helps to enhance resolution and clean up noise so you can see the real shape of the signal under the unavoidable noise, and it does not reduce HF signal content like a filter (or eres mode) does.
Finally, if you have a quality USB-C PD charger (like from a laptop or chromebook), try powering the scope from it, to see if some of the noise is actually conducted from the power supply.