What you see in EEVblog #441 is that Dave disables the output (the classical way is to connect the probe tip to ground at the DUT). That does not require any extra gear or investment. Same with switching off the power supply. Do you still see noise with any of the methods?
Is the power supply grounded? In that case ground loop currents could increase the apparent noise. In that case, using a differential set up would be preferable.
#1 (assuming you use a grounding clip) should work best with a powers supply where both rails have a high impedance to ground (think battery). Common mode signals should not be an issue, and it keeps the loop area between probe tip and ground as small as possible.
#2 As for termination, I don't agree with Dave's method of using a 50 Ohm terminator with 1x probe. It attenuates an already weak signal and screws up the frequency response (look at the probe compensation signal). I would rather ground the DUT using some other method (probe ground shield or even separate grounding wire). In that case this method should work well for DUTs where common mode is an issue (think grounded power supplies or power supplies with a capacitive coupling to ground). However, the loop area between the probes is much greater, increasing the potential for inductive pickup of external signals (e.g. lighthing). Also CH1-CH2 will have a limited common mode rejection.
#3 Should work fine with termination, but only if the output voltage is low enough as not to exceed the rating of the terminator (which might only be 0.5 W) and if the 50 Ohm termination will not load the power supply appreciably. The noise of power supply tends to be higher under load. Try adding a 50 Ohm resistor across the power rails with the other methods to see if it makes a difference. The banana to BNC adapter should be pretty good for inductive pickup (small loop area).