Thanks TheoB -- Good idea! I'm a litttle bit short on SMA / BNC adapters but I managed to route a MiniCircuits 15dB directional coupler into the RF path which now looks like this:
SSA3021X TG -> Coupler IN
Coupler Coupled OUT -> SSA3021X SA IN
Coupler OUT -> BNC T on DS1054Z CH1 with 50 Ohm terminator on the free BNC T port
With this setup, the results look much more reasonable but still impressive: I find the 3dB point of the DS1054Z to be at 195MHz. By swapping the BNC end of my SMA -> BNC adapter cable between the BNC T at the o'scope input and the input of a second SA (Rigol DSA815TG), I verified the level was stable within +- 0.2dB. So this should actually be a pretty reliable test I guess. The only factor that could still lead to too high figures is the ancient BNC 50 Ohms terminator that I used on the BNC T (haven't got a BNC feed-through terminator, should put it on my shopping list...).
Okay, just checked the BNC terminator (with cable but less the scope input) with a return loss bridge to provide at least 25dB of reflection damping at 200MHz, so this should be good enough for this test.
Cheers,
Thomas