As suggested there is lots of info on these measurements, just do a diligent search and you'll find it!!
Regarding the SDG2KX as a potential source for doing the DSO IMD testing. This sort of parameter may not be directly specified (maybe indirectly) but our AWG is not bad, maybe you should test yours with a good instrument (Spectrum Analyzer). Then you'll know if the major source of the IMD is the AWG or DSO, or both. Please remember this is Root Sum Squared result, so you may need to "back out" the IMD if the source and measurement are not >10dB apart.
Here's a quick SDG2KX test we did with a SSA3021X+ operating in it's "Sweet Spot". We used a Mini-Circuits 10dB PAD (always keep this N type PAD in place to protect the SA), and set the AWG CH1 and 2 to 0dBm (50 ohm reference), and frequency CH 1MHz, Ch2 1.01MHz. AWG Wave Combine is used, with output directed to Ch1.
SA center F of 1MHz with 100KHz span, and RBW & VBW of 1Hz. Judge for yourself the results shown below.
Edit: Added a couple more plots at 100KHz and 10MHz Center Frequency.
The point here is if the AWG is measuring this level of IMD on the SA, then you "know" the AWG is at least this good a source for use with such measurements, which as mentioned should be at least 10dB better than what your intended DUT is.
As always, KTI (Know Thy Instrument) and YMMV.
Best,