If the goal is only to press sort and see what comes up, just select save 10-50 times for the same crystal and try it. If the goal is to actually measure crystals, the plot you show doesn't look right. You can waste a lot of time taking bad data. I suggest your time is better spent making sure things are correct before going further.
First thing are those spikes in the frequency. There is no way the crystal should shift frequency like that. It wouldn't recover that fast. Something else is going on, maybe noise. Assuming the cables are tight and you're not touching anything, maybe it's your Nano. Some versions of firmware would show bad readings at random. That's part of the reason for the regression testing. I haven't seen this problem for some time. It should be obvious that running the same firmware removes a lot of questions and may save you a lot of time.
Start by selecting the Xmsn Rectangular from the Main menu. With your short installed in place of the crystal, it should read well within 0.1dB after proper calibration (Capture1.PNG). Make sure you understand the difference between the Nano's internal calibration and the software's. Post a screen shot.
Assuming that the gain and noise look good, I would select Record and let it run for 10 minutes or more. Press Record again to turn off the recording. Select Advanced, Surface, Add Waveform and then load the file you saved the data to. Next select Narrow Band Zoom and select S21 Mag. You should have << 0.1dB noise P-P. Notice mine after touching the setup takes a dive but is still within 0.045dB P-P (Capture2.PNG). Post a screen shot.
Assuming you are not seeing a problem up to this point, select the Main menu (you should still be seeing << 0.1dB) and swap out the short for one of your 10MHz crystals. You should see a VERY smooth curve (Capture3.PNG). The peak is the Attenuation. You show 20dB which seems very low. Post a screen shot.
Select Record and save the data to a new file. Let it run for another 10 minutes or more. Don't touch anything and stay away from the fixture. If you have a lot of air flow in the room, my standard practice it the beach towel. It's too bad that the Nano doesn't have an ambient temperature sensor. People would rather have a worthless battery indicator. I would make a note of the temperature if you plan to try and reproduce the test or are collecting the data over long periods of time. 100ppm/degC with even a 2 degree shift and a 10MHz is easily detected.
Select Record again to stop the recording. Select Surface and select Flush. Now select Add Waveform. Looking at both S21 Mag/Phase, it better be very clean. (Capture4 & 5)
If all of that seems good, I have no idea what those spikes are. If a second crystals shows the same problem and you are running identical firmware, it seems there is a problem with the Nano, cables, not tightening things, external noise source.... You need to figure it out before wasting time measuring parts.
If you find the problem and get similar data, I would next validate that the C0 measurement is working. If you don't have a good RLC meter, I suggest installing a known capacitor. Maybe a few values from 10pF to 100pF. Depending what you have for parts, their tolerances can be pretty wide. Just make sure that the numbers are in the ballpark and you should be fine. If you insert a 100pF capacitor and it reads 50pF, something is obviously wrong. Try another capacitor. This shouldn't be a problem based on the data you posted for your jig. Again, that assumes you have properly calibrated the setup.
***
Very poor grammar for even me.