It's VERY possible the problem is in my software, which is why I wanted to use yours to collect the data. Best to divide these problems in half.
I am not fitting the data. Rather I simply use linear interpolation.
Corrected previous statement, there is no fitter Using this, with a 1GHz full scale signal applied, I measure -36.6dB for THD. The published measured value was -37.65 +/-12dB. Being the lazy person I am, I called it a day.
Let me try adding a gaussian fit like was first mentioned and see if it improves things.
***
A new day.
From the manual:
you can recover the signal voltage V0 by interpolating between the two neighboring voltages where F(V ;Δt) first transitions from below 1/2 to above 1/2.
...
2. If two or more voltages in the CDF are equal, sort the corresponding CDF values (for that voltage value) from lowest to highest. If the interpolation is done in an incorrect order, a large apparent spike may result (example 2, attached).
These considerations do not apply if you directly fit an error function to the CDF data.
We just discovered effect (2) recently, so we may send you an updated firmware file tomorrow that implements this sorting for you.
...
Do you always use a Gaussian fit to get the centroid?
A bit complicated, but details if you're interested: to a first approximation, you can use the interpolation procedure we described in the last post. This usually gets you close, but the noise performance is poor since you're only using two samples.
Fitting a Gaussian error function is theoretically the best thing to do (in the sense of Fisher information), but in practice is overly sensitive to outliers on the wings. What we've found works well in practice is to fit a line to the inverse Gaussian CDF applied to the data, but truncated within a certain region (say 10-90%) of the CDF, with the truncation also seeded by the interpolation method. (This is also faster on the CPU.) You need to weight samples with some care to maintain equal sensitivity to all values (which minimizes the total propagated noise, and improves outlier robustness).
This gives a significantly lower noise floor than the interpolation method alone, but is still robust to spurious errors.
Attached plot shows a typical CDF values (WHITE) that appear out of order. The GigaWave has all the recent patches. If I don't sort them, and weigh CDF values that are outside of the 5-95% at 0 (no effect), all others inside the window weighted at 1, then run a linear fit (RED). Then interpolate the fitted data to find the centroid index. Use that index and linear interpolation to determine the voltage. The two numbers are close. Also shown using 10/90% window, same weighting, with sorting CDF.
First, do I always need to sort the CDF values? Is there a problem that I am seeing what appears to be out of order data?
With this calculation being crucial, it seems that for anyone wanting to reproduce the result obtained using your software, your conversion needs to be well documented.