DMM Noise comparison testing project

[branadic]

I was told by someone that Plotter and Alavar seem to show different results in Allan Variance. Andreas, have you performed a conformance test?

http://www.ulrich-bangert.de/Plotter.zip
http://www.alamath.com/alavar/

[Andreas]

Hello,

I was told by someone that Plotter and Alavar seem to show different results in Allan Variance.

Source?
Significance?

Andreas, have you performed a conformance test?

Why should I?
Do you make conformance tests with every standard tool  that you use?

With best regards

Andreas

[TiN]

More data for volt-nut tinkering: 3458A + A10 preamp, with copper wire short at input. Gain = 100K (hp3458 data column already multiplied, reading in nanovolts). NPLC 10, fixed correction offset -4.5nV. Peak-peak noise ~0.4nV.

K2001 column - unprocessed raw VDC readings at 200mV range.



As one can see from data, there is no need for 3458A/2002 to get most of A10 (purple dots - K2001 samples, green dots - 3458).
Now the difficult question, how to have any DC signal to be AC coupled to preamp, so this kind of numbers are not compromised. 

[Kleinstein]

As the noise of the A10 preamp is so low, it must have a rather low input impedance and likely also quite some current noise. So it only works with a low impedance source (e.g. < 10 Ohms) - in this case AC coupling is really difficult a lower frequencies. So even with large capacitors like a few 1000 µF  the frequency range would be limited to a few 10 Hz and up. How far one might go down could be seem from the current noise, which could be seen if the A10 is used with a resistor (e.g. 1 K, low thermal EMF) instead of the dead short.

If the DC voltage to compensate for it rather small (e.g. a few mV - so still to much to use a high gain), there is an option to use an external DC servo loop - there was a thread a few weeks ago in the design section using such a scheme, though less extreme and still with AC coupling.

With so much gain of the A10 it is not a surprise to have mainly noise from the input stage and thus no need for a high resolution DMM any more.

[TiN]

Here we go, 1.2 KOhm WW Ohmite 41 resistor on preamp input, gain change to 10K.



Noise still very low, ~15nV/pk-pk, if my math is right.
Perhaps I should move this discussion into separate thread?

[Pipelie]

I tested my HP34420 couple day ago, attach the RAW data, for now just tested Channel 1.
 
replace the ZIP file

[3roomlab]

hmmm i think the csv format saved some special characters. due to conversion from chinese character?

[Alex Nikitin]

Here we go, 1.2 KOhm WW Ohmite 41 resistor on preamp input, gain change to 10K.

Noise still very low, ~15nV/pk-pk, if my math is right.
Perhaps I should move this discussion into separate thread?

What is the effective bandwidth in this case?

Cheers

Alex

[Pipelie]

hmmm i think the csv format saved some special characters. due to conversion from chinese character?

i run EZGPIB in XPMODE(English version ) to record data.
it probably happen when i open the CSV file by excel  in win7(chinese version) to change the row name "HP34420A_1" to "HP34420A" and save it.

when I open the CSV file with excel, and than click close, will pop-up a small window ask save the change or not? So,something did change.

[TiN]

Welcome to EEVBlog, Pipelie, thanks for 34420 data. I'll add it into comparison. I think it's time to make 2016 summary with all recent datalogs
So readers have CSV data files pending, feel free to upload them. I'd say deadline by December 25, to make it in finals

Alex Nikitin
Previous log was NPLC 10.
This one NPLC1.
Noise about same, but here you can see also temperature induced drift (AC was turned on in room, A10 preamp in the styrofoam box though).

[3roomlab]

anybody else with keithley want to try a base NPLC of 2 / 3 / 4 and its effects on the noise plateau (in allan variance plot?). ie : NPLC4 x 25 repeating ave to get nominal 100NPLC?

[Kleinstein]

anybody else with keithley want to try a base NPLC of 2 / 3 / 4 and its effects on the noise plateau (in allan variance plot?). ie : NPLC4 x 25 repeating ave to get nominal 100NPLC?

Doing extra averaging (e.g. 25 times 4 PLC to get a simulated 100 PLC) does not help for the Alan variance plots. The exactly same data would be in the curve with the single readings (4 PLC) - it is just a different program to do the average, and you get some extra in between point if you use the data directly.

The Keithley 2015 seems so show the same strange (Keithley typical ?) plateau. So if it supports modes like 2,3,4,5 PLC you are free to take the data (a little longer than the old curves). At least the 10 V data should be rather similar to the stock 2000 / 2015.

[Pipelie]

Thanks, TiN
just finish the test,attach the HP and Agilent 3458A data.

[Andreas]

anybody else with keithley want to try a base NPLC of 2 / 3 / 4 and its effects on the noise plateau (in allan variance plot?). ie : NPLC4 x 25 repeating ave to get nominal 100NPLC?

Doing extra averaging (e.g. 25 times 4 PLC to get a simulated 100 PLC) does not help for the Alan variance plots. The exactly same data would be in the curve with the single readings (4 PLC) - it is just a different program to do the average, and you get some extra in between point if you use the data directly.

The Keithley 2015 seems so show the same strange (Keithley typical ?) plateau. So if it supports modes like 2,3,4,5 PLC you are free to take the data (a little longer than the old curves). At least the 10 V data should be rather similar to the stock 2000 / 2015.

Hello,

mhm,
perhaps the "error" is much simpler.

Has anybody tried to analyze the resolution of the data being output?
Assume they are using single precision floating point arithmethic on the DMMs with 24 Bit resolution mantissa....
In this case it makes a difference if you calculate the averaging on DMM or on PC (with double precision).

When I have a look at Blackdogs HP34461A noise data in 10V-range then suddenly at 10mV the resolution seems to change ...

with best regards

Andreas

[maxwell3e10]

Interesting looking data... Could it have something to do with writing ascii data with certain number of digits, so one digit is lost for  V>10mv?

The digital resolution of DMM7510 is shown here https://www.eevblog.com/forum/metrology/dmm-adc-noise-comparison-testing-project/msg1081650/#msg1081650
It is about 0.0014 ppm (I zoomed in, but the histogram looks gaussian on larger scale).

I noted that comparison of 10V and 1V noise suggests a certain threshold behavior at around 0.03ppm where the AZ starts to work: https://www.eevblog.com/forum/metrology/dmm-adc-noise-comparison-testing-project/msg1083353/#msg1083353
One possibility is that the zero value is calculated with lower digital precision. So, on 10V scale, the AZ adjustments are below digital resolution of zero measurements and no correction is being made. On 1V scale larger amplifier noise causes more bit noise in the zero value and digital steps are  averaged out better due to dither. It would be at about 26 bits. From the timing, it looks like zero measurements take about the same time as real measurements (same PLC) so in principle there is no reason for it to have lower resolution, unless, again, it is a software bug.

[Andreas]

The digital resolution of DMM7510 is shown here https://www.eevblog.com/forum/metrology/dmm-adc-noise-comparison-testing-project/msg1081650/#msg1081650
It is about 0.0014 ppm (I zoomed in, but the histogram looks gaussian on larger scale).

Hello,

I do not remember if it was a zero measurement or a 10V offseted measurement. (which is more likely).

So what happens with your resolution if the values are slightly above 10V?

With best regards

Andreas

[3roomlab]

Has anybody tried to analyze the resolution of the data being output?
Assume they are using single precision floating point arithmethic on the DMMs with 24 Bit resolution mantissa....
In this case it makes a difference if you calculate the averaging on DMM or on PC (with double precision).

When I have a look at Blackdogs HP34461A noise data in 10V-range then suddenly at 10mV the resolution seems to change ...

with best regards

Andreas

from the K7510 plot, it would seem the resolution if at 1 NPLC would be 166.67nV (i counted 150nV over 9stops)?. in my case (K2015, see pic) it is about 500nV. about 3x difference. i think i tried to understand this "problem" before in the K2015 noob mod thread, but i didnt understand the noise vs plateau problem. my thinking of the plateau problem currently is due to the way the NPLC timing noise is a fixed frequency interference to the speed of sampling, but then again this is where i do not know how to continue to decipher. i think i saw lymex did said something about allan variance noise in his 38hot.bbs thread, maybe he has some input on this subject.
in the alavar plot it is hard to tell, so i m not sure what to make of it?

[Kleinstein]

I don't think the noise plateau for the Keithleys is due to limited resolution. If this would be a problem one would see it in the direct plots, but the histogram plots are really spread out. My best guess so far is that the software somehow is not using the zero readings directly, even at 10 PLC. The first gues would be that they use a kind of averaging for the zero readings and maybe even some extrapolation. This could explain the kind of extra low frequency noise even with AZ active. However somehow they missed out the advantage of this: At 10 PLC and short time (0.5 s) the noise with AZ active is still about twice that of running without AZ. This is the same ratio one would expect with simple difference calculation (average of 2 readings compared to difference of 2 readings). Averaging over longer time would have got less noise (e.g. 1.5 times compared to non AZ mode). So maybe a not so clever Version of extrapolation is used.

The extra noise is just in an time / frequency range that if often used to do things like manual polarity switching. It is just the range where the HP meters are at there best. With even the 34465 beating the DMM7510 or 2002.

[maxwell3e10]

I agree that simple digital resolution in the zero measurements would introduce some anomalies in the histogram. I had tried looking for various spikes in the histogram of the differences, but didn't find anything there. On the other hand, it is striking that the Allan variance with AZ starts to go down just when the noise without autozero exceeds about 0.03 ppm, on both 1V and 10v scale. So there appears to be  a threshold behavior, but with some additional processing.
What if they use some version of "Filter window" as described in their manual. If Autozero values are close to each other, they are averaged together, if they deviate by more than a threshold, the average is discarded. This can perhaps explain that when using triggered measurements it works better, if the trigger resets the autozero average.

[maxwell3e10]

Digital resolution: I hooked up both DMM7510 and 34465 to an old lead battery that gives about 10.24 V. It seems reasonably stable. Here is the resolution that the meters write to USB on 10V scale with no relative offset:

DMM7510: 10.24113281872
34465A:    10.2408757   (ignore the offset, the meter hasn't warmed up yet)

For 34465A the resolution is 0.1 uV, which is just adequate since the noise is 0.4uV on 10PLC 10V scale.

I am going to leave them record the battery for a while, see what the noise looks like.

[dvdput]

Hi, long time lurker here. I thought it was time I added some small contribution.

Attached are 3 files of 10.000 readings from a Fluke 8846A. (10.000 readings is the max I can automatically store on a USB stick).
0001 is at 10V, 0002 is at 1V and 0003 is at 100mV. All 3 files were measured with 10NPLC, no digital or analog filter.

I hope that it is useful, let me know if other readings should be added.

Cheers, David

[maxwell3e10]

Here is the comparison of DMM7510 and 34465A for finite input. I used a 10.24 V 7Ah (almost dead) lead battery from a UPS supply and a 3.13 V (primary, moderately used) Li CR123A battery from a wireless alarm sensor. For lead battery, I removed linear self-discharge trend. For Li battery, just found a quiet temperature spot.

As might be expected, at finite voltage 34465A does not perform as well, probably due to limitation of the voltage reference. It would be best to compare 34470A to DMM7510 in this case.

@dvdput: Thanks for posting the data! Looking at the file on 10V scale the digital resolution is 3 uV. This is much worse than 34465A. If there is no way to change this setting it points to a clear superiority of 34465A at a smaller list price. It would be nice to add some of this information to the DMM comparison table, so people can decide which meter to buy with  information that is often not stated on the datasheets.

[Pipelie]

Here is the test result of Channel 2, it's better than channel 1.
Shorting Block: LEMO FFA 1S ,BUT NOT manufacture by LEMO.

[branadic]

Hello,

I was told by someone that Plotter and Alavar seem to show different results in Allan Variance.

Source?
Significance?

Andreas, have you performed a conformance test?

Why should I?
Do you make conformance tests with every standard tool  that you use?

With best regards

Andreas

Hello Andreas,

I was told the results differ by 10e-1 between Plotter and Avar, while Avar seems to deliver the same results as Stable32. I haven't checked myself so far, but there are test data available to prove the results you get are okay. I'm sure you know this website: http://www.wriley.com/paper1ht.htm
and the linked Handbook of Frequency Stability Analysis: http://tf.nist.gov/timefreq/general/pdf/2220.pdf?
You can find a set of test data on this website to check if the results are matching what you expect or not:

The classic set of nine 3-digit numbers from Annex 8.E of NBS Monograph 140: http://www.wriley.com/nbs.dat
A test suite of 1000 pseudo-random frequency data points: http://www.wriley.com/tst_suit.dat

Furthermore, the handbook describes the different plot methodes to analyse the behaviour of your DUT.

[Jorn]

For greater confidence in Allan Variance measurements I decided to perform some experiments whose results I will share with you below.

All tests are carried out in Python with the package allantools found at https://github.com/aewallin/allantools/

Overlapping Allan Deviation is chosen so that the results can be compared with other measurements in this thread.

First test is carried out with the 9-point test set from http://www.ieee-uffc.org/frequency-control/learning-riley.asp (ieee-uffc)
The 9-point test set is normalised by subtracting the mean from each sample. The calculated result for tau=1 and tau=2 is identical to the values listed in table II(ieee-uffc). Here follows the code:
 


The algorithm for a 1000 point pseudo random integer sequence from section "Test Data" in (ieee-uffc) was implemented and executed. The results for tau=1, tau=10 and and tau=100 are identical to the values listed in table III(ieee-uffc). Here follows the code:



Confidence in the allantools python library, and its use, is hereby present.

Moving on to the data file from Dr. Frank: Ltz5_tc_9.txt (find it earlier in this thread and remember to remove "zero" samples)



As can be seen from the plot it is almost identical to Andreas plot from reply #429. Only difference is that Andreas uses tau0=1 as time unit and thus you have to know the sample spacing in order to interpret the graph. I prefer to use rate=0.25 corresponding to tau0=4 seconds (2s acquire (100 NPLC) + 2s auto zero) between samples.

However confidence in the tool (afair DF6JB Plotter) that Andreas uses for allan deviation calculation and graphing is present. 

@brandic: hope you agree, no factor 10 error here :-)

Allan deviation plots of my own reference follows in part II

-Jorn