DMM Noise comparison testing project

[bingo600]

@maxwell3e10

Any chance you could share some info (scripts , data , formats) for using that gfx package ?

I just built it on my linux machine, but would like some "jump starting"

/Bingo

[maxwell3e10]

Its a good idea to post on TEK/Keithley forum, do you mean here? https://forum.tek.com/viewforum.php?f=363
Seems like it would be in their interest to fix it, as it could make DMM7510 an all-around great meter that can maybe compete with HP3458.

I was surprised that autozero off noise is not actually that bad on 1-20 sec timescale, so its not like they need to work hard to cancel it.

I am attaching the Genplot script for calculating Allan variance. It can be a bit slow for >10^5 points because I didn't bother making the step size variable.

[dr.diesel]

Its a good idea to post on TEK/Keithley forum, do you mean here? https://forum.tek.com/viewforum.php?f=363

Done,   

https://forum.tek.com/viewtopic.php?f=363&t=139083#p281815

[maxwell3e10]

Here is a comparison of autozero on and off for 10V range and 1V range.
On 1V range, autozero functions better in removing the drift, which is primarily caused by input amplifier.

It is interesting to note that at say 20 sec time scale, the autozero fractional Allan variance is better on 1V scale, about 0.015 ppm, than on 10V scale, about 0.03ppm. Here is one hypothesis: maybe they set a certain limit (about 0.03 ppm=10^(-7.5)), below which no autozero adjustments are made. In this way on 1V scale, having larger input amplifier noise forces autozero adjustments, while on 10V scale, there are no autozero adjustments until the voltage drifts enough.

[3roomlab]

i want to try to understand allan deviation for my K2015, but i think i got it all wrong, or a K2015 ADC is too noisy?
the data is in the zip. it is last years data (3600 samples over 2100 seconds)
col A = time
col B = AVEDEV calculated
col J = data in nV (after zero)
col K = raw data

the plot is AVEDEV() vs time, which i am guessing is the opencalc equiv version of allan deviation (10v NPLC20 AZ=on)

can anyone familiar take a look at the equation inside the xls and provide a hint?

edit : the NPLC 10 version is in the "1158" zip and as seen in pic 2.gif. (there is some disturbance after 100th second). but overall, it shares trend but in a quirky way lol

in a nutshell is the AVEDEV() equation the correct method?

[maxwell3e10]

The key for Allan deviation is to take difference between successive points. So, first point of the Allan deviation plot is standard deviation of the difference between neighboring points. For next point you average 2 neighboring points together and calculate standard deviation of the difference of the pairs, and so on.

[3roomlab]

lotsa cartoon question marks going off above my head.

so say at the 20th sample, the allan deviation = STDEV of samples number 19 to 21? only 3 samples?
this appears to be the VAR AVEDEV function in opencalc :
https://wiki.openoffice.org/wiki/Documentation/How_Tos/Calc:_VAR_function

in my case example =
VAR AVEDEV(S19; S20; S21)

like that?

[maxwell3e10]

No, 20th point involves averaging data in groups of 21 points, taking difference between successive groups, and calculating standard deviation of that.

[bingo600]

Had a look @Genplot , looks fairly complex.
But also somewhat like gnuplot.

Tried an example from the net, but it complained about missing device, not being able to open the Xdriver  and other stuff.
And when it "coredumped" on overriding these errors , i decided to stick w. gnuplot.


/Bingo

[3roomlab]

No, 20th point involves averaging data in groups of 21 points, taking difference between successive groups, and calculating standard deviation of that.

hmmm ok, i replenished some sugar to make the brain work (or not?). i assume this time my diagram is correct? so the number of differences is (n-1)? and the result being plotted is a progression of the increasing number of differences being included WRT time?

so (rho)^2 = sum [ ((S1 - S2)^2 ) / (n -1) ] ? assuming S1 and S2 are inside n number of samples, but i stumbled on an online pdf which says the divider is  =2* (n-1), which makes (rho)^2 = sum [ ((S1 - S2)^2 ) / (2n -2) ] , the correct version is the later yes? (is there a simple way to know why the additional 2?)

http://www.measurement.sk/PAPERS/Siraya.pdf

**update
if i am understanding allan variance right, which in open calc can be expressed as =(DEVSQ(D1:D2)/(2*n-2))^0.5 ... with 30 samples in the xls file, column k = original data, J = difference, I = AVRA. again, i would appreciate if anyone could tell me if this is the correct method.
however on the flip side, i dont understand why the plot should go lower with AZ=on, shouldnt the overall noise be more or less be a constant over time?

[maxwell3e10]

I don't know how to calculate Allan variance in Excel, but with a little googling I found this stand-alone program http://www.alamath.com/alavar/
It just needs one column of uniformly sampled data in txt format.

[maxwell3e10]

Results from a K2010, in cal, Keithley 4-term shorting block. 

Range 2 on a k2010 is 10v, FYI.

I've been poking around here and found that Dr.Diesel data for the Keithley 2010 DMM look promising. The rms noise on 10V, 10PCL is only 0.5 muV and the Allan variance looks good. But the data were only recorded at 5 sec intervals. Could you take more data at faster polling rate and maybe also lower voltage ranges?

[dr.diesel]

Could you take more data at faster polling rate and maybe also lower voltage ranges?

You bet!  I got a full house today, but will run the full gambit in the next day or two.

[dr.diesel]

Could you take more data at faster polling rate and maybe also lower voltage ranges?

You bet!  I got a full house today, but will run the full gambit in the next day or two.

NPLC 10, 5, 1, 0.1 at ranges 10, 1, 100mV, 1 sec polling:

ftp://xdevs.com/k2010/

[3roomlab]

i am still obnoxiously trying to understand the equation behind the Allan variance, but still, the neurons dont work ...
but using the program indicated by maxwell
this is the plot (from year 2015 log no 1251). how should i "look" at this to understand my DMM? the Y scale should be 100nV. esp the "knee" ?
(im glad i have kept the huge folder, and have documented every log file)

**edit/updated**
pic 3 = 20NPLC, AZ = on, 10v sampling interval approx 1.2 ~ 1.3s (running from RS232) log no 1251 (dec 22nd) modded with new damage
pic 4 = 10NPLC, AZ = on, 10v sampling interval approx 0.8 ~ 0.9s (running from RS232) log no 1158 (dec 22nd) modded with new damage
pic 5 = 10NPLC, AZ = on, 10v sampling interval approx 1.0s (running from RS232) modded w/o damage, log no 0125 (nov 12th)
pic 6 = 10NPLC, AZ = on, 1v sampling interval approx 0.67s (running from RS232) modded w/o damage, log no 1208 (nov 03rd)
pic 7 = 8 NPLC, AZ = on, 1v sampling interval approx 0.5s (running from RS232) modded with new damage. 2nd jan 2016 (only 200 samples)
as i try to recall what i done to the DMM, and the 2nd accidental damage (on about 12dec?).


should the error limiters be flicker FM  or PM? these are all FM.

[maxwell3e10]

You bet!  I got a full house today, but will run the full gambit in the next day or two.
NPLC 10, 5, 1, 0.1 at ranges 10, 1, 100mV, 1 sec polling:
ftp://xdevs.com/k2010/

Thanks. It seems to have somewhat similar behavior, faster polling does not lead to less noise on a given time scale. Overall, the performance seems to be slightly worse than DMM7510.  I wonder if all of Keithley multimeters have this peculiar feature where the noise does not average as 1/sqrt(time).

[maxwell3e10]

how should i "look" at this to understand my DMM?
should the error limiters be flicker FM  or PM? these are all FM.

I am not an expert on all the options, Allan variance originates in the analysis of clock stability, so things like phase modulation are not relevant here. Just look at ADEV, it should start as low as possible and go down as 1/sqrt(time). If you have white, uncorrelated noise, if N points are averaged, the uncertainty goes as 1/sqrt(N).

[3roomlab]

hmmm this is interesting, borrowing tin's log of 10v ref using 3458a
this file "10v_3458_nplc200_mm_01527_clean_reduced.psv"

produces this allan variance . now maybe this is a good tool to use on vref?

** i had tried some new logging format during 2016, but i did not do it extensively, see the 1v log (pic 6 vs pic7 in post #394). i used a diff format. the NPLC i use jumps in "2s" instead of "10s". im not sure if this point to some kind of "logic" problem, but the 2 log of 1v, have drastic difference NPLC 10 vs NPLC 8? maybe somebody else can try. NPLC 2/4/8/16, vs 1/5/10/15?

there seem to be no mention of what FM/PM error means, except here http://www.wriley.com/The%20Evolution%20of%20Frequency%20Stability%20Analysis%20Software.pdf.
which i assume, maybe FM = freq something ? PM = power something? and the corresponding white noise and flicker noise plot. and this is interesting as 99% of my plots are all high in flicker noise.

**update : i found a log which i think corresponds to the "standard" allan variance character slope. log no 1548 30th nov 2015.
100mV range, 3600 samples @ 1 hz, 10NPLC AZ = on, PPM range recorded is 1.49. this is before the short circuit accident.
but the 10v range does not perform the same. log no 0410
10V range, 3600 samples @ 1 hz, 10NPLC AZ = on, PPM range recorded is 0.05. this is before the short circuit accident.

**update : i needed a way to compare, and so i borrowed Dr Diesel's latest 2010 data from xdev ftp --> the NPLC 10 --> 0.1v and 10v
i converted all to nV scale so there is no decimal points to consider on the Y axis, hmmm not enough samples to reach 1000 (i think needs about 2048 samples)

as kleinstein pointed out before, the K2015 is at the limits of its "anti noise" mod. now what would happen if the 2010 is modded?

[dr.diesel]

i converted all to nV scale so there is no decimal points to consider on the Y axis, hmmm not enough samples to reach 1000 (i think needs about 2048 samples)

as kleinstein pointed out before, the K2015 is at the limits of its "anti noise" mod. now what would happen if the 2010 is modded?

I'm happy to do longer runs.

I've not looked in detail, but I'm more than willing to mod my 2010.  (I need more DMMs, too much stuff to log)   

[maxwell3e10]

OK here are results for a K2182A.
Alan

Alan, can you share the raw data taken with K2182? Or take some new files for a few ranges, longer is better.

[dr.diesel]

Overall, the performance seems to be slightly worse than DMM7510.

Today's brief trend of my 731B agrees with that statement.

[3roomlab]

i converted all to nV scale so there is no decimal points to consider on the Y axis, hmmm not enough samples to reach 1000 (i think needs about 2048 samples)

as kleinstein pointed out before, the K2015 is at the limits of its "anti noise" mod. now what would happen if the 2010 is modded?

I'm happy to do longer runs.

I've not looked in detail, but I'm more than willing to mod my 2010.  (I need more DMMs, too much stuff to log)   

1) change bypass caps next to important opamps. 100nF --> 10uF 50v. i used TDK x7r for this or something with very very high ripple character. 1210 sitting upright on 1206 pads. see pic
2) add additional DC reservoir to sections of the DMM, ie : ohms section has its own 100uF rails buffer, etc. i used some OSCON for this or any high ripple char.
3) gnd trace beefing, to bypass the 0.2 to 0.4ohm PCB trace resistance. essentially is = create new star grounding. the PCB ground seem to focus on a testpoint as center.
4) zero buffer have the most noise influence in my case, the entire section is overhauled. LTC1050 -> LTC2057, and the power float buffer all BJT is swapped from SOT23 to TO92. this section in my case heats up and creates alot of noise previously.
5) input impedance test, when bad, the impedance never exceed 11Gohm. when good, it is beyond 50G, this is when i have least noise.
6) front/rear switch, it is quite a huge problem with every touch creates a different offset as it is very old. so i did away with the switch totally. input is direct wired to the input protection section.
7) blocking of air movement / improving of air movement. this 1 is finicky. have pros and cons.

im not sure how to quantify if i did add 0.5digit of accuracy to this old K2015, i have nothing more accurate than this to measure. but noise wise i think it is seems no longer a 6.5digit. i think alex nikitin did the opamp mod too, he might have characterized his DMM with calibrated numbers?
and of cos the K2000 schema https://xdevs.com/fix/kei2000/ this is from the original BBShot 38, there are some discrepencies, so always check before committing changes.
i posted some updates of schema here https://www.eevblog.com/forum/repair/repaired-keithley-2000/, they are in kicad format iirc

option 8 ) if you did mod a bypass from front switch, then this could be interesting
https://www.eevblog.com/forum/projects/diy-low-thermal-emf-switchscanner-for-comparisons-of-voltage-and-resistor-stand/
add a new auto/manual switcher?

option 9 ) PCB top mod see pic here https://www.eevblog.com/forum/beginners/o-qn-on-funny-nplc-multislopenoise/msg808835/#msg808835 <-- these are the i'm bored mods, they dont seem to be very effective capacitor additions

**update @ dr diesel
i remember now, after re-reading this post https://www.eevblog.com/forum/beginners/o-qn-on-funny-nplc-multislopenoise/msg831474/#msg831474
back then, i found some that NPLC setting creates a clock noise which varies with NPLC, can be "seen" in the final noise. and in that experiment, by using a base NPLC that is not 10, seem to create a less jittery noise. this seems to coincide with a later test which i used NPLC 8.

[3roomlab]

@ maxwell and dr diesel

i have a suggestion to look at the noise using different sets of NPLC multipliers, with my existing old data, i manage to find a LIKELY trend using the allan dev program. see pics.
x NPLC times y REPeating average (not moving average). the NPLC setting are as per the pic name. eg : 5 x 5 = 5NPLC x 5 repeated_average (non overlapping samples).

notice the deliberate plateaus/flat-line? and that is missing somewhat in NPLC2 and NPLC4?

so again this ties back to my assumption, the NPLC timing, zero crossing of the multislope (NE5534 section?) creates some kind of noise which is leaked into the AD path or somewhat. then this effect is somewhat spoken about passively in K2700 manual which states that there is least noise not when NPLC is highest, but between 2 and 9, strange? in this case, NPLC 2? NPLC 3? the knee also looks diff on diff voltage ranges, it is like an amplification of offset error? but i dont know for sure.

maybe more experts who know this kind of deviation well can give better analysis of this data (y scale is nV or mV for the 100v case)
and maybe it needs more samples to look at esp between 2 to 9 NPLC for machines outside my home to be sure this plateau happens all the time?

[Kleinstein]

For the Alan deviation analysis there is no need to also look at averaged data. This in already included for the longer times (data are averaged before taking the difference). So there is really no need for those extra curves.

There is a certain dependence of noise on the integration time - usually the very short times are not that good, as more time is lost to switching, maybe the final run down and digital resolution could be limited. At very long integration time there will be flicker noise coming through as the AZ cycle is to slow. So there is an optimum length. Depending on the instrument this could be at 1 PLC, 10 PLC or even much longer (e.g. with the Solartron 1081 best length could be in the minutes). Usually the noise from a single reading at a higher NPLC is lower noise than a single faster reading (with same time in between). But the average of several faster readings could get better at some point. Some instrument do internal averaging already to get longer times.

The plateau seen in the Alan variance curves for some of the Keithley instruments at times in the 1 second range is something odd. Normally one would not expect nothing interesting for a DMM using classical auto zero mode (measure zero, measure signal and output difference). This chopper like operation should take care of most of the flicker noise, especially with a shored input. Anything else than a curve going down like the square root is odd. Only at the very low (long) end there might be something from before the AZ switch (e.g. the input protection, or the input buffer at the K2000 and similar).

One thing that could cause a different curve would a kind of averaging / interpolation used for the zero reading. As a simple example this would be using the zero reading before and after the input reading instead of only the one before (or after) - this would reduce the higher frequency noise (due to averaging), but can also increase the flicker noise from an intermediate frequency range. Averaging / digital filtering of the zero readings shifts the balance between higher and lower frequency noise. In the Alan noise plot this might cause such a plateau be reducing the noise for short times a little, but adding noise for the longer times. It is a little like giving away the advantage one could have from averaging several short AZ readings instead of one very long one, as the AZ can no longer reduce flicker noise with too much averaging. Seeing the plateau for the Keithley Instruments suggests they use a kind of averaging / filtering for AZ - with the DMM7510 they seem to have overdone it, cause quite some extra noise.

[3roomlab]

i have many questions about a noise allan variance plot

http://www.phidgets.com/docs/Allan_Deviation_Primer
http://electronics.stackexchange.com/questions/101045/reading-noise-from-allan-variance-plot-for-mems-sensor-per-ieee-std-952-1997
quite alot of time-nut based pdf info in google

Point A - This y-axis value is the standard deviation of noise for any one single measurement point.
Point B - Averaging over the time spans along the decreasing slope corrects noise which oscillates quickly.
Point C - Eventually, you average enough that the fast-oscillating noise is mostly corrected for. This minimum has both an X and Y value of interest.
Point D - Noise which oscillates over longer time frames begins to influence bigger groups of averaged data.

aside from the plateau problem, this article suggest that allan plot can be used to determine the best amount of integration time needed to reach the lowest noise uncertainty (point C). and since every DMM and location noise is random, which suggest that our sampling times should be adjusted based on this? especially for long term drift negation? which could maximise long term logging accuracy? ie best SnR?

reference to my post #402, in 10NPLC x 10 mode, the integration time is approx more than 200-300 seconds ... hmmm ? sounds logical?
** if NPLC = counter AC noise cycle, point (c) = counter DMM base noise cycle?