DMM Noise comparison testing project

[TiN]

Offset compensation in Keithley works kinda different, and dead time is longer as well.

[3roomlab]

finally some 10v 100NPLC testing log. the entire log wasnt very successful as there is the mechanical switch problem. but still i could extract 1000 samples.
compared to the 3458a noise, K2015 is 3x more noisy  . now maybe i found something to want to mod again?

this segment is taken from the middle of the log
10NPLC x 10repeating-average (100NPLC). 6.5s x 1000samples. AZERO on.
STDEV 302.51nV, p-p noise 1744nV (kurt -0.20, skew -0.03)

the logged width of the p-p noise is just bad 

**edit @kleinstein, i am not sure about output timing consistency, but in the SCPI command page, there is an elaborate trigger and delay system.

[Kleinstein]

With autozero on the single 10 NPLC readings are essentially uncorrelated, except for noise from the reference, which does not really matter much in a zero voltage measurement anyway. So there is not much extra information gained in using simlulated higher NPLC readings.  Still getting about 0.9 ppm noise at 10 PLC for the modifyed K2015 is a good value for a 6 digit meter. But this is still something like 2-3 times the high end meters.

More could be gain from the longer runs, if the individual data are checked for correlations. This would give the information how much the curve will deviate form the square root n slope.

[alanambrose]

Wow, this is a very interesting thread. My own results without auto zero below. I have a coupla questions:

@Dr. Frank

- presumably it was the upward tip of the graph which tipped you off to the need for auto zero?
- if we compare the non-azero vs the a-zeroed results, presumably that gives a measure for the instrument/ref flicker noise? Would you guess this is short term temperature fluctuation or something else?
- did you get your noise spec from reading values off the graph in the datasheet or from somewhere else? (I read mine off the graph)
- the split between the spec and the non-auto-zeroed results seems to occur earlier for 100mV and 1/100V ranges. Any thoughts?
- (also) does it matter that you're not triggering off the PLC? i.e. you can trigger anywhere in the cycle?
- (also) does it matter (re APER and NPLC) that the line frequency isn't an integer at any particular moment or is APER time effectively defined on a line frequency basis (I guess I could RTFM for this one)?

@everyone

Is there a way to run the characterisation for non-zero inputs?

Apologies for more questions that answers, but it does provoke a lot of thought.

Alan

[TiN]

Aha, so I'm not the only one to see worse data with higher NPLCs.
I need to rerun with meter in current condition, as in my case long NPLCs reading were compromised by LTZ's "popcorn" noise (random shifts ~1ppm).

Non-zero inputs require way more thought than zero tests, as it's rather difficult to have sub-ppm stable signals with sub-ppm noise, to match performance of reference in meter itself. Otherwise we will be testing source noise rather than DMMs.

Alan, can you drop your CSV's on my FTP, so we can see if your reference is suspected to jumps as well? That would explain bad results you seeing.

[3roomlab]

but how did Dr franks have such low STDEV?
surely it is something repeatable, to produce similar result if conditions could be similar?

[alanambrose]

>>> Non-zero inputs require way more thought than zero tests, as it's rather difficult to have sub-ppm stable signals with sub-ppm noise, to match performance of reference in meter itself. Otherwise we will be testing source noise rather than DMMs.

Ah yes, I'm thinking the answer is something like:

(1) measure the noise on a lowish noise source at a few chosen levels using an independent method (AN124 etc)
(2) measure the noise using the chosen meter at the same chosen levels using the same methods we've been using for zero levels
(3) RMS-subtract the meter-measured noise from the independently-measured noise to get the meter noise contribution at these levels

... but this is a new area for me.

The reason I bring this up is that it would be interesting (at least IMHO) to establish/measure the meter noise floor for non-zero inputs. This could be done automatically for, say, a dozen key levels and a reasonable stab at a noise surface estimated.

A.

[alanambrose]

>>> Aha, so I'm not the only one to see worse data with higher NPLCs.

Glad to have been of service

>>> I need to rerun with meter in current condition, as in my case long NPLCs reading were compromised by LTZ's "popcorn" noise (random shifts ~1ppm).

Maybe an AZERO on and off comparison would isolate the flicker/popcorn level?

>>> Alan, can you drop your CSV's on my FTP, so we can see if your reference is suspected to jumps as well? That would explain bad results you seeing.

I'm not saving the point-by-point results atm but will try writing them out in your format.

A.

[deadlylover]

but how did Dr franks have such low STDEV?
surely it is something repeatable, to produce similar result if conditions could be similar?

My R6581 got pretty close and it's worse meter. 

I think you guys might be taking too many samples which means we're starting to measure short term drift and not noise, try a run at 100NPLC with only 100 samples with Auto Zero on as a sanity check.

[TiN]

I had runs with 50 samples only, result was very close if not same.
Autozero does nothing to flicker reference noise, as reference voltage is inside the compensation loop already.

[deadlylover]

I had runs with 50 samples only, result was very close if not same.
Autozero does nothing to flicker reference noise, as reference voltage is inside the compensation loop already.

Well, your 3458A has been to hell and back, back to hell, and back again. 

[3roomlab]

try a run at 100NPLC with only 100 samples with Auto Zero on as a sanity check.

using log from this afternoon (unit = uV), numbers all look smaller with smaller samples
0.1v 1NPLC azero=off (3600 samples, SD=0.122, pp=0.822)[200samples, SD=0.115, pp=0.575]
1v 1NPLC azero=off (3600 samples, SD=0.165, pp=1.240)[200samples, SD=0.141, pp=0.806]
10v 1NPLC azero=off (3600 samples, SD=1.010, pp=6.889)[200samples, SD=0.992, pp=4.592]
i still dislike my 10v range noise . cheap used n strange modded DMM cant be that picky i guess

[Dr. Frank]

Wow, this is a very interesting thread. My own results without auto zero below. I have a coupla questions:

@Dr. Frank

1) - presumably it was the upward tip of the graph which tipped you off to the need for auto zero?
2) - if we compare the non-azero vs the a-zeroed results, presumably that gives a measure for the instrument/ref flicker noise? Would you guess this is short term temperature fluctuation or something else?
3) - did you get your noise spec from reading values off the graph in the datasheet or from somewhere else? (I read mine off the graph)
4) - the split between the spec and the non-auto-zeroed results seems to occur earlier for 100mV and 1/100V ranges. Any thoughts?
5) - (also) does it matter that you're not triggering off the PLC? i.e. you can trigger anywhere in the cycle?
6) - (also) does it matter (re APER and NPLC) that the line frequency isn't an integer at any particular moment or is APER time effectively defined on a line frequency basis (I guess I could RTFM for this one)?



Alan

Hi Alan,

1) Answer:
I used AZERO OFF only because I wanted to use DIGITIZE for small aperture times, i.e. getting data at the fastest possible rate, w/o dead time.
My measurements never showed an upward tendency for NPLC > 1, and my results are in good accordance with the 3458A specification, i.e. that the noise is going down for longer integration times.
I also did not get extraordinary better noise figures than the specification.

In the very beginning of this thread, I already made a comment, to avoid acquiring too many data especially for longer integration times. This also causes measurement periods which are also much too long, causing gain drifts of the 3458A.

In the region of NPLC >1 you already reach 1ppm noise level, and 0.001ppm for NPLC 1000!

The instrument has a gain drift of about 0.5ppm/K for the 10V range, and 1.2ppm/K for 1V and 100mV.
ACAL does NOT apply here!

Ambient temperature changes of a few tenths of °C therefore can easily cause drifts of many tenths of ppm.
And that's exactly, what you may see in your measurements.

Therefore I limited my measurements to a few seconds per run, where possible.
Also, the ambient temperature in my basement may be stable to 1/10°C  over hours.

To extract drift from noise, you may use the modified Allan distribution.

2) I made no measurements with AZERO ON, only for NPLC 1000, where the offset drift already affected my noise measurements, equivalently to the explanation above.
You may identify different noise sources by the different time scales.

3) the standard 3458A noise graph originates from the specification

4) which split? can you describe more precisely what you suppose?

5) For NPLC 1, 10, 100, 1000 all measurements are per definition in phase with the line frequency, or in other words, always over multiples of 20msec.
For NPLC < 1, sampling is of course over fractions of one line cycle, and there is no synchronization to the line frequency, so you may of course pick up additional noise.

It doesn't matter, because the pick up noise seems to be minor compared to the other noise sources.

6) I do not understand that question. The situation is as explained in 5)

Frank

[alanambrose]

Hi Dr Frank,

Thanks for your detailed replies.

>>> The instrument has a gain drift of about 0.5ppm/K for the 10V range, and 1.2ppm/K for 1V and 100mV.

Ah this has finally clicked with me now you mention the actual figures So it is critical to just run the minimum # of samples quickly - say to a target standard error / confidence level in the noise figure. (edit) ... or maybe watch the internal temperature and abandon any stats gathered over a detectable temperature change.

>>> For NPLC 1, 10, 100, 1000 all measurements are per definition in phase with the line frequency, or in other words, always over multiples of 20msec.

Apologies my question was a little cryptic. I was meaning as the line frequency is only ~50Hz i.e. +- 1% e.g.

http://www2.nationalgrid.com/uk/services/balancing-services/frequency-response/

... so 1 NPLC is only ~20ms at '50Hz'.

>>> TiN - Autozero does nothing to flicker reference noise, as reference voltage is inside the compensation loop already.

I don't quite get that - is it possible to explain that further?

Regards, Alan

[Kleinstein]

When measuring with zero voltage, the reference noise is very much suppressed  - so you should not see it. It take a low noise source close to the maximum reading in the 10 V range to see the noise from the reference. This might be a 9 V battery, especially if first order drift is compensated for. Also noise from the ADC itself might be higher than at zero reading. Also gain drift (and fluctuations) gets important if measured with a signal.

Using AZ mode eliminates most of the very low frequency noise of the input amplifier and the ADC itselt. This 1/f type noise together with drift is the reason for the curve to go up at higher NPLC values. There may be also some extra low frequency noise before the point the AZ circuit kicks in, e.g. thermal fluctuations at the inputs / range switching relays. So Az is not perfect in elimination all the 1/f noise. The reference is anyway outside the part that AZ is used for.

With more points in a series, you add lower frequeny noise and drift. So the RMS and pp valles should not be lower but are expected to be slightly higher. For the really long runs (e.g. > 2000 points) it would make sense to use fourier transformation or similar to get the noise as a function of frequency.

[alanambrose]

Many thanks for the explanation re ref noise and other noise contributors to the acquisition process - I am beginning to understand now. Also an interesting point about fourier analysis. Below a plot comparing azero on and off (I used the same setting on all nplcs) to counteract drift - presumably the difference is mainly down to temperature drift. With a coupla runs I was able to get an internal temp drift less than 0.1C during the nplc 1,000 run. These are at 60 samples per nplc for a relative standard error in the noise readings of 10% (see error bars). TiN - it would be interesting to run the 'azero on' test on your 'evil' meter - maybe your temperature environment is fairly drifty like mine.

(edit) fixed the error bars on the chart and added the intervening points.

Alan

[TiN]

Hi Dr.Frank,

The instrument has a gain drift of about 0.5ppm/K for the 10V range, and 1.2ppm/K for 1V and 100mV.
ACAL does NOT apply here!

Ambient temperature changes of a few tenths of °C therefore can easily cause drifts of many tenths of ppm.
And that's exactly, what you may see in your measurements.

I'm not sure if that's always correct, as according to recent testing, change of instrument temperature even 2C does not usually cause change more than 1ppm. That's of course if one to trust assumption that Fluke 732A does not drift same direction as three 3458A's with ambient temperature.

And why would ACAL not apply against temperature change, as it's exactly why it was implemented in first place, to battle temperature variations, no?

"Red" 3458A from results linked above was sampling at NPLC20, Autozero ON, 24/7 over temperature/environment variations more than a week long, and barely went outside 0.5ppm pk-pk, mostly staying in 0.2ppm pk-pk zone. That is with no ACAL between.

>>> TiN - Autozero does nothing to flicker reference noise, as reference voltage is inside the compensation loop already.

I don't quite get that - is it possible to explain that further?

Per my understanding, all measurements in 3458A are performed with reference to A9 LTZ1000 output. If that output jumps, or drifts, then all ACAL/CAL functions and readings will follow those variations as well, multiplied by range/function gain. I learned that hard way, observing ~1ppm jumps (7V zener voltage) causing clear jumps in DCV/Ohm measurements as well. So even if ACAL was performed - it would not remove any of these variations from output, as internally 3458A considers A9 voltage output as constant, and have no means to compare it to anything else.

All my tests on 3458A were done with autozero enabled, by default.
But since there were unstability in A9 output, my noise data from 3458A is invalid. I did not retest it again since I changed bad LTZ1000ACH on A9 to LTZ1000CH, but will do it after get GPIB interfacing on Windows PC fixed (we use Raspberry Pi to sample all drifty logs).

[Kleinstein]

The noise tests with zero input signal are not sensitive to reference variations (unless rediculus large) and gain variations. It really does not change much by how much you amplifiy a value near zero. So even 10 % in gain or reference drift would hardy show up. This noise test only shows additive noise and offset drift. Its only with  DC signal that gain fluctuations / drift and reference noise are included.

The ACAL could compensate for gain drift in the 10 V range at least - it can't do it fully in the other modes.

I am wondering why the noise readings are not higher with AZ on - there should be a higher contribution from white noise, as the differece of two measurements is used. The zero measurement might be lower noise than the real input, as not input protection is involved.  So the extra noise from the AZ must not show up with all instruments. Also take into acout that the measurement with AZ active usually takes longer - so a non AZ mode could use a higher NPLC setting (e.g. twice as long) to get the same data rate. With AZ active the data rates seem to be not the same with all the instruments - so comparing just based on NPLC settings can be misleading. At least one must also note the possible data rate at the setting.

[3roomlab]

The variance (or uncertainty) of a set of N measurements is given by stdv/ sqrt(N).

For the 10V range, you may average over about 50 measurements of NPLC 100 to have 1e-9 variance of the result.
4 averaged measurements of NPLC 1000 also give 1e-9.

I doubt that anything below 1e-8 is useful.
At first, the mentioned environmental parameters are worse, and second, the linearity of the A/D is 2e-8 'only'.

Therefore, any higher resolution is useless.

i am trying to understand more about uncertainty/variance
from the K2000 spec, if i measure 10v on 10v range. this (using 24hr figures 15+4ppm), i am expecting 190ppmuV uncertainty based on factory spec.
however, if i base on my shorted noise measure of say 0.36uV (over 100 samples @ 100NPLC), = 3.6e-8. between the self shorted variance and a spec sheet, how should i effectively use the 2 now known variances to apply to the 10v on 10v range reading? should i only use the 3.6e-8 since this is the actual uncertainty measured?

  it is always after not thinking about it, then i come about to figure out after re-reading it again. the 0.36 STDEV = 36uV reading uncertainty. which means my machine uncertainty is 36+40uV in uncertainty for the 10v measured.  did i get this right? or the 36uV doesnt apply because @ 10v, we are talking about a possibly diff value of STDEV? (STDEV possibly scaling linearly with input resistance?)

update, i finally converted the data for last few weeks into a summary plot (it needs to be known the DMM is modified, for NPLC with multiple points - i have plotted several variants of same NPLC) unfortunately i do not have parrallel data on temp

[TiN]

Repeated my tests, this time bit different. Had program to take fixed 100 samples, disregarding NPLC rate, and first 10 samples thrown away.

New data is marked 90SMPL on graphs. Old 10minute fixed time data in light blue line.









Mains voltage = 110V
Mains frequency = 60Hz
TEMP? =  33°?
Ambient temp = 22°?
AZERO ON

[pelule]

Hi,
I just have uploaded my Keithley 2000 noise measurements (CVS raw) via FTP.
Placed it into new created subfolder ".\PeluLe"
File "20160202 Keithley 2000 Noise Test PeLuLe" contains the relevant details.
BR
PeLuLe

[zlymex]

Very good thread and informative data
I had also made some similar tests before, here is the result in the ppm against NPLC manner:


1. It was an old chart of mine, I cannot find all the supporting data now
2. Some of the data points were not measured by me.
3. All these were measurements of 10V Vrefs (not zero, or shorted)
4. More than 500 measurements for each test, select the best 100 portion, and calculate the standard deviation
5. Later, I calculate allan deviation instead, this gives better result when slow drift is present. If there is no drift, the two are equal.
6. For twice, I had six 8.5 DMM in my lab, but I didn't test them in systematic ways.

[Kleinstein]

Also looking at the input current and input current noise makes sense. Though a 10 M "source" resistance is rather high. This reading can also get sensitive to shielding / noise pickup. So it is not that reproducible.

When having the 10 M resistance one also has to take into account the capacitance at the input, as there is also a capacitance to ground. Here different meters can be quite different. This is especially important for fast readings.

For the input stage there are mainly two versions:
1) JFET differential stage and auto zero through a slow switching between the source and zero. (HP3457, 3458,...)
2) some kind chopper stabilized amplifier (CMOS OP or discrete with JFETs) (e.g. Solatron, Keithly 2000,...)

Much of the input current can come from charge injection, so not a smooth current but pulses that might not fully cancel out to zero. Here is also can make a big difference if AZ mode is used or not.

[zlymex]

, I'm seeing something that can only be described as "popcorn noise"-- little positive [or negative] "spikes" in the data that far exceed the input bias current-- seemingly random, but at a far lower rate than the more-or-less normal input noise. 

Is it something similar to my measurement as attached?
This is my femto-amp meter(DIY), the CMOS input with a very large feedback resistor. I cannot find the source of those spikes neither. Once I even suspect it was the cosmic ray or background radiation because I measure the similar background in my Geiger counter.

[alanambrose]

Thanks zlymex for the noise comparison chart - that's v interesting.

A.