Data logging with meters: Average on meter or on logging computer

[TheUnnamedNewbie]

I'm working on a framework in matlab to log data from my keithley 2000 (and other meters lateron). I know there are frameworks in place but I am doing this to learn about the interfacing between instruments and the computer.

One of the features I wanted to add to my meter window was an averaging capability.
My keithley 2000 can already do averaging internally. But then again, the server I am using to do this communications does not exactly have a shortage of CPU cycles...
What are the pros' and cons' of doing averaging on the logging computer vs doing averaging on the meter itself? I imagine if I don't do any averaging on the meter, it will be able to take more measurements every second and increase the amount of data?

[cellularmitosis]

A possible advantage of averaging on the computer is that you have more flexibility with how you average the data.  For example, rather than a simple rolling average, you might apply a Savitzky-Golay filter, which is similar but doesn’t induce a “phase lag”.

Here’s a python script I recently used to add and additional column to a csV file with a savitzky-golay filter.  You’ll want to tune the window size for your application.  https://github.com/cellularmitosis/logs/blob/master/20180205-r-tempcos/savgol.py

[2N3055]

It's a matter of entropy...

You collect original data in it's pure form, and store it completely, in full precision and with defined sampling criteria (sample time, frequency, settings). Than you process is later to whatever extent you want.. Same dataset can be processed in any conceivable way later, even for the purposes you didn't expect or forsee when you collected data.

That's how you do it. And with today's processing power and cheap and ample data storage easily available , quite easy to do.

Regards,

Sinisa

[TheUnnamedNewbie]

You raise good points. I will try to do all averaging on computer once I figure out how to do it properly (ie, without spending a lot of time waiting for the meter etc - this is just a matter of me figuring out how to program things).

On a related note: How does this change when you go from a mathematical average to measurement averaging (IE, number of PLC)? better to have 1 PLC and average on computer over larger number of samples, or 10 PLC with ten times less datapoints?

[2N3055]

You raise good points. I will try to do all averaging on computer once I figure out how to do it properly (ie, without spending a lot of time waiting for the meter etc - this is just a matter of me figuring out how to program things).

On a related note: How does this change when you go from a mathematical average to measurement averaging (IE, number of PLC)? better to have 1 PLC and average on computer over larger number of samples, or 10 PLC with ten times less datapoints?

That will actually depend on inherent noise and characteristics of actual data sampler (voltmeter in this case). For instance, some meters will have maximum 10 NPLC in hardware, and when you put them in 100 NPLC they will average it in software...  There is a topic on this blog regarding meter noise where it is explained in quite detail, with data for many common benchtop meters. For what I remember, for Keithley 2000 10 PLC might be a good choice (if it is fast enough for what you need).

As for how you will implement it on a PC, if you are using Matlab, there is oodles of support for all kinds of data massaging in Matlab..
Matlab is THE tool to do that...

Regards
Sinisa

[ramon]

It's a matter of entropy...

You collect original data in it's pure form, and store it completely, in full precision and with defined sampling criteria (sample time, frequency, settings). Than you process is later to whatever extent you want.. Same dataset can be processed in any conceivable way later, even for the purposes you didn't expect or forsee when you collected data.

That's how you do it. And with today's processing power and cheap and ample data storage easily available , quite easy to do.

Regards,

Sinisa

And after those wise words are said I just drop by to add that there is a software named R that is awesome for statistical computing and graphic visualization of data. And it is completely free.

[dacman]

If the meter is just averaging the data, then I would prefer to have the raw data, unless there is a reason to reduce measurement noise.  The individual measurements could be used to calculate the uncertainty of the measurement, and to calculate a standard error for each averaged set of data, which could be used to identify noisy instruments.

[alm]

That will actually depend on inherent noise and characteristics of actual data sampler (voltmeter in this case). For instance, some meters will have maximum 10 NPLC in hardware, and when you put them in 100 NPLC they will average it in software... 

Even if the meter does due true analog integration for longer integration times, the limited output resolution (and hence quantization noise) may still limit the improvement in noise. I did a test a while ago with a 6.5 digit Prema DMM that could vary its integration time up to 20 seconds (no mention of software averaging in the manual), and would output 6.5 digits to both front panel and GPIB for integration times from 1 to 20 seconds. I measured a 10 V source for something like a week, changing the integration time every 300 seconds (so 300 seconds taking however many samples it could with 1 second integration time, then 2 second, then ... 300 seconds of 20 seconds integrations, and then back to 1 second integration again. This way any external noise sources (like temperature fluctuations) would be spread out evenly across all integration times. At the end I calculated the standard error of the mean over all samples per integration interval.

Integration time (seconds)	Standard error of the mean (ppm of mean)
1	0.2663
2	0.1898
4	0.1027
10	0.1467
20	0.1752

Clearly in this case for this setup, digital averaging (in the meter or on the computer) with an integration time of four seconds gives the most stable average.

[2N3055]

That will actually depend on inherent noise and characteristics of actual data sampler (voltmeter in this case). For instance, some meters will have maximum 10 NPLC in hardware, and when you put them in 100 NPLC they will average it in software... 

Even if the meter does due true analog integration for longer integration times, the limited output resolution (and hence quantization noise) may still limit the improvement in noise. I did a test a while ago with a 6.5 digit Prema DMM that could vary its integration time up to 20 seconds (no mention of software averaging in the manual), and would output 6.5 digits to both front panel and GPIB for integration times from 1 to 20 seconds. I measured a 10 V source for something like a week, changing the integration time every 300 seconds (so 300 seconds taking however many samples it could with 1 second integration time, then 2 second, then ... 300 seconds of 20 seconds integrations, and then back to 1 second integration again. This way any external noise sources (like temperature fluctuations) would be spread out evenly across all integration times. At the end I calculated the standard error of the mean over all samples per integration interval.

Integration time (seconds)	Standard error of the mean (ppm of mean)
1	0.2663
2	0.1898
4	0.1027
10	0.1467
20	0.1752

Clearly in this case for this setup, digital averaging (in the meter or on the computer) with an integration time of four seconds gives the most stable average.

@Alm, well I didn't explain it well enough.. sorry for that.. I did mean that you should find "sweet spot" where you get best results from your meter... Exactly like you did..
And then sampling with longer integration in meter wouldn't make sense, and if you wanted to average further, you do it on PC.......

So in your case , 4 sec integration time would be best for most stable measurements..
Regards,

Sinisa