Multimeter Hi Res digit rounding....or not.

[mjlorton]

In this video I demonstrate how some multimeters with a Hi Res (high resolution) mode round the last digit when switching back to a low resolution mode. In essence they just "hide" the last digit which is not the correct way they should handle this.

Equipment used:

* Brymen TBM867 / BM867
* ISO-Tech IDM207
* Fluke 87-V



Cheers,
Martin.

[slateraptor]

With so many variations of rounding, the "right way" of performing the operation appears to me a moot point.

For example, using the 1.1875V value displayed on your Fluke, I was taught in chemistry that if the least significant digit is 5 and the next decimal over was unknown, then rounding to 4 significant digits would yield 1.187V, justified by uncertainty in 5. In contrast, if the meter read 1.18750V, then this value would be rounded to 1.188V since the uncertainty lies in 0. By this convention, even Fluke's algorithm is "wrong," so to speak. That is to say, what makes floor(x) "wrong" and ceil(x) "right" beyond arbitrary convention?

[olsenn]

I wouldn't trust the least significant digit anyways... I don't even trust the second most significant digit in most cases... I go by the spec (+- 0.05% for example) if I need to know how much I can trust the readings... usually it's much closer to the correct value than that, but there's no point, in my opinion, in taking that 4th digit past the decimal place seriously; it's just a guess really.

[grenert]

With so many variations of rounding, the "right way" of performing the operation appears to me a moot point.

I think the point of the video is that the operation (rounding) isn't being performed at all!  No matter what system you might use for rounding, I think most everyone would prefer that something like .19 get rounded to .2 when you go to low resolution mode, rather than .1.

But it probably has little impact on overall accuracy.

[slateraptor]

I think the point of the video is that the operation (rounding) isn't being performed at all!  No matter what system you might use for rounding, I think most everyone would prefer that something like .19 get rounded to .2 when you go to low resolution mode, rather than .1.

But it probably has little impact on overall accuracy.

Truncating, or floor(x), is rounding...it's just not the kind of rounding that most were taught in grade school (if that is in fact what we're really seeing; the video is inconclusive). Furthermore, you can't gauge a meter's performance in low-res mode based on its hi-res display; not only is that self-referencing, but it also means that in order to acquire a low-res reading the meter would have to acquire a hi-res reading first, which makes no sense from a sampling perspective.

[cybergibbons]

Most meters reduce the resolution of the sample from say, 20 bits down to 16 bits, so that they can sample more quickly and update the display more quickly. It's not that it's hiding the last digit - it isn't measuring it at all.

[slateraptor]

Most meters reduce the resolution of the sample from say, 20 bits down to 16 bits, so that they can sample more quickly and update the display more quickly. It's not that it's hiding the last digit - it isn't measuring it at all.

Sampling rate likely stays the same whether in low-res or hi-res mode; what I suspect varies is the number of samples acquired in conjunction with a simple DSP algorithm to increase ENOB and thus resolution.

[Kilroy]

Rounding errors tend to matter more if they accumulate significantly during multiple subsequent calculations on the initial data.