Author Topic: [Sloved] Percent of range, how is it calculated?  (Read 940 times)

0 Members and 1 Guest are viewing this topic.

Offline ballsystemlordTopic starter

  • Regular Contributor
  • *
  • Posts: 208
  • Country: us
  • Student
[Sloved] Percent of range, how is it calculated?
« on: October 26, 2023, 02:33:22 am »
Hello,
I was reading some specs for DMMs and instead of the typical xx% + xxDigits, I came upon xx% + xx% of range.
So if the range is 1000.00V, and I'm measuring 1V, and the accuracy is 0.1% + 0.01% of the range, then I could get a reading of up to 101.001V?!
(1V*1.001%)+(1000(range)*0.01%) == 11.001V
And that's assuming I'm using only the portion of the range after the period. If not, then 1V could become:
(1*1.001)+(100000*0.01) == 1,001.001V!!!

Now I suspect I'm making a mistake somewhere, please point me in the right direction.

Thanks!
« Last Edit: October 26, 2023, 04:28:27 am by ballsystemlord »
 

Offline Fungus

  • Super Contributor
  • ***
  • Posts: 17184
  • Country: 00
Re: Percent of range, how is it calculated?
« Reply #1 on: October 26, 2023, 02:40:09 am »
I assume you mean when measuring AC voltage.

Many TRMS converters are inaccurate at the low end of their ranges.

That's why we have auto-ranging meters - to switch to a lower range on demand and avoid these problems.  :)
 

Offline ballsystemlordTopic starter

  • Regular Contributor
  • *
  • Posts: 208
  • Country: us
  • Student
Re: Percent of range, how is it calculated?
« Reply #2 on: October 26, 2023, 03:06:33 am »
No, not AC. This is the DC range.
« Last Edit: October 26, 2023, 03:09:37 am by ballsystemlord »
 

Offline mwb1100

  • Frequent Contributor
  • **
  • Posts: 530
  • Country: us
Re: Percent of range, how is it calculated?
« Reply #3 on: October 26, 2023, 03:10:16 am »
You’ve got a decimal point in the wrong place. 

0.01% of the range would be:

    (1000V * 0.0001) = 0.1
 

Offline ballsystemlordTopic starter

  • Regular Contributor
  • *
  • Posts: 208
  • Country: us
  • Student
Re: Percent of range, how is it calculated?
« Reply #4 on: October 26, 2023, 03:24:38 am »
Which if we took the above datasheet picture and applied that for 1V at the 1V range, we'd get ((1*1.01)+(1*0.00005)) = 1.01005. Or the reading could be off by as much as 1% even though the meter's accuracy is 0.01% in the 1V range... Which means that absolute accuracy specs are a bit misleading?
 

Online J-R

  • Super Contributor
  • ***
  • Posts: 1215
  • Country: us
Re: Percent of range, how is it calculated?
« Reply #5 on: October 26, 2023, 04:02:07 am »
(1V*0.0001)+(1V*0.00005)=0.00015V
So in spec is between 0.99985V and ‭1.00015‬V
 

Offline bdunham7

  • Super Contributor
  • ***
  • Posts: 8007
  • Country: us
Re: Percent of range, how is it calculated?
« Reply #6 on: October 26, 2023, 04:02:51 am »
I don't know how you are doing that math?  0.01% means you multiply by 0.0001 to get the tolerance amount.

So for 1V on the 1V range you have (1V x .0001 + 1V x .00005) = 0.00015 allowable error.  So your 1V input would be guaranteed to read between .99985 to 1.00015.

Now 1V on the 1000V range you have (1V x .0001 + 1000V x .00005) = .0501 allowable error.  Your 1V input is now guaranteed by spec to read betwen .9499 and 1.0501, except that you only have a 5.5 digit meter so in the 1000V range, or the 1000.00V range with all the digits shown, your 1V can only read 000.95 to 001.05.  Of course the leading zeroes may be suppressed.
A 3.5 digit 4.5 digit 5 digit 5.5 digit 6.5 digit 7.5 digit DMM is good enough for most people.
 
The following users thanked this post: ballsystemlord

Offline ballsystemlordTopic starter

  • Regular Contributor
  • *
  • Posts: 208
  • Country: us
  • Student
Re: Percent of range, how is it calculated?
« Reply #7 on: October 26, 2023, 04:27:31 am »
I Sorry about that, I keep forgetting that a 0.XX% means to multiply by 0.00XX to get the correct result.
 

Offline AVGresponding

  • Super Contributor
  • ***
  • Posts: 4829
  • Country: england
  • Exploring Rabbit Holes Since The 1970s
Re: [Sloved] Percent of range, how is it calculated?
« Reply #8 on: October 26, 2023, 05:21:36 am »
AC in particular will usually specify the accuracy is only valid for readings within 20-80% of range (or thereabouts) as well.
nuqDaq yuch Dapol?
Addiction count: Agilent-AVO-BlackStar-Brymen-Chauvin Arnoux-Fluke-GenRad-Hameg-HP-Keithley-IsoTech-Mastech-Megger-Metrix-Micronta-Racal-RFL-Siglent-Solartron-Tektronix-Thurlby-Time Electronics-TTi-UniT
 
The following users thanked this post: ballsystemlord


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf