Has this been resolved? Inevitable quantization error is 0.5 LSB in any case, even a perfect, perfect ADC will be "wrong" by 0.5 LSB (2.5mV in this case) simply because if the input voltage is, say, 2.49mV, the ADC has to commit to output = 0 LSB (0mV, 2.49mV error) or output = 1 LSB (5 mV, 2.51mV error). Getting an error of only 0.7mV is like, impossible for many values of analog input ? How is the graph that good? I must be missing something, like results are being averaged or some 12-bit mode is being used or something.
There is no 12bit mode with the ATMEGA328 - 10 bit only. The sampling is averaged so it could filter out random fluctuation. Averaging may give the impression of higher resolution, but more digits in the math doesn't change the resolution of the ADC.
The fact that you were averaging does resolve this dilemma, sorry if you mentioned this earlier in the thread. Believe it or not, averaging many samples
can improve noise below one LSB, and more importantly removes quantization noise as a fundamental obstacle. This is the basis of the "Hi-res" modes on many oscilloscopes that give better resolution than the native 8 bits of the ADC by heavily oversampling and averaging. It's a real gain, not mathematical smoke and mirrors. 1-bit delta-sigma ADCs are the ultimate example of this (loosely).
Since you already expressed your dislike of "reading+-%error" format in another thread, you will not like this graph. Not your cup of tea.
I'm sorry about the other thread, I know I was going on about it a bit. However, in this thread I was just trying to understand how you were seemingly getting better results that physically possible (averaging explains it). BTW, the graph that I'm commenting on is the absolute one, without that division I "dislike" so much
. The datasheet says 0.5 LSB of non-linearity, you're getting 0.2 LSB, other people should assume 0.5 LSB to be safe but it's good to see that your sample is behaving well within the datasheet limits. Few applications (basically, only ones that average or integrate, such as that current measuring one in the other thread) can even theoretically benefit from a non-linearity better than 0.5 LSB.
Thanks!