Shorting the ADC inpupt and not multiplying by 10 (and removing the overange values) I get 2,038uV RMS.
Hello,
the "overrange" values are valid results. So you should not remove them.
Instead calculate the correct sign for it.
(Otherwise you get an unwanted offset when averaging).
And take care when calculating on a small processor.
Usually they have only single precision floats (4 byte) with 24 bit mantissa.
This is not enough to handle 24 bits + sign + overrange (+ 4 sub-LSBs)
And also not enough for averaging.
You get additionally unwanted quantisation noise.
I calculate the results on a PC with double floats (64 bit mantissa).
For 10V range I use a LTC1043 capacitive precision divider.
with best regards
Andreas