OK. Sorry about the length but I dug up various old notes. The service manual is helpful to a point, but it doesn't always seem right and there is a big caveat. Everything here was determined by reverse engineering. There may (will) be mistakes - I'm writing as if all is fact, just because too many "I believe that" gets annoying. For example I do not know what causes the need for the quadratic non-linear correction.
I was primarily interested in how the ADC worked on the DC volts ranges. The various measurement types are split into three classes "DC-like", "AC-like" and "time-like" (classed as 0 to 2 respectively).
The primary hardware is an integrator U402-A. The inputs are summed from (a) signal (which can be switched to ground (MZ), input amp (MC) or pre-charge DAC amplifier (PRE)) or (b) positive fixed current (from a precision voltage source through a precision resistor) (c) negative fixed current. The integrator output feeds to the channel 0 AD input on the 80C196KB. It also feeds indirectly to the ASIC. In operation at all times either the positive (b) or negative current is turned on (c) (this is not made clear, but implied from the calculation in the rundown gain calibration test (error 605). The ASIC checks the output every 2.66us and switches between positive and negative current inputs as required to drive integrator output toward 2.5V (input range is 0-5V).
While one sign (presumed +) of current is enabled, the ASIC counter counts pulses (assumed to be the clock) in a 256 state internal counter. Pulses are gated to not be counted while the opposite sign is active. Overflow from this 256 state counter feeds P2.3 (T2CLK) on the CPU (and because they are at least divided by 256 they fall within the timing requirements of this counter). The CPU for DC measurement is configured to interrupt on T2-capture, but not on internal AD done. By raising a sync pulse (positive edge) on P2.4 of the CPU, T2Capture latches the state of the T2 clock in CPU and also an internal U501 latch captures the value of the 256 state counter in the ASIC into a register mapped to RAM at 1FFA (byte). The values of both counters can the be concatenated bitwise to give a complete 24-bit count. The counter is NOT reset by this operation, the current value is copied to a separate latch (manual suggests otherwise but this wouldn't work for the long period integrations e.g. 100NPLC).
A sampling process is as follows:
Two Timer1 counts [1] before the sync pulse to latch T2 and the ASIC count, the AD in the 80C196KB triggers its AD sample and hold selecting channel 0 (output of the integrator). Because the CPU has clock of 12MHz is "fast" (err manual says fast is >6MHz) the internal AD uses a clock pre-scaler and its sample and hold time would appear to be 15 state times, with 1 state time to start the process. Thus the S+H gate closes at the same time as the count is latched. The sync also triggers a T2 count capture and subsequently a T2 capture interrupt. The handler for this interrupt assembles the complete count data and collects the internal AD result (interestingly no attempt is made to check that the conversion is complete e.g. from AD status, or AD done interrupt). Judging by a pointless left shift 15 places of 0 instruction (which is slow) the code execution path has been made sufficiently long that this is a given (although I have not been able to verify this, partly due to lack of clear data).
All measurements are differential and process is as follows:
* Settling time (200us) after selecting a new input switch
* AD start followed by sync pulse is used to record initial values
* CPU HSO timer used to implement fixed delay e.g. 15000 T1 ticks for 1PLC at 50Hz
* start AD after fixed delay
* 2 T1 ticks after AD raise SYNC
HSO control scheduled this way is accurate +/50ns. Long times > 10PLC are split up (and increments over each period summed) to avoid risks from complete 24-bit counter wrap arounds. In auto-zero mode for these long periods it appears that instead of a block of MZ and a block of MC there are alternating blocks.
Interim results are computed in long integrations (possibly to allow ranging decisions to be made more quickly?) so the actual structure is two nested measurement loops up to 10x 10x in NPLC 100 case, but the finished flag is only set when the full period has elapsed. For reference with a 50Hz supply
NPLC T1-cycles(50Hz) T1-cycles (60Hz) Inner Loop Outer Loop (wrap avoidances and MC/MZ switch)
0.02 300 300 1 1
0.2 2250 2250 1 1
1 15000 12500 1 1
10 15000 12500 10 1
100 15000 12500 10 10
The astute observer will see the T1-counts for < 1PLC are the same for both line frequencies (and the first is correct at 50Hz and the second at 60Hz). I cannot explain this other than by assuming it was decided that for < 1 full cycle the precise timing didn't matter. The longer periods which are integral multiples of the line period are exact.
To subtract two results we need to know about rundown gain. This may be standard but I don't have a reference to hand:
[BEGIN THEORY DIGRESSION]
With input voltage v, if we integrate for a time t clocks, and the count at start is c_0 and end is c_t then voltage at output of integrator V_t will be given by
V_t-V_0 = v* a *t + b*(c_t-c_0) - b(t-(c_t-c_0))
a is proportionality constant for measurement current & integrator capacitor
b is proportionality constant for the fixed voltage and resistor & integrator capacitor used in the multislope converter
So if V=V_t-V_0 is difference in integrator voltage over period
and C =c_t-c_0 is count diff
Then
V = (a*v-b)*t + 2*b*C
Or
(a*v-b)*t = 2*b*C -V = (rundown gain)*C -V
If we now apply v=0 (call this MZ for measure zero, in contrast to MC for measure client) and repeat
-b*t = (rundown gain)*C_MZ - V_MZ
So subtracting the two
a*v*t = ((rundown gain)*C_MC - V_MC) - ((rundown gain)*C_MZ - V_MZ)
From which v may be derived.
[END THEORY DIGRESSION]
Depending on the Auto-Zero setting one MZ measurement is made at the start of a block of measurements or MC and MZ alternating. For long > 10PLC measurements the alternating occurs within the measurement period after blocks of 10PLC. After disconnecting MZ, pre-charge is applied for a fixed time (busy loop) and then the pre-charge switch opened before connecting MC (and thus the input amplifier).
Rundown gain may be estimated by selecting MZ and integrating over two different time periods T_1 and T_2 obtaining internal 80C196KB AD count differences V_1 and V_2 and multislope count differences C_1 and C_2
rundown gain = (V_1*T_2 - V_2*T_1)/(T1*C_1 -T_2*C_2)
This is used to test the AD functionality, but not as part of the calibration, the NVRAM stored value is used to reduce measurements. Invalid results (e.g. same for both times) give errors 605/6. The calibration is repeated 8 times and the similarity of results checked (error 607) to ensure the difference between maximum and minimum values obtained is less than 0x100 otherwise the test fails.
For the purpose of reducing measurements the rundown gain is used as follows. It should be noted that fixed precision arithmetic is used throughout the result reduction process.
Value diff = (count diff)*(rundown gain param #70)/4 - (Ad count diff)
The divide by 4 is simply to allow the gain to be expressed in a finer increment of 0.25 rather than 1.0
So for more practical points:
You can see the gain with a SCPI peek
diag:peek? -2,70,0 (mine is 1694)
and after a measurement (e.g. a READ?) you can see the value diff
diag:peek? -7,0,0
Next stage is to turn this raw count into an expression voltage at the AD input. This is done by first an affine transformation (y=ax+b) in fixed precision arithmetic. This is expressed as
y = round((x-offset) * mult * 2^shift / 2^32)
where 0<=mult<2^32, and as usual in this form of representation to represent a factor c, the shift is chosen to be the smallest integer such that 2^shift>c
which is corrected by the mult/2^32. Two offsets are stored on each range, for front and rear terminals.
The coeffs from NVRAM must be adjusted for
a) integration time (and line period if NPLC)
b) to compensate for the non-linear adjustment at 10V input to ADC
Having thus determined a first value for the input to the ADC the non-linear correction may be applied. This has two terms one quadratic and the other cubic (effectively the cubic is a mid range correction). The graph shows the form.
The input to the adc is expressed as an integer value which is multiplied by 1e-7 to give ADC input in volts. The actual meter inputs may require further multiplication by powers of 10 to correct for input amp gain, or divider ratio. Let the true value in volts be x. The corrections are implemented in fixed point arithmetic but for the sake of comprehension we use floating point expressions:
First stage quadratic correction (in counts) = 0.10077 * nlc1 * x^2
Second stage correction (in counts) =nlc2 * x * ( 2.691209 - x^2 * 0.02712)
The two corrections are added to the ADC input count and this is the result. These fp values are approximate - because these adjustments have a granularity limited to 1 tick there seems little point in writing them out with loads of huge numbers.
It will be noted that only the first correction is significant at full scale, where with an input x=10V, the output reads 10^8 + 0.10077*nlc1*100 = 1e8+10+10*nlc1. Thus the raw multiplier from calib is corrected in presence of non-linear correction
mult = (calib mult) *(plc correction) * 1e8/ (1e8 + nlc1 * 10)
This ensures that the same calib coeffs can be used irrespective of the non-linear correction choices.
Data format
The range cal data can be extracted using diag:peek? -2, set, 0
Set numbers from lowest to highest range are
DCV 75-79
DCI 82-85
OHM 87-92
OHM4 94-99
ACV 104-108
Looking in more detail at one row, the DC 10V range is 77. On my meter (and obviously not yours as the calibration is probably different!) this is
2271461829,1,0,69,77,
The values are:
multiplier, shift left count, 0, offset front terminals, offset rear terminals
I have no idea if the 0 can be given a value, but when in use in the meter this value is replaced by NLC2 (the second global nonlinear AD correction)
These values are applicable for the longest standard integration time. They are converted via factors for NPLC and line frequency (again note the 0.02 and 0.2 NPLC have the same factors for 50 & 60Hz), I suspect this is actually wrong, but since it is < 1 cycle it doesn't really matter).
We have an affine form mult*(x-offset) applicable where x is measured over the longest integration time. If we integrate x over a fraction r<1 of the calibrated time we expect
ADC input count = mult * (x /r - offset) = (mult/r) * (x - offset*r)
This is the NPLC scaling.
mult -> mult/r
offset -> offset*r
Since r<1 this requires no shift and can be expressed as (offset multiplier)/2^32.
The multiplier by 1/r requires a 2^k term since it is greater than unity. For minimal loss of precision it is desirable to ensure that the multiplier (unsigned) has its leading 1 as close as possible to the MSB. This may be achieved by
Scaling
scaled multiplier = mult/r = 2^(shift)*mult * 2^{adjshift} * adjmult / 2^32
scaled offset = offset*r = offset * adjoffset/2^32
The adjustments are as follows
50Hz line
NPLC0.02 adjoffset 0x000d1b71, adjmult 0xa0000000 adjshift 0x3
NPLC0.2 adjoffset 0x00624dd3, adjmult 0xa6aaaaac adjshift 0xa
NPLC 1 adjoffset 0x028f5c29, adjmult 0xc8000000 adjshift 0x7
NPLC 10 adjoffset 0x1999999a, adjmult 0xa0000000 adjshift 0x4
NPLC 100 adjoffset 0xffffffff, adjmult 0x80000000 adjshift 0x1
60Hz
NPLC 0.02 adjoffset 0x000d1b71, adjmult 0xa0000000 adjshift 0x3
NPLC 0.2 adjoffset 0x00624dd3, adtmult 0xa6aaaaac adjshift 0xa
NPLC 1 adjoffset 0x02222222, adjmult 0xf0000000 adjshift 0x7
NPLC 10 adjoffset 0x15555555. adjmult 0xc0000000 adjshift 0x4
NPLC 100 adjoffset 0xd5555555, adjmult 0x9999999a adjshift 0x1
An example might help here. Suppose we wish to use the 10V DC range to measure the voltage of a 1.5V cell using the front panel using a 10NPLC sampling window. The cal data is 2271461829,1,0,69,77, and assume a 50Hz line supply.
The 10NPLC adjust is
NPLC 10 adjoffset 1999999a, adjmult a0000000 adjshift 4
New offset = round (1999999a * 69)/2^32 = round (0.1 * 69) = 7
New shift = 4 + 1 = 5 (subject to adjustment)
New multiplier = 2271461829 * a0000000 / 2^32 = 1,419,663,643.15
The binary representation of this begins 01... so to avoid precision loss we multiply by 2 and drop by one the shift
So
shift=4
multiplier = 2,839,327,286
[So basically offset is divided by 10 and mutliplier multiplied by 10 to account for the shorter integration time]
Now this meter has NLC1=27 and NLC2=4 (decimal) and as a result for a full scale AD input of +/-10V the first nonlinear correction results in a count value of 10^8 = 10^8 + NLC1*10. It should read 10^7 *10 = 10^8 so the multiplier correction required is
1e8/100,000,270
Offset = 7
Multiplier = 2,839,327,286 * 1e8/100,000,270 = 2,839,319,619
Shift =4
In the 7-5-2 firmware the coeffs used can be read out at location 15B2 and these are indeed the ones used.
Let us now reduce a reading; suppose we measure a 1.5V cell and get a raw count of 1510615 (peek -7,0,0 value).
Reading = round(2^(shift)*mult*(count - offset)/2^32) = round( 2^4 * 2,839,319,619 * ( 1510615 - 7) /2^32)
= 15,978,139
Now we must apply the non linear corrections; this suggests x = 1.5978139
first correction = round( 0.10077 * nlc1 * x^2 ) = ropund ( 0.10077*27*(1.5978139)^2) = 7
second correction = int ( nlc2 * x * ( 2.691209 - x^2 * 0.02712) ) = round (4 *(1.5978139)*(2.691209 - (1.5978139)^2*0.02712) = int (16.75) =16
Result = 15,978,139 + 7+16 =15,978,162
So voltage = 1.5978162V
To be continued maybe... pre-charge is worth a discussion
[1] A T1 count is 8 state times. One state time is two clock cycles (at 12MHz)
[2] Did anyone make it here or was it too boring?
[3] I don't have a similar doc on the 3458. Maybe someday if someone gave me one....?