There are quite a few contributions to the noise of those integrating ADCs. As typical with an optimized design it is not one big noise source, but several of similar size. A few obvious ones are:
1) The resistors at the input of the integrator. For the 3458 this are 40 K for the input and about the same for the reference currents. So effectively 80 K and thus about 36 nV/Sqrt(Hz). Other DMMs often use larger resistors.
2) noise voltage of the integrator OPs relative to the input signal. Due to the low frequency (e.g. 25 Hz range or lower) there can be quite some 1/f noise from JFET OPs. A BJT based OP like the OPA177 will add some current noise too.
3) Higher frequency noise of the integrator and following comparator (if used). The comparator might need to be rather fast and thus the effective bandwidth can be rather high and thus quite a noise contribution. Some modern ADCs use and extra ADC chip instead. This might be an advantage due to an effective lower BW. Not sure on how much of the given opportunities they actually use. This would be especially true if an ADC is used in combination with a separate rundown phase ( I don't know such a design).
4) The charge zero phase at the start of conversion might add some noise for the starting point too - this part might be overlooked in some designs. Again continuous versions with ADC might not use a zero Phase.
5) Charge injection from the switches for the reference might also add some noise, especially with fast switching.
6) Clock or control circuit jitter. For a low noise it needs signal jitter well below 1 ns. It is not so much the full time, but with a rather fast reference switching frequency, a single conversion can have several 10000 switching events. So clock jitter adds up. Frequent switching is attractive to keep other noise (and INL) contributions small.
7) The balance of the positive and negative references adds some noise (e.g. the OP and the resistors used for this)
Power supply noise and similar signals coupled in might be a factor too
9) for the critical low frequency part (e.g. 1-100 mHz range) thermal stability can be a factor too - this often is what makes the Alan variance curve go up after some point.
10) Higher frequency noise from the reference (e.g. modulation frequency band) can enter. Though filtering is easy I have not seen it with most DMMs.
11) A non perfect sync to the line frequency can contribute
There are other contributions too, some can be hard to find and specific to some ADCs.
For the AZ mode the second measurement also adds to the noise - it is hard to avoid for the longer time scales.
The typical noise level of a LTZ1000 reference also sets a noise level, that might lead to a common design target. There is not that much advantage of having an ADC that is much lower noise than the reference. Similar there are not many signal sources (e.g. calibration sources) that have a much lower voltage noise.