I dont know if the A23 is galvanically isolated and i think if your well matched (A23 2mV input max) references are galvanically isolated itself to allow difference measurements, then it shouldnt matter.
Apart from the A23-solution may i suggest the Keithley 181 Nanovoltmeter, which comes for about 250€ used on ebay very often. The A23 has 1.3nVpp, the K181 ~15nVpp AFAIR and i dont think that you need the ultralow-noise-voltage of the A23 when you compare voltage references, since even the chinese 2DW-zeners have ~300nVpp and drift due to age/temp.
I dont really see the use-case for the nV-amplifier; is your goal really only noise measurement or do you want to see the tiny reference-drift effects during short scales?
Thinking further:
Wouldnt in this case a Low Noise Amplifier 0.1-10Hz be the better solution, like the ones shown in this thread:
https://www.eevblog.com/forum/metrology/diy-low-frenquency-noise-meter/ ? They are made to determine the low frequency voltage noise of references and are capable of having a amplifier-noise-floor of ~150nVpp RTI (if we take Andreas-Design as an example) and therefore able to measure even the small 2DW-zener-noise.
Sorry if that wont answer your questions regarding the A23, im just curious.