Is it normal for it to be this jumpy or its the noise?
That is a differential amplifier arrangement. I am not sure what bridge you use but assuming it is a popular 350ohm one, you get the equation:
Uout-U3_4 = X*(U1_3 - U2_4)
U1_3 is the voltage on connection of (Rb1 Rb3), U2_4 is on (Rb2 Rb4) and U3_4 is on (Rb3 Rb4) which you have grounded.
The X is called differential gain and in your case that is over 5714
You should not reference the output to U3_4 because this signal eventually is fed to ADC so it should be referenced to GND of ADC and uC. And GND on uC is not the same as U3_4.
If by some chance U1_3 is lower/higher than U2_4 by some 5V/5714=875uV initially then the output hits the top/bottom rail. So you have to null the output somehow (tare).
As for peak to peak and noise - calculate mean and variance of the signal fed to ADC. If the variance is around 0.027 of an ADC bit then that is ok.