In general, unless the noise is a zero mean Gaussian process, averaging will distort the waveform. I apologize for not being interested enough to go through all the many probability density functions to enumerate those which are not zero mean.
A Gaussian assumption is often, but not always valid.
The PDF does not really matter.
Averaging is a linear operation, i.e. we can treat averaging of the noise component and averaging of the wanted signal component separately, and then add the two results.
Averaging N copies of the (non-random) wanted signal component y(t) still results in y(t) - that's identity - there won't be any distortion per se.
Non-zero mean of the noise is not really an issue either - it's just a DC offset, but does not affect the waveform's shape otherwise. For averaging the AC noise component, the Central Limit Theoerm applies. If more and more independent randeom variables are added (ragardless of their PDF), the PDF of the sum tends toward a Gaussian distribution, i.e. the resdiual noise after averaging will be closer to Gaussion and will have a lower standard deviation than before averaging.
However, the wanted signal component gets indeed distorted, when the N averaged copies are
not exactly time-aligned, due to triggering jitter, which is predominantly a consequence of the noise as well. Low-pass finltering of the trigger path may help. An even better alignment could be achieved by calculating cross correlation and finding the time offset which gives the best match of the waveforms - but that's computationally expensive.