Because audio signals are such low bandwidth, and such high level, that noise margins are essentially irrelevant.
You need to pay some attention for analog transducers, where mV, µV even, signal levels are common, and so microphone designers, phonos back in the day, various optical systems nowadays, etc., need careful consideration; a good microphone might have >100dB+ of SNR or dynamic range. Or mastering equipment, where you want to maximize the DR of each channel before summing them all together (while minimizing noise gained by processing steps, effects loops, etc.). But fully mastered media, where at most you're reducing its volume to average listening level? Hardly a problem.
Ultimately what you're doing is impedance-matching the noise of the amplifier. This minimizes the joint effect of voltage and current noise, giving whatever the noise figure is for the amplifier. If the impedance doesn't match, one or the other dominates.
Noise match is generally close to impedance match, though they typically differ by a bit, for unclear reasons.
Now, AM radio specifically, has a lot of tolerance, as the SNR is quite high to begin with. It has to be, because the atmospheric noise background is quite high. We're all bottled in here by ionospheric reflection, both noise from lightning and the transmitters. This means radio stations might have to be 50kW or more, but can also broadcast over a huge area (100s miles radius), even with a typical reduction in power level at night. The power levels mean, despite the long wavelength, quite short antennas can be used; you throw away all your noise margin on antenna gain, and still need a pretty sensitive / low-noise receiver, and come out better on the whole because improving the antenna into a bulky ferrite or loop let alone dipole, gets more expensive way faster than a single JFET or whatever in the receiver, plus an IC doing pretty much all the tuning, IF and detection in one. That is, a "pretty good" receiver plus an awful antenna, is cheaper than a mediocre antenna and mediocre receiver, let alone a "good" antenna.
If you want to throw away some of your noise margin as impedance mismatch rather than antenna factor alone, you can get away with that, to as many dB total between the two (mismatch + antenna) as you would have otherwise. This may make the impedance matching less critical -- but it also means strictly speaking you need a bigger, better antenna, to have optimal reception.
Tim