Amplifiers are often (almost always?) compensated to keep them stable. That is, the bandwidth, or gain at high frequency is limited to keep it from oscillating/introducing distortion.
If you don't want to think about bandwidth, you can think about voltage slew-rate, or dv/dt limitation. As frequency increases, holding amplitude constant, the maximum dv/dt of a given sine wave increases. As amplitude increases, holding frequency constant, the dv/dt also increases. So, if you are operating at the amplifiers maximum dv/dt, then if you want to increase frequency, you need to decrease amplitude, and vice versa.
Probably some inaccuracies in this explanation, but that's how I think of it.