Now try to switch voltage ranges on the scope, to the point where the relay will click (and adjust the signal level if necessary), and you'll very likely see yet another change in the vicinity of the rising/falling edge.
I've never been able to compensate a probe to get an ideal waveform display in every condition. Yes it works fine if the waveform is zoomed out sufficiently: when 5-10 periods fit on a single screen, then it's a perfect square wave. Things change when you start to zoom in.
That's why I stop at "good enough" for the probes. It works for practical purposes, considering all the other limitations of passive probes, but it still does not explain the fine effects of it. I am pretty sure there are good articles on what probe compensation actually does, and they can be used to get a better understanding.
To answer the "is it the probe or is it the signal?" question* you need to compare what you see with the probe with what you see without it: connect the signal generator, assuming it has 50Ω output impedance, using a coax cable with a 50Ω feed-through terminator at the scope input, that will allow you to see the signal with as little distortion as possible. Of course, internal 50Ω termination would be much better, but your scope doesn't have it. External terminator should be fine except very close (few ns) to the edges.
*but not the "is it the scope?" question: that is yet another variable.
p.s. You do understand that a compensated probe is only compensated relative not only to a given scope, but also to a given input of that scope (and as I mentioned above, also, it seems, to a given vertical scale range), don't you? The capacitors and wiring on each input can differ sligthly, so they will require a different amount of probe compensation.