For both channels I use the same settings:
Coupling = DC
Bandwith limit = on
Probe = 10x (Also 10x on the actual probe)
Digital filter = OFF.
For probe calibration, bandwidth limit should be OFF in general. But then again, the BW limit cannot explain the odd effects shown here.
You haven't set both channels identical, at least there is no bandwidth limit on channel 2.
Don't know about the digital filter - make absolutely sure it's really OFF, because it's about the only setting I can think of which could cause such funny results.
I use the squarewave on the scope as input for both channels, and I use the same (one) probe for both channels.
Channel 2 gives me the expected squarewave with 3 volt signal.
Channel 1 however gives me 1/5 = 600 mV and the signal is noway near a squarewave. Se attached photos.
Sorry, but I see something different.
You get a clean squarewave 0 to 300mV (=300mVpp) on channel 2.
On channel 1, there is a triangular waveform 128 ~ 680mV (~550mVpp), which might be a hint on a bandwidth limit set very low (digital filter?)
If the settings are correct and the weird behavior persists, I would suspect something's wrong with the Ch. 1 input.
If it's the usual split path amplifier configuration, it looks like only the DC path (with an OpAmp) is working, whereas the HF amplifier is dead. Maybe a capacitor has failed (unlikely, more likely would be a bad solder joint) or a transistor or two have died and need to be replaced...