I never noticed this, but let's see what others think.
Work well in my DSOX3034A
It seems to be much worse very recently, I am thinking now it may be a physical fault, hopefully a bit of switch cleaner will resolve.
Could also be a cracked solder joint or PCB track near the encoder. One way to diagnose this is to see if it gets better or worse if you apply a slight sideways force in various directions while turning
The symptoms you mention were indeed there, however when I disassembled the unit there was no evidence of any bad joints. Rather than just re-touch them and re-assemble, I had a look in the parts drawers and found I had almost identical device in stock that fits perfectly and functions identically electrically with the right number of detents (24) and a push switch. Even the shaft is the right length and type. So I replaced the knob. It works perfectly.
The part is RS part number 6234237, and the manufacturer's details are Alps EC12E2424407.
The problem has been there since I bought the scope 18 months ago, but it wasn't that bad, I just put it down to Windows CE. It seems like it's been getting worse over time, but only now I've done anything about it as it became bad enough.
Edit: the job took about an hour and a half in total. Getting the old switch out is a chore, I snipped the leads in the end rather than risk ruining the PCB by trying to reflow it.
Edit 2: Coincidentally, this is the same encoder I used on a Rigol MSO1074z for the multi-finction knob mod.