Digging into this TDS794D now. I ran the extended self-test from the utility menu and got a useful error.
"ERROR: diagnostic test failure, digDataFormatDiag, ERROR!!BYTE mode, in demux 198 ,i= 20 memBaseAdr 0x730002a= data = ffffb400, expectedData[i] = 3400"
Looks like my suspected RAM error, I suspect that ffff is what's making the spikes on the display, pegging the data out at maximum for that value. So next step is to try to figure out what chip contains address 0x730002a so I can avoid shotgun replacing these RAM chips..