I researched these HP 859X series of Spectrum Analyzers about two years ago or so, then I stopped.
The #1 problem is the "age" of their internal parts.
The mixer diodes in the first converter will degrade and eventually short or go open, and as time goes on, the DAC range and the accuracy in regards to "amplitude" slowly drifts.
The diodes were made and matched by HP. No individual or packaged diodes that I've seen used as replacements are able to bring the DAC range back into spec.
In the picture you took, the RL-VENR column is the tell-tale. The uppermost number should be between 230-245 and the lowest number should be between 3-20. This scale is in the service manual. Btw, on other machines, I've seen RL-VENR numbers start at the top being as low as 185.
The ERR and SGAIN numbers should reflect a very small fraction, somewhere between 0.00 and .25.
If that isn't enough, the attenuator also degrades. They were once made by Wavetek, so if your attenuator tests bad, you'll need to find another machine to scrounge one from. I've seen a few attenuators on Ebay for ~$100, but remember, they are also used.
I've researched the 856X series as well and it suffers too but to a lesser extent. I believe the 856x's are "lab-grade" analyzers anyway.
I am a beginner in electronics, so the way that I'm describing this may not be 100% accurate.