Thanks, I checked them but while some voltages were absent, I wasn't seeing ripple, really, and I wasn't able to find one making noise.
That said, I did use a sound meter to trace down the sound, and then eventually found the cause and fixed it!
The sound was coming from the transformer, and the sound was generated deep enough inside that touching the tape around the windings or core itself simply did not contact the part making noise. I scoped around for a bit and identified that the power for the controller and PFC controller was low - and when viewed at the main filter cap after the diodes for it - was normal looking except for brief, regular excursions to zero. All the way to zero, with very fast edges for the ~8.5ms period they were happening at, and their width was so narrow, it wasn't showing as AC activity when I had my meter on DC mode using it - it only showed on the scope.
After some tracing, removing and checking parts, and narrowing down, I found that the same drop was present at the switching controller, except that the downward spike actually went negative (this was being blocked from the rest of the rail by a series diode), and there was no negative signal or inductor (aside from the transformer) for it to be just switching to. After some checking, it turned out a little extra solder between two pins had formed a very small bridge that really didn't look it under magnification, and that getting rid of this meant no noise and full startup.
The unit had been broken when I got it, and I had replaced the controller IC, so it was my fault in the end, but I think replacing that, the PFC controller, and a diode or two actually would have fixed it months ago when I made the first attempt, would I have been slightly more sparing with my solder application.