Noisier in what ways? Higher Fsw is easier to filter the ripple (fundamental and low harmonics), at the expense of greater switching loss (and somewhat higher average HF noise). Commutation noise (that associated with the switching event itself) depends only on the layout and filtering (the peak value of the noise, that is).
We typically call the former "EMI", and the latter "RFI".
An EMI choke (common mode) typically has little to no effect on RFI, partly because it is hardly effective at such frequencies (most chokes are effectively small capacitors at that frequency), but also because the equivalent circuit coupling into the conducted or radiated emissions is different at those frequencies.
EMI is easily filtered, through DM and CM filtering and conventional components; RFI is harder to filter, requiring a more carefully constructed filter with a compact star grounding topology and shielding if necessary, or can be treated at the source by dampening the switching edges or improving layout.
Tim