One key issue is the generation of the LO signal. The first limitation is the tuning range of the VCO, you will have to use a varactor with multiple capacitor banks to maximize the tuning range of the VCO and that of course has its limits (think about a LC tank in a VCO, L can't be tuned and there are limits on what you can do using switchable capacitor banks).
Most of these chips are direct conversion transceivers, thus you need to generate quadrature LO signals (I,Q) over a wide freq. range. The wide freq. range prohibits the use poly phase filters to generate IQ signals, so typically you follow your PLL output with a muxed divider section at the output. If you need 6GHz you would have to use a 12GHz signal, divide it by 2, to get IQ.
Thus, the closer you want to get to DC the more insane the divide ratio becomes unless you start integrating multiple PLLs and mux them.
The RX, TX signal chain is not that difficult to design for large BW so NF, IP3, IP2, gain requirements are not 'that difficult' (of course that's a 'relative') to meet.
That's a short, somewhat general explanation.