The output capacitor takes over the output impedance for the higher frequency end. So one should have a fast regulator loop, so that the regulator can generate a suitable low Z even up to higher than normal frequencies.
The output stages also tend to become slower when at relatively low current. So one may consider a low current and higher current part in parallel to also get high speed at low current. It can also help to have the output work as a push pull output - getting a minimum current and being able to go down fast as well. This can also avoid working close to the limit, where anti windup can kick in.
Some kind of anti wind-up can help to speed up the transition between CC and CV modes, but this part can also be tricky.
For the power devices there are relatively fast BJTs made for audio. With a suitable gate driver MOSFETs are also quite fast.
Much capacitance at the output can make the regulator react slow. For stability one likely needs some capacitance with loss (series resistance). It can help to have not more than needed for stability - more usually only makes things slow.
Thank you very much for elaborate answer.
As for slow down at low output currents, I currently addressed it with a current sink on output. So, for 0-100mA output, I have a current sink of 20mA, this way output current changes only 20-120mA, which helps with stability. This approach is not without downsides, one is, the current sink needs to be precise, or current measurements will be off. And if there are multiple current ranges, the current sink should switch ranges too. It's still attractive because I can, say, "auto-zero" current offset. I also haven't figured out how to make CC and CV loops co-exist without oscillation (and I have piles of documents yet to read on opamp compensation and stability).
As of proposed design approaches (push-pull and parallel stages), I currently don't have skills to design a system with push-pull output stage, or with parallel stages. I'm afraid push-pull is too complicated, as it probably needs the current shunt to be on the ground. The good news is, this way it feels like it's easy to get a two-quadrant power supply. Or even four-quadrant? (I'll read AoE).
Parallel stages... do you have an example of a device that does it? If not, can you please explain how it works? Is it like two power supplies, with one (the big one) having slightly lower voltage output that will take the load if small-current supply will go in CC mode under the load?
PS BJTs I consider are TP2955/TP3055 for high-power output, BD139/BD140 for low-power output.