So, I'm familiar with the idea that as you lower the ESR of the output caps after a linear regulator, you increase the chances that it will be unstable (oscillate).
The stability criteria for "normal" and LDO linear regulators are different. The ESR of output caps is not a critical design issue for a normal regulator at all. It all boils down to the general rules of stability as these are common feedback systems where the usual rules and math apply. Thus whatever else, the loop gain must go below 0 dB at 180 degree phase shift for both in order to achieve stability.
A "normal" regulator has an NPN transistor in a common collector configuration. Because the output impedance of a common collector configuration is very low it does not have a pole associated with the output impedance. The naturally occuring pole comes from the gain transition of the pass transistor and it occurs usually at the low MHz. This is high enough to fail the stability criterion. So a pole has to be artificially created and it is called the dominant pole. The knee of the pole is usually at a reasonably low frequency ( in the 100s of Hz). The dominant pole must be so placed that together with the natural pole, sufficient phase/gain margin is achieved and usually this is not difficult (the needed cap is part of the regulator chip).
An LDO regulator is a different beast. The pass transistor is a PNP stone and this makes the circuit a common emitter one where the output impedance is relatively high. This is a bad thing for a number of reasons. A major reason is that now the low frequency pole includes the output capacitance and load resistance. As the pole position wanders on the frequency axis subject to the load characteristics, dimensioning the pole position similarly to the NPN case is just not possible. So you need to do something else.
Recall that a left half plane pole slopes the loop gain down by 20 dB/decade and ultimately retards the phase by 90 degrees. A left half plane zero does the opposite, i.e. it slopes the gain UP 20 dB/decade while advancing the phase up to 90 degrees. Introducing a series resistor to the output cap creates a zero at 1/jwRC, critically advancing the phase to shift the 180 degree crossing to higher frequencies. It does increase the gain as well but judicious placement of the zero will result in unconditionally stable loop _provided the load stays within certain constraints_.
I'd like to explore this problem and I wanted to run some thoughts by the community and see if I'm on the right track.
The above is the mechanism for stability or lack thereof. Explore and understand it, and you are on a firm footing.
Ultimately, is the issue here the gain bandwidth product of the regulator? E.g. if you had more bandwidth, you'd be able to deal with lower ESR caps without oscillating. Is that right?
Not directly or solely. The more gain your loop has, the more potential for instability (depending on the relation to the phase). The critical question is whether the stability criterion is fulfilled or not. The loop gain _must_ be below 0 dB @ 180 degrees. Funnily enough, gain can increase again after the phase lags beyond 180 degrees, but at the exact point this condition must be met.
Another thing which plays into this is the regulator topology. E.g., an NPN pass element has the down-side of requiring more headroom (larger dropout voltage), but is more stable (has more phase margin?) than an LDO / PNP pass element regulator. Does this mean that the ancient LM78XX regulators can deal with lower ESR than modern LDO regulators?
Yes - they can deal with 0 ESR
. No zero needed in the transfer function...
I have read that the gain bandwidth of the LM78XX series is around 100kHz. One way to get more bandwidth would be to go one step closer to discrete, where we use a discrete op-amp and pass transistor. As you use a faster and faster op-amp, does this mean you will be able to handle lower ESR caps? Or would the additional speed make overshoot and ringing more likely?
Would there be any reason to go even more discrete, ditching the op-amp for individual transistors?
Hope the above already gives you tools to answer that one yourself.