No point using huger caps, fix the problem (slow regulator)!
Is this really a problem? I've never bothered to use any of those web design tools. Do they make poor choices for edge cases? Are they just fatally conservative (slow loop = unlikely to oscillate, shitty transient response)?
How important is it that the output is 4.200V? If it's powering e.g. a mobile device, most parts would probably function down to 3V or so (e.g., 2.8-5.0V MCU, backlight controller -- but not a parallel-type LED backlight, and maybe not an LCD panel), or are happy enough with the lowest voltage (LCDs, MCUs, etc. most commonly operating in the 1.8-3.3V range; just use an LDO or buck regulator). If a power circuit (RC, motors/servos?), why not choose motors with minimal voltage ratings and use the controllers to buck down the spare change? (Thus giving a nominal 3V output, but a wide input compliance so regulation doesn't matter.)
I'm oddly reminded of a (RC?) contest that demands the internal supply voltage no higher than 4.20V or something like that...
Tim