Dave, you set a challenge for us to find something simpler.
OK, here is my attempt. One transistor, one FET, one capacitor and four resistors.
I tested it is LTSpice, so that's the reason for the weird switch circuit.
I also added a load as there is one problem with this kind of switch. If the load has filter caps across the supply usually. Those capacitors take a time to discharge. If I increase the load capacitor from 10uF to 100uF, the circuit will not switch off, as at there is enough voltage from the load to turn the circuit back on. I think your circuit has the same problem.
So to make this work, you have to find the worse case decay time of the load, and adjust C1 to suit.
This circuit does not oscillate. If does not matter how long you press the button for.
Richard.