Use a P-channel Jfet as a pull down, Jfet source gets tied to your existing gate signal, Jfet drain gets connected to ground/source though a current limiting resistor. The Jfet gate gets tied directly to the ground/source connection.
When your gate signal is high, the Jfet gate is biased negative, and the Jfet is clamped off other than a couple uA of leakage. This leakage pulls down the big mosfet, slow at first, but as the voltage declines, the resistance of the Jfet decreases also, increasing how fast it pulls the voltage down.
When you turn back on, you have to fight the Jfet at first, (hence the series resistor) but once you get enough voltage on the gate to pinch off the Jfet, its back to a nice low-current state.
Now, what Jfet to use? I cant give solid advice on that, most datasheets I find give currents at a Vds of 15V, so at 4-5v leakage at cut-off could be vastly different, or maybe not.
The MMBFJ176 shows a 10nA drain current with 15Vds, and the cutoff between 1.0 and 4.0 volts, which unfortunately is a pretty wide range, given that this depends on the VGS(off) to set the point at which the pull down changes from low to high current.