24V is above the 2N7002 Vgs(max), it can cope with +/-40V Vgs(max) briefly, but would suggest your application is too much. Could you replace the jumper with a resistor to lower this? Are the jumpers field replaceable?
Would suggest that R3/R5 are quite low also. Depending on the quality of the 10KHz you want you can probably increse these values and lower the current. Also, the 2N7002 is failry well turned on by 2.5V to 3V, so you could re-ratio your dividers a bit.
Q3 is the biggest issue though. It is a P-channel and in your circuit it always has some voltage between gate and source. Vgs for this part is -1.5V to -3.5V, so it's probably off at 5V rail (gate will be 4.1V, so Vgs=-0.9V), however at 24V rail it will be at 19.7V (Vgs=-5.3v) so very much ON.
Vgs(max) is also +/-20V, so you might get away with removing R6 - but not a solution I'd be comfortable with. You are slightly bollox'd if you've done the layout - probably resort to a scalpel to rework it or find a MOSFET that has a bigger Vgs or somesort of protection built in.
Quick other thought - you *might* get away with an N-channel as Q3 if you reverse drain and source, and flip your NPN-PNP assignment, but you won't hit 24V out and may have other complications depending on what you are driving.