I haven't seen your full circuit schematic, so I cannot judge if there are any other possible problems to be solved. The above schematic posted of your circuit seems not to be complete - as it would not work like that in reality.
You do not need a PMOS. Why would you? You can use either a NMOS or NPN in the + rail as the series pass element, as an source or emitter follower respectively. (NMOS having the requirement of higher driving voltage, as the Vgsth is about 3-4V, compared to Vbe of 0.6V for NPN).
IF you need your supply to output up to 5V, using a mosfet as the regulator, the gate to circuit ground voltage required is 5V+Vgsth (if the control circuitry will be referenced to the negative output terminal - which I recommend to do). So you need your voltage regulator opamp to output at least 9V on the gate (measured against circuit ground, negative output terminal). So, if you power the control circuitry from a 12V or 15V I'd rather go for 15V to be on the safe side, any jellybean opamp will do fine. If you need somewhat precision of the power supply, look for an opamp with low offset voltage and low drift. (OP07 comes to mind, as a very cheap precision opamp. Also, same applies for the current regulating opamp!).
If you need further help, please provide us with the full schematic, I (or anyone else) can look through.