It appears to be inverting properly. What were you expecting?
Note that, at startup, the NMOS is a follower, i.e. the gate charges up to about +12V, and the source follows about Vgs(th) below, maybe 9-10V. You won't see this visually (it's a tiny difference in LED brightness) but you can show it with a multimeter or scope. After pressing the switch (and waiting for the LED to fully charge the capacitor -- the first time it's cycled, this will take some milliseconds; not nearly long enough to see visually, but easily checked on the scope), it should saturate properly (source to 12V), until the cap charge leaks away (which could take seconds to hours, or even days with good components?).
If you wanted noninverting operation, try +5V to the 1k base resistor, and switch from base to GND (shunting base current)?
Tim