If you are using the Peltier in a feedback loop to control temperature, would turning the Peltier on and off every few seconds be inefficient?
Yes. On/offing peltiers is not recommended for neither efficiency nor reliability.
First, the Peltier efficiency drops at higher currents (so-called diminishing returns). So basically if you have a heat transfer of, say, 20W at 20W input power (100% COP), you
won't have 40W heat transfer at 40W input power, but maybe only 35W (88% COP). (Made-up example numbers.) This starts to drop quite quickly when getting near to the nameplate power, so if you
have to on/off control it, derate the power.
This means, high current should be used only when high level of cooling is actually needed. 100%-0% control (very slow, or faster PWM, doesn't matter) always runs it at the lowest efficiency possible.
Secondly, despite being marketed as a "solid state device", Peltiers have limited on/off cycles due to thermal stress and may resonate with higher PWM frequencies.
Hence, voltage control is definitely recommended. It doesn't need to be super accurate, and a small amount of ripple won't kill it.
Direct PWM with a MOSFET cannot be recommended for the reasons mentioned, but
open-loop buck, which basically is the same but with added output filter (inductor + capacitor), is well acceptable. (Remember the proper two-switch topology: a freewheeling diode is definitely needed once you add the inductor.) For sizing the inductor and the capacitor, look up for buck converter design aids. You'll find appnotes, spreadsheet calculators, etc.
Although, in that case, just using a bog standard DC/DC converter IC is likely easier, because it implements overcurrent protection (which, again, becomes important when you add
capacitance to the output).
And yes, as a general rule of thumb, there is this physical law regarding Peltiers that the hot side cooling is always 3.14159 times more important and difficult than you thought it is, no matter how well you did your homework.
This is because all experience people have on cooling electronics assumes some dT=60 (so that, for example, the semiconductor junction sits at 100 degC at 40 degC ambient). But using Peltiers efficienctly typically requires optimizing it below dT=10 degC in order to have any meaningful cooling, so it's completely different ball game. If you are dissipating 100W, you need to use solutions you'd normally use for dissipating 1kW.