The problem wouldn't so much be lack of frames, but lack of frame updates. No use staring at last minute's idilic frame buffer when at this very instant you are heading straight towards a big moose/cow/kangaroo/alien*.
* Adjust for local wildlife setting.
My guess is it's a fast shutter - if it's a 60fps imager, losing a few frames is probably tolerable. It could use avaraging over a few shutter images to reduce noise in the reference image if necessary. Could also be a continuous wheel, and higher internal sample rate
It could also analyse the image for motion and only do a cal when there's not much action going on.
As it doesn't care much about actual temperature, Another option may be to analyse the incoming data for pixel-timescale variations and take a long-term avarage to estimate the variation between pixels. You may get occasional visual artifacts if the camera is looking at a stationary image for a while, but you don't really care about this for an automotive vision system as it's movement you're looking for.
Another alternative is they do a factory cal over a range of temps and store the nominal offsets for each pixel at each temperature.