I think it work not at all or poorly at best.
I just checked the datasheet and confirmed my memory that the LM311 response time is 200 nSec. That is probably a typical number (not worst case maximum.) But a complete cycle (one period) of a 4 MHz oscillator is 1/(4MHz) = 250 nSec. The output needs to switch both up and down during one period. This means to me that the response time MUST be no more than 250 nSec/2 = 125 nSec maximum, and probably less than that.
I think that if you want to use a comparator for this purpose, you probably ought to use one with a response time of 50 nSec or less. I have never designed a crystal oscillator and I am not in a position to give you a design unless I were to spend some time working on it.
When using a crystal be careful not to overdrive it....