It looks to be FSK.
But really really really awful FSK. It would be really really awful PSK too.
The abrupt changes in phase where the first derivative goes from positive to negative more or less instantaneously is vile. Such modulation schemes went out with polar relays and 88 mH toroids.
Civilized FSK is not if (D == 0) x = sin(f1 t) else x = sin(f2 t)
The digital waveform is *shaped* with some kind of filter and then converted to a continuous function m(D) between -1 and 1. Then the output is x = sin(f0 + m(D) * delta_F)
That’s not what we see here. Instead we see the instantaneous phase changes that cause enormous clicky sidebands. Those are all the spikes in the spectrum.
But this may be an artifact of your collection scheme? I see that you’ve limited the RX bandwidth to 9.6 kHz. Am I reading that right? For a signal with a shift of 76 kHz deviation, you really want a sample rate above 100 kHz (higher if possible) and you want to open the filter up to at least 50 kHz on either side of the carrier.
Beyond that, one can do a fairly simple FSK demodulator by “beating” the carrier down to baseband with a complex sinusoid tuned to the midpoint between the two tones. This gets you an I and Q representation of the signal. Looking at the IQ values as a vector in the complex plane, the “high” tone is represented by a counter-clockwise rotation of the vector, a “low” tone by a clockwise rotation. This scheme is substantially easier to implement than a BPSK demodulator that needs to track the carrier phase reasonably well. For the FSK demod that I just described, you don’t even have to get the carrier frequency correct, just close.
From eyeballing your traces, this is an FSK signal. But a pretty crappy one.