1 is simply outright bad: ambient noise adds in series with a given cable, and without a ground at the receiving end to pick up that ground reference, the noise is added directly onto the signal. As you say, it might be okay with slow signals -- given the additional provision that the signal is well filtered first! Which is a good principle in general, only use what bandwidth you need to.
2 is the preferred way, and you can improve it further by using a differential receiver so that ground level is not ignored. The relevant case here being, ambient noise (in this case not so much RF thanks to the AC grounding, but mains and slowly varying offsets ("DC")) skews the baseline, adding phase noise.
Use lots of caps in parallel, to approximate a wide ground connection. Just one will be adequate for low frequencies (a few MHz), but you will need many in parallel to cover the full range of interference. Again, this is give or take signal filtering.
This is the preferred solution for dealing with ground loop, in audio applications. (For analog signals, the differential receiver is simply a diff amp.) Or, it would be if they knew about it, but no one seems to care. Go figure?
3 is good of course, when it's applicable. PPS could be converted to a RZ code first, or modulated in any number of ways (ASK, FSK, etc.). Note that an RZ code (basically short pulses up or down) is high bandwidth, so is very susceptible to noise, especially impulsive noise. This can be addressed somewhat with a windowed filter (disable the input until very near the expected arrival time of the next pulse), so there are digital solutions, but ASK and especially FSK offers strong benefits from signal filtering, and you can still do the window filter after detection if you like.
Tim