Doesn't the 4066 give quite a spike/dip (of a few tens to hundreds of nanoseconds length) on the passed-through signal whenever you switch it? So if you're only switching infrequently you're fine, but if you're switching very often you'll find these spikes replacing non-insigificant perods worth of the signal you're switching.