What part of it, exactly, are you trying to pot? If you can keep the potting material away from the signal lines (or outside the shielding), it can be basically whatever you want, because then your dielectric is either the stuff already in the connector or it's air. I don't see why you'd need to get potting compound either in the connector housings or on the header contacts, so I'm not sure the choice of materials would need to be too elaborate.
I think mentioning other connectors as options isn't a bad notion, since a repair could just swap them out, but it may be difficult to find very different ones in that form factor. If the mounting of the XLR plate is solid, then maybe the best way to relieve physical stress on the connectors is actually securing the wires externally - like bundling with other cables or holding down to tie points or something.